Trial implementation of an Item response Based Computerized Test for psychosocial Problems in Dutch Preventive Child Health Care for Children Aged 7-12

-The project is a follow up on a previous successful study in the ZonMw program
*Prevention* and is a necessary step in valorising the results of this. The
primary aim of this study is to establish which factors are important for a
successful implementation of a newly developed effective instrument for the
detection of children with psychosocial problems. Additionally the proposal
aims to test in real life how effective the method is in improving detection by
JGZ. The project also aims to determine the efficiency in terms of demand
placed on the parents (number of questions to be answered) and the ease of use
for the JGZ (for example in terms of a higher specificity of the findings and
the time saved).

-Benefit to society
Society can benefit considerably from this project. Psychosocial problems are
highly prevalent and a good early detection increases the chance of an
effective treatment and a healthy development. This project aims to realize a
fundamental improvement in the quality of this detection using methods that
have already demonstrated their potential. Successful implementation will
therefore result in an improved detection and a more specific advice to
and/or referral of (parents of) the children. Moreover, a good detection by
the JGZ prevents the referral of children who do not have serious problems.
The introduction of this method is therefore a way of implementing the
recommendations of the Invent group. The method is cost-effective for the
JGZ, as no time needs to be spent on manual scoring. In the future, the
results of the measurement can be automatically incorporated in the
Electronic Child File (Dutch acronym EKD for Elektronisch Kind Dossier).

-Improved efficiency and effectiveness compared to standard alternatives
Questionnaires the JGZ currently uses and the questionnaires currently being
developed are based on traditional psychometry. For practical reasons the JGZ
cannot use long questionnaires and short questionnaires are less sensitive
and specific. Also they only provide global information: the exact severity
and, as subscales are even less reliable, the precise nature of the problem
remain unclear. A CAT can therefore lead to a considerable improvement in the
quality and effectiveness of detecting psychosocial problems.

Aim of the project is to determine whether it is possible to implement a CAT
for the identification of children with psychosocial problems in Dutch JGZ.
Additionally the study will evaluate the CAT as it is being used in practice.
The trial implementation will be evaluated and the following questions will be
answered.
1. Which factors can facilitate or impede a successful implementation of an IRT
CAT such as that which has been developed;
2. Can the validity of the IRT CAT as established in the simulation study be
replicated in real life and;
3. Does the use of the method result in an improved detection and more specific
follow-up actions by JGZ.
Based on these results regarding the first question an implementation strategy
well be developed, after which the trial implementation will be carried out.
During this trial, additional information on factors relevant for
implementation will be gathered, mainly based on actual experiences of
individual users with the application.

The study design has four phases:

1 Survey of factors relevant for a succesful adoption and implementation
2 Development of an implementation strategy
3 Trial implementation
4 Analysis and reporting.

Phase 1: Survey of factors relevant for a succesful adoption and implementation
In the first phase four workshops will be organized, spread over the
Netherlands, during which we will discuss relevant implementation factors with
Preventive Child Health Care (PCH) professionals. This after a presentation of
the CAT as it was developed. The discussion will be structured using the
implementation model of Fleuren et al (2004). Questionnaires, based on the
same model will be used to stimulate the participants.

Phase 2: Development of an implementation strategy
The work in this phase is strongly dependent on the results of phase 1, but
will include the development of a training for a co-ordinating staff member of
each individual PCH centre that will participate in phase 3 and of trainings
for each individual staff member who will participate. The existing application
may need some adaptation and possibly UMTS laptops need to be made available,
in order to enable the CAT to be completed on the PCH centre, in stead of at
home.


Phase 3: Trial
After training participating PCH professionals will send parents a letter,
asking them to co-operate in the study. Only parents who are invited for a
regular health check up will be invited for the study. Parents are asked to
fill in and sign an informed consent form and to give that to the PCH
professional.
When parents are willing to cooperate they have to fill in a paper and pencil
questionnaire and, if possible, the CAT via the internet. The questionnaire
consists of the Child Behavior Checklist (CBCL) which is used as the primary
criterion measure. It also contains a limited set of demographic questions.
Finally, those parents who answered the CAT will be asked how they valued it.
Parents who did not answer the CAT, will be asked, why not.
The questionnaires are put in a closed envelope, anonymously, but with a code
enabling linking information from the parents to information provided by PCH
(see below). The envelopes are returned, unopened, to TNO by PCH.
During the regular health examination, PCH professionals use the results from
the CAT for their assessment of the presence of psychosocial problems. They
register whether such problems are present and also which follow up actions are
undertaken. When parents are not willing to participate, they register their
reasons and some demographic characteristics, but only if parents allow them to
do so.
Each participating PCH professionals rates the extent to which the CAT
contributed to the assessment, after 30 health examinations each. These ratings
are not linked to individual children.

After completion of the trial a workshop will be organized during which the
experiences with the CAT will be evaluated and possible improvements are
determined.

Phase 4: Analysis and reporting
De results of the workshops will be described, using descriptive statistical
analyses of the questionnaires answered during the workshops. ROC analysys
between CAT results and CBCL data (dichotomized, first criterion measure) will
establish sensitivity, specificity and Area Under Curve.
We will establish the number of cases for whom psychosocial problems are
detected and for whom follow up actions will be undertaken. It will be tested
whether these percentages exceed those found in an earlier study among children
aged 7-12 (Vogels et al, 2005)

The study takes place in the context of the regular health examination for
which children in the age of 7-12 are regularly invited. For the purpose of
this study, their parents will be asked to fill in a written questionnaire
(about 150 items) and, if possible, to answer the CAT via the Internet (35
items at most). It is estimated that this will take up to 30 minutes and
implies no risk.