Análisis de Riesgos para ambas alternativas

This chapter presents summaries of the highest quality evaluations of parental involvement interventions found in the literature. What it shows is two things. A far higher quality

evaluation of a simple of parental involvement intervention is desperately needed. And there is not yet enough evidence here that any intervention will work. The most promising

elements of these seven interventions are summarised in Chapter Nine, but this is made difficult by the fact that six of them are very complex in structure (and three studies are by the same team of the same intervention). The studies presented in Chapters Five to Eight are of lower quality, and so are not generally described in as much detail. Their overall findings are just as ambiguous as those here.

37

Summary

• Applying our quality criteria, seven studies (seven instead of eight because two were the same study, with the second study analysing the data by gender) of medium and/or near medium quality were found. There were no studies judged to be of high quality.

• Four showed positive effects of parental involvement (PI) on academic outcomes, but these were multi-component interventions of which parental involvement was only one of them. So it is hard to say if PI was the active ingredient. It is possible that a combination of these (e.g. classroom and economic interventions) and PI work together feeding off each other and that is the magic potion. We do not know unless we conduct a series of trials that allows one to analyse the contribution of each component separately.

• Also, three of the four studies were conducted by the same team examining the same programme. These studies examined the effect of the Chicago Child–Parent Center Program, which uses a combination of classroom and parent activities to build a school-based learning environment which actively involves parents.

• Of the other three, one showed PI made no difference to academic outcomes and the other two showed that PI may actually have a negative impact, suggesting that children may actually be better off without the intervention.

• Although the intervention did improve reading motivation in the negative study, this was not translated to performance, indicating that motivation does not lead to performance unless accompanied by competence.

38

CHAPTER FIVE – PARENTAL INVOLVEMENT INTERVENTIONS FOR PRE-SCHOOL CHILDREN

There is a near consensus among policy-makers and practitioners, with support from some research commentators, that identifying children’s problems early on in life and intervening when they are young is more effective and efficient in overcoming their relative disadvantage at school than doing so once they are in school. This is the long-standing rationale behind early intervention programmes such as the Head Start, Sure Start, HIPPY, the Abecedarian and Carolina early intervention projects and other school readiness programmes. According to Heckman (2006), such pre-school interventions,

‘promote schooling, raise the quality of the workforce, enhance the productivity of schools and reduce crime, teenage pregnancy and welfare dependency. They raise earnings and promote social attachment. Focusing solely on earnings gains, returns to dollars invested are as high as 15–17%... a rare public policy initiative that promotes fairness and social justice and at the same time promotes productivity in the economy and in society at large.’

Of all the early interventions, parental training programmes appear to be most prevalent, and their success widely accepted. In fact, C4EO was so confident in their own review analysis that they made engaging parents and parenting programme one of their priority areas (C4EO 2010). This may explain why the largest number of interventions uncovered in this new review relates to work with pre-school children. Many interventions for this age group involve parental training, and sometimes a combination of parental training and home support or home instruction. These include training mothers to interact with their children, providing instructional materials and guidance to support parents, and training parents to use school-related activities to support their children. Unfortunately, the majority of these studies are of low quality. They are grouped here into negative or unclear studies in which there is no clear indication that the parental involvement intervention works, and those that claim or report positive outcomes. In reality, because of the nature of the evidence all are somewhat ‘unclear’.

5.1 Studies with positive outcomes

Boggess (2008) conducted a study of providing parents with educational materials to help them to practise essential skills at home with their children to prepare them for

39

kindergarten/1st grade. Support was given to parents and children to help them along. The intervention children whose parents were provided with extra support and resources

performed better in terms of Georgia Kindergarten Assessment Program – Revised (GKAP- R) results compared to a group matched on standardised maths and reading scores. There were only 18 cases in each group and no randomisation. Participating parents were

volunteers. It was not clear if implementation of intervention at home was monitored. The quality of the study is low.

In another study, positive effects were reported for training parents to use school activities at home for practising State Reading Tests (Calnon 2005). The reported effect size was

medium. However, the 255 disadvantaged families were divided into a treatment group of 114 where the parents volunteered to take part and a ‘comparison’ group of 141 made up of those who refused to take part in the intervention. The researchers incorrectly used

significance testing even though neither the sample nor the allocation to groups was random. A number of parent and student activity logs were missing. Fidelity of treatment is thus questioned, as it is not possible to verify the actual time spent on academic activity at home. Also, the parent surveys were completed by one group only, making comparisons between groups impossible. This study is of low quality and potentially very misleading because of the volunteer bias.

Fagan and Iglesias (1999) evaluated the Head Start programme, which involved a range of activities, but included the training of fathers about literacy and numeracy materials, trips and outdoor activities. The children had an average age of around four and a half. A total of 146 fathers were selected for the study but only 96 (66%) took part, with 55 in the intervention and 41 for comparison. The comparison group were not involved but taken from other geographical areas merely deemed to be equivalent to the sites of the intervention. The researcher reported that fathers on the programme showed an increased amount of time spent with their children compared to the other group. The children of intervention fathers scored higher on applied problems and letter-word identification. However, again, the study used multivariate analysis of variance (MANOVA) inappropriately with these non-random and non-randomised cases. There is no evidence that any subsequent difference was not already inherent in this volunteer group. The quality of the study is low.

A family literacy project known as Project EASE (Early Access to Success in Education) focused on training parents in supporting their children’s language skills by providing scaffolded activities which were stage appropriate (Jordan et al. 2000). Parents of 177 kindergarten children in eight classes attended training sessions in school and were

40

provided with book-mediated activities for use at home to reinforce what was learnt in the parent sessions at school. All children improved on Peabody Picture Vocabulary Test- Revised (PPVT-R) and the Comprehensive Assessment Programme (CAP), but Project EASE participants made greater gains than a comparison group of 71 children in three other classes not receiving the intervention. This included vocabulary, story comprehension, sequence, sound awareness and concept of print. The size of the difference varied

according to the amount of participation measured by completed book-related activities. The change was more impressive for children who scored lower in pre-tests than others. This, of course, could be the result of regression to the mean. Clarity of reporting was deficient for this study. For example, it was not clear how the allocation to groups was decided. It was also not clear whether interventions at home were monitored, so it is hard to say if the intervention was implemented as intended. The study was rated low.

Mullis et al. (2002–2004) reported a study that evaluated a parent support intervention to encourage parent–child interactions promoting dialogic reading, vocabulary development and print awareness among pre-school children. Parents were provided with information about early literacy. Parents also received activity worksheets to encourage them to work together with their children. A convenience sample of 41 children from three childcare centres were chosen to participate in the programme. These were compared with 26 children from two childcare centres not involved in the programme. However, only 35 parents in all took part (n = 13 for intervention; n = 22 for comparison group). Pre- and post- test comparisons reported that the experimental group improved more in print awareness and receptive vocabulary than the comparison group. No improvements were found in basic concepts and other measures of vocabulary. The researchers attributed this to the nature of the vocabulary tests. However, no baseline equivalence was established, so the groups could already have been different, and the researchers were wrong to conduct significance tests with these data. The results could be misleading, and the study quality is low.

Starkey and Klein (2000) described two experimental studies that evaluated the impact of training parents to support their children’s development of maths and early reading skills. Thirty families with pre-school children in the Head Start programme were involved in the study. The families were African American and Latino, selected according to three criteria: a) child did not have special needs; b) at least one parent did not have mental health or

substance abuse; c) family was low income according to US federal guidelines. A pre- and post-test comparison with a comparison group was carried out. The dependent variable was a maths test. As a different type of ‘control‘ the study also tested the pupils in literacy. The hypothesis would be that the scores in math would increase but not in literacy. Positive gains

41

in mathematical knowledge were reported for the intervention group, but not for the

comparison group. There was no difference in literacy. Despite this innovation, the weight of evidence for the study was still rated as low because of the very small sample size (n = 30), the lack of randomisation to treatment conditions and the lack of baseline equivalence between groups as evidenced in initial differences observed in the pre-test scores. This poses a threat to internal validity. Limiting the sample to only African American and Latinos also reduces the generalisability of the study.

Klein (1990) used a quasi-experimental design to re-evaluate a previous study that looked at a programme of parent education. Details of the original programme are not reported. The study used extant data from the previous study to compare 19 first-time parents involved in parent education with 22 other families. Results suggest that children of parents who participated in the parent education and support programme exhibited increased levels of cognitive competency, and treatment children scored higher on all measures of cognitive ability than comparison children. However, no difference was found for composite score and there was little or no observable difference in social competency. This study was rated low because of a combination of small sample size (n = 41), the lack of details about the original study design and the fact that the sample was unlikely to be representative of national population.

Dieterich et al. (2006) started from an assumption that training parents, using Play and Learning Strategies (PALS), does enhance their children’s cognitive and social development (even though the previous section casts doubt on this). PALS is a home-based parenting support programme that helps parents to learn and practise a number of strategies to enhance their children’s cognitive and social development. PALS facilitators were research assistants trained to work with families using PALS strategies. They made weekly visits to homes to coach and train mothers in using PALS parenting concepts. The study compared this approach with M-PALS which adds a mentor to the work of the facilitator. Mentors were recruited from the community and were trained in knowledge of social services and

identifying mothers’ personal and social needs. In addition they also provided support to mothers in implementing the PALS strategies. Dieterich et al. (2006) reported positive effects (d = 0.62) on children’s cognitive skills measured using the Mental Scale from the Bayley Scales of Infant Development, and claimed that this difference was due to the use of mentors. The study initially involved 132 low socio-economic group families with young children aged 4 months to 4 years. Over 11% dropped out, almost entirely in the comparison group, leaving 46 in PALS and 71 in the M-PALS conditions. The two groups were matched on a number of items (not including older siblings), but were heavily unbalanced in terms of

42

ethnicity and marital status. The researchers attempted to ‘correct’ for non-randomisation and bias through the use of propensity scores, and then analysed the differences between groups using F-tests. Again, this use of probability calculations for non-probability samples and groups is a serious error. The study is rated as low quality, due to scale, dropout, inappropriate counterfactual and incorrect analysis.

Pungello et al. (2010) examined the effects of early educational intervention, namely the Abecedarian and Carolina projects, on academic outcomes of young adults. They used data from two longitudinal studies of cohorts to ‘predict’ the effects of early home environment and early educational intervention on the academic outcomes of young adults. The total sample was 137 (67 receiving early educational intervention and 72 controls). There were 104 cases in Abecedarian treatment and control groups combined, and 35 in the CARE groups. The smallest group was only 14 cases in the CARE treatment group. There is no description of response rates or dropout over time. The analysis presented suggests that early intervention was significantly associated with general educational attainment, post- compulsory participation gainful employment. Post-hoc tests of indirect effects suggest that home environment mediated the early risk factors for control group, but not for the treated group(s). The treatment took place in a university research centre and may not be

representative of childcare experience for most children. The researchers were wrong to use significance tests with these non-random groups. Therefore, this study is rated low.

Garcia (2006) evaluated the well-known home-school collaboration project, HIPPY

programme (Home Instruction for Parents of Pre-school Youngsters). HIPPY is a free, two- year home-based early intervention program for four- and five-year old children, designed to provide educational enrichment to at-risk children from poor and immigrant families. It involves a 30-week curriculum using an explicit, direct, instructional program. Lessons are designed to develop a child's skills in language development, problem solving, and sensory and perceptual discrimination. The programme is delivered by trained professionals living in the same, targeted high-need communities as the families they serve. It includes role-playing during biweekly home visits and monthly group meetings to engage parents in learning activities with their children, and help them to view themselves as active agents in their children's education. The intervention group (n = 35) included those who attended an early childhood school as a four year-old and participated in the HIPPY 4 and 5 programmes. The comparator group was made up of those who attended an early childhood school as four year-olds but did not participate in HIPPY (n = 35). Outcome measures were the TAKS (Texas-mandated criterion-referenced Texas Assessment Knowledge and Skills) and TerraNova and TerraNova SUPERA norm-referenced test scores. The researcher used

43

ANOVA and reported a significance difference between groups in five out of six test scores. Given that the groups were not randomly allocated, nor was the treatment group randomly selected, this use of significance testing is incorrect. Due to the lack of attempt to ensure balance between the initial groups and the small sample, this study is rated low.