PRIMERA CLAUSULA ADICIONAL TARJETA AZTECA

2. Basic internal design of study 3. Other threats to validity B. Main effects C. Moderators 1. Patient variables (a) sociodemographics (b) premorbid status psychological variables cancer variables (c) individual differences social support

2. Therapists and therapeutic techniques (a) treatment components

(b) treatment groupings (c) theorized mechanisms

(d) nonspecific therapeutic variables D. The longevity of tx impacts (follow-up) E. Practical cost-effective strategies

46 Preliminary issue re number of studies for a moderator analysis: (Devine & Westlake, 1995) required 10 or more. Shane has not seen any rule. Its about the amount of variability captured – looking for reduced variability, overlap, different means. Three big studies may suffice, whereas 10 little ones may not. He and Jo used about five. Perhaps we could take the rule for „evidence based‟ treatment approval as a lead, ie two big RCT‟s heading the same way make for an „established‟ treatment. We will keep an eye out for an official stance on this.

A. Quality of Methodology 1. External validity issues

 publication bias

o do published, peer reviewed have higher ES? fail safe N? funnel plot (Egger, 1997; Greenhouse & Iyengar, 1994)

o do foreign original language have higher ES?

o do smaller N have higher ES? (Egger, 1997). Draw a line at 100 (Barsevick, Sweeney, & Haney, 2002) cites authority

 age of study

 nationality of participants

 sample selection strategy (representativeness, ie volunteers etc) o age

o ethnicity

 discipline of therapists

Analyses:

o descriptives of all of the above

o significance testing for publication biases of ES

o sensitivity analyses re age of study, age, discipline of therapists, checking impact on main effects (only, ie no moderators)

o comment on generalisability

o feed impact of publication bias into the handling of study quality (below) 2. Basic internal design of study

o Design description

o Sample sizes, incl comparison of N in tx and control

o Strategy for dealing with selection effects (unknown, unmeasured moderators), including o randomization

o concealment of allocation to group

o attrition and missing data (ie where all data has been removed for people who only completed some of the observations over time, bias (Bottomley, 1997) p258]

 numbers  reasons

47 o equivalence of groups (ie whether pretest taken into account in final analysis) o control group

 presence

 nature (impact of attention/placebo v. treatment as usual v. wait list/nothing)

Analyses:

o Descriptives for all of the above

o Sensitivity analyses for selection effect variables over main effects (only), ie which make a sig difference to ES? (excluding repeat measures studies)

 Impact of different types of control group on ES. Is attention itself a big component of therapy (ie placebos erode most of tx ES?). Diffusion/imitation of treatment / control has elements of treatment (incl assessment of Hawthorne effect, or non-specific therapeutic effect of attention) ie nature of control group type impact on main ES? would subgroup ES‟s be important here (diff phases of cancer)?

3. Other internal validity issues  Expectancy effects/Blinding

 participants  therapists  raters

o Screening/floor effect

o Control for somatics that mimic depression

 comparison of depression measures for validity (around illness specificity) ie are differences evident in ES‟s at early and late stage cancer (when

spontaneous improvement / deterioration is expected)?

 control as part of study design * note, this was not raised during quality control discussions

o Treatment fidelity *note, this was not raised during quality control discussions  manualised/replicable

 fidelity checks  Social desirability

o Social desirability *note, this has been brought forward from Main Effect Subgroup Analyses tentatively (without discussion). Which assessment mode (self-report, professional interview etc) produces the biggest ES‟s?

Analyses:

o Descriptives for all of the above

o Sensitivity analyses for selection effect variables over main effects (only) except where mentioned otherwise, ie which make a sig difference to ES?

o Repeat measures can be analysed in relation to the latter listed variables only. Results kept separate, but compared with indep groups. Perhaps this comparison should be kept for later ie repeat measures brought into discussion of moderators, after discussion of main effects, OR kept for entirely separate consideration

48 >> Decide how to deal with impacts of both sets of validity threats, one by one

eg, refer (Devine, 2003) and (Cochrane handbook for systematic reviews of interventions)(and refer Cochrane rationale, annexed)

o if it is shown that the great majority of studies do not guard against a particular threat (eg failing to blind or conceal allocation) then unwilling to loose

variation in data because of this, but to be taken into account in overall ES. o decide which measures for depression to use, or how to adjust ES

o exclude or separate some studies because of clear impact of a quality variable(s) on ES, ie form different level groups according to empirically demonstrated threats

o use cumulative m-a or meta-regression strategies to explore impact of validity on ES

o decide which DV measures to use, or whether we average results from studies, or what

 Compare with Newell ie for those studies that cover same period and are drawn from her reference list. Did she (and other authors eg Meyer, and recommendation of Coyne) appropriately discard all non RCT‟s? Discuss the general approach to quality of study method (eg ex Cochrane, cannot rely on simple aggregates to provide valid correlations with real impacts of measurement factors on validity of outcomes)

 Comment on the practicalities of dealing with this population/this research, going through each of the descriptives, eg impact of lack of blinding, lack of screening („preventative‟ tx).