CAPÍTULO I: FUNDAMENTO TEÓRICO 35
I.2 Estrategia de validación 39
In empirical research, we typically study mechanisms of change with mediation analyses, most appropriately in the context of randomized, controlled trials. Mediators are any intervening variables that we can statistically demonstrate to account for some of the relationship between an independent and dependent variable, in this case between treatment and reduction in PTSS. It is important to stress that while mediators may represent mechanisms of change, not all mediators necessarily do (Kazdin, 2007; Kraemer et al., 2002; Tryon, 2018). The study of mediators serving as proxies for them is still a sensible first step or basic requirement for understanding mechanisms of change (Kazdin, 2009; Tryon, 2018).
A number of statistical techniques exist for demonstrating the mediating role of a variable in the effects of a treatment. Typical approaches in psychological research combine regression or structural equation model estimates of a) the relationship between treatment and mediator and b) the relationship between mediator and outcome, so that an overall estimate of the indirect effect of the treatment on the outcome via the mediator is established (Baron & Kenny, 1986; Kraemer et al., 2002; MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002). The typical approaches to mediation in psychology have a range of challenges and involve assumptions that are unlikely to be satisfied in many situations, which I expound on in the Discussion in Section 9.6 (Bullock, Green, & Ha, 2010).
Besides observing that it mediates treatment effects on outcomes, Kazdin (2007) provided a set of criteria for claiming robust evidence for the role of a mechanism of change in the effects of an intervention. 1) Strong association between intervention and mechanism, and mechanism and outcome. 2) Demonstration of specificity of a mechanism in that a particular construct accounts for therapeutic change, while (many) others do not. 3) Consistency in terms of replicating our
findings on a mechanism across different studies, conditions and types of samples. 4) Experimental manipulation of the mechanism as providing the most solid evidence. 5) Establishing a timeline in that mechanisms change before outcomes do. 6) Demonstrating a dose gradient in responses so that the more a mechanism is activated, the greater the change in the outcome. 7) Plausibility in that a proposed mechanism makes sense in the context of our theoretical thinking and current knowledge base.
Kazdin (2007) and others have particularly emphasized the fifth and seventh criteria and they deserve further discussion. A crucial condition indeed for claiming a mechanistic role is establishing the temporal sequence of changes, i.e., that changes in the mechanism take place before changes in the outcome and lead to them, not vice versa (Johansson & Høglend, 2007; Pek & Hoyle, 2016). Correlational evidence about concurrent changes in mechanisms and outcomes being associated with each other can hint at a mechanism in action, but is not enough to make claims about causal effects. Changes in the proposed mechanism might simply be alternative indicators of success in treatment or correlates or side effects of symptom reduction (Zalta, 2015). Even if the mechanism as such is valid and real, cross-sectional analyses may be misleading and unpredictably biased (Maxwell & Cole, 2007). All this means that to establish a timeline we should assess changes in mechanisms, at a minimum, once before assessing outcomes. Preferably, we should repeat assessments several times during and after treatment or on a session-by-session basis.
That a proposed mechanism makes theoretical sense is also crucial. Ad hoc investigations of a great variety of possible mediators and paths risks spurious findings and confusing rather than clarifying our understanding of change processes. The selection of potential mechanisms to be studied can be based on several sources, including theory on basic psychological processes, existing empirical findings on risk factors and predictors of psychopathology, wide-ranging or specific theories of psychopathology, or the rationales and clinical models of the treatments being studied (Doss, 2004; Kazdin, 2007). Surveys of clinicians or patients might be another source for putative mechanisms. In the ideal case, we would test predictions clinical models make about relevant mechanisms in empirical trials and then refine the models based on our findings. We should keep in mind the possibility, however, that a theory might correctly explain or predict intervention effects, but incorrectly explain a disorder’s etiology (Tryon, 2005).
Relatedly, mechanisms must be both separate enough from ultimate outcomes, i.e., not too confounded with them, and simultaneously not simply part of the definition of the treatment and thus collinear with it (Kraemer et al., 2002). Here, we
must be clear about how we define our outcomes. In this dissertation, I mainly deal with aggregate measures of total PTSS. With such an outcome, it may be, e.g., trivial to demonstrate that a treatment’s effects on intrusive symptoms at posttreatment mediate effects on total PTSS at follow-up, as intrusive symptoms are a core part of PTSS. However, such analyses tell us little about how the treatment lead to alleviation of symptoms in the first place, so change in intrusive symptoms cannot be considered a mechanism of change. More detailed analyses, e.g., at the level of individual symptoms as in network analyses (Fried et al., 2018), of the order and dynamics of symptom alleviation and spread of intervention effects may certainly be informative, but are beyond my focus here. As an example for collinearity with treatment, if an intervention were to contain guided nature walks as part of the protocol, it would be inappropriate to consider the associated increased time spent in nature as a mechanism of change. Something like increased connection with nature could be a potential mechanism of change, however.
Operationalizing mechanisms of change also provides some particular challenges (Doss, 2004). As we are dealing with change, measures of mechanisms of change must not contain historical items but should refer to factors that can in fact change over the course of therapy. Measures should also be amenable to frequent repeated assessments, and contain an adequate range of items to account for the entire course of treatment.
Some researchers have argued that it only makes sense to attempt to find mechanisms of change when we have first shown a treatment to lead to effects on the outcome (Doss, 2004; Kazdin, 2007). However, overall change in the outcome is not necessary for a significant indirect effect via a mediator to exist, as such positive indirect effects might be counteracted by another effect in the opposite direction for all or some subgroup of participants. We can extend this thinking to mechanisms of change as well (Kraemer et al., 2002). Though there may be no overall average benefit, we may identify pathways to effectiveness via some mechanisms that are counteracted by other effects in this particular treatment or sample, but could potentially be successfully exploited in a different intervention. This was part of our rationale for studying the mechanisms of change of TRT in Study II, despite its limited effectiveness in that particular sample. Still, it probably makes most sense to focus our studies on mechanisms of change generally to treatments with some evidence base.
Mechanisms of change are typically studied within the context of RCTs.. So far, this has often meant secondary analyses and afterthoughts, but increasingly plans for analysis of mechanisms are also being included in preregistered protocols of trials,
as we did for Study IV. Still, comprehensive understanding of mechanisms of change is not possible based on a single study. Thus, systematic reviews and meta-analyses such as our Study I may be particularly well suited for bringing together findings on mechanisms (Bullock et al., 2010; Kazdin, 2007).