Desarrollo de mercado

1. Study limitations

For randomized controlled trials, the main criteria for assessing limitations are:

■ whether concealment of allocation to treatment group is adequate; ■ whether participants and investigators were blinded, especially if the

outcomes are measured subjectively and subject to bias; ■ whether an intention-to-treat analysis is reported;

■ whether all withdrawals and patients lost to follow-up are accounted for; ■ whether the trial was stopped early for benefit.

For studies of diagnostic accuracy, additional limitations include:

■ whether patients were consecutively recruited and not classified by disease state;

■ whether both the new test and the reference standard were done in all patients;

■ whether evaluators were blinded to the results of the alternative test and reference standard.

For observational studies, the main criteria depend on the design (i.e. case- control or cohort studies). For both designs, the methods used to select the population in the study and the comparability of the two groups are important. For case-control studies the method of determining exposure to the factor of interest also needs to be evaluated. For cohort studies the method of measuring outcomes should be evaluated.

The evidence summaries should categorize the limitations as follows. a. “No limitations” generally means that the majority of studies meet

all the minimum quality criteria for the design. The implication of this is that the rating of quality of evidence remains the same as the initial assessment.

b. “Minor limitations” applies when minor flaws are found when analyzing how the available studies were designed and performed. If you decide there are minor limitations, these should be noted in a footnote in the evidence profile but they would not usually down- grade the quality.

c. “Serious limitations” means that one of the minimum criteria for quality is not met by the majority of studies in the review. This re- sults in a -1 score for the overall quality rating (e.g. “high” becomes “moderate”).

Chapter 7

d. ‘‘Very serious limitations” means that at least two of the criteria proposed as potential study limitations are present in the majority of studies in the review. This results in a -2 score for quality. 2. Consistency

Consistency relates to whether the results are similar across studies. Differences in the direction, the size, and the significance of the differences in effect, guide the decision about whether important inconsistency exists. If all the results of the studies for one outcome are in the same direction with overlapping confidence intervals, there is unlikely to be significant consistency. To evaluate the degree of consistency of the results of the available studies, the direction and size of the effect for each outcome should be evaluated. If a formal meta-analysis was conducted, the result of the test for heterogeneity can be used to help assess consistency. Variability or inconsistency in results may arise from differences in the populations in the studies, in the interventions or in outcomes.

If there is inconsistency in the results, such as the largest trial showing results that contradict smaller trials, then a -1 score should be applied. If the results are very heterogeneous, “very serious” should be chosen, which will downgrade the evidence for this outcome by two levels. If only one study is present, consistency is not applicable as a criterion.

3. Directness

Directness, generalizability, external validity of study results and applicability are all synonyms. There are two types of indirectness.

■ Indirect comparison occurs when a comparison of intervention A versus B is not available, but A was compared with C and B was also compared with C. Such trials allow indirect comparisons of the magnitude of effect of A versus B. Such evidence is of lower quality than direct comparisons of A and B would provide.

■ Indirect population, intervention, comparator or outcome arise when the question being addressed by the guideline development group or by the authors of a systematic review is different from the available evidence regarding the population, intervention, comparator or outcome.

4. Imprecision

Results are imprecise when studies include relatively few patients and few events and thus have wide confidence intervals around the estimate of the effect. In this case the quality of the evidence is lower than it otherwise would be because of uncertainty in the results.

When event rates are very low, 95 per cent confidence intervals around relative effects can be very wide, but 95 per cent confidence intervals around absolute effects may be narrow. In the latter case, the quality should not be downgraded for imprecision.

5. Reporting bias

Reporting bias (also called publication bias) is a systematic underestimate or overestimate of the underlying beneficial or harmful effect due to the selective publication of studies or selective reporting of outcomes. Reporting bias arises when investigators fail to report studies they have done (typically those that show no effect) or neglect to report outcomes that they have measured (typically those for which they observed no effect).

Despite methods to detect the possibility of publication bias, the authors of systematic reviews and guideline panels must often make assumptions about the extent of this bias. Reporting bias should be considered when published evidence is limited to a small number of trials, all of which were funded by a for-profit organization. In such a situation, consider the extent to which evidence about the magnitude of the effect is uncertain due to selective publication of studies or reporting of outcomes. If this is likely, downgrade the quality rating by one or even two levels.

The criteria that are used for downgrading the quality of evidence and the reason for the assessment should be explained in a footnote to the table.