How to generalize efficacy results of randomized trials: recommendations based on a systematic review of possible approaches

How to generalize efficacy results of randomized trials: recommendations based on a systematic review of possible approaches:

Abstract

Rationale, aims and objectives

Randomized controlled trials (RCTs) are the preferred source for evidence for the effect of treatment. However, patients participating in RCTs often manifest important differences from patients seen in practice. Therefore, guideline developers have to decide whether the results are generalizable to the target population not represented in RCTs.

Method

A systematic review of the literature was undertaken to identify methods to decide whether to generalize the results from RCTs to patients who were not represented in these trials.

Results

One approach is to examine the in- and exclusion criteria of trials and infer from these whether the trial population was sufficiently representative. Other authors suggest, because of the inclusion of a broader range of patients, reliance on observational studies if no direct evidence for the target population is available.

Another approach is to apply the relative effect of treatment found in trials to patients in practice unless there is a compelling reason to believe the results would differ substantially as a function of particular characteristics of those patients. Although there are exceptions, this approach is supported by empirical evidence that, in general, relative effect of treatment on benefit outcomes seldom differs to an important extent across subgroups of patients.

Conclusion

We propose this last approach: focusing on RCTs unless there is a compelling reason not to do so. Compelling reasons will most often be found with respect to issues of rare adverse effects, for which observational studies are likely to provide the best estimates.

abstract: Assumptions of Expected Benefits in Randomized Phase III Trials Evaluating Systemic Treatments for Cancer

Assumptions of Expected Benefits in Randomized Phase III Trials Evaluating Systemic Treatments for Cancer

Background In designing phase III randomized clinical trials (RCTs), the expected magnitude of the benefit of the experimental therapy (δ) determines the number of patients required and the number of person-years of follow-up. We conducted a systematic review to evaluate how reliably δ approximates the observed benefit (B) in RCTs that evaluated cancer treatment. 

Conclusions Investigators consistently make overly optimistic assumptions regarding treatment benefits when designing RCTs. Attempts to reduce the number of negative RCTs should focus on more realistic estimations of δ. Increased use of interim analyses, certain adaptive trial designs, and better biological characterization of patients are potential ways of mitigating this problem.

Editorial :: VIEWPOINT: Randomised controlled trials using invasive ‘placebo’ controls are unethical and should be excluded from Cochrane Reviews - The Cochrane Library

"Placebo controls are frequently used to ‘blind’ participants, trial personnel and outcome assessors to intervention and control in clinical trials. Effective blinding of treatment reduces the risk of performance bias (differences between groups in the care provided apart from the intervention being evaluated) and detection bias (differences between groups in how outcomes are ascertained, diagnosed or verified). A placebo has traditionally been defined as an “inert or innocuous substance”,[1] such as a ‘sugar pill’. However, some randomised controlled trials (RCTs) have been shown to erroneously use the term ‘placebo’ to describe an invasive intervention that exposes participants, allocated to a control group, to risks of serious harm.[2,3] In this context therefore, the term ‘placebo’ describes an invasive intervention which is neither inert nor innocuous.
In a recent study of local anaesthesia RCTs, over half the RCTs used an invasive ‘placebo’ control.[2] The ‘placebo’ interventions mostly involved deep-needle insertion through body tissues with potential damage to nerves, vessels and other structures such as liver and bowel. These interventions exposed control group participants to risks of serious morbidity.[2,3].......

free full text: Reporting of eligibility criteria of randomised trials: cohort study comparing trial protocols with subsequent articles -- Blümle et al. 342 -- bmj.com

Conclusions:
Many users of trial information rely on published journal articles. These articles generally do not reflect the exact definition of the study population as prespecified in the protocol. Incomplete or inadequate reporting of eligibility criteria hampers a proper assessment of the applicability of trial results.

JAMA full free access: Health Outcomes After Stopping Conjugated Equine Estrogens Among Postmenopausal Women With Prior Hysterectomy-randomized control trial

Health Outcomes After Stopping Conjugated
Equine Estrogens Among Postmenopausal
Women With Prior Hysterectomy

abstract/full free access: Improving the interpretation of quality of life evidence in meta-analyses: the application of minimal important difference units

Abstract:

Systematic reviews of randomized trials that include measurements of health-related quality of life potentially provide critical information for patient and clinicians facing challenging health care decisions. When, as is most often the case, individual randomized trials use different measurement instruments for the same construct (such as physical or emotional function), authors typically report differences between intervention and control in standard deviation units (so-called "standardized mean difference" or "effect size"). This approach has statistical limitations (it is influenced by the heterogeneity of the population) and is non-intuitive for decision makers. We suggest an alternative approach: reporting results in minimal important difference units (the smallest difference patients experience as important). This approach provides a potential solution to both the statistical and interpretational problems of existing methods.

abstract: Presenting the Results of Cochrane Systematic Reviews to a Consumer Audience: A Qualitative Study

Results. Participants preferred results presented as words, supplemented by numbers in a table. There was a lack of understanding regarding the difference between a review and an individual study, that the effect is rarely an exact number, that evidence can be of low or high quality, and that level of quality is a separate issue from intervention effect.
 Conclusion. Through testing and iteration the authors identified and addressed several problems, using explanations, rephrasing, and symbols to present scientific concepts. Other problems remain, including how best to present confidence intervals and continuous outcomes. Future research should also test information elements in combination rather than in isolation. The new Plain Language Summary format is being evaluated in a randomized controlled trial.