PLoS ONE: Empty Reviews: A Description and Consideration of Cochrane Systematic Reviews with No Included Studies - your opinions?

Blogger's Note/Opinion: gyn cancers - this topic of discussion should be of interest: Cochrane total gyn cancers reviews = #85; empty reviews = #7; because Cochrane synthesizes and analyses systematic reviews,  often the results are inadequate (for patients/providers) in that studies included are: poor quality; lacking in data or are not randomized clinical trials; we often see this as 'further research is required'; many 'argue' that the value of systematic reviews in gyn cancers are dated (as discussed in this article) but part of this issue is the nature of, in particular, ovarian cancer research itself; the other 'issue' for this blogger is the number of systematic reviews which  continually review reviews; (re: reviews that include Cochrane systematic reviews); there needs to be a streamlining of systematic reviews (randomized clinical trial reviews) nominating or permitting one entity/conglomerate - eg. the philosophy of meaningful use, however, therein lies system wide issues within cancer; observational studies are often not included in the reviews and Cochrane is attempting to address this issue within the parameters of the purpose and philosophy of Cochrane itself, still for quality evidence Cochrane remains the leader; note also that Cochrane has always been one of, if not the leader, in including patients/consumers - your thoughts?

PLoS ONE: Empty Reviews: A Description and Consideration of Cochrane Systematic Reviews with No Included Studies

Introduction  

The Cochrane Library is the largest and perhaps best recognized global collection of health care evidence, currently hosting more than 4,500 systematic reviews in its Cochrane Database of Systematic Reviews (CDSR). However, it has been reported that clinicians find Cochrane reviews of limited relevance to practice decisions. For example, one study found that while Cochrane reviews are highly regarded for their quality, they are used less than other sources for clinical decision-making because of their emphasis on methodology and rigor rather than on clinical relevance [1].

It is not Cochrane's policy to provide guidelines for practice or policy decisions [2]. Instead, it sees itself as the provider of best quality evidence and specifically states that guidelines go “beyond a systematic review and require additional information and informed judgments that are typically the domain of clinical practice guideline developers.”
Systematic reviews that find no studies eligible for inclusion, commonly known as “empty reviews,” may be especially problematic for clinicians and other decision-makers. Little is known about the incidence, prevalence or variation in reporting of such reviews [3]. The little that has been written about them suggests that the reporting of implications for practice may sustain a risk for bias. With no studies meeting criteria for inclusion, these empty reviews may appear to: (1) offer no conclusions, (2) offer conclusions based on referenced excluded studies, (3) offer conclusions based on other evidence, or (4) offer conclusions not based on evidence. Thus, empty reviews may contribute to what appears to be generalized disappointment with The CDSR among some clinicians and policymakers [1], [4]........

Conclusions

The stated purpose of Cochrane reviews is to help healthcare providers, consumers, researchers, and policy makers “make well-informed decisions about health care… by providing a reliable synthesis of the available evidence on a given topic… considering all the evidence on the effect of an intervention” [2]......

Cochrane: YourHealthNet - navigating effective treatments with systematic reviews

YourHealthNet - navigating effective treatments with systematic reviews

The parts of a Cochrane systematic review and the information they contain - learning to navigate a review

Cochrane systematic reviews

Cochrane systematic reviews are the major output of the international organisation The Cochrane Collaboration, and over 4,000 Cochrane reviews are available online on The Cochrane Library. Because Cochrane review authors carry out their research with scientific rigour and follow detailed guidelines Cochrane reviews are considered a high quality source of research evidence.

David Tovey from The Cochrane Collaboration talks about systematic reviews.   » Read a transcript or » Listen to David's comments » See screen shot... This is how a systematic review looks on The Cochrane Library.

Explore the parts of a Cochrane systematic review

Systematic reviews are a complex mix of process and product - they report on the process review authors undertook, and the conclusions authors came to about what they found. All Cochrane systematic reviews follow the same format and methods; they have the same content in the same sections. This ensures their transparency and rigour.....

Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases - The Cochrane Library ((beta-carotene, vitamin A, vitamin C, vitamin E, and selenium)

Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases - The Cochrane Library

 Abstract

"The increased risk of mortality was associated with beta-carotene and possibly vitamin E and vitamin A, but was not associated with the use of vitamin C or selenium. The current evidence does not support the use of antioxidant supplements in the general population or in patients with various diseases."

Background

Our systematic review has demonstrated that antioxidant supplements may increase mortality. We have now updated this review.

Objectives

To assess the beneficial and harmful effects of antioxidant supplements for prevention of mortality in adults.

Systematic Reviews - Why prospective registration of systematic reviews makes sense

Abstract (provisional)

Prospective registration of systematic reviews promotes transparency, helps reduce potential for bias and serves to avoid unintended duplication of reviews. Registration offers advantages to many stakeholders in return for modest additional effort from the researchers registering their reviews.

The complete article is available as a provisional PDF. The fully formatted PDF and HTML versions are in production.

open access: BioMed Central Blog : Connecting the evidence: an “ontology” (vocabulary) for Threaded Publications (unpublished research)

Thursday Feb 02, 2012

Connecting the evidence: an “ontology” for Threaded Publications

"Unpublished research is a serious problem for evidence-based decision making in healthcare, and this was recently highlighted on BBC Radio 4’s Today programme and in an entire issue of the BMJ. Systematic reviews aim to present the totality of the evidence, and a problem for those preparing and maintaining these reviews is how to find unpublished studies and data. But, even when clinical trials are reported in journals and their supplements the formats and descriptions are widely heterogeneous and studies can remain difficult to discover and challenging to compare with similar trials.
Clearly connecting trial-related publications is a way to help with this problem and is a major goal of BioMed Central’s Threaded Publications initiative. To achieve its fundamental aims of connecting all digital published content relating to the evidence about a particular trial, however, Threaded Publications must go beyond a single journal or publisher.
Through our partnership with CrossRef – an organisation founded by publishers, for publishers – and engagement with editors and publishers we hope to achieve interoperability across different publishing platforms.
The desired outcome is that articles reporting the protocol or the findings of a trial published in different journals or by different publishers will be linked in a thread, which should also include the trial’s entry in a research register......."

Systematic Reviews - new open access journal

David Moher - Systematic Reviews & Prospero Register | Open Medicine Blog-PROSPERO International

PROSPERO - International prospective register of systematic reviews

PROSPERO is an international database of prospectively registered systematic reviews in health and social care. Key features from the review protocol are recorded and maintained as a permanent record in PROSPERO. This will provide a comprehensive listing of systematic reviews registered at inception, and enable comparison of reported review findings with what was planned in the protocol.

Substantial changes to the review questions or methods

• It can be difficult to decide whether an update to a review is in fact a new review. There is little published guidance on this. PROSPERO adopts a pragmatic approach. If changes to the review questions or methods are so substantial that they require major changes to the original protocol, this should be regarded as a new review rather than an update.

Examples that would constitute a new review

• addition of new treatment comparisons
  e.g. direct comparison of different drugs, when the old review included only comparisons of drug with placebo
• substantial changes to the population being studied
  e.g. adding adults to a review that was previously restricted to children
• exclusion criteria in the old review become inclusion criteria in the new review
• introduction of new analysis techniques
  e.g. a switch from aggregate data meta-analyses to individual participant meta-analyses.
  

If in doubt, a new record for a new review should be created. This will minimise the complexity of the editing to the original record in PROSPERO and make it easier for users to distinguish between the original review and the later version. Links between the new and original review can be added in the registration form.

free full access: WHO/PLoS Collection “No Health Without Research”

"It seems astonishing that in the 21st century decisions on health care can still be made without a solid grounding in research evidence. This is true even in clinical research, whether for simple or complex interventions [1], where systematic reviews time and time again conclude that the evidence base is inadequate [2]. It is even more true in the areas of health policy and health systems, where quality research is hampered further by a lack of shared definitions, a lack of consensus on guiding principles, poor capacity (especially in low-resource regions), and methodological challenges [3],[4].
The World Health Report (WHR) for 2012 will be on the theme of “No Health without Research”. This flagship report from WHO will, for the first time in its history, focus on research for better health. The primary target audience of the report will be ministers of health in the WHO member states, and the goal of the report is to provide new ideas, innovative thinking, and pragmatic advice for member states on how to strengthen their own health research systems. The report will have the three following aims.

1. To show that research is important for meeting health needs and improving health outcomes;
2. To encourage countries to therefore invest more resources in developing and strengthening their national health research systems;
3. To argue that countries should not see research as an expense or as an afterthought, but as an investment for a better, healthier future.....cont'd

full free access: PLoS Medicine: Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up? Cochrane Collaboration

"To meet the needs of patients, clinicians, and policymakers, unnecessary trials need to be reduced, and systematic reviews need to be prioritised. Streamlining and innovation in methods of systematic reviewing are necessary to enable valid answers to be found for most patient questions. Finally, clinicians and patients require open access to these important resources."

Cochrane Reviews - by sub-topics 'Gynaecological Cancer'

CANCER OF THE OVARY (25) (reviews in process/completed)

Prevention (0)
Screening (0)
Initial management and follow-up (12)
Immunotherapy (0)
Surgery (2)
Laparoscopy versus laparotomy for FIGO Stage I ovarian cancer
Ultraradical surgery for the primary debulking of epithelial ovarian cancer (protocol stage)
Radiotherapy (1)
Chemotherapy and/or radiotherapy after surgery for ovarian carcinosarcoma (protocol stage)
Chemotherapy (5)
Adjuvant (post-surgery) chemotherapy for early stage epithelial ovarian cancer
Chemotherapy and/or radiotherapy after surgery for ovarian carcinosarcoma (protocol stage)
Intraperitoneal chemotherapy for the initial management of primary epithelial ovarian cancer
Maintenance chemotherapy for ovarian cancer (protocol stage)
Topotecan for ovarian cancer
Hormonal therapy (1)
Antigen-specific active immunotherapy for ovarian cancer
Combined modality therapy (2)
Effectiveness of different treatment modalities for the management of adult onset granulosa cell tumours of the ovary (primary and recurrent) (protocol stage)
Interventions for the treatment of borderline ovarian tumours (protocol stage)
Follow-up (1)
Evaluation of follow-up strategies for patients with epithelial ovarian cancer following completion of primary treatment (protocol stage)
Management of advanced disease (13)

full access: PLoS Clinical Trials: Updating Systematic Reviews: An International Survey

Conclusions/Significance:

Most organizations that sponsor and/or carry out SRs (systematic reviews) consider updating important. Despite this recognition, updating practices are not regular, and many organizations lack a formal written policy for updating SRs. This research marks the first baseline data available on updating from an organizational perspective.

PLoS ONE: Bias Due to Changes in Specified Outcomes during the Systematic Review Process

The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews -- Kirkham et al. 340: c365 -- BMJ

"Of the 42 meta-analyses with a statistically significant result only, eight (19%) became non-significant after adjustment for outcome reporting bias and 11 (26%) would have overestimated the treatment effect by 20% or more."

"Conclusions:
Outcome reporting bias is an under-recognised problem that affects the conclusions in a substantial proportion of Cochrane reviews. Individuals conducting systematic reviews need to address explicitly the issue of missing outcome data for their review to be considered a reliable source of evidence...."