How To Write A Systematic Review?

STEP 1: FRAMING THE QUESTION – The research question may initially be stated as a query in free form but reviewers prefer to pose it in a structured and explicit way. The relations between various components of the question and the structure of the research design are shown in Figure 1, This paper focuses only on the question of safety related to the outcomes described below. Structured questions for systematic reviews and relations between question components in a comparative study Box 1 The steps in a systematic review

Step 1: Framing questions for a review The problems to be addressed by the review should be specified in the form of clear, unambiguous and structured questions before beginning the review work. Once the review questions have been set, modifications to the protocol should be allowed only if alternative ways of defining the populations, interventions, outcomes or study designs become apparent Step 2: Identifying relevant work The search for studies should be extensive. Multiple resources (both computerized and printed) should be searched without language restrictions. The study selection criteria should flow directly from the review questions and be specified a priori, Reasons for inclusion and exclusion should be recorded Step 3: Assessing the quality of studies Study quality assessment is relevant to every step of a review. Question formulation (Step 1) and study selection criteria (Step 2) should describe the minimum acceptable level of design. Selected studies should be subjected to a more refined quality assessment by use of general critical appraisal guides and design-based quality checklists (Step 3). These detailed quality assessments will be used for exploring heterogeneity and informing decisions regarding suitability of meta-analysis (Step 4). In addition they help in assessing the strength of inferences and making recommendations for future research (Step 5) Step 4: Summarizing the evidence Data synthesis consists of tabulation of study characteristics, quality and effects as well as use of statistical methods for exploring differences between studies and combining their effects (meta-analysis). Exploration of heterogeneity and its sources should be planned in advance (Step 3). If an overall meta-analysis cannot be done, subgroup meta-analysis may be feasible Step 5: Interpreting the findings The issues highlighted in each of the four steps above should be met. The risk of publication bias and related biases should be explored. Exploration for heterogeneity should help determine whether the overall summary can be trusted, and, if not, the effects observed in high-quality studies should be used for generating inferences. Any recommendations should be graded by reference to the strengths and weaknesses of the evidence

What is the structure of writing a systematic review?

Overarching the systematic review is a superstructure or framework. This structure comprises the following components in order: Title-Abstract-Introduction-Methods-Results-Discussion-References.

Can I write a systematic review alone?

Systematic reviews cannot be performed alone. One investigator is not sufficient to reduce the risk of bias in the review process. It is essential that Cochrane reviews be undertaken by more than one person. This ensures that tasks such as selection of studies for eligibility and data extraction can be performed by at least two people independently, increasing the likelihood that errors are detected.

Include expertise in the pertinent clinical content areas Include expertise in systematic review methods Include expertise in searching for relevant evidence Include expertise in quantitative methods Include other expertise as appropriate

– National Academies of Sciences, Engineering, and Medicine, Finding What Works in Health Care: Standards for Systematic Reviews, chapter 2, 2011.

What are the elements of a good systematic review?

Abstract – This paper examines the subject of systematic reviews from a nursing viewpoint. The history of the evidence-based healthcare movement and the major differences between systematic reviews and traditional literature reviews are discussed. The steps of the process used by those conducting reviews are examined in detail.

What is the difference between a systematic review and a literature review?

Literature reviews usually answer broad and descriptive research questions. Systematic reviews are more comprehensive and precise because they seek to answer specific scientific questions of high importance.

What are the 5 steps of systematic review?

Systematic review/meta-analysis steps include development of research question and its validation, forming criteria, search strategy, searching databases, importing all results to a library and exporting to an excel sheet, protocol writing and registration, title and abstract screening, full-text screening, manual

Are meta-analysis and systematic review the same?

Formulating research questions – A systematic review attempts to gather all available empirical research by using clearly defined, systematic methods to obtain answers to a specific question. A meta-analysis is the statistical process of analyzing and combining results from several similar studies.

• Here, the definition of the word “similar” is not made clear, but when selecting a topic for the meta-analysis, it is essential to ensure that the different studies present data that can be combined.
• If the studies contain data on the same topic that can be combined, a meta-analysis can even be performed using data from only two studies.

However, study selection via a systematic review is a precondition for performing a meta-analysis, and it is important to clearly define the Population, Intervention, Comparison, Outcomes (PICO) parameters that are central to evidence-based research.

How many sources do you need for a systematic review?

LITERATURE SEARCH – A systematic review is only as good as the data on which it is based, that is, the primary studies. To ensure that the widest scope of primary research is identified, a thorough and complete search of the literature is needed. The best way to accomplish this is to have help from a librarian with expertise in the area of systematic reviews in defining the search terms, search strategies, and databases to be used.

The rule of thumb for a systematic review literature search is that more than 2 databases should be used. For the casual reader, the databases that should be used for a particular systematic review can be difficult to judge, but generally databases beyond just MEDLINE should be searched. Ideally, articles in languages other than English should be included, and there should be an attempt to find unpublished research and research that has not been formally published in a journal.

This “grey literature” is the most difficult to find. Grey literature has been defined as “that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers”.5 It can include reports, theses, conference proceedings, technical specifications and standards, noncommercial translations, bibliographies, technical and commercial documentation, and official documents not published commercially (such as government reports).5 The instantaneous nature of the Internet has led to a proliferation of this type of literature, but the challenge lies in finding it.

Searching the websites of agencies and organizations that may be involved in the area of interest is a reasonable way to start. Grey literature repositories and gateways have evolved and offer another method for researchers to find primary evidence not available in the peer-reviewed literature.5 Again, searching for grey literature can be much easier with the help of a librarian.

The Steps of a Systematic Review

In addition to a formal bibliographic search and a grey literature search, hand-searching of reference lists or key journals should be performed, as well as searching of controlled trial registers. This type of search can also be guided by a librarian.

The difficulty with the search stage of a systematic review is keeping all of your potential studies organized. Online or shareable citation software is an easy way to manage the potentially large number of studies being considered. By using an online version, you can allow your co-researchers to have access to the studies.

Given that at least 2 researchers will be needed to review the potentially included studies, online citation management allows for easy access by the multiple researchers.

Why use PRISMA for systematic review?

Using the PRISMA statement and its extensions to write protocols or the completed review report, and completing the PRISMA checklists are likely to let reviewers and readers know what authors did and found, but also to optimize the quality of reporting and make the peer review process more efficient.

Can one person do a systematic review?

The methodology for systematic reviews is designed to be rigorous and isn’t suitable for individual researchers and those working to a tight timescale. To learn more about different review methodologies and understand which type of review will best to undertake for your topic, we recommend you read ths following article: Grant, M.J.

Is there sufficent literature on your topic to warrant a review? As systematic reviews synthesise existing evidence and provide synthesis of published studies, it is a requirement that there is sufficent literature available on a topic for a review to be successful. You can carry out a scoping search (background search to assess the amount of literature available) before deciding on the best type of review to address your research question. If you find there isn’t much literature available on your topic then you may decide to conduct a ‘systematic search and review’ which aims to combine the strengths of the more traditional critical review with a systematic search process.

Do you have the time to complete a systematic review? Systematic reviews of medical interventions are carried out over a long period of time ( mean: 67.3 weeks ) and good quality, rigorous systematic reviews require multiple authors and experts to support the different stages of the review process. Reviewers should search multiple bibliographic databases (at least three relevant databases) to ensure that they have been comprehensive in their approach, and utilise other searching methods such as hand-searching to ensure all trials, or relevant studies are identified. Hand-searching is a manual process whereby an author identifies relevant studies for the review by examining citation lists in journal issues or in grey literature, The search process can be time consuming, so if you are on a tight deadline a different type of review methodology is likely to more appropriate.

Do you have methods to reduce the risk of bias in place? Systematic reviews follow a study protocol which details the methods that will be used in the review. Protocols are essential to ensuring a rigorous approach and can help verify that previous systematic reviews have not already answered the research question.

Do you have support available to work on the review? Systematic reviews should not be carried out by one person as this may lead to increased bias in searching, screening and data selection. A second person should be available for screening and article selection and this will reduce the chance of any errors made. Well conducted systematic reviews are carried out by teams with expertise in the topic under review, with the support of information professionals (librarians or information specialists) advising on the search strategy, database selection and reporting methods. If you are a current student, our team of librarians can advise you on good practice for search strategy formation and provide advice on searching databases efficiently. They will not be able to check your search strategy, or help you generate keywords as your search strategy will be assessed as part of your academic work.

There are many other types of review that you can undertake and often these are defined by the amount of time you have and the amount of literature that is available. Scoping reviews are often used to clarify gaps in knowledge or assess the literature available on a given topic.

• Sometimes, they can be used to determine if a systematic review is necessary.
• This article provides information to help you understand the differences between these review types: Munn, Z., Peters, M.D.J., Stern, C. et al.
• Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach.

BMC Med Res Methodol 18, 143 (2018). https://doi.org/10.1186/s12874-018-0611-x You may find our guide on scoping reviews helpful.

Can one person complete a systematic review?

Content experts are necessary to provide expertise on the topic, in developing the procedures and documentation standards, in searching and documenting procedures, to manage the references, and give advice on statistics. By definition, a systematic review cannot be conducted by one person.

Does a systematic review require 2 reviewers?

The Value of a Second Reviewer for Study Selection in Systematic Reviews Res Synth Methods. Author manuscript; available in PMC 2020 Dec 1. Published in final edited form as: Res Synth Methods.2019 Dec; 10(4): 539–545. Published online 2019 Jul 18. doi: PMCID: PMC6989049 NIHMSID: NIHMS1040153 1 Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, Saint Louis, MO Find articles by 1 Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, Saint Louis, MO Find articles by 2 Brown School, Washington University School of Medicine, Saint Louis, MO Find articles by 3 Behavioral Research Program, Division of Cancer Control & Population Sciences, National Cancer Institute, Bethesda, Maryland Find articles by 3 Behavioral Research Program, Division of Cancer Control & Population Sciences, National Cancer Institute, Bethesda, Maryland Find articles by 1 Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, Saint Louis, MO Find articles by Although dual independent review of search results by two reviewers is generally recommended for systematic reviews, there are not consistent recommendations regarding the timing of the use of the second reviewer.

This study compared the use of a complete dual review approach, with two reviewers in the both the title/abstract screening stage and the full-text screening stage, as compared to a limited dual review approach, with two reviewers only in the full-text stage. This study was performed within the context of a large systematic review.

Two reviewers performed a complete dual review of 15,000 search results and a limited dual review of 15,000 search results. The number of relevant studies mistakenly excluded by highly experienced reviewers in the complete dual review was compared to the number mistakenly excluded during the full-text stage of the limited dual review.

• In the complete dual review approach, an additional 6.6% to 9.1% of eligible studies were identified during the title/abstract stage by using two reviewers, and an additional 6.6% to 11.9% of eligible studies were identified during the full-text stage by using two reviewers.
• In the limited dual review approach, an additional 4.4% to 5.3% of eligible studies were identified with the use of two reviewers.

Using a second reviewer throughout the entire study screening process can increase the number of relevant studies identified for use in a systematic review. Systematic review performers should consider using a complete dual review process to ensure all relevant studies are included in their review.

1. When performing a systematic review, the importance of study selection cannot be overstated.
2. Decisions about which studies to include are considered among the most significant decisions made during the review process.
3. The quality of the study selection process is dependent on two factors, the formation of specific and clear eligibility criteria, and the systematic implementation of these criteria against each record found in the search process.

As a comprehensive search strategy can result in thousands of results that must be screened, the process of screening search results against eligibility criteria can require significant time and resources. The best method for study screening is one that allows for a high level of accuracy, ensuring that no relevant studies are mistakenly excluded, with as much efficiency as possible.

• Studies can be mistakenly excluded during the screening process due to a misapplication or misunderstanding of eligibility criteria or due to random error of the screener.
• In order to reduce this potential for missed studies, it is commonly recommended that two (or more) screeners undertake the screening process.

The Agency for Healthcare Research and Quality (AHRQ), the Center for Reviews and Dissemination (CRD), the Institute of Medicine (IOM), and the Cochrane Collaboration all recommend using two or more members of the review team, working independently, to screen studies.

– The IOM specifically notes that doubling the number of screeners requires significant time and resources, but that the additional expense is justified in order to reduce bias and errors. A previous study explored the impact of using dual reviewers as compared to a single reviewer, and found that the average increase in eligible studies identified using two reviewers was 9%, ranging from 0% to 32%, suggesting a notable impact by the second reviewer.

Additionally, these groups are consistent in recommending that study screening be performed in a two-stage process, in which titles and abstracts are screened first, followed by full-text study reports. However, these groups do not explicitly recommend if the additional reviewer should be involved in both stages.

Cochrane and AHRQ address this briefly. Cochrane suggests that adding a second reviewer at the full-text stage may be sufficient, saying that “Authors must first decide if more than one of them will assess the titles and abstracts of records retrieved from the searchIt is most important that the final selection of studies into the review is undertaken by more than one author.” AHRQ states that, “Some form of dual review should be done at each stage,” however they suggest alternatives to dual review in the title/abstract stage such as having the second reviewer only review the first reviewer’s exclusions, or only conducting dual review on a small percentage of the records in a pilot phase in order to resolve any confusion, and then going on to single review only for the remainder of the title/abstract phase.

However, while performing a pilot phase may help to reduce error due to unclear or misunderstood eligibility criteria, it is unlikely to prevent all error, and it will not prevent random error by the reviewer, so is unlikely to be sufficient to improve accuracy and prevent mistakenly excluded studies.

Using a second reviewer in the screening process represents a significant amount of resources. Using a second reviewer only in the full-text stage of screening may be a way to reduce resources necessary, while still maintaining a lower level of bias. A previous study explored various methods of study selection using a cost-effectiveness analysis and found that using two reviewers in the title/abstract stage and moving all records marked by at least one reviewer as eligible, rather than resolving disagreements, was equally effective and less costly than traditional double screening in a systematic review of effects of undergraduate medical education in UK general practice settings.

However, they conclude that effectiveness of different screening methods is likely to vary between systematic reviews. We set out to compare two methods of study selection, using dual independent reviewers throughout the title/abstract and full-text stages of the screening process versus using dual independent reviewers only in the full-text stage in the context of a systematic review exploring representation of multimorbidity in behavioral intervention randomized controlled trials.

• The objective of this study is to identify if using dual reviewers throughout the entire study screening process produces a clear benefit over using dual reviewers only at the full-text screening stage in a large systematic review.
• This study took place in the context of a large systematic review evaluating the inclusion of participants with multiple chronic conditions in randomized trials of behavioral health interventions.

The methods and results of this systematic review are reported separately, but summarized briefly here. The eligibility criteria of the systematic review were (1) Primary report of a RCT testing the efficacy or effectiveness of behavioral interventions (2) the study reports original data (protocols, post-trial follow-up studies, secondary or separate subgroup analyses were excluded) (3) the RCT targets chronic illness (4) the RCT applied eligibility criteria at the individual level (5) the trial was published in English (6) the RCT enrolled only adult subjects (≥18 years).

• The search strategy of the systematic review was designed to be broad in order to identify all published RCTs in adults that test behavioral health interventions and target chronic illness.
• Due to this broad search strategy, this systematic review involved a large number of search results which provided the ideal setting for the current study.

This search produced 343,123 records of potentially relevant reports. After removing duplicate records, 190,555 records remained. After the search was performed, a sampling strategy was used to produce a representative sample of literature of behavioral intervention RCTs targeting participants with chronic conditions published from 2000 to 2014.

This was done by randomly ordering search results (within three time periods, 2000–2004, 2005–2009, 2010–2014) using the RAND function in Microsoft Excel and performing study selection on the randomly ordered results within each time period until the target sample size (200 studies per time period, 600 studies total) was reached.

For purposes of the current study, the first 15,000 records from the randomly ordered search results (5,000 per time period) were used for the complete dual review approach and the next 15,000 records of the randomly ordered search results (5,000 per time period) were used for the limited dual review approach.

Two experienced reviewers (CS, SI) took part in this study. Both reviewers were involved in the study design of the systematic review and the definition of the eligibility criteria. The reviewers went through a pilot process with the eligibility criteria prior to starting the study to ensure they had a similar understanding of the eligibility criteria.

During the first approach of the study (), reviewers fully performed a dual independent review of all records. Reviewers first independently screened studies by title/abstract and compared results. Records were excluded if both reviewers had excluded them.

• Records were moved to full-text screening if both reviewers indicated they should be kept.
• Records for which reviewers had opposing decisions were reviewed again together and a consensus was made to exclude them or move them to full-text screening.
• Reviewers then independently screened identical lists of studies by reading the full-text of each study report and applying eligibility criteria.

After the screenings were complete, results were compared. Records were excluded if both reviewers had excluded them. Records were included in the review if both reviewers had included them. Records for which reviewers had opposing decisions were reviewed again together and a consensus was formed.

1. This review process was completed in three sets of 5,000 records (total of 15,000 records), with comparison between the results of the two reviewers at the end of each set.
2. In the second approach of the study (), reviewers performed a limited dual review of records.
3. Records were assigned to each reviewer in alternating groups of 2,500 (total of 15,000 records) such that each title/abstract was reviewed by only one reviewer.

Decisions made by the sole reviewer regarding exclusion or moving of the record to full-text screening were considered final. Studies indicated for full-text review by solo review were then independently dually reviewed, following the same full-text review process as in the first approach of the study.