Research: Systematic Reviews Step-by-Step

Step 3: Formulate the question, develop & register the protocol

• Establish a clear & answerable question. Consider using PICOT to focus the question (see Ask the Right Question box). May require several revisions & more scoping of literature.

• Cochrane Handbook: Part 2, Chapter 2: Determining the scope of the review and the questions it will address

• Establish whether the question has already been answered in the published literature or is registered as an ongoing review (search DARE, Cochrane, and PROSPERO, or the Campbell Collaboration for social interventions)
• Write a review protocol to formulate the review question and methods before retrieving the literature, thereby minimizing bias. Include the clear & answerable question; rationale for the review; the search strategy (search terms & resources);  inclusion & exclusion criteria (such as publication date range, language, main focus, explicit methodology, outcome measurements); quality checklist to assess individual studies; data extraction & management strategy;  data synthesis strategy;  dissemination strategy;  timeframe. 
• RevMan – the software used for preparing & maintaining Cochrane protocols & reviews
• **NEW** RevMan Web is now also available for the creation of non-Cochrane reviews (requires a subscription)
• Register your review to ensure others know it is ongoing. Register with Cochrane for interventions or accuracy of diagnostic tests; with Campbell Collaboration for social interventions in education, crime & justice, social welfare; with PROSPERO for reviews in health or social care. New Cochrane protocols are automatically uploaded to PROSPERO. Each organisation will have its own suggestions for the review protocol  

• Cochrane Handbook: Part 1, Chapter 2: Planning a Cochrane review
• Campbell Collaboration - Guidelines for preparation of review protocols
• Standards for Initiating a Systematic Review in Finding What Works in Health Care: Standards for Systematic Reviews, Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, 2011

Step 4: Systematically search the literature

• At least 2 reviewers are required. They first independently screen titles & abstracts based on the research question (in terms of PICOT). Based on this, full-text articles are retrieved for a second stage screening. The final selection is then submitted for data extraction.
• Sources to search include: Cochrane Library; DARE (Database of Abstracts of Reviews of Effects); Medline; Embase; PubMed Clinical Queries; subject specific databases (e.g. CINAHL, PsycInfo); conference proceedings; hand search key journals; bibliographies & references listed in primary sources; guidelines (often based on SRs); grey literature (see our Grey Literature page); forward citation searching of seminal articles; foreign language literature (do not limit searches to English); clinical trials (see Trials Registries tab on Grey Literature page ), contact scholars/experts in the field, search the Internet. See the following for more details:

• Cochrane Handbook: Part 2: Chapter 4. Searching for and selecting studies
• Standards for Finding and Assessing Individual Studies (In Finding what works in health care: standards for systematic reviews)

• The search process should be meticulously documented as it develops. Include:​

1. databases used
2. date of search
3. dates of coverage provided by each database
4. search terms used
5. total number of publications found
6. number of relevant publications
7. limits applied

Include the PRISMA flowchart in the review. Use one of the following online tools for managing your search results:

• EndNote (freely available to Department of Health staff);
• Covidence ("improves healthcare evidence synthesis by improving the efficiency and experience of creating and maintaining Systematic Reviews." Supported by The Alfred Hospital, Monash University, National ICT Australia & University of London);
• EPPI-Reviewer 4 (software for all types of literature reviews including systematic reviews & meta-analyses)

Step 5: Data extraction

• Devise a  standardized form to assist with data extraction.
• This will generally include details of the reference, study objectives, study design, population, intervention, control, outcome, comments on study quality.

• Cochrane Handbook: Part 2, Chapter 5: Collecting data

Step 6: Quality appraisal

Step 7: Data synthesis

• After including & excluding studies according to the quality appraisal, the analysis of data & results should initially begin with a simple descriptive evaluation of each study, often presented in a table.
• If the study results are very heterogeneous it may be more appropriate to summarise the data narratively.
• A statistical analysis (meta-analysis) should include numerical and graphical presentations of the data, look at the strength and consistency of the evidence and investigate reasons for inconsistencies.
• See Calculate box and

•  Cochrane Handbook: Part  2, Chapter 10: Analysing data and undertaking meta-analyses
•  Cochrane Handbook: Part 2, Chapter 13: Assessing risk of bias due to missing results in a synthesis
• Standards for Synthesizing the Body of Evidence (In Finding what works in health care: standards for systematic reviews)

Step 8: Interpreting results, presenting the review

• Most information for this can be presented in the analysis of data & results table. However, the strengths & weaknesses of the included studies should be discussed.

• Cochrane Handbook: Part 2, Chapter 14: Completing ‘Summary of findings’ tables and grading the certainty of evidence
• Cochrane Handbook: Part 2, Chapter 15: Interpreting results and drawing conclusions

• It is essential that your review is presented clearly & in accordance with current best practice. View the guidance of the organisation to which you intend to submit your review .Also see the following:
  • PRISMA checklist
  • Standards for Reporting Systematic Reviews (In Finding what works in health care: standards for systematic reviews)

(References: Higgins, J. & Thomas, J., (2019). Cochrane Handbook for Systematic Reviews of Interventions ; Wright, R.W.,  Brand, R.A., Dunn,W. & Spindler, K.P. (2007). How to Write a Systematic Review. Clinical Orthopaedics and Related Research, 455, pp. 23–29; Simons, M. (2011).  Curtin University: Systematic Reviews; Duke University: Systematic Reviews: The Process; University of Edinburgh, Systematic reviews and meta-analyses: a step-by-step guide )