Skip to Main Content
Banner Image
WNHS Internet |   WNHS Hub |  HealthPoint |   WA Health Library Sites
Contact Us!

Research: Systematic Reviews Step-by-Step

Systematic Reviews Step-by-Step

Use the tabs above for information about the following steps:

  1. Scope the literature
  2. Form a team
  3. Formulate the question, develop & register the protocol
  4. Systematically search the literature
  5. Data extraction
  6. Quality appraisal
  7. Data synthesis
  8. Interpreting results, presenting the review

 

Step 1: Scope the literature

A systematic review is most useful where:

  • there is a substantive research question
  • several empirical studies have been published 
  • there is uncertainty about the results

 

Step 2: Form a team

The highest quality reviews will have input from experts in:

  • the subject being reviewed
  • systematic review methodology
  • information retrieval
  • statistics
  • reference management
  • other aspects such as qualitative research methods if needed

 

(References: Higgins, J. & Thomas, J. (2019). Cochrane Handbook for Systematic Reviews of Interventions Wright, R.W.,  Brand, R.A., Dunn,W. & Spindler, K.P. (2007). How to Write a Systematic Review. Clinical Orthopaedics and Related Research, 455, pp. 23–29; Simons, M. (2011).  Curtin University Systematic Reviews; Duke University: Systematic Reviews: The Process; University of Edinburgh, Systematic reviews and meta-analyses: a step-by-step guide )

Step 3: Formulate the question, develop & register the protocol

  • Establish a clear & answerable question. Consider using PICOT to focus the question (see Ask the Right Question box). May require several revisions & more scoping of literature.
  • Establish whether the question has already been answered in the published literature or is registered as an ongoing review (search DARECochrane, and PROSPERO, or the Campbell Collaboration for social interventions)
  • Write a review protocol to formulate the review question and methods before retrieving the literature, thereby minimizing bias. Include the clear & answerable question; rationale for the review; the search strategy (search terms & resources);  inclusion & exclusion criteria (such as publication date range, language, main focus, explicit methodology, outcome measurements); quality checklist to assess individual studies; data extraction & management strategy;  data synthesis strategy;  dissemination strategy;  timeframe. 
  • RevMan – the software used for preparing & maintaining Cochrane protocols & reviews
  • **NEW** RevMan Web is now also available for the creation of non-Cochrane reviews (requires a subscription)
  • Register your review to ensure others know it is ongoing. Register with Cochrane for interventions or accuracy of diagnostic tests; with Campbell Collaboration for social interventions in education, crime & justice, social welfare; with PROSPERO for reviews in health or social care. New Cochrane protocols are automatically uploaded to PROSPERO. Each organisation will have its own suggestions for the review protocol  

Step 4: Systematically search the literature

  • At least 2 reviewers are required. They first independently screen titles & abstracts based on the research question (in terms of PICOT). Based on this, full-text articles are retrieved for a second stage screening. The final selection is then submitted for data extraction.
  • Sources to search include: Cochrane Library; DARE (Database of Abstracts of Reviews of Effects); Medline; Embase; PubMed Clinical Queries; subject specific databases (e.g. CINAHL, PsycInfo); conference proceedings; hand search key journals; bibliographies & references listed in primary sources; guidelines (often based on SRs); grey literature (see our Grey Literature page); forward citation searching of seminal articles; foreign language literature (do not limit searches to English); clinical trials (see Trials Registries tab on Grey Literature page ), contact scholars/experts in the field, search the Internet. See the following for more details:
  • The search process should be meticulously documented as it develops. Include:
  1. databases used
  2. date of search
  3. dates of coverage provided by each database
  4. search terms used
  5. total number of publications found
  6. number of relevant publications
  7. limits applied

Include the PRISMA flowchart in the review. Use one of the following online tools for managing your search results:

  • EndNote (freely available to Department of Health staff);
  • Covidence ("improves healthcare evidence synthesis by improving the efficiency and experience of creating and maintaining Systematic Reviews." Supported by The Alfred Hospital, Monash University, National ICT Australia & University of London);
  • EPPI-Reviewer 4 (software for all types of literature reviews including systematic reviews & meta-analyses)

 

Step 5: Data extraction

  • Devise a  standardized form to assist with data extraction.
  • This will generally include details of the reference, study objectives, study design, population, intervention, control, outcome, comments on study quality.

 

Step 6: Quality appraisal

Each study should be evaluated in terms of its:
  • Methodological quality - the extent to which the design & conduct are likely to have prevented systematic errors (bias)
  • Precision - a measure of the likelihood of random errors (confidence interval)
  • External validity - the extent to which the results are generalisable to a particular target population
  • See the appropriate checklists in the Appraise & Calculate boxes. Also see:

Step 7: Data synthesis

  • After including & excluding studies according to the quality appraisal, the analysis of data & results should initially begin with a simple descriptive evaluation of each study, often presented in a table.
  • If the study results are very heterogeneous it may be more appropriate to summarise the data narratively.
  • A statistical analysis (meta-analysis) should include numerical and graphical presentations of the data, look at the strength and consistency of the evidence and investigate reasons for inconsistencies.
  • See Calculate box and

 

Step 8: Interpreting results, presenting the review

  • Most information for this can be presented in the analysis of data & results table. However, the strengths & weaknesses of the included studies should be discussed.

 

(References: Higgins, J. & Thomas, J., (2019). Cochrane Handbook for Systematic Reviews of Interventions Wright, R.W.,  Brand, R.A., Dunn,W. & Spindler, K.P. (2007). How to Write a Systematic ReviewClinical Orthopaedics and Related Research, 455, pp. 23–29; Simons, M. (2011).  Curtin University: Systematic Reviews; Duke University: Systematic Reviews: The Process; University of Edinburgh, Systematic reviews and meta-analyses: a step-by-step guide )

JBI SUMARI

JBI SUMARI facilitates the entire systematic review process from protocol to report, and includes team and contributor management for effective and efficient collaboration. Access to JBI SUMARI is through the KEMH Library's subscription. Note, you need to create a personal login to use this tool.  See here for further information and help.