Skip to main content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Medicine: Systematic Reviews

This guide is intended to assist students and researchers to find resources in Medicine

What is a Systematic Review?

A systematic review is a review of a clearly formulated question that uses systematic and reproducible methods to identify, select and critically appraise all relevant research, and to collect and analyse data from the studies that are included in the review.

A systematic review:

  • Answers a focused research question
  • Employs a comprehensive, reproducible search strategy
  • Identifies ALL relevant studies (both published and unpublished)                                                         
  • Assesses all results for inclusion/exclusion, and for quality
  • Presents an unbiased, balanced summary of findings
  • Involves a team of researchers looking at a complex research question
  • Can take months, or even years, to complete. 

A systematic review can be either quantitative or qualitative.

  • quantitative systematic review will include studies that have numerical data.
  • qualitative systematic review derives data from observation, interviews, or verbal interactions and focuses on the meanings and interpretations of the participants. It will include focus groups, interviews, observations and diaries. 

Steps in Systematic Review

A systematic review involves the following steps:  

  1. Check for existing reviews/protocols. If a systematic review answering your question has been conducted, or is being undertaken, you may need to amend or refine your question
  2. Formulate a specific research question that is clear and focused. Use the PICO tool (for quantitative reviews) or PICo (for qualitative reviews)
  3. Develop and register your protocol, including the rationale for the review, and eligibility criteria
  4. Design a robust search strategy that is explicit and reproducible
  5. Conduct a comprehensive search of  the literature by searching the relevant databases and other sources  
  6. Select and critically appraise the quality of included studies
  7. Extract relevant data from individual studies and use established methods to synthesise the data
  8. Interpret your results and prepare a comprehensive report on all aspects of your systematic review. Present your findings so that they can be translated into clinical practice,

Step 1: Finding reviews/protocols

Checking existing reviews/protocols ensures that you are not repeating someone else's work and that you are not wasting resources. It is always necessary to check whether a systematic review answering your question has already been conducted, or is currently being undertaken. This may help you in choosing or refining a review topic. These sources are useful for determining whether a recent systematic review has already been performed on your topic: 

PROSPERO International register of prospective systematic reviews
Cochrane Database of Systematic Reviews Covers all areas of medicine and health
Joanna Briggs Institute EBP Database (OVID) Primarily covers nursing disciplines, with more medical and allied health content available recently
DoPHER Covers both systematic and non-systematic reviews of effectiveness of health promotion and public health worldwide

DARE (Database of Abstracts of Reviews of Effects)   

NOTE: no longer active - records available until 31st March 2015                     

Contains thousands of systematic reviews including several quality assessed reviews
The Campbell Collaboration online library Covers the fields of education, crime and justice, social welfare, international development and nutrition
EPPI-Centre knowledge library  Covers a wide range of topics including health conditions, education and social policy, health promotion and public health and health systems and development
PsycBITE Covers cognitive, behavioural and other treatments for psychological problems in Acquired Brain Impairment
OTseeker Occupational therapy interventions 
PeDRO Physiotherapy evidence database
SpeechBITE Speech pathology treatment evidence
TRIP (Turning research into practice) Clinical search engine which located high-quality research evidence, including systematic reviews 

Step 2: Formulate a specific question

Systematic reviews require focused clinical questions. PICO is a useful tool for formulating such questions.

PICO example for quantitative studies

The PICO (Patient, Intervention, Comparison, Outcome) framework is commonly used to develop focused clinical questions for quantitative systematic reviews.    

Patient, Population or Problem

What are the characteristics of the patient or problem? What is the condition or disease you are interested in?

Intervention or exposure What do you want to do with this patient (e.g. treat, diagnose, observe)?
Comparison What is the alternative to the intervention (e.g. placebo, different drug, surgery)?
Outcome What are the relevant outcomes (e.g. morbidity, death, complications)?

Sample topic: 

In middle aged women suffering migraines, is Botulinium toxin type A compared to placebo effective at decreasing migraine frequency?

P - Middle aged women suffering migraines

I - Botulinium toxin type A

C - Placebo

O - Decreased migraine frequency

Step 2: Formulate a specific question

PICO Framework

Without a well-focused question, it can be very difficult and time consuming to identify appropriate resources and search for relevant evidence. Practitioners of Evidence-Based Practice (EBP) often use a specialized framework, called PICO, to form the question and facilitate the literature search.1 PICO stands for:
 

  • Patient Problem, (or Population)

  • Intervention,

  • Comparison or Control, and

  • Outcome

Framework item: Think about: Example:
Patient Problem (or Population) What are the patient's demographics such as age, gender and ethnicity?  Or what is the or problem type? Work-related neck muscle pain
Intervention What type of intervention is being considered? For example is this a medication of some type, or exercise, or rest? Strength training of the painful muscle
Comparison or Control Is there a camparison treatment to be considered? The comparison may be with another medication, another form of treatment such as exercise, or no treatment at all. Rest
Outcome What would be the desired effect you would like to see? What effects are not wanted? Are there any side effects involved with this form of testing or treatment? Pain relief

When forming your question using PICO, keep the following points in mind:

  • Your Patient is a member of a population as well as a person with (or at risk of) a health problem. So, in addition to age and gender, you may also need to consider ethnicity, socioeconomic status or other demographic variables.

  • A Comparison is not always present in a PICO analysis.

  • Outcomes should be measurable as the best evidence comes from rigorous studies with statistically significant findings.

  • An Outcome ideally measures clinical wellbeing or quality of life, and not alternates such as laboratory test results.

Step 3: Registering your protocol

Registering your protocol is useful because it can: 

  • reduce the risk of duplicate reviews
  • potentially increase visibility of your research to potential researchers or editors globally.

Protocol registries

Popular systematic review registries include:

  • PROSPERO
  • JBI (Joanna  Briggs Institute) - SUMARI 
    JBI SUMARI (System for the Unified Management of the Assessment and Review of Information) is a software package designed to assist in the conduct of JBI systematic reviews.
    Access to SUMARI is available through the Library's subscription to the
    Joanna Briggs Institute EBP database (Ovid)
  • Go to EBP Tools at the top of the screen and select SUMARI.
  • At the login screen click on login through EBP Network/Ovid.

See the SUMARI knowledge base to find information on how to set up your systematic review project and protocol.

Step 4: Developing search strategy

Once you have developed your PICO and well-formed clinical question you can begin to build your search strategy by translating the significant concepts of the PICO into a concept grid. 

It is not necessary to include all of the PICO concepts in the search strategy. It is preferable to search for those concepts that can be clearly defined and translated into search terms. Although a research question may address particular populations, settings or outcomes, these concepts may not be well described in the title or abstract of an article and are often not well indexed with controlled vocabulary terms. It is useful to start with a broad search using the Population and Intervention elements of the PICO. 

For example:

In middle aged women suffering migraines, is Botulinium toxin type A compared to placebo effective at decreasing migraine frequency?

Concept 1: Middle aged women Concept 2: Migraines  Concept 3: Botulinium toxin type A 


Alternative (similar) keywords

Authors often use different terms to describe the same concept. When searching it is important to consider alternative terms (synonyms) and spelling variations which may be used.

Think about:

  • Medical vs. Common terms e.g. Varicella Zoster / Chicken pox
  • Acronyms/Abbreviations e.g. COPD / Chronic Obstructive Pulmonary Disease
  • Generic vs. Brand name drugs e.g. Acetaminophen / Panadol
  • English vs. American terminology e.g. Tumour / Tumor
  • Broader vs. Narrower terms e.g. Obesity surgery / Bariatric surgery

Similar terms can be added to the grid beneath the relevant concept, for example:

Concept 1: Middle aged women Concept 2: Migraine  Concept 3: Botulinium toxin type A 

Women

Woman

Female

Migraine

Migraine disorders

Migraine headaches

Botulinium toxin type A

Botulinium toxins

Clostridium botulinium toxins


Combining search terms with AND and OR

You can structure your search using AND and OR to combine your keywords:

  • AND - use to combine keywords that reflect different concepts e.g. women AND migraine
  • OR - use to combine keywords that reflect similar concepts e.g. woman OR female

Database search tips

  • Truncation (usually *) can be used to find alternate endings of a word e.g. educat* for educate, educated, education, educational etc.
  • Phrase searching (" ") can be used to search for two or more terms as a phrase rather than individually e.g. "migraine headaches"
  • Wildcards  (usually ?) can be used for spelling variations e.g. wom?n for woman and women
  • Title and abstract searching   can be used to narrow search results and increase precision e.g. breast feeding.ti,ab. (Ovid databases)
  • Proximity searching  can be used to achieve greater precision than phrase searching. Use the operators NEAR, NEXT OR ADJ . This allows you to retrieve records that contain your terms (in any order) within a specified number (n) of words of each other. e.g. Community ADJ3 pharmacy.

Note: Truncation symbols, wildcard symbols and proximity operators can vary between databases. See the Help section in the databases to find out which symbols and operators are used.

Step 5: Document your search strategy

When you are conducting your searches, keep track of what you are doing by documenting your search process in enough detail to ensure that it can be reported correctly in the review.

Documentation of your search strategy should include:

  • databases used
  • date of search
  • dates of coverage provided by each database
  • search terms used
  • total publications found
  • number of relevant publications
  • limits applied

Saving searches in a database

It is also possible to save your search history within each database. In order to do this you need to set up a personal account in each relevant database. For details on how to set up an account, check the Help pages. Saving a search in a database will allow you to run the search again at a later date. You also have the option to create an alert for your search.

Manage your search results

We recommend that you use a bibliographic management tool such as EndNote to manage your search results. 

With EndNote you can:

  • export references from databases into your EndNote library
  • store full text PDFs
  • add your own research notes to references
  • insert citations into a Word document and have them formatted in your chosen referencing style
  • create a bibliography in a selected referencing style
  • organise your references into groups.

Step 6: Select and appraise studies

What is critical appraisal?

Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, its value and relevance in a particular context (Burls, 2009).           

Why do we need to critically appraise the literature? We do this to:

  • Weigh up the evidence for usefulness
  • Assess benefits and strengths for research against flaws and weaknesses
  • Assess the research process and results - Are the findings reliable? What do the results mean in the context of the decision we are making?

References

Critical appraisal websites

Step 7: Data extraction

Once multiple team members have screened the entire list of references, you will be left with a core group of studies to be included in your review. The next step is to extract the data from each of the studies in order to synthesize their results. The extraction process should be tracked using a standardized data extraction form (see examples below). Data can also be coded for computer analysis. For more information about data extraction, check out this subject guide by the Himmelfarb Health Sciences Library at George Washington University:

Data Extraction Resources

Step 8: Interpret and present findings

Effectively Sharing Your Research

The best way to share your systematic review findings with the research community is to prepare a well-written report. Clarity is key: readers should be able to follow, and potentially replicate, every step of your review process. There are a number of well-known standards, handbooks, and guides available for you to follow; on this page, we outline a few of the basics.

The PRISMA Checklist and Flow Diagram

The PRISMA Checklist can serve as an outline for your systematic review report. There are 27 items on the checklist. It begins with the simple inclusion of a title and abstract, and goes on to outline the introduction, methods, results, and discussion portions of your report. It also suggests that you clearly state all sources of funding.

To download an editable template of the PRISMA Checklist, click below:

Existing hand in hand with the PRISMA Checklist, the PRISMA Flow Diagram facilitates accurate reporting of your search process. It displays the number of studies involved at each stage of your research. Click below to download an editable template:

Handbooks and Standards

Reporting Your Search Methods

Your systematic review should be designed with the research community in mind. Other researchers might want to explore the details of your search. Future research teams might want to replicate your review in order to follow up on your findings. To make these things possible, you must report every detail of your search methodology.

Your search methods should be explained at varying levels of detail in multiple areas of your report.

Abstract

In a dedicated section in your review abstract, briefly explain your search methods. State the databases used and the timeframe of your searches. You may also include a very brief description of your research question, core concepts, search criteria, and search process.

Methods Section

An entire section of your report should be dedicated to explaining the methods used to complete the review. Within this section, expand upon the search methods outlined in your abstract. This is also the place to thoroughly detail your search strategy, outline your inclusion and exclusion criteria, state the number of results at each phase of your search process, explain your screening procedures, and describe how data was extracted and analysed. If a methodological expert (librarian, statistician, etc.) contributed to the search or analyses, they may be best suited to write the relevant parts of this section to reflect their contributions accurately. If you used the PRISMA Checklist and Diagram, you can include these in the review's appendix.

Appendix

In your review's appendix, include your entire search strategy. This is the best way to make your review reproducible by others. We recommend displaying your strategy in table format, with separate columns for each database. Below is an example of a search strategy table:

The Forest Plot

Generic forest plot

James Grellier / CC-BY-SA-3.0

Forest plots are a standard way of visualizing the results of a systematic review. The summary measure, represented by a diamond in the above example, is the overall statistical result of the data analysis.

Forest plots are recommended for inclusion in systematic reviews by the Cochrane Collaboration. RevMan, Cochrane's free reference manager, can also be used to produce forest plots.

How to Interpret a Forest Plot

Video opens in a new tab.