A systematic review is a review of a clearly formulated question that uses systematic and reproducible methods to identify, select and critically appraise all relevant research, and to collect and analyse data from the studies that are included in the review.
A systematic review:
A systematic review can be either quantitative or qualitative.
A systematic review involves the following steps: 
Checking existing reviews/protocols ensures that you are not repeating someone else's work and that you are not wasting resources. It is always necessary to check whether a systematic review answering your question has already been conducted, or is currently being undertaken. This may help you in choosing or refining a review topic. These sources are useful for determining whether a recent systematic review has already been performed on your topic:
| PROSPERO | International register of prospective systematic reviews |
| Cochrane Database of Systematic Reviews | Covers all areas of medicine and health |
| Joanna Briggs Institute EBP Database (OVID) | Primarily covers nursing disciplines, with more medical and allied health content available recently |
| DoPHER | Covers both systematic and non-systematic reviews of effectiveness of health promotion and public health worldwide |
|
DARE (Database of Abstracts of Reviews of Effects) NOTE: no longer active - records available until 31st March 2015 |
Contains thousands of systematic reviews including several quality assessed reviews |
| The Campbell Collaboration online library | Covers the fields of education, crime and justice, social welfare, international development and nutrition |
| EPPI-Centre knowledge library | Covers a wide range of topics including health conditions, education and social policy, health promotion and public health and health systems and development |
| PsycBITE | Covers cognitive, behavioural and other treatments for psychological problems in Acquired Brain Impairment |
| OTseeker | Occupational therapy interventions |
| PeDRO | Physiotherapy evidence database |
| SpeechBITE | Speech pathology treatment evidence |
| TRIP (Turning research into practice) | Clinical search engine which located high-quality research evidence, including systematic reviews |
Systematic reviews require focused clinical questions. PICO is a useful tool for formulating such questions.
PICO example for quantitative studies
The PICO (Patient, Intervention, Comparison, Outcome) framework is commonly used to develop focused clinical questions for quantitative systematic reviews.
| Patient, Population or Problem |
What are the characteristics of the patient or problem? What is the condition or disease you are interested in? |
| Intervention or exposure | What do you want to do with this patient (e.g. treat, diagnose, observe)? |
| Comparison | What is the alternative to the intervention (e.g. placebo, different drug, surgery)? |
| Outcome | What are the relevant outcomes (e.g. morbidity, death, complications)? |
Sample topic:
In middle aged women suffering migraines, is Botulinium toxin type A compared to placebo effective at decreasing migraine frequency?
P - Middle aged women suffering migraines
I - Botulinium toxin type A
C - Placebo
O - Decreased migraine frequency
PICO Framework
Without a well-focused question, it can be very difficult and time consuming to identify appropriate resources and search for relevant evidence. Practitioners of Evidence-Based Practice (EBP) often use a specialized framework, called PICO, to form the question and facilitate the literature search.1 PICO stands for:
Patient Problem, (or Population)
Intervention,
Comparison or Control, and
Outcome
| Framework item: | Think about: | Example: |
|---|---|---|
| Patient Problem (or Population) | What are the patient's demographics such as age, gender and ethnicity? Or what is the or problem type? | Work-related neck muscle pain |
| Intervention | What type of intervention is being considered? For example is this a medication of some type, or exercise, or rest? | Strength training of the painful muscle |
| Comparison or Control | Is there a camparison treatment to be considered? The comparison may be with another medication, another form of treatment such as exercise, or no treatment at all. | Rest |
| Outcome | What would be the desired effect you would like to see? What effects are not wanted? Are there any side effects involved with this form of testing or treatment? | Pain relief |
When forming your question using PICO, keep the following points in mind:
Your Patient is a member of a population as well as a person with (or at risk of) a health problem. So, in addition to age and gender, you may also need to consider ethnicity, socioeconomic status or other demographic variables.
A Comparison is not always present in a PICO analysis.
Outcomes should be measurable as the best evidence comes from rigorous studies with statistically significant findings.
An Outcome ideally measures clinical wellbeing or quality of life, and not alternates such as laboratory test results.
Registering your protocol is useful because it can:
Protocol registries
Popular systematic review registries include:
See the SUMARI knowledge base to find information on how to set up your systematic review project and protocol.
Once you have developed your PICO and well-formed clinical question you can begin to build your search strategy by translating the significant concepts of the PICO into a concept grid.
It is not necessary to include all of the PICO concepts in the search strategy. It is preferable to search for those concepts that can be clearly defined and translated into search terms. Although a research question may address particular populations, settings or outcomes, these concepts may not be well described in the title or abstract of an article and are often not well indexed with controlled vocabulary terms. It is useful to start with a broad search using the Population and Intervention elements of the PICO.
For example:
In middle aged women suffering migraines, is Botulinium toxin type A compared to placebo effective at decreasing migraine frequency?
| Concept 1: Middle aged women | Concept 2: Migraines | Concept 3: Botulinium toxin type A |
|
|
Alternative (similar) keywords
Authors often use different terms to describe the same concept. When searching it is important to consider alternative terms (synonyms) and spelling variations which may be used.
Think about:
Similar terms can be added to the grid beneath the relevant concept, for example:
| Concept 1: Middle aged women | Concept 2: Migraine | Concept 3: Botulinium toxin type A |
|
Women Woman Female |
Migraine Migraine disorders Migraine headaches |
Botulinium toxin type A Botulinium toxins Clostridium botulinium toxins |
Combining search terms with AND and OR
You can structure your search using AND and OR to combine your keywords:
Database search tips
Note: Truncation symbols, wildcard symbols and proximity operators can vary between databases. See the Help section in the databases to find out which symbols and operators are used.
When you are conducting your searches, keep track of what you are doing by documenting your search process in enough detail to ensure that it can be reported correctly in the review.
Documentation of your search strategy should include:
Saving searches in a database
It is also possible to save your search history within each database. In order to do this you need to set up a personal account in each relevant database. For details on how to set up an account, check the Help pages. Saving a search in a database will allow you to run the search again at a later date. You also have the option to create an alert for your search.
Manage your search results
We recommend that you use a bibliographic management tool such as EndNote to manage your search results.
With EndNote you can:
What is critical appraisal?
Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, its value and relevance in a particular context (Burls, 2009). 
Why do we need to critically appraise the literature? We do this to:
References
Critical appraisal websites
Once multiple team members have screened the entire list of references, you will be left with a core group of studies to be included in your review. The next step is to extract the data from each of the studies in order to synthesize their results. The extraction process should be tracked using a standardized data extraction form (see examples below). Data can also be coded for computer analysis. For more information about data extraction, check out this subject guide by the Himmelfarb Health Sciences Library at George Washington University:
The best way to share your systematic review findings with the research community is to prepare a well-written report. Clarity is key: readers should be able to follow, and potentially replicate, every step of your review process. There are a number of well-known standards, handbooks, and guides available for you to follow; on this page, we outline a few of the basics.
The PRISMA Checklist can serve as an outline for your systematic review report. There are 27 items on the checklist. It begins with the simple inclusion of a title and abstract, and goes on to outline the introduction, methods, results, and discussion portions of your report. It also suggests that you clearly state all sources of funding.
To download an editable template of the PRISMA Checklist, click below:
Existing hand in hand with the PRISMA Checklist, the PRISMA Flow Diagram facilitates accurate reporting of your search process. It displays the number of studies involved at each stage of your research. Click below to download an editable template:
Your systematic review should be designed with the research community in mind. Other researchers might want to explore the details of your search. Future research teams might want to replicate your review in order to follow up on your findings. To make these things possible, you must report every detail of your search methodology.
Your search methods should be explained at varying levels of detail in multiple areas of your report.
In a dedicated section in your review abstract, briefly explain your search methods. State the databases used and the timeframe of your searches. You may also include a very brief description of your research question, core concepts, search criteria, and search process.
An entire section of your report should be dedicated to explaining the methods used to complete the review. Within this section, expand upon the search methods outlined in your abstract. This is also the place to thoroughly detail your search strategy, outline your inclusion and exclusion criteria, state the number of results at each phase of your search process, explain your screening procedures, and describe how data was extracted and analysed. If a methodological expert (librarian, statistician, etc.) contributed to the search or analyses, they may be best suited to write the relevant parts of this section to reflect their contributions accurately. If you used the PRISMA Checklist and Diagram, you can include these in the review's appendix.
In your review's appendix, include your entire search strategy. This is the best way to make your review reproducible by others. We recommend displaying your strategy in table format, with separate columns for each database. Below is an example of a search strategy table:
James Grellier / CC-BY-SA-3.0
Forest plots are a standard way of visualizing the results of a systematic review. The summary measure, represented by a diamond in the above example, is the overall statistical result of the data analysis.
Forest plots are recommended for inclusion in systematic reviews by the Cochrane Collaboration. RevMan, Cochrane's free reference manager, can also be used to produce forest plots.
Video opens in a new tab.
