methodological systematic literature review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Methodology

Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

Formulate a research question
Develop a protocol
Search for all relevant studies
Apply the selection criteria
Extract the data
Synthesize the data
Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism. Run a free check.

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

Academic style
Vague sentences
Style consistency

See an example

methodological systematic literature review

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
Access to databases and journal archives. Often, your educational institution provides you with access.
Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

They minimize research bias by considering all available evidence and evaluating each study for bias.
Their methods are transparent , so they can be scrutinized by others.
They’re thorough : they summarize all available evidence.
They can be replicated and updated by others.

Systematic reviews also have a few cons .

They’re time-consuming .
They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

Allow you to more effectively communicate your research to other researchers and practitioners
Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

Population(s) or problem(s)
Intervention(s)
Comparison(s)

You can rearrange these four components to write your research question:

What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

Type of study design(s)
The population of patients with eczema
The intervention of probiotics
In comparison to no treatment, placebo , or non-probiotic treatment
The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
Randomized control trials, a type of study design

Their research question was:

What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

Background information : Provide the context of the research question, including why it’s important.
Research objective (s) : Rephrase your research question as an objective.
Selection criteria: State how you’ll decide which studies to include or exclude from your review.
Search strategy: Discuss your plan for finding studies.
Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
Handsearch: Conference proceedings and reference lists of articles
Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

Abstract : A summary of the review
Introduction : Including the rationale and objectives
Methods : Including the selection criteria, search method, data extraction method, and synthesis method
Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
Discussion : Including interpretation of the results and limitations of the review
Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

Student’s t -distribution
Normal distribution
Null and Alternative Hypotheses
Chi square tests
Confidence interval
Quartiles & Quantiles
Cluster sampling
Stratified sampling
Data cleansing
Reproducibility vs Replicability
Peer review
Prospective cohort study

Research bias

Implicit bias
Cognitive bias
Placebo effect
Hawthorne effect
Hindsight bias
Affect heuristic
Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other academic texts , with an introduction , a main body, and a conclusion .

An annotated bibliography is a list of source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a paper .

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved July 4, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, what is your plagiarism score.

Which review is that? A guide to review types

Which review is that?
Review Comparison Chart
Decision Tool
Critical Review
Integrative Review
Narrative Review
State of the Art Review
Narrative Summary
Systematic Review
Meta-analysis
Comparative Effectiveness Review
Diagnostic Systematic Review
Network Meta-analysis
Prognostic Review
Psychometric Review
Review of Economic Evaluations
Systematic Review of Epidemiology Studies
Living Systematic Reviews
Umbrella Review
Review of Reviews
Rapid Review
Rapid Evidence Assessment
Rapid Realist Review
Qualitative Evidence Synthesis
Qualitative Interpretive Meta-synthesis
Qualitative Meta-synthesis
Qualitative Research Synthesis
Framework Synthesis - Best-fit Framework Synthesis
Meta-aggregation
Meta-ethnography
Meta-interpretation
Meta-narrative Review
Meta-summary
Thematic Synthesis
Mixed Methods Synthesis
Narrative Synthesis
Bayesian Meta-analysis
EPPI-Centre Review
Critical Interpretive Synthesis
Realist Synthesis - Realist Review
Scoping Review
Mapping Review
Systematised Review
Concept Synthesis
Expert Opinion - Policy Review
Technology Assessment Review

Methodological Review

Systematic Search and Review

A methodological review is a type of systematic secondary research (i.e., research synthesis) which focuses on summarising the state-of-the-art methodological practices of research in a substantive field or topic" (Chong et al, 2021).

Methodological reviews "can be performed to examine any methodological issues relating to the design, conduct and review of research studies and also evidence syntheses". Munn et al, 2018)

How to write the methods section of a systematic review

Home | Blog | How To | How to write the methods section of a systematic review

Covidence breaks down how to write a methods section

The methods section of your systematic review describes what you did, how you did it, and why. Readers need this information to interpret the results and conclusions of the review. Often, a lot of information needs to be distilled into just a few paragraphs. This can be a challenging task, but good preparation and the right tools will help you to set off in the right direction 🗺️🧭.

Systematic reviews are so-called because they are conducted in a way that is rigorous and replicable. So it’s important that these methods are reported in a way that is thorough, clear, and easy to navigate for the reader – whether that’s a patient, a healthcare worker, or a researcher.

Like most things in a systematic review, the methods should be planned upfront and ideally described in detail in a project plan or protocol. Reviews of healthcare interventions follow the PRISMA guidelines for the minimum set of items to report in the methods section. But what else should be included? It’s a good idea to consider what readers will want to know about the review methods and whether the journal you’re planning to submit the work to has expectations on the reporting of methods. Finding out in advance will help you to plan what to include.

Describe what happened

While the research plan sets out what you intend to do, the methods section is a write-up of what actually happened. It’s not a simple case of rewriting the plan in the past tense – you will also need to discuss and justify deviations from the plan and describe the handling of issues that were unforeseen at the time the plan was written. For this reason, it is useful to make detailed notes before, during, and after the review is completed. Relying on memory alone risks losing valuable information and trawling through emails when the deadline is looming can be frustrating and time consuming!

Keep it brief

The methods section should be succinct but include all the noteworthy information. This can be a difficult balance to achieve. A useful strategy is to aim for a brief description that signposts the reader to a separate section or sections of supporting information. This could include datasets, a flowchart to show what happened to the excluded studies, a collection of search strategies, and tables containing detailed information about the studies.This separation keeps the review short and simple while enabling the reader to drill down to the detail as needed. And if the methods follow a well-known or standard process, it might suffice to say so and give a reference, rather than describe the process at length.

Follow a structure

A clear structure provides focus. Use of descriptive headings keeps the writing on track and helps the reader get to key information quickly. What should the structure of the methods section look like? As always, a lot depends on the type of review but it will certainly contain information relating to the following areas:

Selection criteria ⭕
Data collection and analysis 👩‍💻
Study quality and risk of bias ⚖️

Let’s look at each of these in turn.

1. Selection criteria ⭕

The criteria for including and excluding studies are listed here. This includes detail about the types of studies, the types of participants, the types of interventions and the types of outcomes and how they were measured.

2. Search 🕵🏾‍♀️

Comprehensive reporting of the search is important because this means it can be evaluated and replicated. The search strategies are included in the review, along with details of the databases searched. It’s also important to list any restrictions on the search (for example, language), describe how resources other than electronic databases were searched (for example, non-indexed journals), and give the date that the searches were run. The PRISMA-S extension provides guidance on reporting literature searches.

Systematic reviewer pro-tip:

Copy and paste the search strategy to avoid introducing typos

3. Data collection and analysis 👩‍💻

This section describes:

how studies were selected for inclusion in the review
how study data were extracted from the study reports
how study data were combined for analysis and synthesis

To describe how studies were selected for inclusion , review teams outline the screening process. Covidence uses reviewers’ decision data to automatically populate a PRISMA flow diagram for this purpose. Covidence can also calculate Cohen’s kappa to enable review teams to report the level of agreement among individual reviewers during screening.

To describe how study data were extracted from the study reports , reviewers outline the form that was used, any pilot-testing that was done, and the items that were extracted from the included studies. An important piece of information to include here is the process used to resolve conflict among the reviewers. Covidence’s data extraction tool saves reviewers’ comments and notes in the system as they work. This keeps the information in one place for easy retrieval ⚡.

To describe how study data were combined for analysis and synthesis, reviewers outline the type of synthesis (narrative or quantitative, for example), the methods for grouping data, the challenges that came up, and how these were dealt with. If the review includes a meta-analysis, it will detail how this was performed and how the treatment effects were measured.

4. Study quality and risk of bias ⚖️

Because the results of systematic reviews can be affected by many types of bias, reviewers make every effort to minimise it and to show the reader that the methods they used were appropriate. This section describes the methods used to assess study quality and an assessment of the risk of bias across a range of domains.

Steps to assess the risk of bias in studies include looking at how study participants were assigned to treatment groups and whether patients and/or study assessors were blinded to the treatment given. Reviewers also report their assessment of the risk of bias due to missing outcome data, whether that is due to participant drop-out or non-reporting of the outcomes by the study authors.

Covidence’s default template for assessing study quality is Cochrane’s risk of bias tool but it is also possible to start from scratch and build a tool with a set of custom domains if you prefer.

Careful planning, clear writing, and a structured approach are key to a good methods section. A methodologist will be able to refer review teams to examples of good methods reporting in the literature. Covidence helps reviewers to screen references, extract data and complete risk of bias tables quickly and efficiently. Sign up for a free trial today!

Laura Mellor. Portsmouth, UK

Perhaps you'd also like....

Top 5 Tips for High-Quality Systematic Review Data Extraction

Data extraction can be a complex step in the systematic review process. Here are 5 top tips from our experts to help prepare and achieve high quality data extraction.

How to get through study quality assessment Systematic Review

Find out 5 tops tips to conducting quality assessment and why it’s an important step in the systematic review process.

How to extract study data for your systematic review

Learn the basic process and some tips to build data extraction forms for your systematic review with Covidence.

Better systematic review management

Head office, working for an institution or organisation.

Find out why over 350 of the world’s leading institutions are seeing a surge in publications since using Covidence!

Request a consultation with one of our team members and start empowering your researchers:

By using our site you consent to our use of cookies to measure and improve our site’s performance. Please see our Privacy Policy for more information.

Locations and Hours
UCLA Library
Research Guides
Biomedical Library Guides

Systematic Reviews

Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

Planning Your Systematic Review
Database Searching
Creating the Search
Search Filters and Hedges
Grey Literature
Managing and Appraising Results
Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x


	Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or mode	Seeks to identify most significant items in the field	No formal quality assessment. Attempts to evaluate according to contribution	Typically narrative, perhaps conceptual or chronological	Significant component: seeks to identify conceptual contribution to embody existing or derive new theory
	Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness. May include research findings	May or may not include comprehensive searching	May or may not include quality assessment	Typically narrative	Analysis may be chronological, conceptual, thematic, etc.
Mapping review/ systematic map	Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature	Completeness of searching determined by time/scope constraints	No formal quality assessment	May be graphical and tabular	Characterizes quantity and quality of literature, perhaps by study design and other key features. May identify need for primary or secondary research
	Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results	Aims for exhaustive, comprehensive searching. May use funnel plot to assess completeness	Quality assessment may determine inclusion/ exclusion and/or sensitivity analyses	Graphical and tabular with narrative commentary	Numerical analysis of measures of effect assuming absence of heterogeneity
	Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies	Requires either very sensitive search to retrieve all studies or separately conceived quantitative and qualitative strategies	Requires either a generic appraisal instrument or separate appraisal processes with corresponding checklists	Typically both components will be presented as narrative and in tables. May also employ graphical means of integrating quantitative and qualitative studies	Analysis may characterise both literatures and look for correlations between characteristics or use gap analysis to identify aspects absent in one literature but missing in the other
	Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics	May or may not include comprehensive searching (depends whether systematic overview or not)	May or may not include quality assessment (depends whether systematic overview or not)	Synthesis depends on whether systematic or not. Typically narrative but may include tabular features	Analysis may be chronological, conceptual, thematic, etc.
	Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies	May employ selective or purposive sampling	Quality assessment typically used to mediate messages not for inclusion/exclusion	Qualitative, narrative synthesis	Thematic analysis, may include conceptual models
	Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research	Completeness of searching determined by time constraints	Time-limited formal quality assessment	Typically narrative and tabular	Quantities of literature and overall quality/direction of effect of literature
	Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research)	Completeness of searching determined by time/scope constraints. May include research in progress	No formal quality assessment	Typically tabular with some narrative commentary	Characterizes quantity and quality of literature, perhaps by study design and other key features. Attempts to specify a viable review
	Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives	Aims for comprehensive searching of current literature	No formal quality assessment	Typically narrative, may have tabular accompaniment	Current state of knowledge and priorities for future investigation and research
	Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review	Aims for exhaustive, comprehensive searching	Quality assessment may determine inclusion/exclusion	Typically narrative with tabular accompaniment	What is known; recommendations for practice. What remains unknown; uncertainty around findings, recommendations for future research
	Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis’	Aims for exhaustive, comprehensive searching	May or may not include quality assessment	Minimal narrative, tabular summary of studies	What is known; recommendations for practice. Limitations
	Attempt to include elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment	May or may not include comprehensive searching	May or may not include quality assessment	Typically narrative with tabular accompaniment	What is known; uncertainty around findings; limitations of methodology
	Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results	Identification of component reviews, but no search for primary studies	Quality assessment of studies within component reviews and/or of reviews themselves	Graphical and tabular with narrative commentary	What is known; recommendations for practice. What remains unknown; recommendations for future research

<< Previous: Home
Next: Planning Your Systematic Review >>
Last Updated: Apr 17, 2024 2:02 PM
URL: https://guides.library.ucla.edu/systematicreviews

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

Methodology of a systematic review

Affiliations.

1 Hospital Universitario La Paz, Madrid, España. Electronic address: [email protected].
2 Hospital Universitario Fundación Alcorcón, Madrid, España.
3 Instituto Valenciano de Oncología, Valencia, España.
4 Hospital Universitario de Cabueñes, Gijón, Asturias, España.
5 Hospital Universitario Ramón y Cajal, Madrid, España.
6 Hospital Universitario Gregorio Marañón, Madrid, España.
7 Hospital Universitario de Canarias, Tenerife, España.
8 Hospital Clínic, Barcelona, España; EAU Guidelines Office Board Member.
PMID: 29731270
DOI: 10.1016/j.acuro.2018.01.010

Context: The objective of evidence-based medicine is to employ the best scientific information available to apply to clinical practice. Understanding and interpreting the scientific evidence involves understanding the available levels of evidence, where systematic reviews and meta-analyses of clinical trials are at the top of the levels-of-evidence pyramid.

Acquisition of evidence: The review process should be well developed and planned to reduce biases and eliminate irrelevant and low-quality studies. The steps for implementing a systematic review include (i) correctly formulating the clinical question to answer (PICO), (ii) developing a protocol (inclusion and exclusion criteria), (iii) performing a detailed and broad literature search and (iv) screening the abstracts of the studies identified in the search and subsequently of the selected complete texts (PRISMA).

Synthesis of the evidence: Once the studies have been selected, we need to (v) extract the necessary data into a form designed in the protocol to summarise the included studies, (vi) assess the biases of each study, identifying the quality of the available evidence, and (vii) develop tables and text that synthesise the evidence.

Conclusions: A systematic review involves a critical and reproducible summary of the results of the available publications on a particular topic or clinical question. To improve scientific writing, the methodology is shown in a structured manner to implement a systematic review.

Keywords: Meta-analysis; Metaanálisis; Methodology; Metodología; Revisión sistemática; Systematic review.

PubMed Disclaimer

LinkOut - more resources

Full text sources.

Elsevier Science

Other Literature Sources

scite Smart Citations

Research Materials

NCI CPTC Antibody Characterization Program
Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Communicative Sciences and Disorders

Online Learners: Quick Links
ASHA Journals
Research Tip 1: Define the Research Question
Reference Resources
Evidence Summaries & Clinical Guidelines
Drug Information
Health Data & Statistics
Patient/Consumer Facing Materials
Images/Streaming Video
Database Tutorials
Crafting a Search
Cited Reference Searching
Research Tip 4: Find Grey Literature
Research Tip 5: Save Your Work
Cite and Manage Your Sources
Critical Appraisal
What are Literature Reviews?
Conducting & Reporting Systematic Reviews
Finding Systematic Reviews
Tutorials & Tools for Literature Reviews
Point of Care Tools (Mobile Apps)

Choosing a Review Type

For guidance related to choosing a review type, see:

"What Type of Review is Right for You?" - Decision Tree (PDF) This decision tree, from Cornell University Library, highlights key difference between narrative, systematic, umbrella, scoping and rapid reviews.
Reviewing the literature: choosing a review design Noble, H., & Smith, J. (2018). Reviewing the literature: Choosing a review design. Evidence Based Nursing, 21(2), 39–41. https://doi.org/10.1136/eb-2018-102895
What synthesis methodology should I use? A review and analysis of approaches to research synthesis Schick-Makaroff, K., MacDonald, M., Plummer, M., Burgess, J., & Neander, W. (2016). What synthesis methodology should I use? A review and analysis of approaches to research synthesis. AIMS Public Health, 3 (1), 172-215. doi:10.3934/publichealth.2016.1.172 More information less... ABSTRACT: Our purpose is to present a comprehensive overview and assessment of the main approaches to research synthesis. We use "research synthesis" as a broad overarching term to describe various approaches to combining, integrating, and synthesizing research findings.
Right Review - Decision Support Tool Not sure of the most suitable review method? Answer a few questions and be guided to suitable knowledge synthesis methods. Updated in 2022 and featured in the Journal of Clinical Epidemiology 10.1016/j.jclinepi.2022.03.004

Types of Evidence Synthesis / Literature Reviews

Literature reviews are comprehensive summaries and syntheses of the previous research on a given topic. While narrative reviews are common across all academic disciplines, reviews that focus on appraising and synthesizing research evidence are increasingly important in the health and social sciences.

Most evidence synthesis methods use formal and explicit methods to identify, select and combine results from multiple studies, making evidence synthesis a form of meta-research.

The review purpose, methods used and the results produced vary among different kinds of literature reviews; some of the common types of literature review are detailed below.

Common Types of Literature Reviews 1

Narrative (literature) review.

A broad term referring to reviews with a wide scope and non-standardized methodology
Search strategies, comprehensiveness of literature search, time range covered and method of synthesis will vary and do not follow an established protocol

Integrative Review

A type of literature review based on a systematic, structured literature search
Often has a broadly defined purpose or review question
Seeks to generate or refine and theory or hypothesis and/or develop a holistic understanding of a topic of interest
Relies on diverse sources of data (e.g. empirical, theoretical or methodological literature; qualitative or quantitative studies)

Systematic Review

Systematically and transparently collects and categorize existing evidence on a question of scientific, policy or management importance
Follows a research protocol that is established a priori
Some sub-types of systematic reviews include: SRs of intervention effectiveness, diagnosis, prognosis, etiology, qualitative evidence, economic evidence, and more.
Time-intensive and often takes months to a year or more to complete
The most commonly referred to type of evidence synthesis; sometimes confused as a blanket term for other types of reviews

Meta-Analysis

Statistical technique for combining the findings from disparate quantitative studies
Uses statistical methods to objectively evaluate, synthesize, and summarize results
Often conducted as part of a systematic review

Scoping Review

Systematically and transparently collects and categorizes existing evidence on a broad question of scientific, policy or management importance
Seeks to identify research gaps, identify key concepts and characteristics of the literature and/or examine how research is conducted on a topic of interest
Useful when the complexity or heterogeneity of the body of literature does not lend itself to a precise systematic review
Useful if authors do not have a single, precise review question
May critically evaluate existing evidence, but does not attempt to synthesize the results in the way a systematic review would
May take longer than a systematic review

Rapid Review

Applies a systematic review methodology within a time-constrained setting
Employs methodological "shortcuts" (e.g., limiting search terms and the scope of the literature search), at the risk of introducing bias
Useful for addressing issues requiring quick decisions, such as developing policy recommendations

Umbrella Review

Reviews other systematic reviews on a topic
Often defines a broader question than is typical of a traditional systematic review
Most useful when there are competing interventions to consider

1. Adapted from:

Eldermire, E. (2021, November 15). A guide to evidence synthesis: Types of evidence synthesis. Cornell University LibGuides. https://guides.library.cornell.edu/evidence-synthesis/types

Nolfi, D. (2021, October 6). Integrative Review: Systematic vs. Scoping vs. Integrative. Duquesne University LibGuides. https://guides.library.duq.edu/c.php?g=1055475&p=7725920

Delaney, L. (2021, November 24). Systematic reviews: Other review types. UniSA LibGuides. https://guides.library.unisa.edu.au/SystematicReviews/OtherReviewTypes

Integrative Reviews

"The integrative review method is an approach that allows for the inclusion of diverse methodologies (i.e. experimental and non-experimental research)." (Whittemore & Knafl, 2005, p. 547).

The integrative review: Updated methodology Whittemore, R., & Knafl, K. (2005). The integrative review: Updated methodology. Journal of Advanced Nursing, 52 (5), 546–553. doi:10.1111/j.1365-2648.2005.03621.x More information less... ABSTRACT: The aim of this paper is to distinguish the integrative review method from other review methods and to propose methodological strategies specific to the integrative review method to enhance the rigour of the process....An integrative review is a specific review method that summarizes past empirical or theoretical literature to provide a more comprehensive understanding of a particular phenomenon or healthcare problem....Well-done integrative reviews present the state of the science, contribute to theory development, and have direct applicability to practice and policy.

Conducting integrative reviews: A guide for novice nursing researchers Dhollande, S., Taylor, A., Meyer, S., & Scott, M. (2021). Conducting integrative reviews: A guide for novice nursing researchers. Journal of Research in Nursing, 26(5), 427–438. https://doi.org/10.1177/1744987121997907
Rigour in integrative reviews Whittemore, R. (2007). Rigour in integrative reviews. In C. Webb & B. Roe (Eds.), Reviewing Research Evidence for Nursing Practice (pp. 149–156). John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470692127.ch11

Scoping Reviews

Scoping reviews are evidence syntheses that are conducted systematically, but begin with a broader scope of question than traditional systematic reviews, allowing the research to 'map' the relevant literature on a given topic.

Scoping studies: Towards a methodological framework Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8 (1), 19-32. doi:10.1080/1364557032000119616 More information less... ABSTRACT: We distinguish between different types of scoping studies and indicate where these stand in relation to full systematic reviews. We outline a framework for conducting a scoping study based on our recent experiences of reviewing the literature on services for carers for people with mental health problems.
Scoping studies: Advancing the methodology Levac, D., Colquhoun, H., & O'Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science, 5 (1), 69. doi:10.1186/1748-5908-5-69 More information less... ABSTRACT: We build upon our experiences conducting three scoping studies using the Arksey and O'Malley methodology to propose recommendations that clarify and enhance each stage of the framework.
Methodology for JBI scoping reviews Peters, M. D. J., Godfrey, C. M., McInerney, P., Baldini Soares, C., Khalil, H., & Parker, D. (2015). The Joanna Briggs Institute reviewers’ manual: Methodology for JBI scoping reviews [PDF]. Retrieved from The Joanna Briggs Institute website: http://joannabriggs.org/assets/docs/sumari/Reviewers-Manual_Methodology-for-JBI-Scoping-Reviews_2015_v2.pdf More information less... ABSTRACT: Unlike other reviews that address relatively precise questions, such as a systematic review of the effectiveness of a particular intervention based on a precise set of outcomes, scoping reviews can be used to map the key concepts underpinning a research area as well as to clarify working definitions, and/or the conceptual boundaries of a topic. A scoping review may focus on one of these aims or all of them as a set.

Systematic vs. Scoping Reviews: What's the Difference?

YouTube Video 4 minutes, 45 seconds

Rapid Reviews

Rapid reviews are systematic reviews that are undertaken under a tighter timeframe than traditional systematic reviews.

Evidence summaries: The evolution of a rapid review approach Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: The evolution of a rapid review approach. Systematic Reviews, 1 (1), 10. doi:10.1186/2046-4053-1-10 More information less... ABSTRACT: Rapid reviews have emerged as a streamlined approach to synthesizing evidence - typically for informing emergent decisions faced by decision makers in health care settings. Although there is growing use of rapid review "methods," and proliferation of rapid review products, there is a dearth of published literature on rapid review methodology. This paper outlines our experience with rapidly producing, publishing and disseminating evidence summaries in the context of our Knowledge to Action (KTA) research program.
What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments Harker, J., & Kleijnen, J. (2012). What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments. International Journal of Evidence‐Based Healthcare, 10 (4), 397-410. doi:10.1111/j.1744-1609.2012.00290.x More information less... ABSTRACT: In recent years, there has been an emergence of "rapid reviews" within Health Technology Assessments; however, there is no known published guidance or agreed methodology within recognised systematic review or Health Technology Assessment guidelines. In order to answer the research question "What is a rapid review and is methodology consistent in rapid reviews of Health Technology Assessments?", a study was undertaken in a sample of rapid review Health Technology Assessments from the Health Technology Assessment database within the Cochrane Library and other specialised Health Technology Assessment databases to investigate similarities and/or differences in rapid review methodology utilised.
Rapid Review Guidebook Dobbins, M. (2017). Rapid review guidebook. Hamilton, ON: National Collaborating Centre for Methods and Tools.
NCCMT Summary and Tool for Dobbins' Rapid Review Guidebook National Collaborating Centre for Methods and Tools. (2017). Rapid review guidebook. Hamilton, ON: McMaster University. Retrieved from http://www.nccmt.ca/knowledge-repositories/search/308
<< Previous: Literature Reviews
Next: Conducting & Reporting Systematic Reviews >>
Last Updated: Jun 26, 2024 3:00 PM
URL: https://guides.nyu.edu/speech

Open access
Published: 02 July 2024

Unravelling the complexity of ventilator-associated pneumonia: a systematic methodological literature review of diagnostic criteria and definitions used in clinical research

Markus Fally 1 ,
Faiuna Haseeb 2 , 3 ,
Ahmed Kouta 2 , 3 ,
Jan Hansel 3 , 4 ,
Rebecca C. Robey 2 , 3 ,
Thomas Williams 5 ,
Tobias Welte 6 ,
Timothy Felton 2 , 3 , 5 &
Alexander G. Mathioudakis 2 , 3

Critical Care volume 28 , Article number: 214 ( 2024 ) Cite this article

62 Accesses

Metrics details

Ventilator-associated pneumonia (VAP) is a prevalent and grave hospital-acquired infection that affects mechanically ventilated patients. Diverse diagnostic criteria can significantly affect VAP research by complicating the identification and management of the condition, which may also impact clinical management.

We conducted this review to assess the diagnostic criteria and the definitions of the term “ventilator-associated” used in randomised controlled trials (RCTs) of VAP management.

Search methods

Based on the protocol (PROSPERO 2019 CRD42019147411), we conducted a systematic search on MEDLINE/PubMed and Cochrane CENTRAL for RCTs, published or registered between 2010 and 2024.

Selection criteria

We included completed and ongoing RCTs that assessed pharmacological or non-pharmacological interventions in adults with VAP.

Data collection and synthesis

Data were collected using a tested extraction sheet, as endorsed by the Cochrane Collaboration. After cross-checking, data were summarised in a narrative and tabular form.

In total, 7,173 records were identified through the literature search. Following the exclusion of records that did not meet the eligibility criteria, 119 studies were included. Diagnostic criteria were provided in 51.2% of studies, and the term “ventilator-associated” was defined in 52.1% of studies. The most frequently included diagnostic criteria were pulmonary infiltrates (96.7%), fever (86.9%), hypothermia (49.1%), sputum (70.5%), and hypoxia (32.8%). The different criteria were used in 38 combinations across studies. The term “ventilator-associated” was defined in nine different ways.

Conclusions

When provided, diagnostic criteria and definitions of VAP in RCTs display notable variability. Continuous efforts to harmonise VAP diagnostic criteria in future clinical trials are crucial to improve quality of care, enable accurate epidemiological assessments, and guide effective antimicrobial stewardship.

Ventilator-associated pneumonia (VAP) stands as the most prevalent and serious hospital-acquired infection observed in intensive care units [ 1 ]. VAP prolongs hospital stays, durations of mechanical ventilation, and is associated with considerable mortality and an increase in healthcare costs [ 2 , 3 ].

Diagnosing VAP can be challenging for clinicians as it shares clinical signs and symptoms with other forms of pneumonia as well as non-infectious conditions [ 4 ]. The most recent international clinical guidelines define VAP as the presence of respiratory infection signs combined with new radiographic infiltrates in a patient who has been ventilated for at least 48 h [ 5 , 6 ]. While the guidelines developed by ERS/ESICM/ESCMID/ALAT do not provide a detailed definition of signs of respiratory infection [ 5 ], the ATS/IDSA guidelines mention that clinical signs may include the new onset of fever, purulent sputum, leucocytosis, and decline in oxygenation [ 6 ]. However, the ATS/IDSA guideline panel also acknowledges that there is no gold standard for the diagnosis of VAP [ 6 ]. This lack of a standardised definition is further highlighted by the varying, surveillance-based definitions of VAP provided by the Centre for Disease Control (CDC) and the European Centre for Disease Control (ECDC) [ 7 , 8 ]. These definitions, focusing on a combination of clinical, radiological, and microbiological signs to identify cases of VAP, were established to standardise reporting and facilitate the monitoring of infections in healthcare settings. However, the criteria given by the CDC and ECDC may not always align with the diagnostic criteria used by clinicians to confirm or rule out the condition [ 9 , 10 , 11 ].

Variations in the eligibility criteria applied to VAP can have a significant impact on systematic reviews and meta-analyses that assess different interventions, primarily due to the potential lack of comparability among the studied populations [ 12 ]. Furthermore, the incidence of VAP may be underestimated when excessively strict diagnostic criteria are employed [ 13 , 14 ].

A recent systematic review conducted by Weiss et al. focused on inclusion and judgment criteria used in randomised controlled trials (RCTs) on nosocomial pneumonia and found considerable heterogeneity [ 15 ]. However, the authors only considered RCTs evaluating antimicrobial treatment as interventions, did not distinguish between hospital-acquired pneumonia (HAP) and VAP, and did not evaluate definitions of the term "ventilator-associated".

The objective of this systematic review was to provide a concise overview of the diagnostic criteria for VAP recently used in RCTs, as well as the definitions attributed to the term "ventilator-associated". Its findings will provide valuable insights to a forthcoming task force, which aims to establish a uniform definition and diagnostic criteria for VAP in clinical trials. The task force will be made up of representatives from prominent international societies with an interest in VAP, as well as patient partners with lived experience. The harmonisation of the diagnostic criteria for VAP in upcoming clinical research are vital for enhancing patient care, enabling accurate epidemiological studies, and guiding successful antimicrobial stewardship programs.

Protocol and registration

The protocol for this systematic review was registered in advance with the International Prospective Register of Systematic Reviews (PROSPERO 2019 CRD42019147411), encompassing a broad review focusing on pneumonia outcomes and diagnostic criteria in RCTs. Recognising the limitations of discussing all findings in one manuscript, we opted to produce several focused and comprehensive manuscripts, all employing the same fundamental methodology, as registered with PROSPERO. While a previous publication focused on outcomes reported in RCTs on pneumonia management [ 16 ], the current submission specifically addresses diagnostic criteria for VAP.

Eligibility criteria

We included RCTs that were registered, planned, and/or completed that: (1) enrolled adults with VAP; and (2) assessed the safety, efficacy and/or effectiveness of pharmacological or non-pharmacological interventions for treating VAP.

We have excluded systematic reviews, meta-analyses, narrative reviews, post hoc analyses from RCTs, observational studies, case reports, editorials, conference proceedings, and studies that do not exclusively focus on pneumonia (such as trials including patients with pneumonia alongside other diseases). Additionally, studies on pneumonia subtypes other than VAP, such as pneumonia without specifying a subtype, community-acquired pneumonia (CAP), healthcare-associated pneumonia (HCAP), and HAP, have also been excluded. To maintain focus and relevance, studies on Coronavirus Disease 2019 (COVID-19) were excluded from this systematic review, as the viral aetiology and distinct clinical management protocols differ significantly from the nature and treatment strategies of VAP. RCT protocols were only included if the results have not been previously published in another article included in this systematic review. Due to resource constraints and the lack of multilingual expertise within the review team, this systematic review was restricted to English-language RCTs.

Information sources and search

On 20 May 2024, we searched MEDLINE/PubMed, and the Cochrane Register of Controlled Trials (CENTRAL) for RCTs published between 1 January 2010 and 19 May 2024. We used electronic algorithms introducing a combination of controlled vocabulary and search terms as reported in the Appendix.

Study selection

Two reviewers (FH, MF) independently screened titles and abstracts to identify eligible studies using Rayyan [ 17 ]. In case of disagreement, a third reviewer was consulted (AGM). After immediate exclusion of duplicates using EndNote X9, four reviewers (AGM, FH, JH, MF) independently checked for eligibility at full-text level. The results of the selection process are reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 18 ].

Data collection process

We developed an extraction sheet as endorsed by the Cochrane Collaboration [ 19 ]. The extraction sheet was independently tested by three reviewers (AGM, FH, MF) on five randomly selected studies and adapted to ensure good inter-reviewer agreement. The extraction sheet contained the following elements: (1) study ID, name, reference and NCT number; (2) type of pneumonia: CAP, HCAP, HAP and/or VAP; (3) diagnostic criteria for pneumonia; (4) definition of setting; (5) study origin, design, populations, interventions, and outcomes.

Four reviewers (AGM, FH, JH, MF) extracted data from the eligible studies. Data were extracted sequentially from either a manuscript containing published results, a published protocol, or, upon obtaining a trial registration number from CENTRAL, from one of the designated trial registries, such as ClinicalTrials.gov, the Clinical Trials Registry India (CTRI), the Chinese Clinical Trial Registry (ChiCTR), the European Clinical Trials Database (EudraCT), the Iranian Registry of Clinical Trials (IRCT), the Japan Primary Registries Network (JPRN), and the Japanese University Hospital Medical Information Network Clinical Trials Registry (UMIN-CTR). Cross-checking of all extracted data was performed by a second reviewer (AGM, AK, MF, RR, TW). Disagreements regarding data collection were resolved by discussion between all reviewers.

Synthesis of results

The findings were consolidated through a combination of narrative and tabular formats. The presentation encompassed the quantitative representation of each diagnostic criterion in terms of numerical values and proportions. Additionally, we provide an analysis of the various combinations of diagnostic criteria employed in RCTs in a sunburst diagram and a tabular format, along with an examination of the definitions attributed to the term "ventilator-associated".

Risk of bias

The main goal of this systematic review was to explore the diagnostic criteria used in clinical trials for diagnosing VAP. It covered trials with published protocols and/or results, as well as those only registered in a trial database. The varying levels and gaps in the information provided by the various sources made it difficult to conduct a reliable and meaningful risk of bias assessment for all included studies. However, for RCTs with published data, risk of bias was evaluated by four reviewers (AGM, JH, MF, RR) using the Risk of Bias in Randomized Trials 2 tool (RoB-2 tool), as endorsed by the Cochrane Collaboration [ 20 ].

Study selection and characteristics

A total of 7173 records were identified through the databases MEDLINE and CENTRAL, as illustrated in Fig. 1 . Following the removal of duplicate entries, a screening process involving the evaluation of titles and abstracts was conducted on 5652 records. Among these, 650 records were deemed potentially eligible for inclusion. Ultimately, our review included 119 studies that specifically focused on VAP (Table S1 in the Appendix, the full dataset is available online [ 21 ]).

PRISMA flowchart showing study selection

The total number of patients in the 119 identified studies was 21,289. Among these studies, 83 focused exclusively on VAP, while the remaining studies encompassed various subtypes of pneumonia in addition to VAP (see Table 1 ). The majority of these studies were registered, and their protocols were accessible either through publication in a journal article or on a clinical trial platform. Results were accessible in 56.3% of cases, while both results and the protocol were accessible in 36.9% of cases. In 40.3% of the included studies, data could only be obtained from a trial registry platform, with ClinicalTrials.gov being the primary platform in 36 out of 48 cases, and ChiCTR (n = 2), CTRI (n = 3), EudraCT (n = 3), IRCT (n = 2), JPRN (n = 1) and UMIN-CTR (n = 1) in the remaining cases.

Diagnostic criteria were provided in 51.2% and the term “ventilator-associated” was defined in 52.1% of the studies, respectively. Of the 20 studies (16.8%) that referred to previously published diagnostic criteria, 13 cited the Clinical Pulmonary Infection Score (CPIS) [ 22 ], while the remaining referred to national and international guidelines.

We evaluated the risk of bias in 67 studies with published results using the RoB-2 tool. The overall assessment showed that 25% of the studies were at high risk of bias, 30% were at low risk of bias, and the remaining 45% had some concerns about potential bias. These results indicate variability in the methodological quality of the studies included in the review. The overall risk of bias and the detailed results of our assessments for the 67 studies are displayed in the Appendix (Figures SF1-SF2).

Diagnostic criteria for VAP

Pulmonary infiltrates.

Of the 61 studies on VAP that provided diagnostic criteria, 59 (96.7%) included the radiological evidence of a new or progressive pulmonary infiltrate.

Clinical signs and symptoms

The most frequently included clinical signs and symptoms were fever (86.9%), hypothermia (49.1%), sputum (70.5%), and hypoxia (32.8%). Different cut-off values were employed to define fever and hypothermia, as indicated in Table 2 . The majority of studies, accounting for 45.2%, utilised a cut-off of > 38 degrees Celsius (°C) to define fever, while 13.2% of studies used a cut-off of ≥ 38°C. In the case of hypothermia, the most commonly employed cut-off value was < 35°C, which was utilised in 43.3% of studies that included hypothermia as a criterion. Only a minority of studies provided information on the site of temperature measurement. Oral measurement was the most frequently employed method, followed by axillary and core temperature measurements (further details are displayed in Table S2 in the Appendix).

Biochemistry criteria

Fifty-four studies (88.5%) incorporated white blood count abnormalities as part of their diagnostic criteria for VAP. Conversely, only one study included an elevation of procalcitonin (PCT) as a diagnostic factor, and none of the identified studies included C-reactive protein (CRP). The specific thresholds for leucocytosis and leucopoenia varied across studies, with leucocyte counts ranging from greater than 10,000/mm3 to greater than 12,000/mm3 for leucocytosis, and less than 3,500/mm3 to less than 4,500/mm3 for leucopoenia (Table 3 ).

Combinations of diagnostic criteria

All definitions of pneumonia were composite in nature and required the fulfilment of a minimum number of predetermined criteria for the diagnosis to be established. In 90.2% of the studies the presence of a new pulmonary infiltrate was a mandatory criterion. Two studies did not include an infiltrate as criterion, whereas the remaining studies (n = 4) included the presence of an infiltrate in their criteria, it was, however, not required for a diagnosis.

The most commonly employed set of diagnostic criteria (18/61, 29.5%) consisted of a pulmonary infiltrate along with two or more additional criteria. However, these additional criteria varied across studies (Fig. 2 ). A quarter (17/61) of the included studies that provided diagnostic criteria required the fulfilment of all individual criteria for diagnosis, including an infiltrate. An infiltrate and one or more additional criteria were used to establish a diagnosis of VAP in 14.8% of studies (9/61). A total of 38 different combinations of diagnostic criteria for VAP were used in the 61 identified studies. A full set of these criteria is displayed in Table S3 in the Appendix.

The different combinations of diagnostic criteria used in VAP RCTs. CXR radiological evidence of a new infiltrate; T temperature criterion; WBC white blood count criterion; dys/tach dyspnoea and/or tachypnoea; O2 hypoxia; auscultation auscultation abnormalities

Definition of “ventilator-associated”

We noted that 52.1% of included studies incorporated a specific definition of the term “ventilator-associated” (Table 4 ). A total of nine distinct definitions were identified across 62 RCTs. The definition most commonly used was “onset after > 48 h of mechanical ventilation” (82.3%). Other definitions employed varying time thresholds, ranging from 24 h to seven days. Additionally, certain studies introduced supplementary criteria to further delineate the concept of “ventilator-associated”, such as administration of antibiotics prior to mechanical ventilation, duration of hospitalisation, or the timing of extubation.

Summary of evidence

This systematic review provides a concise overview of the diagnostic criteria for VAP used in RCTs and the definitions attributed to the term “ventilator-associated”. A total of 119 studies on VAP, published or registered between 2010 and 2024, were included, spanning a total of 21,289 patients. The majority of studies focused exclusively on VAP, while some also included other subtypes of pneumonia alongside VAP. Diagnostic criteria were provided in only 51.2% of the studies, and the term “ventilator-associated” was defined in only 52.1% of the studies. The most commonly utilised definition for “ventilator-associated” was “onset after > 48 h of mechanical ventilation”, used by 82.3% of studies providing a definition.

In clinical practice, the diagnosis of VAP is often based on a combination of clinical signs, laboratory results, and imaging findings, yet these are not without their limitations [ 8 ]. Our systematic review revealed considerable heterogeneity among diagnostic criteria for VAP in recent RCTs. Various combinations of specific criteria were employed to define VAP, leading to significant variability. Moreover, commonly used criteria were defined in different ways, with variations observed in the thresholds set for fever/hypothermia, as well as leucocytosis/leucopoenia.

Several criteria that were used in the studies included in our review have been shown to be insufficient for confirming a diagnosis of VAP. One of the most important criteria, included in the majority of reviewed RCTs, a new or progressive pulmonary infiltrate, has previously been reported to be of limited diagnostic value due to a lack of specificity [ 14 ]. Additionally, criteria like fever/hypothermia and the measurement of biomarkers such as leukocytes, CRP, and PCT may not be effective in diagnosing or excluding VAP in various clinical settings [ 4 , 23 , 24 ]. Despite this, CRP is widely used and has demonstrated some clinical value in predicting VAP [ 25 ]. It is, therefore, surprising that none of the RCTs included in our review employed CRP as a diagnostic criterion.

Overall, the findings of our systematic review underline the diverse nature of VAP, with different diagnostic criteria increasing the risk of both over- and underdiagnosis of VAP [ 14 , 26 ]. There have been attempts to diagnose VAP more objectively, one of these being the development of the CPIS in 1991, a six-component score that 10.9% of studies included in our review referred to [ 27 ]. This score includes different cut-offs for body temperature, leucocyte counts, tracheal secretion appearances, oxygenation levels and radiographical changes to estimate the risk for VAP. However, the CPIS has been shown not to be superior to other diagnostic criteria, and, therefore, its application remains controversial [ 8 , 11 , 22 , 28 ]. Other commonly applied criteria, such as the surveillance-based criteria by the ECDC and CDC, did not seem to be accurate enough to detect true cases of VAP either [ 9 , 10 , 11 ]. Furthermore, there is limited agreement between the two surveillance-based criteria, which has previously resulted in different estimates of VAP events [ 29 ].

In lieu of definitive diagnostic scores or sets of diagnostic criteria to detect all true cases of VAP, the findings of our systematic review indicate the need for more homogeneous diagnostic criteria in future RCTs, to assure their comparability. Currently, international guidelines avoid providing clear diagnostic criteria for VAP [ 5 , 6 ]. Given the significance of establishing strong consensus definitions for high-risk conditions like VAP, it is essential to emphasise even further that a uniform definition is crucial not only for advancing therapeutic research but also, and perhaps more importantly, for refining diagnostic methods. Together with core outcome sets, these definitions can help to improve the likelihood of attaining robust and reliable findings in forthcoming systematic reviews and meta-analyses [ 16 , 30 ].

Strengths and limitations

We used a comprehensive search strategy which included multiple databases and a wide range of search terms, ensuring broad identification of all potentially relevant trials. Additionally, the inclusion criteria were clearly defined, and the study selection process was conducted independently by multiple reviewers to minimise bias. The extraction sheet used for data collection was tested for inter-reviewer agreement and adapted accordingly. Another strength is the open availability of the complete dataset, maximising the transparency and reproducibility of our findings.

However, the following limitations need to be acknowledged. Firstly, the review only included RCTs conducted in English, which may have introduced language bias. This approach was adopted to ensure feasible and reliable data analysis within the scope of the resources available.

Additionally, the exclusion of studies focusing on pneumonia subtypes other than VAP may limit the generalisability of our findings. Furthermore, the lack of diagnostic criteria and definitions in a significant proportion of included studies suggests a potential reporting bias. This might be reinforced by the fact that 40.3% of data were received from trial registry platforms. Compared to final manuscript publications, reporting of eligibility criteria is often incomplete on registry platforms, therefore this must be highlighted as a limitation [ 31 ].

This systematic review provides an overview of diagnostic criteria for VAP used in RCTs and the definitions attributed to the term “ventilator-associated”. Our findings highlight the heterogeneity and lack of standardisation in commonly used diagnostic criteria, as well as the variability in definitions of "ventilator-associated" across clinical trials. We emphasise the need for a uniform definition of VAP to enable better comparability between studies and interventions. The results of this review will inform the work of an upcoming task force aimed at establishing such standardised criteria.

Availability of data and materials

Raw data are accessible via the Open Science Framework (OSF) at osf.io/v3 × 42. This link is referenced in our manuscript (Ref. 21).

Torres A, Cilloniz C, Niederman MS, et al. Pneumonia. Nat Rev Dis Primers. 2021;7(1):28. https://doi.org/10.1038/s41572-021-00259-0 .

Article Google Scholar

Muscedere JG, Day A, Heyland DK. Mortality, attributable mortality, and clinical events as end points for clinical trials of ventilator-associated pneumonia and hospital-acquired pneumonia. Clin Infect Dis. 2010;51:S120–5. https://doi.org/10.1086/653060 .

Article PubMed Google Scholar

Melsen WG, Rovers MM, Groenwold RH, et al. Attributable mortality of ventilator-associated pneumonia: a meta-analysis of individual patient data from randomised prevention studies. Lancet Infect Dis. 2013;13:665–71. https://doi.org/10.1016/S1473-3099(13)70081-1 .

Alagna L, Palomba E, Chatenoud L, et al. Comparison of multiple definitions for ventilator-associated pneumonia in patients requiring mechanical ventilation for non-pulmonary conditions: preliminary data from PULMIVAP, an Italian multi-centre cohort study. J Hosp Infect. 2023;140:90–5. https://doi.org/10.1016/j.jhin.2023.07.023 .

Article CAS PubMed Google Scholar

Torres A, Niederman MS, Chastre J, et al. International ERS/ESICM/ESCMID/ALAT guidelines for the management of hospital-acquired pneumonia and ventilator-associated pneumonia. Eur Respir J. 2017;50:1700582. https://doi.org/10.1183/13993003.00582-2017 .

Kalil AC, Metersky ML, Klompas M, et al. Management of Adults With Hospital-acquired and Ventilator-associated Pneumonia: 2016 Clinical Practice Guidelines by the Infectious Diseases Society of America and the American Thoracic Society. Clin Infect Dis. 2016;63:e61–111. https://doi.org/10.1093/cid/ciw353 .

Article PubMed PubMed Central Google Scholar

Plachouras D, Lepape A, Suetens C. ECDC definitions and methods for the surveillance of healthcare-associated infections in intensive care units. Intensive Care Med. 2018;44:2216–8. https://doi.org/10.1007/s00134-018-5113-0 .

Nair GB, Niederman MS. Ventilator-associated pneumonia: present understanding and ongoing debates. Intensive Care Med. 2015;41:34–48. https://doi.org/10.1007/s00134-014-3564-5 .

Ramírez-Estrada S, Lagunes L, Peña-López Y, et al. Assessing predictive accuracy for outcomes of ventilator-associated events in an international cohort: the EUVAE study. Intensive Care Med. 2018;44:1212–20. https://doi.org/10.1007/s00134-018-5269-7 .

Waltrick R, Possamai DS, de Aguiar FP, et al. Comparison between a clinical diagnosis method and the surveillance technique of the Center for Disease Control and Prevention for identification of mechanical ventilator-associated pneumonia. Rev Bras Ter Intensiva. 2015;27:260. https://doi.org/10.5935/0103-507X.20150047 .

Rahimibashar F, Miller AC, Yaghoobi MH, Vahedian-Azimi A. A comparison of diagnostic algorithms and clinical parameters to diagnose ventilator-associated pneumonia: a prospective observational study. BMC Pulm Med. 2021;21:161. https://doi.org/10.1186/s12890-021-01527-1 .

Article CAS PubMed PubMed Central Google Scholar

Malmivaara A. Methodological considerations of the GRADE method. Ann Med. 2015;47:1–5.

Al-Omari B, McMeekin P, Allen AJ, et al. Systematic review of studies investigating ventilator associated pneumonia diagnostics in intensive care. BMC Pulm Med. 2021;21:196. https://doi.org/10.1186/s12890-021-01560-0 .

Fernando SM, Tran A, Cheng W, et al. Diagnosis of ventilator-associated pneumonia in critically ill adult patients—a systematic review and meta-analysis. Intensive Care Med. 2020;46:1170–9. https://doi.org/10.1007/s00134-020-06036-z .

Weiss E, Essaied W, Adrie C, et al. Treatment of severe hospital-acquired and ventilator-associated pneumonia: a systematic review of inclusion and judgment criteria used in randomized controlled trials. Crit Care. 2017. https://doi.org/10.1186/s13054-017-1755-5 .

Mathioudakis AG, Fally M, Hansel J, et al. Clinical trials of pneumonia management assess heterogeneous outcomes and measurement instruments. J Clin Epidemiol. 2023;164:88–95. https://doi.org/10.1016/j.jclinepi.2023.10.011 .

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5:210. https://doi.org/10.1186/s13643-016-0384-4 .

Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Ann Intern Med. 2009;151:264–9.

Li T, Higgins J, Deeks J (editors) (2019) Chapter 5: Collecting data | Cochrane Training. In: Cochrane Handbook for Systematic Reviews of Interventions version 6.0. https://training.cochrane.org/handbook/current/chapter-05 . Accessed 21 Jul 2020

Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366: l4898. https://doi.org/10.1136/bmj.l4898 .

ERS COS Pneumonia dataset. http://osf.io/v3x42

Zilberberg MD, Shorr AF. Ventilator-associated pneumonia: the clinical pulmonary infection score as a surrogate for diagnostics and outcome. Clin Infect Dis. 2010;51:S131–5. https://doi.org/10.1086/653062 .

Huang H-B, Peng J-M, Weng L, et al. Procalcitonin-guided antibiotic therapy in intensive care unit patients: a systematic review and meta-analysis. Ann Intensive Care. 2017;7:114. https://doi.org/10.1186/s13613-017-0338-6 .

Palazzo SJ, Simpson T, Schnapp L. Biomarkers for ventilator-associated pneumonia: review of the literature. Heart Lung. 2011;40:293–8. https://doi.org/10.1016/j.hrtlng.2010.11.003 .

Póvoa P, Martin-Loeches I, Ramirez P, et al. Biomarker kinetics in the prediction of VAP diagnosis: results from the BioVAP study. Ann Intensive Care. 2016;6:32. https://doi.org/10.1186/s13613-016-0134-8 .

Johnstone J, Muscedere J, Dionne J, et al. Definitions, rates and associated mortality of ICU-acquired pneumonia: a multicenter cohort study. J Crit Care. 2023;75:154284. https://doi.org/10.1016/j.jcrc.2023.154284 .

Pugin J, Auckenthaler R, Mili N, et al. Diagnosis of ventilator-associated pneumonia by bacteriologic analysis of bronchoscopic and nonbronchoscopic “blind” bronchoalveolar lavage fluid. Am Rev Respir Dis. 1991;143:1121–9. https://doi.org/10.1164/ajrccm/143.5_Pt_1.1121 .

Fàbregas N, Ewig S, Torres A, et al. Clinical diagnosis of ventilator associated pneumonia revisited: comparative validation using immediate post-mortem lung biopsies. Thorax. 1999;54:867–73.

Craven TH, Wojcik G, McCoubrey J, et al. Lack of concordance between ECDC and CDC systems for surveillance of ventilator associated pneumonia. Intensive Care Med. 2018;44:265–6. https://doi.org/10.1007/s00134-017-4993-8 .

Mathioudakis AG, Khaleva E, Fally M, et al. Core outcome sets, developed collaboratively with patients, can improve the relevance and comparability of clinical trials. Eur Respir J. 2023;61:2202107. https://doi.org/10.1183/13993003.02107-2022 .

Speich B, Gloy VL, Klatte K, et al. Reliability of trial information across registries for trials with multiple registrations. JAMA Netw Open. 2021;4:e2128898. https://doi.org/10.1001/jamanetworkopen.2021.28898 .

Download references

Acknowledgements

We would like to acknowledge and honour the contributions of Prof. Tobias Welte, who was a vital member of our research team and co-author of this manuscript. Prof. Welte passed away after the initial submission of this work but before its final acceptance. His insights and expertise were invaluable to the development of this research, and he remains deeply missed by the team. We dedicate this work to his memory.

Open access funding provided by Copenhagen University This study was partly supported by the NIHR Manchester Biomedical Research Centre (BRC, NIHR203308) as well as the Capital Region of Denmark (Region Hovedstaden). The funders had no role in study design, data collection or analysis, decision to publish, nor preparation of the manuscript. Dr Jan Hansel was supported by an NIHR Academic Clinical Fellowship in Intensive Care Medicine. Dr Rebecca Robey was supported by an NIHR Academic Clinical Fellowship in Respiratory Medicine. Dr Alexander G. Mathioudakis was supported by an NIHR Clinical Lectureship in Respiratory Medicine. All authors have completed a ICMJE uniform disclosure form detailing any conflicts of interest outside the submitted work that they may have. None of the authors have conflicts directly related to this work.

Author information

Authors and affiliations.

Department of Respiratory Medicine and Infectious Diseases, Copenhagen University Hospital – Bispebjerg and Frederiksberg, Copenhagen, Denmark

Markus Fally

North West Lung Centre, Wythenshawe Hospital, Manchester University NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK

Faiuna Haseeb, Ahmed Kouta, Rebecca C. Robey, Timothy Felton & Alexander G. Mathioudakis

Division of Immunology, Immunity to Infection and Respiratory Medicine, School of Biological Sciences, The University of Manchester, Manchester, UK

Faiuna Haseeb, Ahmed Kouta, Jan Hansel, Rebecca C. Robey, Timothy Felton & Alexander G. Mathioudakis

North West School of Intensive Care Medicine, Health Education England North West, Manchester, UK

Acute Intensive Care Unit, Wythenshawe Hospital, Manchester University NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK

Thomas Williams & Timothy Felton

Department of Respiratory Medicine and German Centre of Lung Research (DZL), Hannover Medical School, Hannover, Germany

Tobias Welte

You can also search for this author in PubMed Google Scholar

Contributions

MF: conceptualisation, methodology, software, formal analysis, investigation, data curation, writing—original draft, visualisation, project administration. FH: conceptualisation, investigation, data curation, validation, writing—review and editing. AK, JH, RCR and TWI: data curation, validation, writing—review and editing. TWE: conceptualisation, investigation, methodology, resources, validation, writing—review and editing. TF: conceptualisation, investigation, methodology, resources, validation, writing—review and editing, supervision. AGM: conceptualisation, investigation, methodology, software, resources, validation, writing—review and editing, project administration, supervision, funding acquisition, project administration.

Corresponding author

Correspondence to Markus Fally .

Ethics declarations

Ethics approval and consent to participate.

Not applicable, as this was a methodological systematic review without patient involvement/participation.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file1 (docx 807 kb), search strategy, medline/pubmed.

#1: pneumonia [mh]

#2: bronchopneumonia [mh]

#3: pleuropneumonia [mh]

#4: Healthcare-Associated Pneumonia [mh]

#5: Ventilator-Associated Pneumonia [mh]

#6: pneumonia [ti]

#7: pneumonia* [ti]

#8: bronchopneumonia [ti]

#9: pleuropneumonia [ti]

#10: #1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7 OR #8 OR #9

#11: randomized controlled trial [pt]

#12: controlled clinical trial [pt]

#13: randomized [tiab]

#14: placebo [tiab]

#15: clinical trials as topic [mesh: noexp]

#16: randomly [tiab]

#17: trial [ti]

#18: #11 OR #12 OR #13 OR #14 OR #15 OR #16 OR #17

#19: animals [mh] NOT humans [mh]

#20: children [mh] NOT adults [mh]

#21: COVID-19 [mh] or (covid[ti]) or (coronavirus [ti]) or (sars-cov-2[ti]) or (covid-19[ti]) or (pandemic[ti])

#22: #19 OR #20 OR #21

#23: #18 NOT #22

#24: #10 AND #23

#25: Publication date: 2010 –2024

Cochrane library

#1: MeSH descriptor: [Pneumonia] explode all trees

#2: pneumonia*:ti

#3: #1 or #2

#4: MeSH descriptor: [COVID-19] explode all trees

#5: COVID-19:ti

#6: covid:ti

#7: coronavirus:ti

#8: sars-cov-2:ti

#9: #4 or #5 or #6 or #7 or #8

#10: #3 not #9

#11: Limit: Publication Date from 2010–2024

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Fally, M., Haseeb, F., Kouta, A. et al. Unravelling the complexity of ventilator-associated pneumonia: a systematic methodological literature review of diagnostic criteria and definitions used in clinical research. Crit Care 28 , 214 (2024). https://doi.org/10.1186/s13054-024-04991-3

Download citation

Received : 28 February 2024

Accepted : 15 June 2024

Published : 02 July 2024

DOI : https://doi.org/10.1186/s13054-024-04991-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Diagnostic criteria
Inclusion criteria
Clinical trial
Ventilator-associated pneumonia
Systematic review

Critical Care

ISSN: 1364-8535

Submission enquiries: [email protected]

Search Menu
Sign in through your institution
Advance Articles
Editor's Choice
Collections
Supplements
InSight Papers
BSR Registers Papers
Virtual Roundtables
Author Guidelines
Submission Site
Open Access Options
Self-Archiving Policy
About Rheumatology
About the British Society for Rheumatology
Editorial Board
Advertising and Corporate Services
Journals Career Network
Dispatch Dates
Terms and Conditions
Journals on Oxford Academic
Books on Oxford Academic

Article Contents

Introduction, supplementary material, data availability, acknowledgements.

< Previous

Comparative efficacy and safety of bimekizumab in psoriatic arthritis: a systematic literature review and network meta-analysis

Article contents
Figures & tables
Supplementary Data

Philip J Mease, Dafna D Gladman, Joseph F Merola, Peter Nash, Stacy Grieve, Victor Laliman-Khara, Damon Willems, Vanessa Taieb, Adam R Prickett, Laura C Coates, Comparative efficacy and safety of bimekizumab in psoriatic arthritis: a systematic literature review and network meta-analysis, Rheumatology , Volume 63, Issue 7, July 2024, Pages 1779–1789, https://doi.org/10.1093/rheumatology/kead705

Permissions Icon Permissions

To understand the relative efficacy and safety of bimekizumab, a selective inhibitor of IL-17F in addition to IL-17A, vs other biologic and targeted synthetic DMARDs (b/tsDMARDs) for PsA using network meta-analysis (NMA).

A systematic literature review (most recent update conducted on 1 January 2023) identified randomized controlled trials (RCTs) of b/tsDMARDs in PsA. Bayesian NMAs were conducted for efficacy outcomes at Weeks 12–24 for b/tsDMARD-naïve and TNF inhibitor (TNFi)-experienced patients. Safety at Weeks 12–24 was analysed in a mixed population. Odds ratios (ORs) and differences of mean change with the associated 95% credible interval (CrI) were calculated for the best-fitting models, and the surface under the cumulative ranking curve (SUCRA) values were calculated to determine relative rank.

The NMA included 41 RCTs for 22 b/tsDMARDs. For minimal disease activity (MDA), bimekizumab ranked 1st in b/tsDMARD-naïve patients and 2nd in TNFi-experienced patients. In b/tsDMARD-naïve patients, bimekizumab ranked 6th, 5th and 3rd for ACR response ACR20/50/70, respectively. In TNFi-experienced patients, bimekizumab ranked 1st, 2nd and 1st for ACR20/50/70, respectively. For Psoriasis Area and Severity Index 90/100, bimekizumab ranked 2nd and 1st in b/tsDMARD-naïve patients, respectively, and 1st and 2nd in TNFi-experienced patients, respectively. Bimekizumab was comparable to b/tsDMARDs for serious adverse events.

Bimekizumab ranked favourably among b/tsDMARDs for efficacy on joint, skin and MDA outcomes, and showed comparable safety, suggesting it may be a beneficial treatment option for patients with PsA.

For joint efficacy, bimekizumab ranked highly among approved biologic/targeted synthetic DMARDs (b/tsDMARDs).

Bimekizumab provides better skin efficacy (Psoriasis Area and Severity Index, PASI100 and PASI90) than many other available treatments in PsA.

For minimal disease activity, bimekizumab ranked highest of all available b/tsDMARDs in b/tsDMARD-naïve and TNF inhibitor–experienced patients.

PsA is a chronic, systemic, inflammatory disease in which patients experience a high burden of illness [ 1–3 ]. PsA has multiple articular and extra-articular disease manifestations including peripheral arthritis, axial disease, enthesitis, dactylitis, skin psoriasis (PSO) and psoriatic nail disease [ 4 , 5 ]. Patients with PsA can also suffer from related inflammatory conditions, uveitis and IBD [ 4 , 5 ]. Approximately one fifth of all PSO patients, increasing to one quarter of patients with moderate to severe PSO, will develop PsA over time [ 6 , 7 ].

The goal of treatment is to control inflammation and prevent structural damage to minimize disease burden, normalize function and social participation, and maximize the quality of life of patients [ 1 , 4 ]. As PsA is a heterogeneous disease, the choice of treatment is guided by individual patient characteristics, efficacy against the broad spectrum of skin and joint symptoms, and varying contraindications to treatments [ 1 , 4 ]. There are a number of current treatments classed as conventional DMARDs such as MTX, SSZ, LEF; biologic (b) DMARDs such as TNF inhibitors (TNFi), IL inhibitors and cytotoxic T lymphocyte antigen 4 (CTLA4)-immunoglobulin; and targeted synthetic (ts) DMARDs which include phosphodiesterase-4 (PDE4) and Janus kinase (JAK) inhibitors [ 1 , 8 ].

Despite the number of available treatment options, the majority of patients with PsA report that they do not achieve remission and additional therapeutic options are needed [ 9 , 10 ]. Thus, the treatment landscape for PsA continues to evolve and treatment decisions increase in complexity, especially as direct comparative data are limited [ 2 ].

Bimekizumab is a monoclonal IgG1 antibody that selectively inhibits IL-17F in addition to IL-17A, which is approved for the treatment of adults with active PsA in Europe [ 11 , 12 ]. Both IL-17A and IL-17F are pro-inflammatory cytokines implicated in PsA [ 11 , 13 ]. IL-17F is structurally similar to IL-17A and expressed by the same immune cells; however, the mechanisms that regulate expression and kinetics differ [ 13 , 14 ]. IL-17A and IL-17F are expressed as homodimers and as IL-17A–IL-17F heterodimers that bind to and signal via the same IL-17 receptor A/C complex [ 13 , 15 ].

In vitro studies have demonstrated that the dual inhibition of both IL-17A and IL-17F with bimekizumab was more effective at suppressing PsA inflammatory genes and T cell and neutrophil migration, and periosteal new bone formation, than blocking IL-17A alone [ 11 , 14 , 16 , 17 ]. Furthermore, IL-17A and IL-17F protein levels are elevated in psoriatic lesions and the superiority of bimekizumab 320 mg every 4 weeks (Q4W) or every 8 weeks (Q8W) over the IL-17A inhibitor, secukinumab, in complete clearance of psoriatic skin was demonstrated in a head-to-head trial in PSO [ 16 , 18 ]. Collectively, this evidence suggests that neutralizing both IL-17F and IL-17A may provide more potent abrogation of IL-17-mediated inflammation than IL-17A alone.

Bimekizumab 160 mg Q4W demonstrated significant improvements in efficacy outcomes compared with placebo, and an acceptable safety profile in adults with PsA in the phase 3 RCTs BE OPTIMAL (NCT03895203) (b/tsDMARD-naïve patients) and BE COMPLETE (NCT03896581) (TNFi inadequate responders) [ 19 , 20 ].

The objective of this study was to establish the comparative efficacy and safety of bimekizumab 160 mg Q4W vs other available PsA treatments, using network meta-analysis (NMA).

Search strategy

A systematic literature review (SLR) was conducted according to the Preferred Reporting Items for Systematic Reviews (PRISMA) guidelines [ 21 ] and adhered to the principles outlined in the Cochrane Handbook for Systematic Reviews of Interventions, Centre for Reviews and Dissemination’s Guidance for Undertaking Reviews in Healthcare, and Methods for the Development of National Institute of Health and Care Excellence (NICE) Public Health Guidance [ 22–24 ]. The SLR of English-language publications was originally conducted on 3 December 2015, with updates on 7 January 2020, 2 May 2022 and 1 January 2023 in Medical Literature Analysis and Retrieval System Online (MEDLINE ® ), Excerpta Medica Database (Embase ® ) and the Cochrane Central Register of Controlled Trials (CENTRAL) for literature published from January 1991 onward using the Ovid platform. Additionally, bibliographies of SLRs and meta-analyses identified through database searches were reviewed to ensure any publications not identified in the initial search were included in this SLR. Key clinical conference proceedings not indexed in Ovid (from October 2019 to current) and ClinicalTrials.gov were also manually searched. The search strategy is presented in Supplementary Table S1 (available at Rheumatology online).

Study inclusion

Identified records were screened independently and in duplicate by two reviewers and any discrepancies were reconciled via discussion or a third reviewer. The SLR inclusion criteria were defined by the Patient populations, Interventions, Comparators, Outcome measures, and Study designs (PICOS) Statement ( Supplementary Table S2 , available at Rheumatology online). The SLR included published studies assessing approved therapies for the treatment of PsA. Collected data included study and patient population characteristics, interventions, comparators, and reported clinical and patient-reported outcomes relevant to PsA. For efficacy outcomes, pre-crossover data were extracted in studies where crossover occurred. All publications included in the analysis were evaluated according to the Cochrane risk-of-bias tool for randomized trials as described in the Cochrane Handbook [ 25 ].

Network meta-analysis methods

NMA is the quantitative assessment of relative treatment effects and associated uncertainty of two or more interventions [ 26 , 27 ]. It is used frequently in health technology assessment, guideline development and to inform treatment decision making in clinical practice [ 26 ].

Bimekizumab 160 mg Q4W was compared with current b/tsDMARDs at regulatory-approved doses ( Table 1 ) by NMA. All comparators were selected on the basis they were relevant to clinical practice, i.e. recommended by key clinical guidelines, licensed by key regulatory bodies and/or routinely used.

NMA intervention and comparators

Therapeutic class .	Drug dose and frequency of administration .
Intervention
IL-17A/17Fi	Bimekizumab 160 mg Q4W
Comparators
IL-17Ai	Secukinumab 150 mg with or without loading dose Q4W or 300 mg Q4W, ixekizumab 80 mg Q4W
IL-23i	Guselkumab 100 mg every Q4W or Q8W, risankizumab 150 mg Q4W
IL-12/23i	Ustekinumab 45 mg or 90 mg Q12W
TNFi	Adalimumab 40 mg Q2W, certolizumab pegol 200 mg Q2W or 400 mg Q4W pooled, etanercept 25 mg twice a week, golimumab 50 mg s.c. Q4W or 2 mg/kg i.v. Q8W, infliximab 5 mg/kg on weeks 0, 2, 6, 14, 22
CTLA4-Ig	Abatacept 150 mg Q1W
JAKi	Tofacitinib 5 mg BID, upadacitinib 15 mg QD
PDE-4i	Apremilast 30 mg BID
Other	Placebo

Therapeutic class .	Drug dose and frequency of administration .
Intervention
IL-17A/17Fi	Bimekizumab 160 mg Q4W
Comparators
IL-17Ai	Secukinumab 150 mg with or without loading dose Q4W or 300 mg Q4W, ixekizumab 80 mg Q4W
IL-23i	Guselkumab 100 mg every Q4W or Q8W, risankizumab 150 mg Q4W
IL-12/23i	Ustekinumab 45 mg or 90 mg Q12W
TNFi	Adalimumab 40 mg Q2W, certolizumab pegol 200 mg Q2W or 400 mg Q4W pooled, etanercept 25 mg twice a week, golimumab 50 mg s.c. Q4W or 2 mg/kg i.v. Q8W, infliximab 5 mg/kg on weeks 0, 2, 6, 14, 22
CTLA4-Ig	Abatacept 150 mg Q1W
JAKi	Tofacitinib 5 mg BID, upadacitinib 15 mg QD
PDE-4i	Apremilast 30 mg BID
Other	Placebo

See Supplementary Table S4 , available at Rheumatology online for additional dosing schedules used in included studies. BID: twice daily; CTLA4-Ig: cytotoxic T lymphocyte antigen 4-immunoglobulin; IL-17A/17Fi: IL-17A/17F inhibitor; IL-17Ai: IL-17A inhibitor; IL-12/23i: IL-12/23 inhibitor; IL-23i: IL-23 inhibitor; JAKi: Janus kinase inhibitor; NMA: network meta-analysis; PDE-4i: phosphodiesterase-4 inhibitor; Q1W: once weekly; Q2W: every 2 weeks; Q4W: every 4 weeks; Q8W: every 8 weeks; Q12W: every 12 weeks; QD: once daily; TNFi: TNF inhibitor.

Two sets of primary analyses were conducted, one for a b/tsDMARD-naïve PsA population and one for a TNFi-experienced PsA population. Prior treatment with TNFis has been shown to impact the response to subsequent bDMARD treatments [ 28 ]. In addition, most trials involving b/tsDMARDs for the treatment of PsA (including bimekizumab) report separate data on both b/tsDMARD-naïve and TNFi-experienced subgroups, making NMA in each of these patient populations feasible.

For each population the following outcomes were analysed: American College of Rheumatology response (ACR20/50/70), Psoriasis Area and Severity Index (PASI90/100), and minimal disease activity (MDA). The analysis of serious adverse events (SAE) was conducted using a mixed population (i.e. b/tsDMARD-naïve, TNFi-experienced and mixed population data all were included) as patients’ previous TNFI exposure was not anticipated to impact safety outcomes following discussions with clinicians. The NMA included studies for which data were available at week 16, if 16-week data were not available (or earlier crossover occurred), data available at weeks 12, 14 or 24 were included. Pre-crossover data were included in the analyses for efficacy outcomes to avoid intercurrent events.

Heterogeneity between studies for age, sex, ethnicity, mean time since diagnosis, concomitant MTX, NSAIDs or steroid use was assessed using Grubb’s test, also called the extreme Studentized deviate method, to identify outlier studies.

All univariate analyses involved a 10 000 run-in iteration phase and a 10 000-iteration phase for parameter estimation. All calculations were performed using the R2JAGS package to run Just Another Gibbs Sampler (JAGS) 3.2.3 and the code reported in NICE Decision Support Unit (DSU) Technical Support Document Series [ 29–33 ]. Convergence was confirmed through inspection of the ratios of Monte-Carlo error to the standard deviations of the posteriors; values >5% are strong signs of convergence issues [ 31 ]. In some cases, trials reported outcome results of zero (ACR70, PASI100, SAE) in one or more arms for which a continuity correction was applied to mitigate the issue, as without the correction most models were not convergent or provided a large posterior distribution making little clinical sense [ 31 ].

Four NMA models [fixed effects (FE) unadjusted, FE baseline risk-adjusted, random effects (RE) unadjusted and RE baseline risk-adjusted] were assessed and the best-fit models were chosen using methods described in NICE DSU Technical Support Document 2 [ 31 ]. Odds ratios (ORs) and differences of mean change (MC) with the associated 95% credible intervals (CrIs) were calculated for each treatment comparison in the evidence network for the best fitting models and presented in league tables and forest plots. In addition, the probability of bimekizumab 160 mg Q4W being better than other treatments was calculated using surface under the cumulative ranking curve (SUCRA) to determine relative rank. Conclusions (i.e. better/worse or comparable) for bimekizumab 160 mg Q4W vs comparators were based on whether the pairwise 95% CrIs of the ORs/difference of MC include 1 (dichotomous outcomes), 0 (continuous outcomes) or not. In the case where the 95% CrI included 1 or 0, then bimekizumab 160 mg Q4W and the comparator were considered comparable. If the 95% CrI did not include 1 or 0, then bimekizumab 160 mg Q4W was considered either better or worse depending on the direction of the effect.

Compliance with ethics guidelines

This article is based on previously conducted studies and does not contain any new studies with human participants or animals performed by any of the authors.

Study and patient characteristics

The SLR identified 4576 records through databases and 214 records through grey literature, of which 3143 were included for abstract review. Following the exclusion of a further 1609 records, a total of 1534 records were selected for full-text review. A total of 66 primary studies from 246 records were selected for data extraction. No trial was identified as having a moderate or high risk of bias ( Supplementary Table S3 , available at Rheumatology online).

Of the 66 studies identified in the SLR, 41 studies reported outcomes at weeks 12, 16 or 24 and met the criteria for inclusion in the NMA in either a b/tsDMARD-naïve population ( n = 20), a TNFi-experienced population ( n = 5), a mixed population with subgroups ( n = 13) or a mixed PsA population without subgroups reported ( n = 3). The PRISMA diagram is presented in Fig. 1 . Included and excluded studies are presented in Supplementary Tables S4 and S5 , respectively (available at Rheumatology online).

PRISMA flow diagram. The PRISMA flow diagram for the SLR conducted to identify published studies assessing approved treatments for the treatment of PsA. cDMARD: conventional DMARD; NMA: network meta-analysis; NR: not reported; PD: pharmacodynamic; PK: pharmacokinetic; PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCT: randomized controlled trial; SLR: systematic literature review

The baseline study and patient characteristics (where reported) are presented in Supplementary Table S6 (available at Rheumatology online). There were 20–483 patients included in treatment arms. The median age of patients was 48.9 years, the median percentage of males was 50.3% and a median of 92.3% of patients were Caucasian. Patients had a mean time since diagnosis of 7.6 years and a mean PASI score of 8.7. The mean (range) use of concomitant MTX, NSAIDs and steroids were 53.9% (29.1% to 84.0%), 72.4% (33.3% to 100.0%) and 16.8% (9.2% to 30.0%), respectively. Heterogeneity was generally low across studies except for the concomitant use of MTX, NSAIDs and steroids. Using an approach consistent with established NMA methods in PsA [ 34–36 ], a meta-regression model using JAGS code reported in NICE DSU Technical Support Document 3 [ 33 ] was used to account for variation in placebo responses when model-fit statistics suggested that baseline risk-adjusted models provided a better fit to the data.

NMA results

The network diagrams for ACR50 in b/tsDMARD-naïve and TNFi-experienced patients are presented in Fig. 2A and B with network diagrams for other outcomes presented in Supplementary Fig. S1 (available at Rheumatology online). The networks for ACR response were larger, in terms of both number of studies and patients included, than the networks for PASI. Similarly, the networks for b/tsDMARD-naïve patients were larger than TNFi-experienced patients across all outcomes analysed. Placebo was used as a common comparator in all networks and there were a few studies that included more than two arms (OPAL-Broaden, Select-PsA-1, SPIRIT-P1 and BE OPTIMAL) that included adalimumab as the reference arm in b/tsDMARD-naïve patients. Lastly, networks included studies where the primary outcome was evaluated at time points longer than 16 weeks (e.g. EXCEED study at 52 weeks) but as per the methods, 16-week data formed the network.

Network of evidence for ACR50. ( A ) b/tsDMARD-naïve patients. ( B ) TNFi-experienced patients. The size of the circle representing each intervention is proportional to the number of patients included in the analysis. The line width is proportional to the number of studies connecting the interventions. ABA: abatacept; ADA: adalimumab; APR: apremilast; b/tsDMARD-naïve: biologic and targeted synthetic DMARD-naïve; BKZ: bimekizumab; CZP: certolizumab pegol; ETA: etanercept; GOL: golimumab; GUS: guselkumab; IFX: infliximab; IV: intravenous; IXE: ixekizumab; PBO: placebo; Q4W: every 4 weeks; Q8W: every 8 weeks; RIS: risankizumab; SEC: secukinumab; TNFi-experienced: TNF inhibitor–experienced; TOF: tofacitinib; UPA: upadacitinib; UST: ustekinumab; w/o LD: without loading dose

The best-fit model is noted for each outcome with full model fit statistics for all outcomes presented in Supplementary Table S7 (available at Rheumatology online). Forest plots for ACR50 and PASI100 are presented in Figs 3 and 4 , with forest plots for other outcomes, along with the league tables in Supplementary Fig. S2 and Table S8 , respectively (available at Rheumatology online).

ACR50. The results for the NMA on ACR50 at week 16. ( A ) b/tsDMARD-naïve patients including forest plot and SUCRA values. FE baseline–adjusted model DIC = 469.59. ( B ) TNFi-experienced patients including forest plot and SUCRA values. RE-unadjusted model DIC = 205.33. a Week 24 data were used as week 16 data was not available. * The 95% CrI does not include 1; bimekizumab 160 mg Q4W is considered either better or worse depending on the direction of the effect. ABA: abatacept; ADA: adalimumab; APR: apremilast; b/tsDMARD-naïve: biologic and targeted synthetic DMARD-naïve; BKZ: bimekizumab; CrI: credible interval; CZP: certolizumab pegol; DIC: deviance information criterion; ETA: etanercept; FE: fixed effects; GOL: golimumab; GUS: guselkumab; IFX: infliximab; IV: intravenous; IXE: ixekizumab; NMA: network meta-analysis; PBO: placebo; Q4W: every 4 weeks; Q8W: every 8 weeks; RE: random effects; RIS: risankizumab; SEC: secukinumab; SUCRA: surface under the cumulative ranking curve; TNFi-experienced: TNF inhibitor–experienced; TOF: tofacitinib; UPA: upadacitinib; UST: ustekinumab; w/o LD: without loading dose

PASI100. The results for the NMA on PASI100 at week 16: ( A ) b/tsDMARD-naïve patients including forest plot and SUCRA values. FE baseline–adjusted model DIC = 150.27. ( B ) TNFi-experienced patients including forest plot and SUCRA values. RE-unadjusted model DIC = 81.76. a Week 24 data were used as week 16 data was not available. * The 95% CrI does not include 1; bimekizumab 160 mg 4W is considered better. ADA: adalimumab; b/tsDMARD-naïve: biologic and targeted synthetic DMARD-naïve; BKZ, bimekizumab; CrI, credible interval; CZP, certolizumab pegol; DIC, deviance information criterion; FE, fixed effects; GOL, golimumab; GUS, guselkumab; IXE, ixekizumab; NMA, network meta-analysis; PASI, Psoriasis Area and Severity Index; PBO, placebo; Q4W, every 4 weeks; Q8W, every 8 weeks; RE, random effects; SEC, secukinumab; SUCRA, surface under the cumulative ranking curve; TNFi-experienced, TNF inhibitor–experienced; UPA, upadacitinib

Joint outcomes

For ACR50 outcomes, the best-fit models for b/tsDMARD-naïve and TNFi-experienced were the FE baseline–adjusted model and RE-unadjusted model, respectively.

b/tsDMARD-naïve patients

Bimekizumab 160 mg Q4W ranked 6th for ACR20 (SUCRA = 0.75), 5th for ACR50 (SUCRA = 0.74) ( Fig. 3A ) and 3rd for ACR70 (SUCRA = 0.80) among 21 treatments. For ACR50, bimekizumab 160 mg Q4W was better than placebo, abatacept 125 mg, guselkumab 100 mg Q4W, ustekinumab 45 mg, risankizumab 150 mg, guselkumab 100 mg Q8W and ustekinumab 90 mg; worse than golimumab 2 mg i.v.; and comparable to the remaining treatments in the network ( Fig. 3A ).

TNFi-experienced patients

Bimekizumab 160 mg Q4W ranked 1st among 16 treatments for ACR20 (SUCRA = 0.96), 2nd among 15 treatments for ACR50 (SUCRA = 0.84) ( Fig. 3B ) and 1st among 16 treatments for ACR70 (SUCRA = 0.83). Bimekizumab 160 mg Q4W was better than placebo, abatacept 125 mg, secukinumab 150 mg without loading dose, tofacitinib 5 mg and secukinumab 150 mg; and comparable to the remaining treatments in the network on ACR50 ( Fig. 3B ).

Skin outcomes

For PASI100 outcomes, the best-fit models for b/tsDMARD-naïve and TNFi-experienced were the FE baseline–adjusted model and RE-unadjusted model, respectively.

Bimekizumab 160 mg Q4W ranked 2nd among 15 treatments (SUCRA = 0.89) for PASI90 and 1st among 11 treatments (SUCRA = 0.95) for PASI100 ( Fig. 4A ). Bimekizumab 160 mg Q4W was better than placebo, certolizumab pegol pooled, golimumab 2 mg i.v., secukinumab 150 mg, adalimumab 40 mg, upadacitinib 15 mg, secukinumab 300 mg and ixekizumab 80 mg Q4W; and comparable to the remaining treatments in the network on PASI100 ( Fig. 4A ).

Bimekizumab 160 mg Q4W ranked 1st among 10 treatments (SUCRA = 0.85) for PASI90 and 2nd among 7 treatments (SUCRA = 0.79) for PASI100 ( Fig. 4B ). Bimekizumab 160 mg Q4W was better than placebo, ixekizumab 80 mg Q4W and upadacitinib 15 mg; and comparable to the remaining treatments in the network on PASI100 ( Fig. 4B ).

For MDA, the best-fit models for b/tsDMARD-naïve and TNFi-experienced were the FE baseline–adjusted model and RE-unadjusted model, respectively.

Bimekizumab 160 mg Q4W ranked 1st among 13 treatments (SUCRA = 0.91) and was better than placebo [OR (95% CrI) 6.31 (4.61–8.20)], guselkumab 100 mg Q4W [2.06 (1.29–3.10)], guselkumab 100 mg Q8W [1.76 (1.09–2.69)], risankizumab 150 mg [1.99 (1.40–2.76)] and adalimumab 40 mg [1.41 (1.01–1.93)]; and comparable to the remaining treatments in the network ( Supplementary Fig. S2G , available at Rheumatology online).

Bimekizumab 160 mg Q4W ranked 1st among 11 treatments (SUCRA = 0.83) and was better than placebo [12.10 (5.31–28.19)] and tofacitinib 5 mg [6.81 (2.14–21.35)]; and comparable to the remaining treatments in the network ( Supplementary Fig. S2H , available at Rheumatology online).

The network for SAEs for a mixed population included 23 treatments and the best-fit model was an RE-unadjusted model (due to study populations and time point reporting heterogeneity). Bimekizumab 160 mg Q4W showed comparable safety to all treatments in the network ( Supplementary Fig. S2I , available at Rheumatology online).

The treatment landscape for PsA is complex, with numerous treatment options and limited direct comparative evidence. Bimekizumab 160 mg Q4W has recently been approved for the treatment of active PsA by the European Medicines Agency and recommended by NICE in the UK, and the published phase 3 results warrant comparison with existing therapies by NMA.

This NMA included 41 studies evaluating 22 b/tsDMARDs including the novel IL-17F and IL-17A inhibitor, bimekizumab. Overall, bimekizumab 160 mg Q4W ranked favourably among b/tsDMARDS for efficacy in joint, skin and disease activity outcomes in PsA across both b/tsDMARD-naïve and TNFi-experienced populations. The safety of bimekizumab 160 mg Q4W was similar to the other b/tsDMARDs.

The Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA) and EULAR provide evidence-based recommendations for the treatment of PsA [ 1 , 2 ]. To treat peripheral arthritis symptoms in PsA, efficacy across the classes of current b/tsDMARDs are considered similar by both GRAPPA and EULAR, in part due to a lack of data comparing licensed therapies in a head-to-head trial setting [ 1 , 2 ]. EULAR recommends the use of JAK inhibitors in the case of inadequate response, intolerance or when a bDMARD is not appropriate [ 1 ]. This recommendation was made when tofacitinib was the only available JAK inhibitor, but reflects current marketing authorizations for tofacitinib and upadacitinib which indicate use in patients with an inadequate response or prior intolerance to TNFis (USA) or bDMARDs (Europe) [ 37–40 ]. This NMA suggests that bimekizumab 160 mg Q4W may have an advantage over current treatments, including IL-23 inhibitors in b/tsDMARD naïve patients, and secukinumab 150 mg and tofacitinib in TNFi-experienced patients, as evidenced by our analysis of ACR50 for which the pairwise comparisons were significantly in favour of bimekizumab 160 mg Q4W.

For the treatment of skin symptoms in PsA, IL-23, IL-12/23 and IL-17A inhibitors are currently recommended due to their greater efficacy compared with TNFis [ 1 , 4 ]. GRAPPA also suggests considering efficacy demonstrated in direct comparative studies in PSO when selecting a treatment for PsA skin symptoms [ 2 ]. In our analysis of complete skin clearance as measured by PASI100, bimekizumab 160 mg Q4W demonstrated the likelihood of significantly greater efficacy than IL-17A, JAK inhibitors and TNFis in b/tsDMARD-naïve patients and IL-17A and JAK inhibitors in TNFi-experienced patients. Furthermore, the NMA results for skin clearance in PsA are in alignment with previous studies in PSO that demonstrated superiority of bimekizumab 320 mg Q4W or Q8W vs secukinumab, ustekinumab and adalimumab ( P < 0.001) (note that the dosing of bimekizumab in PSO differs from that in PsA) [ 12 , 18 , 41 , 42 ].

There are similarities between our results and other recently published NMAs of b/tsDMARDs in PsA, although methodological heterogeneity across all NMAs makes comparisons challenging [ 34–36 , 43–45 ]. Among recent NMAs, the largest evaluated 21 treatments [ 34 ] and only four considered subgroups of b/tsDMARD-naïve and TNFi-experienced patients or those with inadequate response [ 35 , 36 , 43 , 45 ]. Furthermore, different or pooled levels of response were evaluated for ACR and PASI outcomes.

Previous NMAs also support IL-17, IL-12/23 and IL-23 inhibitors having greater efficacy for skin symptoms than TNFis [ 35 , 36 ]. In an overall PsA population, McInnes et al. demonstrated that secukinumab 300 mg, ixekizumab 80 mg Q4W, and ustekinumab 45 mg and 90 mg were likely more efficacious than TNFis for PASI90 [ 35 ]. In another NMA by Ruyssen-Witrand et al. , results suggested that ixekizumab 80 mg Q4W had significantly greater efficacy than adalimumab, certolizumab pegol pooled, and etanercept 25 mg twice weekly/50 mg once weekly for any PASI score (50%, 75%, 90% and 100% reduction) in bDMARD-naïve patients [ 36 ].

For joint outcomes, Mease et al. compared guselkumab Q4W and Q8W with other b/tsDMARDs in a network of 21 treatments in an overall PsA population for ACR50 [ 34 ]. Both guselkumab dosing schedules were better than abatacept and apremilast, but golimumab 2 mg i.v. had a higher likelihood of ACR50 response than guselkumab Q8W [ 34 ]. Despite MDA being assessed in clinical trials for bDMARD therapies and a treatment target in PsA [ 46 ], evidence for comparative efficacy for this outcome is limited. None of the most recent NMAs before this one included an analysis of MDA [ 34–36 ]. With regard to safety outcomes, previous NMAs evaluating SAEs also resulted in either no difference between b/tsDMARDs vs placebo or other b/tsDMARDs [ 34 , 36 , 44 , 45 ].

This study has a number of strengths. To our knowledge this NMA represents the most comprehensive and in-depth comparative efficacy analysis of approved treatments in PsA to date. The evidence was derived from a recent SLR, ensuring that new RCTs and updated results from previously published RCTs were included. It is also the first NMA to include the phase 3 BE COMPLETE and BE OPTIMAL trials of bimekizumab [ 19 , 20 ]. Our NMA used robust methods and accounted for variation in placebo response through network meta-regression in accordance with NICE DSU Technical Support Documents [ 31–33 ]. As an acknowledgement of the evolution of treatment advances, separate analyses of b/tsDMARD-naïve and TNFi-experienced subgroups were conducted with the intent to assist healthcare decision-making in different clinical settings. In addition, a panel of clinical experts were consulted from project inception and are authors of this paper, ensuring inclusion of a comprehensive set of clinically meaningful outcomes, including the composite, treat-to-target outcome of MDA.

Despite the robust evidence base and methodology, this NMA has limitations. Indirect treatment comparisons such as this NMA are not a substitute for head-to-head trials. There was heterogeneity in the endpoints and reporting in the included studies. Fewer studies reporting PASI outcomes resulted in smaller networks compared with the network of studies evaluating ACR response criteria. Not all trials reported outcomes at the same timepoint, thereby reducing the comparability of trial results, which has been transparently addressed by noting where week 24 data were used vs week 12, 14 or 16 data. The analyses for the TNFi-experienced population were limited by potential heterogeneity, especially in the analyses where fewer studies were included in the networks, as this group could include patients who had an inadequate response to TNFi or discontinued TNFi treatment due to other reasons (e.g. lost access). Also, in the analyses for the TNFi-experienced population, very low patient numbers for some treatments resulted in less statistical power. Additionally, the data included in the analysis were derived exclusively from RCTs, for which the study populations may not reflect a typical patient population seen in real-world practice. For example, trial results may be different in patients with oligoarthritis who are not well-represented in clinical trials.

Over the years covering our SLR, we acknowledge that patient populations and the PsA treatment landscape have evolved. After a thorough review of baseline patient characteristics, no significant differences were observed across the studies included in the NMA. To further mitigate uncertainty, baseline regression was used to actively correct for changes in the placebo rate over time ensuring a consistent and fair comparison across all included treatments. In addition, our analyses were conducted in separate b/tsDMARD-naïve and TNFi-experienced populations that reflect the evolving PsA patient population over time. Radiographic progression was not within the purview of this NMA because the NMA focused on a shorter timeframe than the 52-week duration typically recommended by the literature for investigating radiographic progression. Furthermore, there is existing literature on this topic, as exemplified by the work of Wang et al. in 2022 [ 47 ]. Nevertheless, the comprehensive and current evidence base, examination of multiple endpoints, and consistency with previous reported NMAs lend credence to our results.

Overall, the results of this NMA demonstrated the favourable relative efficacy and safety of bimekizumab 160 mg Q4W vs all approved treatments for PsA. Bimekizumab ranked high in terms of efficacy on joint, skin and MDA outcomes in both b/tsDMARD-naïve and TNFi-experienced patient populations, and showed comparable safety to other treatments. In the evolving PsA treatment landscape, bimekizumab 160 mg Q4W is a potentially beneficial treatment option for patients with PsA.

Supplementary material is available at Rheumatology online.

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

This study was funded in full by UCB Pharma.

Disclosure statement : P.J.M.: has received research grants from AbbVie, Amgen, BMS, Eli Lilly, Gilead, Janssen, Novartis, Pfizer, Sun Pharma and UCB Pharma; consultancy fees from AbbVie, Acelyrin, Aclaris, Amgen, BMS, Boehringer Ingelheim, Eli Lilly, Galapagos, Gilead, GSK, Janssen, Moonlake Pharma, Novartis, Pfizer, Sun Pharma and UCB Pharma; and speakers’ bureau for AbbVie, Amgen, Eli Lilly, Janssen, Novartis, Pfizer and UCB Pharma. L.C.C.: received grants/research support from AbbVie, Amgen, Celgene, Eli Lilly, Janssen, Novartis, Pfizer and UCB; worked as a paid consultant for AbbVie, Amgen, Bristol Myers Squibb, Celgene, Eli Lilly, Gilead, Galapagos, Janssen, Moonlake, Novartis, Pfizer and UCB; and has been paid as a speaker for AbbVie, Amgen, Biogen, Celgene, Eli Lilly, Galapagos, Gilead, GSK, Janssen, Medac, Novartis, Pfizer and UCB. D.D.G.: consultant and/or received grant support from Abbvie, Amgen, BMS, Celgene, Eli Lilly, Galapagos, Gilead, Janssen, Novartis, Pfizer and UCB. J.F.M.: consultant and/or investigator for AbbVie, Amgen, Biogen, BMS, Dermavant, Eli Lilly, Janssen, LEO Pharma, Novartis, Pfizer, Regeneron, Sanofi, Sun Pharma and UCB Pharma. P.N.: research grants, clinical trials and honoraria for advice and lectures on behalf of AbbVie, Boehringer Ingelheim, BMS, Eli Lilly, Galapagos/Gilead, GSK, Janssen, Novartis, Pfizer, Samsung, Sanofi and UCB Pharma. S.G. and V.L.-K.: employees of Cytel, Inc. which served as a consultant on the project. A.R.P., D.W. and V.T.: employees and stockholders of UCB Pharma.

The authors acknowledge Leah Wiltshire of Cytel for medical writing and editorial assistance based on the authors’ input and direction, Heather Edens (UCB Pharma, Smyrna, GA, USA) for publication coordination and Costello Medical for review management, which were funded by UCB Pharma. This analysis was funded by UCB Pharma in accordance with Good Publication Practice (GPP 2022) guidelines ( http://www.ismpp.org/gpp-2022 ). Data were previously presented at ISPOR-US 2023 (Boston, MA, USA, 7–10 May 2023).

Gossec L , Baraliakos X , Kerschbaumer A et al. EULAR recommendations for the management of psoriatic arthritis with pharmacological therapies: 2019 update . Ann Rheum Dis 2020 ; 79 : 700 – 12 .

Google Scholar

Coates LC , Soriano ER , Corp N et al. ; GRAPPA Treatment Recommendations domain subcommittees . Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA): updated treatment recommendations for psoriatic arthritis 2021 . Nat Rev Rheumatol 2022 ; 18 : 465 – 79 .

Fitzgerald O , Ogdie A , Chandran V et al. Psoriatic arthritis . Nat Rev Dis Primers 2021 ; 7 : 59 .

Coates LC , Soriano ER , Corp N et al. Treatment recommendations domain subcommittees. Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA): updated treatment recommendations for psoriatic arthritis 2021 . Nat Rev Rheumatol 2022 ; 27 : 1 – 15 .

Najm A , Goodyear CS , McInnes IB , Siebert S. Phenotypic heterogeneity in psoriatic arthritis: towards tissue pathology-based therapy . Nat Rev Rheumatol 2023 ; 19 : 153 – 65 .

Ogdie A , Weiss P. The epidemiology of psoriatic arthritis . Rheum Dis Clin North Am 2015 ; 41 : 545 – 68 .

Alinaghi F , Calov M , Kristensen LE et al. Prevalence of psoriatic arthritis in patients with psoriasis: a systematic review and meta-analysis of observational and clinical studies . J Am Acad Dermatol 2019 ; 80 : 251 – 65.e19 .

Singh JA , Guyatt G , Ogdie A et al. Special Article: 2018 American college of rheumatology/national psoriasis foundation guideline for the treatment of psoriatic arthritis . Arthritis Rheumatol 2019 ; 71 : 5 – 32 .

Coates LC , Robinson DE , Orbai AM et al. What influences patients' opinion of remission and low disease activity in psoriatic arthritis? Principal component analysis of an international study . Rheumatology (Oxford) 2021 ; 60 : 5292 – 9 .

Gondo G , Mosca M , Hong J et al. Demographic and clinical factors associated with patient-reported remission in psoriatic arthritis . Dermatol Ther (Heidelb) 2022 ; 12 : 1885 – 95 .

Glatt S , Baeten D , Baker T et al. Dual IL-17A and IL-17F neutralisation by bimekizumab in psoriatic arthritis: evidence from preclinical experiments and a randomised placebo-controlled clinical trial that IL-17F contributes to human chronic tissue inflammation . Ann Rheum Dis 2018 ; 77 : 523 – 32 .

UCB Pharma S.A . Bimzelx ® (bimekizumab): Summary of Product Characteristics. 2023 . https://www.ema.europa.eu/en/medicines/human/EPAR/bimzelx (26 June 2023, date last accessed).

Adams R , Maroof A , Baker T et al. Bimekizumab, a novel humanized IgG1 antibody that neutralizes both IL-17A and IL-17F . Front Immunol 2020 ; 11 : 1894 .

Burns LA , Maroof A , Marshall D et al. Presence, function, and regulation of IL-17F-expressing human CD4(+) T cells . Eur J Immunol 2020 ; 50 : 568 – 80 .

Kuestner R , Taft D , Haran A et al. Identification of the IL-17 receptor related molecule IL-17RC as the receptor for IL-17F . J Immunol 2007 ; 179 : 5462 – 73 .

Johansen C , Usher PA , Kjellerup RB et al. Characterization of the interleukin-17 isoforms and receptors in lesional psoriatic skin . Br J Dermatol 2009 ; 160 : 319 – 24 .

Shah M , Maroof A , Gikas P et al. Dual neutralisation of IL-17F and IL-17A with bimekizumab blocks inflammation-driven osteogenic differentiation of human periosteal cells . RMD Open 2020 ; 6 : e001306 .

Reich K , Warren RB , Lebwohl M et al. Bimekizumab versus Secukinumab in Plaque Psoriasis . New Engl J Med 2021 ; 385 : 142 – 52 .

McInnes IB , Asahina A , Coates LC et al. Bimekizumab in patients with psoriatic arthritis, naive to biologic treatment: a randomised, double-blind, placebo-controlled, phase 3 trial (BE OPTIMAL) . Lancet 2023 ; 401 : 25 – 37 .

Merola JF , Landewe R , McInnes IB et al. Bimekizumab in patients with active psoriatic arthritis and previous inadequate response or intolerance to tumour necrosis factor-alpha inhibitors: a randomised, double-blind, placebo-controlled, phase 3 trial (BE COMPLETE) . Lancet 2023 ; 401 : 38 – 48 .

Page MJ , McKenzie JE , Bossuyt PM et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews . BMJ 2021 ; 372 : n71 .

Higgins JP. Cochrane handbook for systematic reviews of interventions. Vol. 2. 2nd edn. Chichester, UK: John Wiley & Sons, 2019 .

National Institute for Health and Care Excellence . The guidelines manual: Process and methods [PMG6]. 2012 . https://www.nice.org.uk/process/pmg6/chapter/introduction (3 March 2023, date last accessed).

Booth AM , Wright KE , Outhwaite H. Centre for Reviews and Dissemination databases: value, content, and developments . Int J Technol Assess Health Care 2010 ; 26 : 470 – 2 .

Sterne JAC , Savovic J , Page MJ et al. RoB 2: a revised tool for assessing risk of bias in randomised trials . BMJ 2019 ; 366 : l4898 .

Daly C , Dias S , Welton N , Anwer S , Ades A. NICE Guidelines Technical Support Unit. Meta-Analysis: Guideline Methodology Document 1 (Version 1). 2021 . http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/mpes/nice/guideline-methodology-documents-gmds/ (1 March 2023, date last accessed).

Dias S , Caldwell DM. Network meta-analysis explained . Archi Dis Childhood Fetal Neonatal Ed 2019 ; 104 : F8 – F12 .

Merola JF , Lockshin B , Mody EA. Switching biologics in the treatment of psoriatic arthritis . Semin Arthritis Rheum 2017 ; 47 : 29 – 37 .

Openbugs (website) 2014 . http://www.openbugs.net/w/FrontPage (6 April 2023, date last accessed).

Lunn D , Spiegelhalter D , Thomas A , Best N. The BUGS project: evolution, critique and future directions . Stat Med 2009 ; 28 : 3049 – 67 .

Dias S , Welton NJ , Sutton AJ , Ades AE. NICE DSU technical support document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. London. 2016 . https://www.sheffield.ac.uk/nice-dsu/tsds/full-list (25 January 2023, date last accessed).

Dias S , Welton N , Sutton AJ , Caldwell DM , Lu G , Ades AE. NICE DSU technical support document 4: inconsistency in networks of evidence based on randomised controlled trials. 2011 . https://www.sheffield.ac.uk/nice-dsu/tsds/full-list (25 January 2023, date last accessed).

Dias S , Sutton AJ , Welton N , Ades AE. NICE DSU Technical support document 3: heterogeneity: subgroups, meta-regression, bias and bias-adjustment. 2011 . https://www.sheffield.ac.uk/nice-dsu/tsds/full-list (25 January 2023, date last accessed).

Mease PJ , McInnes IB , Tam LS et al. Comparative effectiveness of guselkumab in psoriatic arthritis: results from systematic literature review and network meta-analysis . Rheumatology (Oxford) 2021 ; 60 : 2109 – 21 .

McInnes IB , Sawyer LM , Markus K et al. Targeted systemic therapies for psoriatic arthritis: a systematic review and comparative synthesis of short-term articular, dermatological, enthesitis and dactylitis outcomes . RMD Open 2022 ; 8 : e002074 .

Ruyssen-Witrand A , Perry R , Watkins C et al. Efficacy and safety of biologics in psoriatic arthritis: a systematic literature review and network meta-analysis . RMD Open 2020 ; 6 : e001117 .

Pfizer Inc . XELJANZ ® (tofacitinib): Summary of Product Characteristics. 2022 . https://www.ema.europa.eu/en/medicines/human/EPAR/xeljanz (4 May 2023, date last accessed).

Pfizer Inc . XELJANZ ® (tofacitinib): US Prescribing Information. 2022 . https://labeling.pfizer.com/ShowLabeling.aspx?id=959 (4 May 2023, date last accessed).

Abbvie Inc . RINVOQ ® (upadacitinib) extended-release tablets, for oral use: US Prescribing Information. 2023 . https://www.rxabbvie.com/pdf/rinvoq_pi.pdf (4 May 2023, date last accessed).

AbbVie Deutschland GmbH & Co . KG. RINVOQ ® (upadacitinib): Summary of Product Characteristics. 2023 . https://www.ema.europa.eu/en/medicines/human/EPAR/rinvoq (4 May 2023, date last accessed).

Warren RB , Blauvelt A , Bagel J et al. Bimekizumab versus Adalimumab in Plaque Psoriasis . N Engl J Med 2021 ; 385 : 130 – 41 .

Reich K , Papp KA , Blauvelt A et al. Bimekizumab versus ustekinumab for the treatment of moderate to severe plaque psoriasis (BE VIVID): efficacy and safety from a 52-week, multicentre, double-blind, active comparator and placebo controlled phase 3 trial . Lancet 2021 ; 397 : 487 – 98 .

Gladman DD , Orbai AM , Gomez-Reino J et al. Network meta-analysis of tofacitinib, biologic disease-modifying antirheumatic drugs, and apremilast for the treatment of psoriatic arthritis . Curr Ther Res Clin Exp 2020 ; 93 : 100601 .

Qiu M , Xu Z , Gao W et al. Fourteen small molecule and biological agents for psoriatic arthritis: a network meta-analysis of randomized controlled trials . Medicine (Baltimore) 2020 ; 99 : e21447 .

Kawalec P , Holko P , Mocko P , Pilc A. Comparative effectiveness of abatacept, apremilast, secukinumab and ustekinumab treatment of psoriatic arthritis: a systematic review and network meta-analysis . Rheumatol Int 2018 ; 38 : 189 – 201 .

Gossec L , McGonagle D , Korotaeva T et al. Minimal disease activity as a treatment target in psoriatic arthritis: a review of the literature . J Rheumatol 2018 ; 45 : 6 – 13 .

Wang SH , Yu CL , Wang TY , Yang CH , Chi CC. Biologic disease-modifying antirheumatic drugs for preventing radiographic progression in psoriatic arthritis: a systematic review and network meta-analysis . Pharmaceutics 2022 ; 14 .

Supplementary data

Month:	Total Views:
January 2024	715
February 2024	637
March 2024	596
April 2024	535
May 2024	462
June 2024	350
July 2024	96

Email alerts

Citing articles via.

Rheumatology Twitter
BSR Twitter
BSR Facebook
Recommend to Your Librarian

Affiliations

Online ISSN 1462-0332
Print ISSN 1462-0324
Copyright © 2024 British Society for Rheumatology
About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Copyright © 2024 Oxford University Press
Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
J Otolaryngol Head Neck Surg

Systematic and other reviews: criteria and complexities

Robert t. sataloff.

1 Editor-in-Chief, Journal of Voice, Philadephia, USA

2 Editor Emeritus, Ear, Nose and Throat Journal, Philadephia, USA

Matthew L. Bush

3 Assistant Editor, Otology & Neurotology, Lexington, USA

Rakesh Chandra

4 Editor-in-Chief, Ear, Ear, Nose and Throat Journal, Nashville, USA

Douglas Chepeha

5 Editor-in-Chief, Journal of Otolaryngology – Head & Neck Surgery, Toronto, Canada

Brian Rotenberg

6 Editor-in-Chief, Journal of Otolaryngology – Head & Neck Surgery, London, Canada

Edward W. Fisher

7 Senior Editor, Journal of Laryngology and Otology, Birmingham, UK

David Goldenberg

8 Editor-in-Chief, Operative Techniques in Otolaryngology – Head and Neck Surgery, Hershey, USA

Ehab Y. Hanna

9 Editor-in-Chief, Head & Neck, Houston, USA

Joseph E. Kerschner

10 Editor-in-Chief, International Journal of Pediatric Otorhinolaryngology, Milwaukee, USA

Dennis H. Kraus

11 Co-Editor-in-Chief, Journal of Neurological Surgery Part B: Skull Base, New York, USA

John H. Krouse

12 Editor-in-Chief, Otolaryngology – Head and Neck Surgery, Philadelphia, USA

13 Editor-in-Chief, OTO-Open, Philadelphia, USA

14 Editor-in-Chief, Journal for Oto-Rhino-Laryngology, Head and Neck Surgery, Philadelphia, USA

15 Editor-in-Chief, World Journal of Otorhinolaryngology – Head and Neck Surgery, Philadelphia, USA

Michael Link

16 Co-Editor-in-Chief, Journal of Neurological Surgery Part B: Skull Base, Rochester, USA

Lawrence R. Lustig

17 Editor-in-Chief, Otology & Neurotology, New York, USA

Samuel H. Selesnick

18 Editor-in-Chief, The Laryngoscope, New York, USA

Raj Sindwani

19 Editor-in-Chief, American Journal of Rhinology & Allergy, Cleveland, USA

Richard J. Smith

20 Editor-in-Chief, Annals of Otology, Rhinology & Laryngology, Iowa City, USA

James Tysome

21 Editor-in-Chief, Clinical Otolaryngology, Cambridge, UK

Peter C. Weber

22 Editor-in-Chief, American Journal of Otolaryngology, Boston, USA

D. Bradley Welling

23 Editor-in-Chief, Laryngoscope Investigative Otolaryngology, Boston, USA

Review articles can be extremely valuable. They synthesize information for readers, often provide clarity and valuable insights into a topic; and good review articles tend to be cited frequently. Review articles do not require Institutional Review Board (IRB) approval if the data reviewed are public (including private and government databases) and if the articles reviewed have received IRB approval previously. However, some institutions require IRB review and exemption for review articles. So, authors should be familiar with their institution’s policy. In assessing and interpreting review articles, it is important to understand the article’s methodology, scholarly purpose and credibility. Many readers, and some journal reviewers, are not aware that there are different kinds of review articles with different definitions, criteria and academic impact [ 1 ]. In order to understand the importance and potential application of a review article, it is valuable for readers and reviewers to be able to classify review articles correctly.

Systematic reviews

Authors often submit articles that include the term “systematic” in the title without realizing that that term requires strict adherence to specific criteria. A systematic review follows explicit methodology to answer a well-defined research question by searching the literature comprehensively, evaluating the quantity and quality of research evidence rigorously, and analyzing the evidence to synthesize an answer to the research question. The evidence gathered in systematic reviews can be qualitative or quantitative. However, if adequate and comparable quantitative data are available then a meta-analysis can be performed to assess the weighted and summarized effect size of the studies included. Depending on the research question and the data collected, systematic reviews may or may not include quantitative meta-analyses; however, meta-analyses should be performed in the setting of a systematic review to ensure that all of the appropriate data were accessed. The components of a systematic review can be found in an important article by Moher et al. published in 2009 that defined requirements for systematic reviews and meta-analyses [ 2 ].

In order to optimize reporting of meta-analyses, an international group developed the Quality of Reporting of Meta-Analyses (QUOROM) statement at a meeting in 1996 that led to publication of the QUOROM statement in 1999 [ 3 ]. Moher et al. revised that document and re-named the guidelines the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The PRISMA statement included both meta-analyses and systematic reviews, and the authors incorporated definitions established by the Cochrane Collaboration [ 4 ]. The PRISMA statement established the current standard for systematic reviews. To qualify as a systematic review, the methods section should acknowledge use of the PRISMA guidelines, and all PRISMA components should be incorporated strictly in all facets of the paper from the research question to the discussion. The PRISMA statement includes a checklist of 27 items that must be included when reporting a systematic review or meta-analysis [ 2 ]. A downloadable version of this checklist can be used by authors, reviewers, and journal editorial staff to ensure compliance with recommended components [ 5 ]. All 27 will not be listed in this brief editorial (although authors and reviewers are encouraged to consult the article by Moher et al. and familiarize themselves with all items), but a few will be highlighted.

The research question, as reflected in the title, should be a hypothesis-based specific research inquiry. The introduction must describe the rationale for the review and provide a specific goal or set of goals to be addressed. The type of systematic review, according to the Cochrane Collaboration, is based on the research question being asked and may assess diagnostic test accuracy, review prognostic studies evidence, evaluate intervention effect, scrutinize research methodology, or summarize qualitative evidence [ 6 ].

In the methods section, the participants, interventions, comparisons, outcomes and study design (PICOS) must be put forward. In addition to mentioning compliance with PRISMA, the methods section should state whether a review protocol exists and, if so, where it can be accessed (including a registration number). Systematic reviews are eligible for registration in the International Prospective Register of Systematic Reviews (PROSPERO) as established at the University of York (York, UK). When PROSPERO is used (it is available but not required for systematic reviews), registration should occur at the initial protocol stage of the review, and the final paper should direct to the information in the register. The methods section also must include specific study characteristics including databases used, years considered, languages of articles included, specific inclusion and exclusion criteria for studies; and rationale for each criterion must be included. Which individuals specifically performed searches should be noted. Electronic search strategy (with a full description of at least one electronic search strategy sufficient to allow replication of the search), process for article selection, data variables sought, assumptions and simplifications, methods for assessing bias risk of each individual study (such as selective reporting in individual studies) and utilization of this information in data synthesis, principal summary measures (risk ratio, hazard ratio, difference in means, etc.), methods of data management and combining study results, outcome level assessment, and other information should be reported.

The results section should include the number of studies identified, screened, evaluated for eligibility (including rationale for exclusion), and those included in the final synthesis. A PRISMA flow diagram should be included to provide this information succinctly [ 7 ]. The results also should include the study characteristics, study results, risk of bias within and across studies, and a qualitative or quantitative synthesis of the results of the included studies. This level of rigor in acquiring and evaluating the evidence of each individual study is one of the criteria that distinguishes systematic reviews from other categories. If the systematic review involves studies with paired samples and quantitative data, a summary of data should be provided for each intervention group along with effect estimates and confidence intervals for all outcomes of each study. If a meta-analysis is performed, then synthesized effect size should be reported with confidence intervals and measures of consistency (i.e. – data heterogeneity such as I 2 ) for each meta-analysis, and assessment of bias risk across studies. A forest plot, which provides a graphical presentation of the meta-analysis results, should be included.

The discussion section should summarize the main findings commenting on the strength of evidence for each outcome, as well as relevance to healthcare providers, policymakers and other key stake-holders; limitations of the study and outcomes; and conclusions highlighting the interpretation of results in the context of other research, and implications for future research.

Without adhering to of all of these criteria and the others listed in the PRISMA statement and checklist, the review does not qualify to be classified as “systematic”.

Meta-analyses

Meta-analyses, when feasible based on available and comparable quantitative data, supplement a systematic review evaluation, by adding a secondary statistical analysis of the pooled weighted outcomes of similar studies. This adds a level of objectivity in the synthesis of the review’s findings. Meta-analyses are appropriate when at least 2 individual studies contain paired samples (experimental group and control group) and provide quantitative outcome data and sample size. Studies that lack a control group may over-estimate the effect size of the experimental intervention or condition being studied and are not ideal for meta-analyses [ 8 ]. It also should be remembered that the conclusions of a meta-analysis are only as valid as the data on which the analysis is based. If the articles included are flawed, then the conclusions of the meta-analysis also may be flawed. Systematic reviews and meta-analyses are the most rigorous categories of review.

Other types of reviews

Mixed methods reviews.

Systematic reviews typically contain a single type of data, either qualitative or quantitative; however, mixed methods reviews bring together a combination of data types or study types. This approach may be utilized when quantitative data, in the setting of an intervention study, only provide a narrow perspective of the efficacy or effectiveness of the intervention. The addition of qualitative data or qualitative studies may provide a more complete picture of the knowledge, attitudes, and behaviors of clinicians, patients or researchers regarding that intervention. This type of review could involve collecting either the quantitative or the qualitative data using systematic review methodology, but often the qualitative data are gathered using a convenience sampling. Many qualitative studies provide useful insights into clinical management and/or implementation of research interventions; and incorporating them into a mixed methods review may provide valuable perspective on a wide range of literature. Mixed methods reviews are not necessarily systematic in nature; however, authors conducting mixed methods reviews should follow systematic review methodology, when possible.

Literature and narrative reviews

Literature reviews include peer-reviewed original research, systematic reviews, and meta-analyses, but also may include conference abstracts, books, graduate degree theses, and other non-peer reviewed publications. The methods used to identify and evaluate studies should be specified, but they are less rigorous and comprehensive than those required for systematic reviews. Literature reviews can evaluate a broad topic but do not specifically articulate a specific question, nor do they synthesize the results of included studies rigorously. Like mixed method reviews, they provide an overview of published information on the topic, although they may be less comprehensive than integrative reviews; and, unlike systematic reviews, they do not need to support evidence-based clinical or research practices, or highlight high-quality evidence for the reader. Narrative reviews are similar to literature reviews and evaluate the same scope of literature. The terms sometimes are used interchangeably, and author bias in article selection and data interpretation is a potential concern in literature and narrative reviews.

Umbrella reviews

An umbrella review integrates previously published, high-quality reviews such as systematic reviews and meta-analyses. Its purpose is to synthesize information in previously published systematic reviews and meta-analyses into one convenient paper.

Rapid review

A rapid review uses systematic review methodology to evaluate existing research. It provides a quick synthesis of evidence and is used most commonly to assist in emergent decision-making such as that required to determine whether COVID-19 vaccines should receive emergent approval.

Scoping, mapping, and systematized reviews

If literature has not been reviewed comprehensively in a specific subject that is varied and complex, a mapping review (also called scoping review) may be useful to organize initial understanding of the topic and its available literature. While mapping reviews may be helpful in crystallizing research findings and may be published, they are particularly useful in helping to determine whether a topic is amenable to systematic review, and to help organize and direct the approach of the systematic review or other reviews of the subject. Systematized reviews are used most commonly by students. The systematized review provides initial assessment of a topic that is potentially appropriate for a systematic review, but a systematized review does not meet the rigorous criteria of a systematic review and has substantially more limited value. Additional types of reviews exist including critical review, state-of-the-art review, and others.

Reviews can be invaluable; but they also can be misleading. Systematic reviews and meta-analyses provide readers with the greatest confidence that rigorous efforts have attempted to eliminate bias and ensure validity, but even they have limitations based upon the strengths and weaknesses of the literature that they have assessed (and the skill and objectivity with which the authors have executed the review). Risks of bias, incomplete information and misinformation increase as the rigor of review methodology decreases. While review articles may summarize research related to a topic for readers, non-systematic reviews lack the rigor to answer adequately hypothesis-driven research questions that can influence evidence-based practice. Journal authors, reviewers, editorial staff, and should be cognizant of the strengths and weaknesses of review methodology and should consider them carefully as they assess the value of published review articles, particularly as they determine whether the information presented should alter their patient care.

Authors’ contributions

The author(s) read and approved the final manuscript.

Declarations

The authors declare no competing interests.

This article is co-published in the following journals: Journal of Voice, Otology & Neurotology, Ear, Nose and Throat Journal, Journal of Laryngology and Otology, Operative Techniques in Otolaryngology – Head and Neck Surgery, Head & Neck, International Journal of Pediatric Otorhinolaryngology, Journal of Neurological Surgery Part B: Skull Base, Otolaryngology – Head and Neck Surgery, World Journal of Otorhinolaryngology – Head and Neck Surgery, The Laryngoscope, American Journal of Rhinology & Allergy, Annals of Otology, Rhinology & Laryngology, Clinical Otolaryngology, American Journal of Otolaryngology, Laryngoscope Investigative Otolaryngology.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The Role of Fintech in Promoting Financial Inclusion to Achieve Sustainable Development: An Integrated Bibliometric Analysis and Systematic Literature Review

Published: 03 July 2024

Cite this article

Kriti Kishor ORCID: orcid.org/0009-0006-2808-1633 1 ,
Sanjeev K. Bansal ORCID: orcid.org/0009-0009-8585-0429 1 &
Roshan Kumar 2

Fintech’s ability to enhance efficiency and reduce costs in financial services can promote greater financial inclusion (FI), which in turn serves as a foundation for sustainable and equitable development. Due to the dearth of thorough summaries in the body of existing literature, this systematic review and bibliometric analysis aim to present quantitative and qualitative information about the comprehensive relationship between fintech, FI, and sustainability development in an organised way. The review includes 189 publications from peer-reviewed journals of Scopus and Web of Science (WoS) databases up to 2023. The article was compiled based on the Scientific Procedures and Rationales for Systematic Literature Reviews (SPAR‐4‐SLR) protocol and the theory-context-characteristics-methodology (TCCM) framework. Bibliometric analysis has identified the leading journals, authors, nations, articles, and themes. A conceptual model has been designed to illustrate the entire scope, following which potential study areas have been proposed. This study aims to provide academic researchers, policymakers, and regulators with a detailed understanding of the relationship between fintech, financial inclusion, and sustainable development. The analysis demonstrates that FI is an essential requirement of our society and a vital pathway to achieve sustainable development. In the content analysis, we identify an integrative framework of four variables on this nexus. We found a very few conceptual, qualitative, and mixed method papers on this interaction, which provide potential avenues for further research. We recommend that scholars consider adopting a multi-theory perspective. We propose a comprehensive framework on this nexus. It will also pinpoint specific areas that require further investigation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

Get 10 units per month
Download Article/Chapter or Ebook
1 Unit = 1 Article or 1 Chapter
Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data Availability

The data that supports the findings of this study is available on request.

Adegbite, O. O., & Machethe, C. L. (2020). Bridging the financial inclusion gender gap in smallholder agriculture in Nigeria: An untapped potential for sustainable development. World Development, 127 , 104755.

Article Google Scholar

Afjal, M. (2023). Bridging the financial divide: A bibliometric analysis on the role of digital financial services within FinTech in enhancing financial inclusion and economic development. Humanities and Social Sciences Communications, 10 (1), 1–27.

Ahelegbey, D., Giudici, P., & Pediroda, V. (2023). A network based fintech inclusion platform. Socio-Economic Planning Sciences, 87 , 101555.

Akolgo, I. A. (2023). On the contradictions of Africa’s fintech boom: evidence from Ghana. Review of International Political Economy , 30 (5), 1639–1659. https://doi.org/10.1080/09692290.2023.2225142

Alshater, M. M., Saba, I., Supriani, I., & Rabbani, M. R. (2022). Fintech in Islamic finance literature: A review. Heliyon , 8(9). https://doi.org/10.1016/j.heliyon.2022.e10385

Ameen, N., Sharma, G. D., Tarba, S., Rao, A., & Chopra, R. (2022). Toward advancing theory on creativity in marketing and artificial intelligence. Psychology & Marketing, 39 (9), 1802–1825.

Arner, D. W., Buckley, R. P., Zetzsche, D. A., & Veidt, R. (2020). Sustainability, FinTech and financial inclusion. European Business Organization Law Review, 21 , 7–35.

Banna, H., Mia, M. A., Nourani, M., & Yarovaya, L. (2022). Fintech-based financial inclusion and risk-taking of microfinance institutions (MFIs): Evidence from Sub-Saharan Africa. Finance Research Letters, 45 , 102149.

Ben Slimane, S., Coeurderoy, R., & Mhenni, H. (2022). Digital transformation of small and medium enterprises: A systematic literature review and an integrative framework. International Studies of Management & Organization, 52 (2), 96–120.

Bhatt, A., Joshipura, M., & Joshipura, N. (2022). Decoding the trinity of Fintech, digitalization and financial services: An integrated bibliometric analysis and thematic literature review approach. Cogent Economics & Finance, 10 (1), 2114160.

Carè, R., Boitan, I. A., & Fatima, R. (2023). How do FinTech companies contribute to the achievement of SDGs? Insights from case studies. Research in International Business and Finance, 66 , 102072.

Chinoda, T., & Mashamba, T. (2021). Fintech, financial inclusion and income inequality nexus in Africa. Cogent Economics & Finance, 9 (1), 1986926. https://doi.org/10.1080/23322039.2021.1986926

Chowdhury, E. K., & Chowdhury, R. (2023). Role of financial inclusion in human development: Evidence from Bangladesh, India and Pakistan. Journal of the Knowledge Economy , 1–26. https://doi.org/10.1007/s13132-023-01366-x

Chung, S., Kim, K., Lee, C. H., & Oh, W. (2023). Interdependence between online peer-to-peer lending and cryptocurrency markets and its effects on financial inclusion. Production and Operations Management, 32 (6), 1939–1957.

Coffie, C. P. K., & Hongjiang, Z. (2023). FinTech market development and financial inclusion in Ghana: The role of heterogeneous actors. Technological Forecasting and Social Change, 186 , 122127.

Comerio, N., & Strozzi, F. (2019). Tourism and its economic impact: A literature review using bibliometric tools. Tourism Economics, 25 (1), 109–131. https://doi.org/10.1177/1354816618793762

Danladi, S., Prasad, M. S. V., Modibbo, U. M., Ahmadi, S. A., & Ghasemi, P. (2023). Attaining Sustainable Development Goals through financial inclusion: Exploring collaborative approaches to fintech adoption in developing economies. Sustainability, 15 (17), 13039.

David-West, O., Iheanachor, N., & Umukoro, I. (2020). Sustainable business models for the creation of mobile financial services in Nigeria. Journal of Innovation & Knowledge, 5 (2), 105–116.

Di Vaio, A., Hassan, R., & Palladino, R. (2023). Blockchain technology and gender equality: A systematic literature review. International Journal of Information Management, 68 , 102517.

Dong, Y., Chung, M., Zhou, C., & Venkataraman, S. (2018). Banking on “mobile money”: The implications of mobile money services on the value chain. Manufacturing & Service Operations Management , 21(2). https://doi.org/10.1287/msom.2018.0717

Ellili, N. O. D. (2023). Is there any association between FinTech and sustainability? Evidence from bibliometric review and content analysis. Journal of Financial Services Marketing, 28 (4), 748–762.

Fu, J., & Mishra, M. (2022). Fintech in the time of COVID− 19: Technological adoption during crises. Journal of Financial Intermediation, 50 , 100945.

Gálvez-Sánchez, F. J., Lara-Rubio, J., Verdú-Jóver, A. J., & Meseguer-Sánchez, V. (2021). Research advances on financial inclusion: A bibliometric analysis. Sustainability, 13 (6), 3156.

GPFI. (2010). G20 Principles for innovative financial inclusion - executive brief. Accessed 18 November 2023. Available at: http://www.gpfi.org/publications/g20-principles-innovative-financial-inclusion-executive-brief

Gulati, A., & Singh, S. (2024). Financial self-efficacy of consumers: A review and research agenda. International Journal of Consumer Studies, 48 (2), e13024.

Gupta, S., Yun, H., Xu, H., & Kim, H. W. (2017). An exploratory study on mobile banking adoption in Indian metropolitan and urban areas: A scenario-based experiment. Information Technology for Development, 23 (1), 127–152.

Han, H., & Gu, X. (2021). Linkage between inclusive digital finance and high-tech enterprise innovation performance: Role of debt and equity financing. Frontiers in Psychology, 12 , 814408.

Hasan, M., Le, T., & Hoque, A. (2021). How does financial literacy impact on inclusive finance? Financial Innovation, 7 (1), 1–23.

Hasan, M., Noor, T., Gao, J., Usman, M., & Abedin, M. Z. (2023). Rural consumers’ financial literacy and access to FinTech services. Journal of the Knowledge Economy, 14 (2), 780–804.

Hashemizadeh, A., Ashraf, R. U., Khan, I., & Zaidi, S. A. H. (2023). Digital financial inclusion, environmental quality, and economic development: the contributions of financial development and investments in OECD countries. Environmental Science and Pollution Research, 30 (54), 116336–116347. https://doi.org/10.1007/s11356-023-30275-4

Hulland, J., & Houston, M. B. (2020). Why systematic review papers and meta-analyses matter: An introduction to the special issue on generalizations in marketing. Journal of the Academy of Marketing Science, 48 , 351–359. https://doi.org/10.1007/s11747-020-00721-7

Hwa, G. (2019). Global FinTech Adoption Index 2019. Accessed on 1 October 2023. Available at https://assets.ey.com/content/dam/ey-sites/ey-com/en_gl/topics/banking-and-capital-markets/ey-global-fintech-adoption-index.pdf

Iheanachor, N., David-West, Y., & Umukoro, I. O. (2021). Business model innovation at the bottom of the pyramid–A case of mobile money agents. Journal of Business Research, 127 , 96–107.

Karim, Z. A., Nizam, R., Law, S. H., & Hassan, M. K. (2022). Does financial inclusiveness affect economic growth? New evidence using a dynamic panel threshold regression. Finance Research Letters, 46 , 102364.

Kemal, A. A. (2019). Mobile banking in the government-to-person payment sector for financial inclusion in Pakistan. Information Technology for Development, 25 (3), 475–502.

Khando, K., Islam, M. S., & Gao, S. (2022). The emerging technologies of digital payments and associated challenges: A systematic literature review. Future Internet, 15 (1), 21.

Kim, M., Zoo, H., Lee, H., & Kang, J. (2018). Mobile financial services, financial inclusion, and development: A systematic review of academic literature. The Electronic Journal of Information Systems in Developing Countries, 84 (5), e12044.

Koomson, I., Martey, E., & Etwire, P. M. (2023). Mobile money and entrepreneurship in East Africa: The mediating roles of digital savings and access to digital credit. Information Technology & People, 36 (3), 996–1019.

Lagna, A., & Ravishankar, M. N. (2022). Making the world a better place with fintech research. Information Systems Journal, 32 (1), 61–102.

Latif, N., Safdar, N., Liaquat, M., Younas, K., Nazeer, N., & Rafeeq, R. (2023). The role of institutional quality in assessing the environmental externality of financial inclusion: A DCCE approach. Frontiers in Environmental Science, 11 , 65.

Lee, C. C., Lou, R., & Wang, F. (2023). Digital financial inclusion and poverty alleviation: Evidence from the sustainable development of China. Economic Analysis and Policy, 77 , 418–434.

Li, J., Wei, R., & Guo, Y. (2022). How can the financing constraints of SMEs be eased in China?-Effect analysis, heterogeneity test and mechanism identification based on digital inclusive finance. Frontiers in Environmental Science, 10 , 949164.

Lim, W. M., Yap, S.-F., & Makkar, M. (2021). Home sharing in marketing and tourism at a tipping point: What do we know, how do we know, and where should we be heading? Journal of Business Research, 122 (September 2020), 534–566. https://doi.org/10.1016/j.jbusres.2020.08.051

Liu, X., Zhan, F. B., Hong, S., Niu, B., & Liu, Y. (2012). A bibliometric study of earthquake research: 1900–2010. Scientometrics, 92 (3), 747–765. https://doi.org/10.1007/s11192-011-0599-z

Liu, S., Gao, L., Latif, K., Dar, A. A., Zia-UR-Rehman, M., & Baig, S. A. (2021). The behavioral role of digital economy adaptation in sustainable financial literacy and financial inclusion. Frontiers in Psychology, 12 , 742118.

Liu, A., Urquía-Grande, E., López-Sánchez, P., & Rodríguez-López, Á. (2023). Research into microfinance and ICTs: A bibliometric analysis. Evaluation and Program Planning, 97 , 102215.

Louman, B., Girolami, E. D., Shames, S., Primo, L. G., Gitz, V., Scherr, S. J., & Brady, M. (2022). Access to landscape finance for small-scale producers and local communities: A literature review. Land, 11 (9), 1444.

Mapanje, O., Karuaihe, S., Machethe, C., & Amis, M. (2023). Financing sustainable agriculture in sub-Saharan Africa: A review of the role of financial technologies. Sustainability, 15 (5), 4587.

Michael, B., Koroleska, N., Tai, A., & Wong, D. W. H. (2022). A critical look at using financial technology policy to promote the sustainable development goals. Sustainable Development, 30 (6), 1911–1920.

Mishra, V., & Bisht, S. S. (2013). Mobile banking in a developing economy: A customer-centric model for policy formulation. Telecommunications Policy, 37 (6–7), 503–514.

Morgan, P. J. (2022). Fintech and financial inclusion in Southeast Asia and India. Asian Economic Policy Review, 17 (2), 183–208.

Mpofu, F. Y. (2022). Industry 4.0 in financial services: Mobile money taxes, revenue mobilisation, financial inclusion, and the realisation of sustainable development goals (SDGs) in Africa. Sustainability, 14 (14), 8667.

N’dri, L. M., & Kakinaka, M. (2020). Financial inclusion, mobile money, and individual welfare: The case of Burkina Faso. Telecommunications Policy, 44 (3), 101926.

Niankara, I. (2023). The impact of financial inclusion on digital payment solution uptake within the Gulf Cooperation Council Economies. International Journal of Innovation Studies, 7 (1), 1–17.

Ozili, P. K. (2018). Impact of digital finance on financial inclusion and stability. Borsa Istanbul Review, 18 (4), 329–340.

Pal, A., De’, R., & Herath, T. (2020). The role of mobile payment technology in sustainable and human-centric development: Evidence from the post-demonetization period in India. Information Systems Frontiers, 22 , 607–631.

Paul, J., & Barari, M. (2022). Meta-analysis and traditional systematic literature reviews—What, why, when, where, and how? Psychology & Marketing, 39 (6), 1099–1115. https://doi.org/10.1002/mar.21657

Paul, J., & Criado, A. R. (2020). The art of writing literature review: What do we know and what do we need to know? International Business Review, 29 (4), 101717. https://doi.org/10.1016/j.ibusrev.2020.101717

Paul, J., & Rosado-Serrano, A. (2019). Gradual Internationalization vs Born-Global/International new venture models: A review and research agenda. International Marketing Review, 36 (6), 830–858. https://doi.org/10.1108/IMR-10-2018-0280

Paul, J., Lim, W. M., O’Cass, A., Hao, A. W., & Bresciani, S. (2021a). Scientific procedures and rationales for systematic literature reviews (SPAR-4-SLR). International Journal of Consumer Studies, 45 (45), 1–16. https://doi.org/10.1111/ijcs.12695

Paul, J., Merchant, A., Dwivedi, Y. K., & Rose, G. (2021b). Writing an impactful review article: What do we know and what do we need to know? Journal of Business Research, 133 , 337–340. https://doi.org/10.1016/j.jbusres.2021.05.005

Paul, J., Khatri, P., & Kaur Duggal, H. (2023). Frameworks for developing impactful systematic literature reviews and theory building: What, Why and How?. Journal of Decision Systems , 1–14. https://doi.org/10.1080/12460125.2023.2197700

Pittaway, L., Holt, R., & Broad, J. (2014). Synthesising knowledge in entrepreneurship research-The role of systematic literature reviews. In Handbook of research on small business and entrepreneurship (pp. 83–105). Edward Elgar Publishing. https://doi.org/10.4337/9781849809245.00014

Pradhan, R. P., Arvin, M. B., Nair, M. S., Hall, J. H., & Bennett, S. E. (2021). Sustainable economic development in India: The dynamics between financial inclusion, ICT development, and economic growth. Technological Forecasting and Social Change, 169 , 120758.

Puschmann, T. (2017). Fintech. Business & Information Systems Engineering, 59 , 69–76. https://doi.org/10.1007/s12599-017-0464-6

Raksmey, U., Lin, C. Y., & Kakinaka, M. (2022). Macroprudential regulation and financial inclusion: Any difference between developed and developing countries? Research in International Business and Finance, 63 , 101759.

Rohman, P. S., Fianto, B. A., Shah, S. A. A., Kayani, U. N., Suprayogi, N., & Supriani, I. (2021). A review on literature of Islamic microfinance from 2010–2020: Lesson for practitioners and future directions. Heliyon, 7 (12). https://doi.org/10.1016/j.heliyon.2021.e08549

Roy, P., & Patro, B. (2022). Financial inclusion of women and gender gap in access to finance: A systematic literature review. Vision, 26 (3), 282–299.

Sahabuddin, M., Sakib, M. N., Rahman, M. M., Jibir, A., Fahlevi, M., Aljuaid, M., & Grabowska, S. (2023). The evolution of FinTech in scientific research: A bibliometric analysis. Sustainability, 15 (9), 7176.

Schilling, L., & Seuring, S. (2023). Mobile financial service-enabled micro-businesses driving sustainable value creation in emerging markets. Technological Forecasting and Social Change, 192 , 122596.

Senyo, P. K., & Osabutey, E. L. (2020). Unearthing antecedents to financial inclusion through FinTech innovations. Technovation, 98 , 102155.

Senyo, P. K., Karanasios, S., Gozman, D., & Baba, M. (2022). FinTech ecosystem practices shaping financial inclusion: The case of mobile money in Ghana. European Journal of Information Systems, 31 (1), 112–127.

Setiawan, B., Phan, T. D., Medina, J., Wieriks, M., Nathan, R. J., & Fekete-Farkas, M. (2023). Quest for financial inclusion via digital financial services (Fintech) during COVID-19 pandemic: Case study of women in Indonesia. Journal of Financial Services Marketing , 1–15. https://doi.org/10.1057/s41264-023-00217-9

Shaikh, A. A., Glavee-Geo, R., Karjaluoto, H., & Hinson, R. E. (2023). Mobile money as a driver of digital financial inclusion. Technological Forecasting and Social Change, 186 , 122158.

Sharma, R., Kamble, S., Gupta, S., Belhadi, A., Rana, N. P., & Kumar, K. (2023). Interlinkages between digital-social entrepreneurship and technological capabilities for sustainable value creation. Journal of Global Information Management (JGIM), 31 (1), 1–26.

Siddik, A. B., Rahman, M. N., & Yong, L. (2023). Do fintech adoption and financial literacy improve corporate sustainability performance? The mediating role of access to finance. Journal of Cleaner Production , 421 , 137658. https://doi.org/10.1016/j.jclepro.2023.137658

Soetan, T. O., Mogaji, E., & Nguyen, N. P. (2021). Financial services experience and consumption in Nigeria. Journal of Services Marketing, 35 (7), 947–961.

Sultana, N., Chowdhury, R. S., & Haque, A. (2023). Gravitating towards Fintech: A study on undergraduates using extended UTAUT model. Heliyon, 9 (10). https://doi.org/10.1016/j.heliyon.2023.e2073

Tay, L. Y., Tai, H. T., & Tan, G. S. (2022). Digital financial inclusion: A gateway to sustainable development. Heliyon 8 (6). https://doi.org/10.1016/j.heliyon.2022.e09766

Tepe, G., Geyikci, U. B., & Sancak, F. M. (2021). Fintech companies: A bibliometric analysis. International Journal of Financial Studies, 10 (1), 2.

Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14 (3), 207–222.

Truby, J. (2020). Fintech and the city: Sandbox 2.0 policy and regulatory reform proposals. International Review of Law, Computers & Technology , 34 (3), 277–309. https://doi.org/10.1080/13600869.2018.1546542

Úbeda, F., Mendez, A., & Forcadell, F. J. (2023). The sustainable practices of multinational banks as drivers of financial inclusion in developing countries. Finance Research Letters, 51 , 103278.

UNSGSA. (2018) Igniting SDG progress through digital financial inclusion. Accessed on 7 October 2023. Available online: https://sustainabledevelopment.un.org/index.php?page=view&type=400&nr=2655&menu=1515

Vasile, V., Panait, M., & Apostu, S. A. (2021). Financial inclusion paradigm shift in the post pandemic period. Digital-divide and gender gap. International Journal of Environmental Research and Public Health, 18 (20), 10938.

Wang, L., Wu, Y., Huang, Z., & Wang, Y. (2022). How big data drives green economic development: Evidence from China. Frontiers in Environmental Science, 10 , 1055162.

Xue, L., Dong, J., & Zha, Y. (2023). How does digital finance affect firm environmental, social and governance (ESG) performance?—Evidence from Chinese listed firms. Heliyon, 9 (10). https://doi.org/10.1016/j.heliyon.2023.e20800

Yang, L., Chen, Z., Liu, T., Gong, Z., Yu, Y., & Wang, J. (2013). Global trends of solid waste research from 1997 to 2011 by using bibliometric analysis. Scientometrics, 96 (1), 133–146. https://doi.org/10.1007/s11192-012-0911-6

Zerucha, T. (2023, May 05). “ The key factors driving financial inclusion”: Fintech Nexus. Accessed on May 28, 2024. https://www.fintechnexus.com/the-key-factors-driving-financial-inclusion/

Download references

Author information

Authors and affiliations.

I.K. Gujral Punjab Technical University, Kapurthala, Punjab, India

Kriti Kishor & Sanjeev K. Bansal

Banaras Hindu University, Varanasi, India

Roshan Kumar

You can also search for this author in PubMed Google Scholar

Contributions

All authors made contributions to the conceptualisation and design of this study. The authors collaborated as a team to carry out material preparation, SLR data collection, and analysis. The initial version of the paper had been written by the collective team of authors, who actively engaged in providing feedback. The final manuscript was read and approved by every author.

Corresponding author

Correspondence to Kriti Kishor .

Ethics declarations

Conflict of interest.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Kishor, K., Bansal, S.K. & Kumar, R. The Role of Fintech in Promoting Financial Inclusion to Achieve Sustainable Development: An Integrated Bibliometric Analysis and Systematic Literature Review. J Knowl Econ (2024). https://doi.org/10.1007/s13132-024-02168-5

Download citation

Received : 03 April 2024

Accepted : 14 June 2024

Published : 03 July 2024

DOI : https://doi.org/10.1007/s13132-024-02168-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Financial Inclusion
Sustainability
Bibliometric
Systematic Literature Review
Find a journal
Publish with us
Track your research

Open access
Published: 03 July 2024

The impact of evidence-based nursing leadership in healthcare settings: a mixed methods systematic review

Maritta Välimäki 1 , 2 ,
Shuang Hu 3 ,
Tella Lantta 1 ,
Kirsi Hipp 1 , 4 ,
Jaakko Varpula 1 ,
Jiarui Chen 3 ,
Gaoming Liu 5 ,
Yao Tang 3 ,
Wenjun Chen 3 &
Xianhong Li 3

BMC Nursing volume 23 , Article number: 452 ( 2024 ) Cite this article

Metrics details

The central component in impactful healthcare decisions is evidence. Understanding how nurse leaders use evidence in their own managerial decision making is still limited. This mixed methods systematic review aimed to examine how evidence is used to solve leadership problems and to describe the measured and perceived effects of evidence-based leadership on nurse leaders and their performance, organizational, and clinical outcomes.

We included articles using any type of research design. We referred nurses, nurse managers or other nursing staff working in a healthcare context when they attempt to influence the behavior of individuals or a group in an organization using an evidence-based approach. Seven databases were searched until 11 November 2021. JBI Critical Appraisal Checklist for Quasi-experimental studies, JBI Critical Appraisal Checklist for Case Series, Mixed Methods Appraisal Tool were used to evaluate the Risk of bias in quasi-experimental studies, case series, mixed methods studies, respectively. The JBI approach to mixed methods systematic reviews was followed, and a parallel-results convergent approach to synthesis and integration was adopted.

Thirty-one publications were eligible for the analysis: case series ( n = 27), mixed methods studies ( n = 3) and quasi-experimental studies ( n = 1). All studies were included regardless of methodological quality. Leadership problems were related to the implementation of knowledge into practice, the quality of nursing care and the resource availability. Organizational data was used in 27 studies to understand leadership problems, scientific evidence from literature was sought in 26 studies, and stakeholders’ views were explored in 24 studies. Perceived and measured effects of evidence-based leadership focused on nurses’ performance, organizational outcomes, and clinical outcomes. Economic data were not available.

Conclusions

This is the first systematic review to examine how evidence is used to solve leadership problems and to describe its measured and perceived effects from different sites. Although a variety of perceptions and effects were identified on nurses’ performance as well as on organizational and clinical outcomes, available knowledge concerning evidence-based leadership is currently insufficient. Therefore, more high-quality research and clinical trial designs are still needed.

Trail registration

The study was registered (PROSPERO CRD42021259624).

Peer Review reports

Global health demands have set new roles for nurse leaders [ 1 ].Nurse leaders are referred to as nurses, nurse managers, or other nursing staff working in a healthcare context who attempt to influence the behavior of individuals or a group based on goals that are congruent with organizational goals [ 2 ]. They are seen as professionals “armed with data and evidence, and a commitment to mentorship and education”, and as a group in which “leaders innovate, transform, and achieve quality outcomes for patients, health care professionals, organizations, and communities” [ 3 ]. Effective leadership occurs when team members critically follow leaders and are motivated by a leader’s decisions based on the organization’s requests and targets [ 4 ]. On the other hand, problems caused by poor leadership may also occur, regarding staff relations, stress, sickness, or retention [ 5 ]. Therefore, leadership requires an understanding of different problems to be solved using synthesizing evidence from research, clinical expertise, and stakeholders’ preferences [ 6 , 7 ]. If based on evidence, leadership decisions, also referred as leadership decision making [ 8 ], could ensure adequate staffing [ 7 , 9 ] and to produce sufficient and cost-effective care [ 10 ]. However, nurse leaders still rely on their decision making on their personal [ 11 ] and professional experience [ 10 ] over research evidence, which can lead to deficiencies in the quality and safety of care delivery [ 12 , 13 , 14 ]. As all nurses should demonstrate leadership in their profession, their leadership competencies should be strengthened [ 15 ].

Evidence-informed decision-making, referred to as evidence appraisal and application, and evaluation of decisions [ 16 ], has been recognized as one of the core competencies for leaders [ 17 , 18 ]. The role of evidence in nurse leaders’ managerial decision making has been promoted by public authorities [ 19 , 20 , 21 ]. Evidence-based management, another concept related to evidence-based leadership, has been used as the potential to improve healthcare services [ 22 ]. It can guide nursing leaders, in developing working conditions, staff retention, implementation practices, strategic planning, patient care, and success of leadership [ 13 ]. Collins and Holton [ 23 ] in their systematic review and meta-analysis examined 83 studies regarding leadership development interventions. They found that leadership training can result in significant improvement in participants’ skills, especially in knowledge level, although the training effects varied across studies. Cummings et al. [ 24 ] reviewed 100 papers (93 studies) and concluded that participation in leadership interventions had a positive impact on the development of a variety of leadership styles. Clavijo-Chamorro et al. [ 25 ] in their review of 11 studies focused on leadership-related factors that facilitate evidence implementation: teamwork, organizational structures, and transformational leadership. The role of nurse managers was to facilitate evidence-based practices by transforming contexts to motivate the staff and move toward a shared vision of change.

As far as we are aware, however, only a few systematic reviews have focused on evidence-based leadership or related concepts in the healthcare context aiming to analyse how nurse leaders themselves uses evidence in the decision-making process. Young [ 26 ] targeted definitions and acceptance of evidence-based management (EBMgt) in healthcare while Hasanpoor et al. [ 22 ] identified facilitators and barriers, sources of evidence used, and the role of evidence in the process of decision making. Both these reviews concluded that EBMgt was of great importance but used limitedly in healthcare settings due to a lack of time, a lack of research management activities, and policy constraints. A review by Williams [ 27 ] showed that the usage of evidence to support management in decision making is marginal due to a shortage of relevant evidence. Fraser [ 28 ] in their review further indicated that the potential evidence-based knowledge is not used in decision making by leaders as effectively as it could be. Non-use of evidence occurs and leaders base their decisions mainly on single studies, real-world evidence, and experts’ opinions [ 29 ]. Systematic reviews and meta-analyses rarely provide evidence of management-related interventions [ 30 ]. Tate et al. [ 31 ] concluded based on their systematic review and meta-analysis that the ability of nurse leaders to use and critically appraise research evidence may influence the way policy is enacted and how resources and staff are used to meet certain objectives set by policy. This can further influence staff and workforce outcomes. It is therefore important that nurse leaders have the capacity and motivation to use the strongest evidence available to effect change and guide their decision making [ 27 ].

Despite of a growing body of evidence, we found only one review focusing on the impact of evidence-based knowledge. Geert et al. [ 32 ] reviewed literature from 2007 to 2016 to understand the elements of design, delivery, and evaluation of leadership development interventions that are the most reliably linked to outcomes at the level of the individual and the organization, and that are of most benefit to patients. The authors concluded that it is possible to improve individual-level outcomes among leaders, such as knowledge, motivation, skills, and behavior change using evidence-based approaches. Some of the most effective interventions included, for example, interactive workshops, coaching, action learning, and mentoring. However, these authors found limited research evidence describing how nurse leaders themselves use evidence to support their managerial decisions in nursing and what the outcomes are.

To fill the knowledge gap and compliment to existing knowledgebase, in this mixed methods review we aimed to (1) examine what leadership problems nurse leaders solve using an evidence-based approach and (2) how they use evidence to solve these problems. We also explored (3) the measured and (4) perceived effects of the evidence-based leadership approach in healthcare settings. Both qualitative and quantitative components of the effects of evidence-based leadership were examined to provide greater insights into the available literature [ 33 ]. Together with the evidence-based leadership approach, and its impact on nursing [ 34 , 35 ], this knowledge gained in this review can be used to inform clinical policy or organizational decisions [ 33 ]. The study is registered (PROSPERO CRD42021259624). The methods used in this review were specified in advance and documented in a priori in a published protocol [ 36 ]. Key terms of the review and the search terms are defined in Table 1 (population, intervention, comparison, outcomes, context, other).

In this review, we used a mixed methods approach [ 37 ]. A mixed methods systematic review was selected as this approach has the potential to produce direct relevance to policy makers and practitioners [ 38 ]. Johnson and Onwuegbuzie [ 39 ] have defined mixed methods research as “the class of research in which the researcher mixes or combines quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study.” Therefore, we combined quantitative and narrative analysis to appraise and synthesize empirical evidence, and we held them as equally important in informing clinical policy or organizational decisions [ 34 ]. In this review, a comprehensive synthesis of quantitative and qualitative data was performed first and then discussed in discussion part (parallel-results convergent design) [ 40 ]. We hoped that different type of analysis approaches could complement each other and deeper picture of the topic in line with our research questions could be gained [ 34 ].

Inclusion and exclusion criteria

Inclusion and exclusion criteria of the study are described in Table 1 .

Search strategy

A three-step search strategy was utilized. First, an initial limited search with #MEDLINE was undertaken, followed by analysis of the words used in the title, abstract, and the article’s key index terms. Second, the search strategy, including identified keywords and index terms, was adapted for each included data base and a second search was undertaken on 11 November 2021. The full search strategy for each database is described in Additional file 1 . Third, the reference list of all studies included in the review were screened for additional studies. No year limits or language restrictions were used.

Information sources

The database search included the following: CINAHL (EBSCO), Cochrane Library (academic database for medicine and health science and nursing), Embase (Elsevier), PsycINFO (EBSCO), PubMed (MEDLINE), Scopus (Elsevier) and Web of Science (academic database across all scientific and technical disciplines, ranging from medicine and social sciences to arts and humanities). These databases were selected as they represent typical databases in health care context. Subject headings from each of the databases were included in the search strategies. Boolean operators ‘AND’ and ‘OR’ were used to combine the search terms. An information specialist from the University of Turku Library was consulted in the formation of the search strategies.

Study selection

All identified citations were collated and uploaded into Covidence software (Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia www.covidence.org ), and duplicates were removed by the software. Titles and abstracts were screened and assessed against the inclusion criteria independently by two reviewers out of four, and any discrepancies were resolved by the third reviewer (MV, KH, TL, WC). Studies meeting the inclusion criteria were retrieved in full and archived in Covidence. Access to one full-text article was lacking: the authors for one study were contacted about the missing full text, but no full text was received. All remaining hits of the included studies were retrieved and assessed independently against the inclusion criteria by two independent reviewers of four (MV, KH, TL, WC). Studies that did not meet the inclusion criteria were excluded, and the reasons for exclusion were recorded in Covidence. Any disagreements that arose between the reviewers were resolved through discussions with XL.

Assessment of methodological quality

Eligible studies were critically appraised by two independent reviewers (YT, SH). Standardized critical appraisal instruments based on the study design were used. First, quasi-experimental studies were assessed using the JBI Critical Appraisal Checklist for Quasi-experimental studies [ 44 ]. Second, case series were assessed using the JBI Critical Appraisal Checklist for Case Series [ 45 ]. Third, mixed methods studies were appraised using the Mixed Methods Appraisal Tool [ 46 ].

To increase inter-reviewer reliability, the review agreement was calculated (SH) [ 47 ]. A kappa greater than 0.8 was considered to represent a high level of agreement (0–0.1). In our data, the agreement was 0.75. Discrepancies raised between two reviewers were resolved through discussion and modifications and confirmed by XL. As an outcome, studies that met the inclusion criteria were proceeded to critical appraisal and assessed as suitable for inclusion in the review. The scores for each item and overall critical appraisal scores were presented.

Data extraction

For data extraction, specific tables were created. First, study characteristics (author(s), year, country, design, number of participants, setting) were extracted by two authors independently (JC, MV) and reviewed by TL. Second, descriptions of the interventions were extracted by two reviewers (JV, JC) using the structure of the TIDIeR (Template for Intervention Description and Replication) checklist (brief name, the goal of the intervention, material and procedure, models of delivery and location, dose, modification, adherence and fidelity) [ 48 ]. The extractions were confirmed (MV).

Third, due to a lack of effectiveness data and a wide heterogeneity between study designs and presentation of outcomes, no attempt was made to pool the quantitative data statistically; the findings of the quantitative data were presented in narrative form only [ 44 ]. The separate data extraction tables for each research question were designed specifically for this study. For both qualitative (and a qualitative component of mixed-method studies) and quantitative studies, the data were extracted and tabulated into text format according to preplanned research questions [ 36 ]. To test the quality of the tables and the data extraction process, three authors independently extracted the data from the first five studies (in alphabetical order). After that, the authors came together to share and determine whether their approaches of the data extraction were consistent with each other’s output and whether the content of each table was in line with research question. No reason was found to modify the data extraction tables or planned process. After a consensus of the data extraction process was reached, the data were extracted in pairs by independent reviewers (WC, TY, SH, GL). Any disagreements that arose between the reviewers were resolved through discussion and with a third reviewer (MV).

Data analysis

We were not able to conduct a meta-analysis due to a lack of effectiveness data based on clinical trials. Instead, we used inductive thematic analysis with constant comparison to answer the research question [ 46 , 49 ] using tabulated primary data from qualitative and quantitative studies as reported by the original authors in narrative form only [ 47 ]. In addition, the qualitizing process was used to transform quantitative data to qualitative data; this helped us to convert the whole data into themes and categories. After that we used the thematic analysis for the narrative data as follows. First, the text was carefully read, line by line, to reveal topics answering each specific review question (MV). Second, the data coding was conducted, and the themes in the data were formed by data categorization. The process of deriving the themes was inductive based on constant comparison [ 49 ]. The results of thematic analysis and data categorization was first described in narrative format and then the total number of studies was calculated where the specific category was identified (%).

Stakeholder involvement

The method of reporting stakeholders’ involvement follows the key components by [ 50 ]: (1) people involved, (2) geographical location, (3) how people were recruited, (4) format of involvement, (5) amount of involvement, (6) ethical approval, (7) financial compensation, and (8) methods for reporting involvement.

In our review, stakeholder involvement targeted nurses and nurse leader in China. Nurse Directors of two hospitals recommended potential participants who received a personal invitation letter from researchers to participate in a discussion meeting. Stakeholders’ participation was based on their own free will. Due to COVID-19, one online meeting (1 h) was organized (25 May 2022). Eleven participants joined the meeting. Ethical approval was not applied and no financial compensation was offered. At the end of the meeting, experiences of stakeholders’ involvement were explored.

The meeting started with an introductory presentation with power points. The rationale, methods, and preliminary review results were shared with the participants [ 51 ].The meeting continued with general questions for the participants: (1) Are you aware of the concepts of evidence-based practice or evidence-based leadership?; (2) How important is it to use evidence to support decisions among nurse leaders?; (3) How is the evidence-based approach used in hospital settings?; and (4) What type of evidence is currently used to support nurse leaders’ decision making (e.g. scientific literature, organizational data, stakeholder views)?

Two people took notes on the course and content of the conversation. The notes were later transcripted in verbatim, and the key points of the discussions were summarised. Although answers offered by the stakeholders were very short, the information was useful to validate the preliminary content of the results, add the rigorousness of the review, and obtain additional perspectives. A recommendation of the stakeholders was combined in the Discussion part of this review increasing the applicability of the review in the real world [ 50 ]. At the end of the discussion, the value of stakeholders’ involvement was asked. Participants shared that the experience of participating was unique and the topic of discussion was challenging. Two authors of the review group further represented stakeholders by working together with the research team throughout the review study.

Search results

From seven different electronic databases, 6053 citations were identified as being potentially relevant to the review. Then, 3133 duplicates were removed by an automation tool (Covidence: www.covidence.org ), and one was removed manually. The titles and abstracts of 3040 of citations were reviewed, and a total of 110 full texts were included (one extra citation was found on the reference list but later excluded). Based on the eligibility criteria, 31 studies (32 hits) were critically appraised and deemed suitable for inclusion in the review. The search results and selection process are presented in the PRISMA [ 52 ] flow diagram Fig. 1 . The full list of references for included studies can be find in Additional file 2 . To avoid confusion between articles of the reference list and studies included in the analysis, the studies included in the review are referred inside the article using the reference number of each study (e.g. ref 1, ref 2).

Search results and study selection and inclusion process [ 52 ]

Characteristics of included studies

The studies had multiple purposes, aiming to develop practice, implement a new approach, improve quality, or to develop a model. The 31 studies (across 32 hits) were case series studies ( n = 27), mixed methods studies ( n = 3) and a quasi-experimental study ( n = 1). All studies were published between the years 2004 and 2021. The highest number of papers was published in year 2020.

Table 2 describes the characteristics of included studies and Additional file 3 offers a narrative description of the studies.

Methodological quality assessment

Quasi-experimental studies.

We had one quasi-experimental study (ref 31). All questions in the critical appraisal tool were applicable. The total score of the study was 8 (out of a possible 9). Only one response of the tool was ‘no’ because no control group was used in the study (see Additional file 4 for the critical appraisal of included studies).

Case series studies . A case series study is typically defined as a collection of subjects with common characteristics. The studies do not include a comparison group and are often based on prevalent cases and on a sample of convenience [ 53 ]. Munn et al. [ 45 ] further claim that case series are best described as observational studies, lacking experimental and randomized characteristics, being descriptive studies, without a control or comparator group. Out of 27 case series studies included in our review, the critical appraisal scores varied from 1 to 9. Five references were conference abstracts with empirical study results, which were scored from 1 to 3. Full reports of these studies were searched in electronic databases but not found. Critical appraisal scores for the remaining 22 studies ranged from 1 to 9 out of a possible score of 10. One question (Q3) was not applicable to 13 studies: “Were valid methods used for identification of the condition for all participants included in the case series?” Only two studies had clearly reported the demographic of the participants in the study (Q6). Twenty studies met Criteria 8 (“Were the outcomes or follow-up results of cases clearly reported?”) and 18 studies met Criteria 7 (“Q7: Was there clear reporting of clinical information of the participants?”) (see Additional file 4 for the critical appraisal of included studies).

Mixed-methods studies

Mixed-methods studies involve a combination of qualitative and quantitative methods. This is a common design and includes convergent design, sequential explanatory design, and sequential exploratory design [ 46 ]. There were three mixed-methods studies. The critical appraisal scores for the three studies ranged from 60 to 100% out of a possible 100%. Two studies met all the criteria, while one study fulfilled 60% of the scored criteria due to a lack of information to understand the relevance of the sampling strategy well enough to address the research question (Q4.1) or to determine whether the risk of nonresponse bias was low (Q4.4) (see Additional file 4 for the critical appraisal of included studies).

Intervention or program components

The intervention of program components were categorized and described using the TiDier checklist: name and goal, theory or background, material, procedure, provider, models of delivery, location, dose, modification, and adherence and fidelity [ 48 ]. A description of intervention in each study is described in Additional file 5 and a narrative description in Additional file 6 .

Leadership problems

In line with the inclusion criteria, data for the leadership problems were categorized in all 31 included studies (see Additional file 7 for leadership problems). Three types of leadership problems were identified: implementation of knowledge into practice, the quality of clinical care, and resources in nursing care. A narrative summary of the results is reported below.

Implementing knowledge into practice

Eleven studies (35%) aimed to solve leadership problems related to implementation of knowledge into practice. Studies showed how to support nurses in evidence-based implementation (EBP) (ref 3, ref 5), how to engage nurses in using evidence in practice (ref 4), how to convey the importance of EBP (ref 22) or how to change practice (ref 4). Other problems were how to facilitate nurses to use guideline recommendations (ref 7) and how nurses can make evidence-informed decisions (ref 8). General concerns also included the linkage between theory and practice (ref 1) as well as how to implement the EBP model in practice (ref 6). In addition, studies were motivated by the need for revisions or updates of protocols to improve clinical practice (ref 10) as well as the need to standardize nursing activities (ref 11, ref 14).

The quality of the care

Thirteen (42%) focused on solving problems related to the quality of clinical care. In these studies, a high number of catheter infections led a lack of achievement of organizational goals (ref 2, ref 9). A need to reduce patient symptoms in stem cell transplant patients undergoing high-dose chemotherapy (ref 24) was also one of the problems to be solved. In addition, the projects focused on how to prevent pressure ulcers (ref 26, ref 29), how to enhance the quality of cancer treatment (ref 25) and how to reduce the need for invasive constipation treatment (ref 30). Concerns about patient safety (ref 15), high fall rates (ref 16, ref 19), dissatisfaction of patients (ref 16, ref 18) and nurses (ref 16, ref 30) were also problems that had initiated the projects. Studies addressed concerns about how to promote good contingency care in residential aged care homes (ref 20) and about how to increase recognition of human trafficking problems in healthcare (ref 21).

Resources in nursing care

Nurse leaders identified problems in their resources, especially in staffing problems. These problems were identified in seven studies (23%), which involved concerns about how to prevent nurses from leaving the job (ref 31), how to ensure appropriate recruitment, staffing and retaining of nurses (ref 13) and how to decrease nurses’ burden and time spent on nursing activities (ref 12). Leadership turnover was also reported as a source of dissatisfaction (ref 17); studies addressed a lack of structured transition and training programs, which led to turnover (ref 23), as well as how to improve intershift handoff among nurses (ref 28). Optimal design for new hospitals was also examined (ref 27).

Main features of evidence-based leadership

Out of 31 studies, 17 (55%) included all four domains of an evidence-based leadership approach, and four studies (13%) included evidence of critical appraisal of the results (see Additional file 8 for the main features of evidence-based Leadership) (ref 11, ref 14, ref 23, ref 27).

Organizational evidence

Twenty-seven studies (87%) reported how organizational evidence was collected and used to solve leadership problems (ref 2). Retrospective chart reviews (ref 5), a review of the extent of specific incidents (ref 19), and chart auditing (ref 7, ref 25) were conducted. A gap between guideline recommendations and actual care was identified using organizational data (ref 7) while the percentage of nurses’ working time spent on patient care was analyzed using an electronic charting system (ref 12). Internal data (ref 22), institutional data, and programming metrics were also analyzed to understand the development of the nurse workforce (ref 13).

Surveys (ref 3, ref 25), interviews (ref 3, ref 25) and group reviews (ref 18) were used to better understand the leadership problem to be solved. Employee opinion surveys on leadership (ref 17), a nurse satisfaction survey (ref 30) and a variety of reporting templates were used for the data collection (ref 28) reported. Sometimes, leadership problems were identified by evidence facilitators or a PI’s team who worked with staff members (ref 15, ref 17). Problems in clinical practice were also identified by the Nursing Professional Council (ref 14), managers (ref 26) or nurses themselves (ref 24). Current practices were reviewed (ref 29) and a gap analysis was conducted (ref 4, ref 16, ref 23) together with SWOT analysis (ref 16). In addition, hospital mission and vision statements, research culture established and the proportion of nursing alumni with formal EBP training were analyzed (ref 5). On the other hand, it was stated that no systematic hospital-specific sources of data regarding job satisfaction or organizational commitment were used (ref 31). In addition, statements of organizational analysis were used on a general level only (ref 1).

Scientific evidence identified

Twenty-six studies (84%) reported the use of scientific evidence in their evidence-based leadership processes. A literature search was conducted (ref 21) and questions, PICO, and keywords were identified (ref 4) in collaboration with a librarian. Electronic databases, including PubMed (ref 14, ref 31), Cochrane, and EMBASE (ref 31) were searched. Galiano (ref 6) used Wiley Online Library, Elsevier, CINAHL, Health Source: Nursing/Academic Edition, PubMed, and the Cochrane Library while Hoke (ref 11) conducted an electronic search using CINAHL and PubMed to retrieve articles.

Identified journals were reviewed manually (ref 31). The findings were summarized using ‘elevator speech’ (ref 4). In a study by Gifford et al. (ref 9) evidence facilitators worked with participants to access, appraise, and adapt the research evidence to the organizational context. Ostaszkiewicz (ref 20) conducted a scoping review of literature and identified and reviewed frameworks and policy documents about the topic and the quality standards. Further, a team of nursing administrators, directors, staff nurses, and a patient representative reviewed the literature and made recommendations for practice changes.

Clinical practice guidelines were also used to offer scientific evidence (ref 7, ref 19). Evidence was further retrieved from a combination of nursing policies, guidelines, journal articles, and textbooks (ref 12) as well as from published guidelines and literature (ref 13). Internal evidence, professional practice knowledge, relevant theories and models were synthesized (ref 24) while other study (ref 25) reviewed individual studies, synthesized with systematic reviews or clinical practice guidelines. The team reviewed the research evidence (ref 3, ref 15) or conducted a literature review (ref 22, ref 28, ref 29), a literature search (ref 27), a systematic review (ref 23), a review of the literature (ref 30) or ‘the scholarly literature was reviewed’ (ref 18). In addition, ‘an extensive literature review of evidence-based best practices was carried out’ (ref 10). However, detailed description how the review was conducted was lacking.

Views of stakeholders

A total of 24 studies (77%) reported methods for how the views of stakeholders, i.e., professionals or experts, were considered. Support to run this study was received from nursing leadership and multidisciplinary teams (ref 29). Experts and stakeholders joined the study team in some cases (ref 25, ref 30), and in other studies, their opinions were sought to facilitate project success (ref 3). Sometimes a steering committee was formed by a Chief Nursing Officer and Clinical Practice Specialists (ref 2). More specifically, stakeholders’ views were considered using interviews, workshops and follow-up teleconferences (ref 7). The literature review was discussed with colleagues (ref 11), and feedback and support from physicians as well as the consensus of staff were sought (ref 16).

A summary of the project findings and suggestions for the studies were discussed at 90-minute weekly meetings by 11 charge nurses. Nurse executive directors were consulted over a 10-week period (ref 31). An implementation team (nurse, dietician, physiotherapist, occupational therapist) was formed to support the implementation of evidence-based prevention measures (ref 26). Stakeholders volunteered to join in the pilot implementation (ref 28) or a stakeholder team met to determine the best strategy for change management, shortcomings in evidence-based criteria were discussed, and strategies to address those areas were planned (ref 5). Nursing leaders, staff members (ref 22), ‘process owners (ref 18) and program team members (ref 18, ref 19, ref 24) met regularly to discuss the problems. Critical input was sought from clinical educators, physicians, nutritionists, pharmacists, and nurse managers (ref 24). The unit director and senior nursing staff reviewed the contents of the product, and the final version of clinical pathways were reviewed and approved by the Quality Control Commission of the Nursing Department (ref 12). In addition, two co-design workshops with 18 residential aged care stakeholders were organized to explore their perspectives about factors to include in a model prototype (ref 20). Further, an agreement of stakeholders in implementing continuous quality services within an open relationship was conducted (ref 1).

Critical appraisal

In five studies (16%), a critical appraisal targeting the literature search was carried out. The appraisals were conducted by interns and teams who critiqued the evidence (ref 4). In Hoke’s study, four areas that had emerged in the literature were critically reviewed (ref 11). Other methods were to ‘critically appraise the search results’ (ref 14). Journal club team meetings (ref 23) were organized to grade the level and quality of evidence and the team ‘critically appraised relevant evidence’ (ref 27). On the other hand, the studies lacked details of how the appraisals were done in each study.

The perceived effects of evidence-based leadership

Perceived effects of evidence-based leadership on nurses’ performance.

Eleven studies (35%) described perceived effects of evidence-based leadership on nurses’ performance (see Additional file 9 for perceived effects of evidence-based leadership), which were categorized in four groups: awareness and knowledge, competence, ability to understand patients’ needs, and engagement. First, regarding ‘awareness and knowledge’, different projects provided nurses with new learning opportunities (ref 3). Staff’s knowledge (ref 20, ref 28), skills, and education levels improved (ref 20), as did nurses’ knowledge comprehension (ref 21). Second, interventions and approaches focusing on management and leadership positively influenced participants’ competence level to improve the quality of services. Their confidence level (ref 1) and motivation to change practice increased, self-esteem improved, and they were more positive and enthusiastic in their work (ref 22). Third, some nurses were relieved that they had learned to better handle patients’ needs (ref 25). For example, a systematic work approach increased nurses’ awareness of the patients who were at risk of developing health problems (ref 26). And last, nurse leaders were more engaged with staff, encouraging them to adopt the new practices and recognizing their efforts to change (ref 8).

Perceived effects on organizational outcomes

Nine studies (29%) described the perceived effects of evidence-based leadership on organizational outcomes (see Additional file 9 for perceived effects of evidence-based leadership). These were categorized into three groups: use of resources, staff commitment, and team effort. First, more appropriate use of resources was reported (ref 15, ref 20), and working time was more efficiently used (ref 16). In generally, a structured approach made implementing change more manageable (ref 1). On the other hand, in the beginning of the change process, the feedback from nurses was unfavorable, and they experienced discomfort in the new work style (ref 29). New approaches were also perceived as time consuming (ref 3). Second, nurse leaders believed that fewer nursing staff than expected left the organization over the course of the study (ref 31). Third, the project helped staff in their efforts to make changes, and it validated the importance of working as a team (ref 7). Collaboration and support between the nurses increased (ref 26). On the other hand, new work style caused challenges in teamwork (ref 3).

Perceived effects on clinical outcomes

Five studies (16%) reported the perceived effects of evidence-based leadership on clinical outcomes (see Additional file 9 for perceived effects of evidence-based leadership), which were categorized in two groups: general patient outcomes and specific clinical outcomes. First, in general, the project assisted in connecting the guideline recommendations and patient outcomes (ref 7). The project was good for the patients in general, and especially to improve patient safety (ref 16). On the other hand, some nurses thought that the new working style did not work at all for patients (ref 28). Second, the new approach used assisted in optimizing patients’ clinical problems and person-centered care (ref 20). Bowel management, for example, received very good feedback (ref 30).

The measured effects of evidence-based leadership

The measured effects on nurses’ performance.

Data were obtained from 20 studies (65%) (see Additional file 10 for measured effects of evidence-based leadership) and categorized nurse performance outcomes for three groups: awareness and knowledge, engagement, and satisfaction. First, six studies (19%) measured the awareness and knowledge levels of participants. Internship for staff nurses was beneficial to help participants to understand the process for using evidence-based practice and to grow professionally, to stimulate for innovative thinking, to give knowledge needed to use evidence-based practice to answer clinical questions, and to make possible to complete an evidence-based practice project (ref 3). Regarding implementation program of evidence-based practice, those with formal EBP training showed an improvement in knowledge, attitude, confidence, awareness and application after intervention (ref 3, ref 11, ref 20, ref 23, ref 25). On the contrary, in other study, attitude towards EBP remained stable ( p = 0.543). and those who applied EBP decreased although no significant differences over the years ( p = 0.879) (ref 6).

Second, 10 studies (35%) described nurses’ engagement to new practices (ref 5, ref 6, ref 7, ref 10, ref 16, ref 17, ref 18, ref 21, ref 25, ref 27). 9 studies (29%) studies reported that there was an improvement of compliance level of participants (ref 6, ref 7, ref 10, ref 16, ref 17, ref 18, ref 21, ref 25, ref 27). On the contrary, in DeLeskey’s (ref 5) study, although improvement was found in post-operative nausea and vomiting’s (PONV) risk factors documented’ (2.5–63%), and ’risk factors communicated among anaesthesia and surgical staff’ (0–62%), the improvement did not achieve the goal. The reason was a limited improvement was analysed. It was noted that only those patients who had been seen by the pre-admission testing nurse had risk assessments completed. Appropriate treatment/prophylaxis increased from 69 to 77%, and from 30 to 49%; routine assessment for PONV/rescue treatment 97% and 100% was both at 100% following the project. The results were discussed with staff but further reasons for a lack of engagement in nursing care was not reported.

And third, six studies (19%) reported nurses’ satisfaction with project outcomes. The study results showed that using evidence in managerial decisions improved nurses’ satisfaction and attitudes toward their organization ( P < 0.05) (ref 31). Nurses’ overall job satisfaction improved as well (ref 17). Nurses’ satisfaction with usability of the electronic charting system significantly improved after introduction of the intervention (ref 12). In handoff project in seven hospitals, improvement was reported in all satisfaction indicators used in the study although improvement level varied in different units (ref 28). In addition, positive changes were reported in nurses’ ability to autonomously perform their job (“How satisfied are you with the tools and resources available for you treat and prevent patient constipation?” (54%, n = 17 vs. 92%, n = 35, p < 0.001) (ref 30).

The measured effects on organizational outcomes

Thirteen studies (42%) described the effects of a project on organizational outcomes (see Additional file 10 for measured effects of evidence-based leadership), which were categorized in two groups: staff compliance, and changes in practices. First, studies reported improved organizational outcomes due to staff better compliance in care (ref 4, ref 13, ref 17, ref 23, ref 27, ref 31). Second, changes in organization practices were also described (ref 11) like changes in patient documentation (ref 12, ref 21). Van Orne (ref 30) found a statistically significant reduction in the average rate of invasive medication administration between pre-intervention and post-intervention ( p = 0.01). Salvador (ref 24) also reported an improvement in a proactive approach to mucositis prevention with an evidence-based oral care guide. On the contrary, concerns were also raised such as not enough time for new bedside report (ref 16) or a lack of improvement of assessment of diabetic ulcer (ref 8).

The measured effects on clinical outcomes

A variety of improvements in clinical outcomes were reported (see Additional file 10 for measured effects of evidence-based leadership): improvement in patient clinical status and satisfaction level. First, a variety of improvement in patient clinical status was reported. improvement in Incidence of CAUTI decreased 27.8% between 2015 and 2019 (ref 2) while a patient-centered quality improvement project reduced CAUTI rates to 0 (ref 10). A significant decrease in transmission rate of MRSA transmission was also reported (ref 27) and in other study incidences of CLABSIs dropped following of CHG bathing (ref 14). Further, it was possible to decrease patient nausea from 18 to 5% and vomiting to 0% (ref 5) while the percentage of patients who left the hospital without being seen was below 2% after the project (ref 17). In addition, a significant reduction in the prevalence of pressure ulcers was found (ref 26, ref 29) and a significant reduction of mucositis severity/distress was achieved (ref 24). Patient falls rate decreased (ref 15, ref 16, ref 19, ref 27).

Second, patient satisfaction level after project implementation improved (ref 28). The scale assessing healthcare providers by consumers showed improvement, but the changes were not statistically significant. Improvement in an emergency department leadership model and in methods of communication with patients improved patient satisfaction scores by 600% (ref 17). In addition, new evidence-based unit improved patient experiences about the unit although not all items improved significantly (ref 18).

Stakeholder involvement in the mixed-method review

To ensure stakeholders’ involvement in the review, the real-world relevance of our research [ 53 ], achieve a higher level of meaning in our review results, and gain new perspectives on our preliminary findings [ 50 ], a meeting with 11 stakeholders was organized. First, we asked if participants were aware of the concepts of evidence-based practice or evidence-based leadership. Responses revealed that participants were familiar with the concept of evidence-based practice, but the topic of evidence-based leadership was totally new. Examples of nurses and nurse leaders’ responses are as follows: “I have heard a concept of evidence-based practice but never a concept of evidence-based leadership.” Another participant described: “I have heard it [evidence-based leadership] but I do not understand what it means.”

Second, as stakeholder involvement is beneficial to the relevance and impact of health research [ 54 ], we asked how important evidence is to them in supporting decisions in health care services. One participant described as follows: “Using evidence in decisions is crucial to the wards and also to the entire hospital.” Third, we asked how the evidence-based approach is used in hospital settings. Participants expressed that literature is commonly used to solve clinical problems in patient care but not to solve leadership problems. “In [patient] medication and care, clinical guidelines are regularly used. However, I am aware only a few cases where evidence has been sought to solve leadership problems.”

And last, we asked what type of evidence is currently used to support nurse leaders’ decision making (e.g. scientific literature, organizational data, stakeholder views)? The participants were aware that different types of information were collected in their organization on a daily basis (e.g. patient satisfaction surveys). However, the information was seldom used to support decision making because nurse leaders did not know how to access this information. Even so, the participants agreed that the use of evidence from different sources was important in approaching any leadership or managerial problems in the organization. Participants also suggested that all nurse leaders should receive systematic training related to the topic; this could support the daily use of the evidence-based approach.

To our knowledge, this article represents the first mixed-methods systematic review to examine leadership problems, how evidence is used to solve these problems and what the perceived and measured effects of evidence-based leadership are on nurse leaders and their performance, organizational, and clinical outcomes. This review has two key findings. First, the available research data suggests that evidence-based leadership has potential in the healthcare context, not only to improve knowledge and skills among nurses, but also to improve organizational outcomes and the quality of patient care. Second, remarkably little published research was found to explore the effects of evidence-based leadership with an efficient trial design. We validated the preliminary results with nurse stakeholders, and confirmed that nursing staff, especially nurse leaders, were not familiar with the concept of evidence-based leadership, nor were they used to implementing evidence into their leadership decisions. Our data was based on many databases, and we screened a large number of studies. We also checked existing registers and databases and found no registered or ongoing similar reviews being conducted. Therefore, our results may not change in the near future.

We found that after identifying the leadership problems, 26 (84%) studies out of 31 used organizational data, 25 (81%) studies used scientific evidence from the literature, and 21 (68%) studies considered the views of stakeholders in attempting to understand specific leadership problems more deeply. However, only four studies critically appraised any of these findings. Considering previous critical statements of nurse leaders’ use of evidence in their decision making [ 14 , 30 , 31 , 34 , 55 ], our results are still quite promising.

Our results support a previous systematic review by Geert et al. [ 32 ], which concluded that it is possible to improve leaders’ individual-level outcomes, such as knowledge, motivation, skills, and behavior change using evidence-based approaches. Collins and Holton [ 23 ] particularly found that leadership training resulted in significant knowledge and skill improvements, although the effects varied widely across studies. In our study, evidence-based leadership was seen to enable changes in clinical practice, especially in patient care. On the other hand, we understand that not all efforts to changes were successful [ 56 , 57 , 58 ]. An evidence-based approach causes negative attitudes and feelings. Negative emotions in participants have also been reported due to changes, such as discomfort with a new working style [ 59 ]. Another study reported inconvenience in using a new intervention and its potential risks for patient confidentiality. Sometimes making changes is more time consuming than continuing with current practice [ 60 ]. These findings may partially explain why new interventions or program do not always fully achieve their goals. On the other hand, Dubose et al. [ 61 ] state that, if prepared with knowledge of resistance, nurse leaders could minimize the potential negative consequences and capitalize on a powerful impact of change adaptation.

We found that only six studies used a specific model or theory to understand the mechanism of change that could guide leadership practices. Participants’ reactions to new approaches may be an important factor in predicting how a new intervention will be implemented into clinical practice. Therefore, stronger effort should be put to better understanding the use of evidence, how participants’ reactions and emotions or practice changes could be predicted or supported using appropriate models or theories, and how using these models are linked with leadership outcomes. In this task, nurse leaders have an important role. At the same time, more responsibilities in developing health services have been put on the shoulders of nurse leaders who may already be suffering under pressure and increased burden at work. Working in a leadership position may also lead to role conflict. A study by Lalleman et al. [ 62 ] found that nurses were used to helping other people, often in ad hoc situations. The helping attitude of nurses combined with structured managerial role may cause dilemmas, which may lead to stress. Many nurse leaders opt to leave their positions less than 5 years [ 63 ].To better fulfill the requirements of health services in the future, the role of nurse leaders in evidence-based leadership needs to be developed further to avoid ethical and practical dilemmas in their leadership practices.

It is worth noting that the perceived and measured effects did not offer strong support to each other but rather opened a new venue to understand the evidence-based leadership. Specifically, the perceived effects did not support to measured effects (competence, ability to understand patients’ needs, use of resources, team effort, and specific clinical outcomes) while the measured effects could not support to perceived effects (nurse’s performance satisfaction, changes in practices, and clinical outcomes satisfaction). These findings may indicate that different outcomes appear if the effects of evidence-based leadership are looked at using different methodological approach. Future study is encouraged using well-designed study method including mixed-method study to examine the consistency between perceived and measured effects of evidence-based leadership in health care.

There is a potential in nursing to support change by demonstrating conceptual and operational commitment to research-based practices [ 64 ]. Nurse leaders are well positioned to influence and lead professional governance, quality improvement, service transformation, change and shared governance [ 65 ]. In this task, evidence-based leadership could be a key in solving deficiencies in the quality, safety of care [ 14 ] and inefficiencies in healthcare delivery [ 12 , 13 ]. As WHO has revealed, there are about 28 million nurses worldwide, and the demand of nurses will put nurse resources into the specific spotlight [ 1 ]. Indeed, evidence could be used to find solutions for how to solve economic deficits or other problems using leadership skills. This is important as, when nurses are able to show leadership and control in their own work, they are less likely to leave their jobs [ 66 ]. On the other hand, based on our discussions with stakeholders, nurse leaders are not used to using evidence in their own work. Further, evidence-based leadership is not possible if nurse leaders do not have access to a relevant, robust body of evidence, adequate funding, resources, and organizational support, and evidence-informed decision making may only offer short-term solutions [ 55 ]. We still believe that implementing evidence-based strategies into the work of nurse leaders may create opportunities to protect this critical workforce from burnout or leaving the field [ 67 ]. However, the role of the evidence-based approach for nurse leaders in solving these problems is still a key question.

Limitations

This study aimed to use a broad search strategy to ensure a comprehensive review but, nevertheless, limitations exist: we may have missed studies not included in the major international databases. To keep search results manageable, we did not use specific databases to systematically search grey literature although it is a rich source of evidence used in systematic reviews and meta-analysis [ 68 ]. We still included published conference abstract/proceedings, which appeared in our scientific databases. It has been stated that conference abstracts and proceedings with empirical study results make up a great part of studies cited in systematic reviews [ 69 ]. At the same time, a limited space reserved for published conference publications can lead to methodological issues reducing the validity of the review results [ 68 ]. We also found that the great number of studies were carried out in western countries, restricting the generalizability of the results outside of English language countries. The study interventions and outcomes were too different across studies to be meaningfully pooled using statistical methods. Thus, our narrative synthesis could hypothetically be biased. To increase transparency of the data and all decisions made, the data, its categorization and conclusions are based on original studies and presented in separate tables and can be found in Additional files. Regarding a methodological approach [ 34 ], we used a mixed methods systematic review, with the core intention of combining quantitative and qualitative data from primary studies. The aim was to create a breadth and depth of understanding that could confirm to or dispute evidence and ultimately answer the review question posed [ 34 , 70 ]. Although the method is gaining traction due to its usefulness and practicality, guidance in combining quantitative and qualitative data in mixed methods systematic reviews is still limited at the theoretical stage [ 40 ]. As an outcome, it could be argued that other methodologies, for example, an integrative review, could have been used in our review to combine diverse methodologies [ 71 ]. We still believe that the results of this mixed method review may have an added value when compared with previous systematic reviews concerning leadership and an evidence-based approach.

Our mixed methods review fills the gap regarding how nurse leaders themselves use evidence to guide their leadership role and what the measured and perceived impact of evidence-based leadership is in nursing. Although the scarcity of controlled studies on this topic is concerning, the available research data suggest that evidence-based leadership intervention can improve nurse performance, organizational outcomes, and patient outcomes. Leadership problems are also well recognized in healthcare settings. More knowledge and a deeper understanding of the role of nurse leaders, and how they can use evidence in their own managerial leadership decisions, is still needed. Despite the limited number of studies, we assume that this narrative synthesis can provide a good foundation for how to develop evidence-based leadership in the future.

Implications

Based on our review results, several implications can be recommended. First, the future of nursing success depends on knowledgeable, capable, and strong leaders. Therefore, nurse leaders worldwide need to be educated about the best ways to manage challenging situations in healthcare contexts using an evidence-based approach in their decisions. This recommendation was also proposed by nurses and nurse leaders during our discussion meeting with stakeholders.

Second, curriculums in educational organizations and on-the-job training for nurse leaders should be updated to support general understanding how to use evidence in leadership decisions. And third, patients and family members should be more involved in the evidence-based approach. It is therefore important that nurse leaders learn how patients’ and family members’ views as stakeholders are better considered as part of the evidence-based leadership approach.

Future studies should be prioritized as follows: establishment of clear parameters for what constitutes and measures evidence-based leadership; use of theories or models in research to inform mechanisms how to effectively change the practice; conducting robust effectiveness studies using trial designs to evaluate the impact of evidence-based leadership; studying the role of patient and family members in improving the quality of clinical care; and investigating the financial impact of the use of evidence-based leadership approach within respective healthcare systems.

Data availability

The authors obtained all data for this review from published manuscripts.

World Health Organization. State of the world’s nursing 2020: investing in education, jobs and leadership. 2020. https://www.who.int/publications/i/item/9789240003279 . Accessed 29 June 2024.

Hersey P, Campbell R. Leadership: a behavioral science approach. The Center for; 2004.

Cline D, Crenshaw JT, Woods S. Nurse leader: a definition for the 21st century. Nurse Lead. 2022;20(4):381–4. https://doi.org/10.1016/j.mnl.2021.12.017 .

Article Google Scholar

Chen SS. Leadership styles and organization structural configurations. J Hum Resource Adult Learn. 2006;2(2):39–46.

Google Scholar

McKibben L. Conflict management: importance and implications. Br J Nurs. 2017;26(2):100–3.

Article PubMed Google Scholar

Haghgoshayie E, Hasanpoor E. Evidence-based nursing management: basing Organizational practices on the best available evidence. Creat Nurs. 2021;27(2):94–7. https://doi.org/10.1891/CRNR-D-19-00080 .

Majers JS, Warshawsky N. Evidence-based decision-making for nurse leaders. Nurse Lead. 2020;18(5):471–5.

Tichy NM, Bennis WG. Making judgment calls. Harvard Business Rev. 2007;85(10):94.

Sousa MJ, Pesqueira AM, Lemos C, Sousa M, Rocha Á. Decision-making based on big data analytics for people management in healthcare organizations. J Med Syst. 2019;43(9):1–10.

Guo R, Berkshire SD, Fulton LV, Hermanson PM. %J L in HS. Use of evidence-based management in healthcare administration decision-making. 2017;30(3): 330–42.

Liang Z, Howard P, Rasa J. Evidence-informed managerial decision-making: what evidence counts?(part one). Asia Pac J Health Manage. 2011;6(1):23–9.

Hasanpoor E, Janati A, Arab-Zozani M, Haghgoshayie E. Using the evidence-based medicine and evidence-based management to minimise overuse and maximise quality in healthcare: a hybrid perspective. BMJ evidence-based Med. 2020;25(1):3–5.

Shingler NA, Gonzalez JZ. Ebm: a pathway to evidence-based nursing management. Nurs 2022. 2017;47(2):43–6.

Farokhzadian J, Nayeri ND, Borhani F, Zare MR. Nurse leaders’ attitudes, self-efficacy and training needs for implementing evidence-based practice: is it time for a change toward safe care? Br J Med Med Res. 2015;7(8):662.

Article PubMed PubMed Central Google Scholar

American Nurses Association. ANA leadership competency model. Silver Spring, MD; 2018.

Royal College of Nursing. Leadership skills. 2022. https://www.rcn.org.uk/professional-development/your-career/nurse/leadership-skills . Accessed 29 June 2024.

Kakemam E, Liang Z, Janati A, Arab-Zozani M, Mohaghegh B, Gholizadeh M. Leadership and management competencies for hospital managers: a systematic review and best-fit framework synthesis. J Healthc Leadersh. 2020;12:59.

Liang Z, Howard PF, Leggat S, Bartram T. Development and validation of health service management competencies. J Health Organ Manag. 2018;32(2):157–75.

World Health Organization. Global Strategic Directions for Nursing and Midwifery. 2021. https://apps.who.int/iris/bitstream/handle/10665/344562/9789240033863-eng.pdf . Accessed 29 June 2024.

NHS Leadership Academy. The nine leadership dimensions. 2022. https://www.leadershipacademy.nhs.uk/resources/healthcare-leadership-model/nine-leadership-dimensions/ . Accessed 29 June 2024.

Canadian Nurses Association. Evidence-informed decision-making and nursing practice: Position statement. 2018. https://hl-prod-ca-oc-download.s3-ca-central-1.amazonaws.com/CNA/2f975e7e-4a40-45ca-863c-5ebf0a138d5e/UploadedImages/documents/Evidence_informed_Decision_making_and_Nursing_Practice_position_statement_Dec_2018.pdf . Accessed 29 June 2024.

Hasanpoor E, Hajebrahimi S, Janati A, Abedini Z, Haghgoshayie E. Barriers, facilitators, process and sources of evidence for evidence-based management among health care managers: a qualitative systematic review. Ethiop J Health Sci. 2018;28(5):665–80.

PubMed PubMed Central Google Scholar

Collins DB, Holton EF III. The effectiveness of managerial leadership development programs: a meta-analysis of studies from 1982 to 2001. Hum Res Dev Q. 2004;15(2):217–48.

Cummings GG, Lee S, Tate K, Penconek T, Micaroni SP, Paananen T, et al. The essentials of nursing leadership: a systematic review of factors and educational interventions influencing nursing leadership. Int J Nurs Stud. 2021;115:103842.

Clavijo-Chamorro MZ, Romero-Zarallo G, Gómez-Luque A, López-Espuela F, Sanz-Martos S, López-Medina IM. Leadership as a facilitator of evidence implementation by nurse managers: a metasynthesis. West J Nurs Res. 2022;44(6):567–81.

Young SK. Evidence-based management: a literature review. J Nurs Adm Manag. 2002;10(3):145–51.

Williams LL. What goes around comes around: evidence-based management. Nurs Adm Q. 2006;30(3):243–51.

Fraser I. Organizational research with impact: working backwards. Worldviews Evidence-Based Nurs. 2004;1:S52–9.

Roshanghalb A, Lettieri E, Aloini D, Cannavacciuolo L, Gitto S, Visintin F. What evidence on evidence-based management in healthcare? Manag Decis. 2018;56(10):2069–84.

Jaana M, Vartak S, Ward MM. Evidence-based health care management: what is the research evidence available for health care managers? Eval Health Prof. 2014;37(3):314–34.

Tate K, Hewko S, McLane P, Baxter P, Perry K, Armijo-Olivo S, et al. Learning to lead: a review and synthesis of literature examining health care managers’ use of knowledge. J Health Serv Res Policy. 2019;24(1):57–70.

Geerts JM, Goodall AH, Agius S, %J SS. Medicine. Evidence-based leadership development for physicians: a systematic literature review. 2020;246: 112709.

Barends E, Rousseau DM, Briner RB. Evidence-based management: The basic principles. Amsterdam; 2014. https://research.vu.nl/ws/portalfiles/portal/42141986/complete+dissertation.pdf#page=203 . Accessed 29 June 2024.

Stern C, Lizarondo L, Carrier J, Godfrey C, Rieger K, Salmond S, et al. Methodological guidance for the conduct of mixed methods systematic reviews. JBI Evid Synthesis. 2020;18(10):2108–18. https://doi.org/10.11124/JBISRIR-D-19-00169 .

Lancet T. 2020: unleashing the full potential of nursing. Lancet (London, England). 2019. p. 1879.

Välimäki MA, Lantta T, Hipp K, Varpula J, Liu G, Tang Y, et al. Measured and perceived impacts of evidence-based leadership in nursing: a mixed-methods systematic review protocol. BMJ Open. 2021;11(10):e055356. https://doi.org/10.1136/bmjopen-2021-055356 .

The Joanna Briggs Institute. Joanna Briggs Institute reviewers’ manual: 2014 edition. Joanna Briggs Inst. 2014; 88–91.

Pearson A, White H, Bath-Hextall F, Salmond S, Apostolo J, Kirkpatrick P. A mixed-methods approach to systematic reviews. JBI Evid Implement. 2015;13(3):121–31.

Johnson RB, Onwuegbuzie AJ. Mixed methods research: a research paradigm whose time has come. Educational Researcher. 2004;33(7):14–26.

Hong, Pluye P, Bujold M, Wassef M. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Syst Reviews. 2017;6(1):61. https://doi.org/10.1186/s13643-017-0454-2 .

Ramis MA, Chang A, Conway A, Lim D, Munday J, Nissen L. Theory-based strategies for teaching evidence-based practice to undergraduate health students: a systematic review. BMC Med Educ. 2019;19(1):1–13.

Sackett DL, Rosenberg WM, Gray JM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. Bmj. British Medical Journal Publishing Group; 1996. pp. 71–2.

Goodman JS, Gary MS, Wood RE. Bibliographic search training for evidence-based management education: a review of relevant literatures. Acad Manage Learn Educ. 2014;13(3):322–53.

Aromataris E, Munn Z. Chapter 3: Systematic reviews of effectiveness. JBI Manual for Evidence Synthesis. 2020; https://synthesismanual.jbi.global .

Munn Z, Barker TH, Moola S, Tufanaru C, Stern C, McArthur A et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. 2020;18(10): 2127–33.

Hong Q, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Mixed methods Appraisal Tool (MMAT) Version 2018: user guide. Montreal: McGill University; 2018.

McKenna J, Jeske D. Ethical leadership and decision authority effects on nurses’ engagement, exhaustion, and turnover intention. J Adv Nurs. 2021;77(1):198–206.

Maxwell M, Hibberd C, Aitchison P, Calveley E, Pratt R, Dougall N, et al. The TIDieR (template for intervention description and replication) checklist. The patient Centred Assessment Method for improving nurse-led biopsychosocial assessment of patients with long-term conditions: a feasibility RCT. NIHR Journals Library; 2018.

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77–101.

Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, et al. Stakeholder involvement in systematic reviews: a scoping review. Syst Reviews. 2018;7:1–26.

Braye S, Preston-Shoot M. Emerging from out of the shadows? Service user and carer involvement in systematic reviews. Evid Policy. 2005;1(2):173–93.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Reviews. 2021;10(1):1–11.

Porta M. Pilot investigation, study. A dictionary of epidemiology. Oxford University Press Oxford; 2014. p. 215.

Kreis J, Puhan MA, Schünemann HJ, Dickersin K. Consumer involvement in systematic reviews of comparative effectiveness research. Health Expect. 2013;16(4):323–37.

Joseph ML, Nelson-Brantley HV, Caramanica L, Lyman B, Frank B, Hand MW, et al. Building the science to guide nursing administration and leadership decision making. JONA: J Nurs Adm. 2022;52(1):19–26.

Gifford W, Davies BL, Graham ID, Tourangeau A, Woodend AK, Lefebre N. Developing Leadership Capacity for Guideline Use: a pilot cluster Randomized Control Trial: Leadership Pilot Study. Worldviews Evidence-Based Nurs. 2013;10(1):51–65. https://doi.org/10.1111/j.1741-6787.2012.00254.x .

Hsieh HY, Henker R, Ren D, Chien WY, Chang JP, Chen L, et al. Improving effectiveness and satisfaction of an electronic charting system in Taiwan. Clin Nurse Specialist. 2016;30(6):E1–6. https://doi.org/10.1097/NUR.0000000000000250 .

McAllen E, Stephens K, Swanson-Biearman B, Kerr K, Whiteman K. Moving Shift Report to the Bedside: an evidence-based Quality Improvement Project. OJIN: Online J Issues Nurs. 2018;23(2). https://doi.org/10.3912/OJIN.Vol23No02PPT22 .

Thomas M, Autencio K, Cesario K. Positive outcomes of an evidence-based pressure injury prevention program. J Wound Ostomy Cont Nurs. 2020;47:S24.

Cullen L, Titler MG. Promoting evidence-based practice: an internship for Staff nurses. Worldviews Evidence-Based Nurs. 2004;1(4):215–23. https://doi.org/10.1111/j.1524-475X.2004.04027.x .

DuBose BM, Mayo AM. Resistance to change: a concept analysis. Nursing forum. Wiley Online Library; 2020. pp. 631–6.

Lalleman PCB, Smid GAC, Lagerwey MD, Shortridge-Baggett LM, Schuurmans MJ. Curbing the urge to care: a bourdieusian analysis of the effect of the caring disposition on nurse middle managers’ clinical leadership in patient safety practices. Int J Nurs Stud. 2016;63:179–88.

Article CAS PubMed Google Scholar

Martin E, Warshawsky N. Guiding principles for creating value and meaning for the next generation of nurse leaders. JONA: J Nurs Adm. 2017;47(9):418–20.

Griffiths P, Recio-Saucedo A, Dall’Ora C, Briggs J, Maruotti A, Meredith P, et al. The association between nurse staffing and omissions in nursing care: a systematic review. J Adv Nurs. 2018;74(7):1474–87. https://doi.org/10.1111/jan.13564 .

Lúanaigh PÓ, Hughes F. The nurse executive role in quality and high performing health services. J Nurs Adm Manag. 2016;24(1):132–6.

de Kok E, Weggelaar-Jansen AM, Schoonhoven L, Lalleman P. A scoping review of rebel nurse leadership: descriptions, competences and stimulating/hindering factors. J Clin Nurs. 2021;30(17–18):2563–83.

Warshawsky NE. Building nurse manager well-being by reducing healthcare system demands. JONA: J Nurs Adm. 2022;52(4):189–91.

Paez A. Gray literature: an important resource in systematic reviews. J Evidence-Based Med. 2017;10(3):233–40.

McAuley L, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228–31.

Sarah S. Introduction to mixed methods systematic reviews. https://jbi-global-wiki.refined.site/space/MANUAL/4689215/8.1+Introduction+to+mixed+methods+systematic+reviews . Accessed 29 June 2024.

Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546–53.

Download references

Acknowledgements

We want to thank the funding bodies, the Finnish National Agency of Education, Asia Programme, the Department of Nursing Science at the University of Turku, and Xiangya School of Nursing at the Central South University. We also would like to thank the nurses and nurse leaders for their valuable opinions on the topic.

The work was supported by the Finnish National Agency of Education, Asia Programme (grant number 26/270/2020) and the University of Turku (internal fund 26003424). The funders had no role in the study design and will not have any role during its execution, analysis, interpretation of the data, decision to publish, or preparation of the manuscript.

Author information

Authors and affiliations.

Department of Nursing Science, University of Turku, Turku, FI-20014, Finland

Maritta Välimäki, Tella Lantta, Kirsi Hipp & Jaakko Varpula

School of Public Health, University of Helsinki, Helsinki, FI-00014, Finland

Maritta Välimäki

Xiangya Nursing, School of Central South University, Changsha, 410013, China

Shuang Hu, Jiarui Chen, Yao Tang, Wenjun Chen & Xianhong Li

School of Health and Social Services, Häme University of Applied Sciences, Hämeenlinna, Finland

Hunan Cancer Hospital, Changsha, 410008, China

Gaoming Liu

You can also search for this author in PubMed Google Scholar

Contributions

Study design: MV, XL. Literature search and study selection: MV, KH, TL, WC, XL. Quality assessment: YT, SH, XL. Data extraction: JC, MV, JV, WC, YT, SH, GL. Analysis and interpretation: MV, SH. Manuscript writing: MV. Critical revisions for important intellectual content: MV, XL. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Xianhong Li .

Ethics declarations

Ethics approval and consent to participate.

No ethical approval was required for this study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Differences between the original protocol

We modified criteria for the included studies: we included published conference abstracts/proceedings, which form a relatively broad knowledge base in scientific knowledge. We originally planned to conduct a survey with open-ended questions followed by a face-to-face meeting to discuss the preliminary results of the review. However, to avoid extra burden in nurses due to COVID-19, we decided to limit the validation process to the online discussion only.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, supplementary material 6, supplementary material 7, supplementary material 8, supplementary material 9, supplementary material 10, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Välimäki, M., Hu, S., Lantta, T. et al. The impact of evidence-based nursing leadership in healthcare settings: a mixed methods systematic review. BMC Nurs 23 , 452 (2024). https://doi.org/10.1186/s12912-024-02096-4

Download citation

Received : 28 April 2023

Accepted : 13 June 2024

Published : 03 July 2024

DOI : https://doi.org/10.1186/s12912-024-02096-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Evidence-based leadership
Health services administration
Organizational development
Quality in healthcare

BMC Nursing

ISSN: 1472-6955

General enquiries: [email protected]

To read this content please select one of the options below:

Please note you do not have access to teaching notes, a contemporary systematic literature review of equestrian tourism: emerging advancements and future insights.

Journal of Hospitality and Tourism Insights

ISSN : 2514-9792

Article publication date: 2 July 2024

Horse-based tourism stands at the intersection of cultural heritage, leisure activities, and eco-friendly travel, captivating enthusiasts and researchers alike with its diverse facets and impacts. This study examines the horse-based tourism literature to provide an overview of horse-based tourism publications.

Design/methodology/approach

Using a systematic literature review (SLR) method, pertinent journal articles published over the past 3 decades were retrieved and analyzed. Based on the review process, 44 papers were identified and analyzed by publication year, journal distribution, research method, and lead author. Using Leximancer software, a thematic analysis was undertaken to determine the major themes of horse-based tourism.

The findings revealed a rising trend of horse-based tourism articles and the appearance of an increasing number of studies in tourism-oriented journals. In addition, it was discovered that the majority of available studies are qualitative, whereas quantitative research is few and limited.

Research limitations/implications

Our research establishes a foundational resource for future studies and scholarly discourse on the multifaceted contributions of horse-based tourism.

Practical implications

This study can assist decision-makers in understanding the potential of horse-based tourism in the sustainable development of destinations. Moreover, it provides clear direction on implementing appropriate strategies to manage horse-based tourism.

Originality/value

This study distinguishes itself as the inaugural comprehensive literature review encompassing the breadth of horse-based tourism publications and research domains. By pioneering this endeavor, we not only contribute a unique perspective to the existing body of knowledge in the field but also emphasize the vital role of horse-based tourism in fostering economic and social sustainability for the countries involved.

Horse-based tourism
Equestrian tourism
Systematic literature review
Research domains
Thematic analysis

Rezapouraghdam, H. , Saydam, M.B. , Altun, O. , Roudi, S. and Nosrati, S. (2024), "A contemporary systematic literature review of equestrian tourism: emerging advancements and future insights", Journal of Hospitality and Tourism Insights , Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/JHTI-01-2024-0046

Emerald Publishing Limited

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Advanced Search

Shedding light on the dark side – A systematic literature review of the issues in agile software development methodology use

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options, recommendations, software process improvement in agile software development a systematic literature review.

It is recognized the relevance and importance that software process improvement (SPI) and agile development have gained in the field of software engineering. Both are approaches that increase the efficiency and effectiveness of a software development ...

Systematic literature review on agile practices in global software development

Developing software in distributed development environments exhibits coordination, control and communication challenges. Agile practices, which demand frequent communication and self-organization between remote sites, are increasingly ...

Challenges in Combining Agile Development and CMMI: A Systematic Literature Review

Recently, Agile Development has emerged as an alternative approach in software engineering. The Agile Software Development (ASD) process provides the ability to cope with ever-changing requirements. On the other hand, the Capability Maturity Model ...

Information

Published in.

Elsevier Science Inc.

United States

Publication History

Author tags.

Agile software development methodologies
Issues in ASD
Dark side of ASD
Information systems development agility
Systematic Literature Review
Research-article

Contributors

Other metrics, bibliometrics, article metrics.

0 Total Citations
0 Total Downloads
Downloads (Last 12 months) 0
Downloads (Last 6 weeks) 0

View options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
Download citation
Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

IMAGES

Systematic Literature Review Methodology
Systematic literature review methodology
Research Methodology of Systematic Literature Review
The methodology of the systematic literature review. Four phases of the
Systematic literature review phases.
example of systematic review methodology

VIDEO

CONDUCTING SYSTEMATIC LITERATURE REVIEW
Literature Review, Systematic Literature Review, Meta
Systematic Literature Review Part2 March 20, 2023 Joseph Ntayi
Introduction Systematic Literature Review-Various frameworks Bibliometric Analysis
Systematic Literature Review
Mastering the Art of Systematic Literature Review and Meta-Analysis

COMMENTS

An overview of methodological approaches in systematic reviews
1. INTRODUCTION. Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the "gold standard" of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search ...
How-to conduct a systematic literature review: A quick guide for
Method details Overview. A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure [12].An SLR updates the reader with current literature about a subject [6].The goal is to review critical points of current knowledge on a ...
Methodological Approaches to Literature Review
This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis.
Guidance on Conducting a Systematic Literature Review
Literature reviews establish the foundation of academic inquires. However, in the planning field, we lack rigorous systematic reviews. In this article, through a systematic search on the methodology of literature review, we categorize a typology of literature reviews, discuss steps in conducting a systematic literature review, and provide suggestions on how to enhance rigor in literature ...
Systematic Review
A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review. In 2008, Dr. Robert Boyle and his colleagues published a systematic review in ...
Literature review as a research methodology: An ...
This paper discusses literature review as a methodology for conducting research and offers an overview of different types of reviews, as well as some guidelines to how to both conduct and evaluate a literature review paper. ... While the methodology for systematic reviews is straightforward and follows highly strict rules and standards ...
How to Do a Systematic Review: A Best Practice Guide for ...
Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question.
How to do a systematic review
A systematic review aims to bring evidence together to answer a pre-defined research question. This involves the identification of all primary research relevant to the defined review question, the critical appraisal of this research, and the synthesis of the findings.13 Systematic reviews may combine data from different.
Methodological guidance for the conduct of mixed methods systematic reviews
Objective: The objective of this paper is to outline the updated methodological approach for conducting a JBI mixed methods systematic review with a focus on data synthesis; specifically, methods related to how data are combined and the overall integration of the quantitative and qualitative evidence. Introduction: Mixed methods systematic reviews provide a more complete basis for complex ...
Systematic reviews: Brief overview of methods, limitations, and
CONCLUSION. Siddaway 16 noted that, "The best reviews synthesize studies to draw broad theoretical conclusions about what the literature means, linking theory to evidence and evidence to theory" (p. 747). To that end, high quality systematic reviews are explicit, rigorous, and reproducible. It is these three criteria that should guide authors seeking to write a systematic review or editors ...
Methodological Review
A typology of literature reviews. A methodological review is a type of systematic secondary research (i.e., research synthesis) which focuses on summarising the state-of-the-art methodological practices of research in a substantive field or topic" (Chong et al, 2021).
PDF Systematic Literature Reviews: an Introduction
Cite this article: Lame, G. (2019) 'Systematic Literature Reviews: An Introduction', in Proceedings of the 22nd International Conference on Engineering Design (ICED19), Delft, The Netherlands, 5-8 August 2019. DOI:10.1017/ ... systematic methods to reduce bias in the selection and inclusion of studies, to appraise the quality of the ...
The ABC of systematic literature review: the basic methodological
There is a need for more methodological-based articles on systematic literature review (SLR) for non-health researchers to address issues related to the lack of methodological references in SLR and less suitability of existing methodological guidance. With that, this study presented a beginner's guide to basic methodological guides and key points to perform SLR, especially for those from non ...
(PDF) Systematic Literature Reviews: An Introduction
Cite this article: Lame, G. (2019) 'Systematic Literature Reviews: An Introduction', in Pr oceedings of the 22nd International Conference on Engineering Design (ICED19), Delft, The Netherlands ...
Guidance to best tools and practices for systematic reviews
Methods and guidance to produce a reliable evidence synthesis. Several international consortiums of EBM experts and national health care organizations currently provide detailed guidance (Table (Table1). 1).They draw criteria from the reporting and methodological standards of currently recommended appraisal tools, and regularly review and update their methods to reflect new information and ...
A methodological review of systematic literature reviews in higher
The methodological framework presented in this review is the outcome of an inductive analysis of methodological stages and steps (each methodological stage can contain numerous steps) of 160 systematic literature reviews in leading higher education journals, presenting a coherent methodological framework using consistent terminologies while ...
How to write the methods section of a systematic review
Keep it brief. The methods section should be succinct but include all the noteworthy information. This can be a difficult balance to achieve. A useful strategy is to aim for a brief description that signposts the reader to a separate section or sections of supporting information. This could include datasets, a flowchart to show what happened to ...
Types of Literature Reviews
Rapid review. Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research. Completeness of searching determined by time constraints. Time-limited formal quality assessment. Typically narrative and tabular.
Methodology of a systematic review
A systematic review involves a critical and reproducible summary of the results of the available publications on a particular topic or clinical question. To improve scientific writing, the methodology is shown in a structured manner to implement a systematic review. Methodology of a systematic review ... broad literature search and (iv ...
What are Literature Reviews?
Relies on diverse sources of data (e.g. empirical, theoretical or methodological literature; qualitative or quantitative studies) Systematic Review. Systematically and transparently collects and categorize existing evidence on a question of scientific, policy or management importance; Follows a research protocol that is established a priori
How-to conduct a systematic literature review: A quick guide for
Overview. A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure .An SLR updates the reader with current literature about a subject .The goal is to review critical points of current knowledge on a topic about research ...
Unravelling the complexity of ventilator-associated pneumonia: a
Unravelling the complexity of ventilator-associated pneumonia: a systematic methodological literature review of diagnostic criteria and definitions used in clinical research. Markus Fally 1, Faiuna Haseeb 2,3, Ahmed Kouta 2,3, Jan Hansel 3,4, Rebecca C. Robey 2,3, Thomas Williams 5, Tobias Welte 6, Timothy Felton 2,3,5 & … Alexander G ...
Comparative efficacy and safety of bimekizumab in psoriatic arthritis
A systematic literature review (most recent update conducted on 1 January 2023) identified randomized controlled trials (RCTs) of b/tsDMARDs in PsA. Bayesian NMAs were conducted for efficacy outcomes at Weeks 12-24 for b/tsDMARD-naïve and TNF inhibitor (TNFi)-experienced patients. Safety at Weeks 12-24 was analysed in a mixed population.
Method for conducting systematic literature review and meta-analysis
Method details: the six basic steps Protocol - SLR methodology step 1. The need for a research protocol for SLR is for the consideration of transparency, transferability, and replicability of the work, which are the characteristics that make a literature review systematic [12].This helps to minimize the bias by conducting exhaustive literature searches.
Systematic and other reviews: criteria and complexities
A systematic review follows explicit methodology to answer a well-defined research question by searching the literature comprehensively, evaluating the quantity and quality of research evidence rigorously, and analyzing the evidence to synthesize an answer to the research question. The evidence gathered in systematic reviews can be qualitative ...
TCCM Framework-Based Review of the Studies
The article was compiled based on the Scientific Procedures and Rationales for Systematic Literature Reviews (SPAR‐4‐SLR) protocol and the theory-context-characteristics-methodology (TCCM) framework. Bibliometric analysis has identified the leading journals, authors, nations, articles, and themes. A conceptual model has been designed to ...
The impact of evidence-based nursing leadership in healthcare settings
The central component in impactful healthcare decisions is evidence. Understanding how nurse leaders use evidence in their own managerial decision making is still limited. This mixed methods systematic review aimed to examine how evidence is used to solve leadership problems and to describe the measured and perceived effects of evidence-based leadership on nurse leaders and their performance ...
A contemporary systematic literature review of equestrian tourism
Design/methodology/approach. Using a systematic literature review (SLR) method, pertinent journal articles published over the past 3 decades were retrieved and analyzed. Based on the review process, 44 papers were identified and analyzed by publication year, journal distribution, research method, and lead author.
Methodological choices in applications of Q methodology: A systematic
A systematic review of the literature can serve as a frame of reference for (new) users of a methodology by providing insight into what is common in the literarure regarding such methodological choices. In addition, a systematic review can demonstrate the current state of research, identify gaps in evidence and, hence, provide a research agenda.
Shedding light on the dark side
The systematization shows what aspects constitute the dark side of ASD, emphasizing its multidimensional nature along issues such as reduced developer well-being, product quality and development productivity. The analysis of how its complexity is defined reveals that customer misbehavior and delivery pressure are significant origins of other issues.

Have a language expert improve your writing

Systematic Review | Definition, Example & Guide

Table of contents

Prevent plagiarism. Run a free check.

Receive feedback on language, structure, and formatting

Step 1: Formulate a research question

Step 2: Develop a protocol

Step 3: Search for all relevant studies

Step 4: Apply the selection criteria

Step 5: Extract the data

Step 6: Synthesize the data

Step 7: Write and publish a report

Cite this Scribbr article

Is this article helpful?

Shaun Turney

Which review is that? A guide to review types

Methodological Review

Further Reading/Resources

How to write the methods section of a systematic review

Covidence breaks down how to write a methods section

Describe what happened

Keep it brief

Follow a structure

1. Selection criteria ⭕

2. Search 🕵🏾‍♀️

3. Data collection and analysis 👩‍💻

4. Study quality and risk of bias ⚖️

Laura Mellor. Portsmouth, UK

Top 5 Tips for High-Quality Systematic Review Data Extraction

How to get through study quality assessment Systematic Review

How to extract study data for your systematic review

Better systematic review management

Systematic Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

Save citation to file

Add to My Bibliography

Methodology of a systematic review

Similar articles

LinkOut - more resources

Other Literature Sources

Research Materials

Communicative Sciences and Disorders

Choosing a Review Type

Types of Evidence Synthesis / Literature Reviews

Common Types of Literature Reviews 1

Integrative Review

Systematic Review

Meta-Analysis

Scoping Review

Rapid Review

Umbrella Review

Further Reading: Exploring Different Types of Literature Reviews

Integrative Reviews

Scoping Reviews

Systematic vs. Scoping Reviews: What's the Difference?

Rapid Reviews

Unravelling the complexity of ventilator-associated pneumonia: a systematic methodological literature review of diagnostic criteria and definitions used in clinical research

Search methods

Selection criteria

Data collection and synthesis

Conclusions

Protocol and registration

Eligibility criteria

Information sources and search

Study selection

Data collection process

Synthesis of results

Risk of bias

Study selection and characteristics

Diagnostic criteria for VAP

Clinical signs and symptoms

Biochemistry criteria

Combinations of diagnostic criteria

Definition of “ventilator-associated”

Summary of evidence

Strengths and limitations

Availability of data and materials

Acknowledgements

Author information

Contributions