examples of a published literature review

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process.

What is the purpose of literature review?
a. Habitat Loss and Species Extinction:
b. Range Shifts and Phenological Changes:
c. Ocean Acidification and Coral Reefs:
d. Adaptive Strategies and Conservation Efforts:
How to write a good literature review
Choose a Topic and Define the Research Question:
Decide on the Scope of Your Review:
Select Databases for Searches:
Conduct Searches and Keep Track:
Review the Literature:
Organize and Write Your Literature Review:
Frequently asked questions

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

examples of a published literature review

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2

Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge.
Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field.
Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research.
Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered.
Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research.
Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature.

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic.

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies:

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements.

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources.

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems.

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning.

How to write a good literature review

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements.

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review.

Opening Statement: Begin with a general statement about the broader topic and its significance in the field.
Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address.
Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research.
Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic.
Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review.

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria.

Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement.
Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic.
Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature.
Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps.
Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review.

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research.

Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective.
Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field.
Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice.
Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field.
Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community.

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1

Choose a Topic and Define the Research Question:

Select a topic that is relevant to your field of study.
Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore?

Decide on the Scope of Your Review:

Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview?
Consider the geographical scope. Is your review global, or are you focusing on a specific region?
Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude?

Select Databases for Searches:

Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar.
Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic.

Conduct Searches and Keep Track:

Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques.
Record and document your search strategy for transparency and replicability.
Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references.

Review the Literature:

Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies.
Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research.
Summarize key findings and arguments from each source. Compare and contrast different perspectives.
Identify areas where there is a consensus in the literature and where there are conflicting opinions.
Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research?

Organize and Write Your Literature Review:

Literature review outline should be based on themes, chronological order, or methodological approaches.
Write a clear and coherent narrative that synthesizes the information gathered.
Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.).
Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research.

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses.

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic.

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods.

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers. Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved. Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic. Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings. Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject. It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review: Introduction: Provide an overview of the topic. Define the scope and purpose of the literature review. State the research question or objective. Body: Organize the literature by themes, concepts, or chronology. Critically analyze and evaluate each source. Discuss the strengths and weaknesses of the studies. Highlight any methodological limitations or biases. Identify patterns, connections, or contradictions in the existing research. Conclusion: Summarize the key points discussed in the literature review. Highlight the research gap. Address the research question or objective stated in the introduction. Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows:

References

Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234.
Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis.
Cantero, C. (2019). How to write a literature review.  San José State University Writing Center .

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai, what are scholarly sources and where can you..., how to write a hypothesis types and examples , measuring academic success: definition & strategies for excellence, what is academic writing: tips for students, why traditional editorial process needs an upgrade.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

An introductory paragraph that explains what your working topic and thesis is
A forecast of key topics or texts that will appear in the review
Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
Critically Evaluate: Mention the strengths and weaknesses of your sources
Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

Summarize the key findings you have taken from the literature and emphasize their significance
Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
Qualitative versus quantitative research
Empirical versus theoretical scholarship
Divide the research by sociological, historical, or cultural sources
Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Literature Review Guide: Examples of Literature Reviews

What is a Literature Review?
How to start?
Search strategies and Databases
Examples of Literature Reviews
How to organise the review
Library summary
Emerald Infographic

All good quality journal articles will include a small Literature Review after the Introduction paragraph. It may not be called a Literature Review but gives you an idea of how one is created in miniature.

Sample Literature Reviews as part of a articles or Theses

Sample Literature Review on Critical Thinking (Gwendolyn Reece, American University Library)
Hackett, G and Melia, D . The hotel as the holiday/stay destination:trends and innovations. Presented at TRIC Conference, Belfast, Ireland- June 2012 and EuroCHRIE Conference

Links to sample Literature Reviews from other libraries

Sample literature reviews from University of West Florida

Standalone Literature Reviews

Attitudes towards the Disability in Ireland
Martin, A., O'Connor-Fenelon, M. and Lyons, R. (2010). Non-verbal communication between nurses and people with an intellectual disability: A review of the literature. Journal of Intellectual Diabilities, 14(4), 303-314.

Irish Theses

Phillips, Martin (2015) European airline performance: a data envelopment analysis with extrapolations based on model outputs. Master of Business Studies thesis, Dublin City University.
The customers’ perception of servicescape’s influence on their behaviours, in the food retail industry : Dublin Business School 2015
Coughlan, Ray (2015) What was the role of leadership in the transformation of a failing Irish Insurance business. Masters thesis, Dublin, National College of Ireland.
<< Previous: Search strategies and Databases
Next: Tutorials >>
Last Updated: Feb 27, 2024 4:07 PM
URL: https://ait.libguides.com/literaturereview

Literature Review Example/Sample

Detailed Walkthrough + Free Literature Review Template

If you’re working on a dissertation or thesis and are looking for an example of a strong literature review chapter , you’ve come to the right place.

In this video, we walk you through an A-grade literature review from a dissertation that earned full distinction . We start off by discussing the five core sections of a literature review chapter by unpacking our free literature review template . This includes:

The literature review opening/ introduction section
The theoretical framework (or foundation of theory)
The empirical research
The research gap
The closing section

We then progress to the sample literature review (from an A-grade Master’s-level dissertation) to show how these concepts are applied in the literature review chapter. You can access the free resources mentioned in this video below.

PS – If you’re working on a dissertation, be sure to also check out our collection of dissertation and thesis examples here .

FAQ: Literature Review Example

Literature review example: frequently asked questions, is the sample literature review real.

Yes. The literature review example is an extract from a Master’s-level dissertation for an MBA program. It has not been edited in any way.

Can I replicate this literature review for my dissertation?

As we discuss in the video, every literature review will be slightly different, depending on the university’s unique requirements, as well as the nature of the research itself. Therefore, you’ll need to tailor your literature review to suit your specific context.

You can learn more about the basics of writing a literature review here .

Where can I find more examples of literature reviews?

The best place to find more examples of literature review chapters would be within dissertation/thesis databases. These databases include dissertations, theses and research projects that have successfully passed the assessment criteria for the respective university, meaning that you have at least some sort of quality assurance.

The Open Access Thesis Database (OATD) is a good starting point.

How do I get the literature review template?

You can access our free literature review chapter template here .

Is the template really free?

Yes. There is no cost for the template and you are free to use it as you wish.

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

Example of two research proposals (Masters and PhD-level)

What will it take for you to guide me in my Ph.D research work?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Print Friendly

Reference management. Clean and simple.

Literature review

How to write a literature review in 6 steps

How do you write a good literature review? This step-by-step guide on how to write an excellent literature review covers all aspects of planning and writing literature reviews for academic papers and theses.

How to write a systematic literature review [9 steps]

How do you write a systematic literature review? What types of systematic literature reviews exist and where do you use them? Learn everything you need to know about a systematic literature review in this guide

What is a literature review? [with examples]

Not sure what a literature review is? This guide covers the definition, purpose, and format of a literature review.

15 Literature Review Examples

literature review examples, types, and definition, explained below

Literature reviews are a necessary step in a research process and often required when writing your research proposal . They involve gathering, analyzing, and evaluating existing knowledge about a topic in order to find gaps in the literature where future studies will be needed.

Ideally, once you have completed your literature review, you will be able to identify how your research project can build upon and extend existing knowledge in your area of study.

Generally, for my undergraduate research students, I recommend a narrative review, where themes can be generated in order for the students to develop sufficient understanding of the topic so they can build upon the themes using unique methods or novel research questions.

If you’re in the process of writing a literature review, I have developed a literature review template for you to use – it’s a huge time-saver and walks you through how to write a literature review step-by-step:

Get your time-saving templates here to write your own literature review.

Literature Review Examples

For the following types of literature review, I present an explanation and overview of the type, followed by links to some real-life literature reviews on the topics.

1. Narrative Review Examples

Also known as a traditional literature review, the narrative review provides a broad overview of the studies done on a particular topic.

It often includes both qualitative and quantitative studies and may cover a wide range of years.

The narrative review’s purpose is to identify commonalities, gaps, and contradictions in the literature .

I recommend to my students that they should gather their studies together, take notes on each study, then try to group them by themes that form the basis for the review (see my step-by-step instructions at the end of the article).

Example Study

Title: Communication in healthcare: a narrative review of the literature and practical recommendations

Citation: Vermeir, P., Vandijck, D., Degroote, S., Peleman, R., Verhaeghe, R., Mortier, E., … & Vogelaers, D. (2015). Communication in healthcare: a narrative review of the literature and practical recommendations. International journal of clinical practice , 69 (11), 1257-1267.

Source: https://onlinelibrary.wiley.com/doi/pdf/10.1111/ijcp.12686

Overview: This narrative review analyzed themes emerging from 69 articles about communication in healthcare contexts. Five key themes were found in the literature: poor communication can lead to various negative outcomes, discontinuity of care, compromise of patient safety, patient dissatisfaction, and inefficient use of resources. After presenting the key themes, the authors recommend that practitioners need to approach healthcare communication in a more structured way, such as by ensuring there is a clear understanding of who is in charge of ensuring effective communication in clinical settings.

Other Examples

Burnout in United States Healthcare Professionals: A Narrative Review (Reith, 2018) – read here
Examining the Presence, Consequences, and Reduction of Implicit Bias in Health Care: A Narrative Review (Zestcott, Blair & Stone, 2016) – read here
A Narrative Review of School-Based Physical Activity for Enhancing Cognition and Learning (Mavilidi et al., 2018) – read here
A narrative review on burnout experienced by medical students and residents (Dyrbye & Shanafelt, 2015) – read here

2. Systematic Review Examples

This type of literature review is more structured and rigorous than a narrative review. It involves a detailed and comprehensive plan and search strategy derived from a set of specified research questions.

The key way you’d know a systematic review compared to a narrative review is in the methodology: the systematic review will likely have a very clear criteria for how the studies were collected, and clear explanations of exclusion/inclusion criteria.

The goal is to gather the maximum amount of valid literature on the topic, filter out invalid or low-quality reviews, and minimize bias. Ideally, this will provide more reliable findings, leading to higher-quality conclusions and recommendations for further research.

You may note from the examples below that the ‘method’ sections in systematic reviews tend to be much more explicit, often noting rigid inclusion/exclusion criteria and exact keywords used in searches.

Title: The importance of food naturalness for consumers: Results of a systematic review

Citation: Roman, S., Sánchez-Siles, L. M., & Siegrist, M. (2017). The importance of food naturalness for consumers: Results of a systematic review. Trends in food science & technology , 67 , 44-57.

Source: https://www.sciencedirect.com/science/article/pii/S092422441730122X

Overview: This systematic review included 72 studies of food naturalness to explore trends in the literature about its importance for consumers. Keywords used in the data search included: food, naturalness, natural content, and natural ingredients. Studies were included if they examined consumers’ preference for food naturalness and contained empirical data. The authors found that the literature lacks clarity about how naturalness is defined and measured, but also found that food consumption is significantly influenced by perceived naturalness of goods.

A systematic review of research on online teaching and learning from 2009 to 2018 (Martin, Sun & Westine, 2020) – read here
Where Is Current Research on Blockchain Technology? (Yli-Huumo et al., 2016) – read here
Universities—industry collaboration: A systematic review (Ankrah & Al-Tabbaa, 2015) – read here
Internet of Things Applications: A Systematic Review (Asghari, Rahmani & Javadi, 2019) – read here

3. Meta-analysis

This is a type of systematic review that uses statistical methods to combine and summarize the results of several studies.

Due to its robust methodology, a meta-analysis is often considered the ‘gold standard’ of secondary research , as it provides a more precise estimate of a treatment effect than any individual study contributing to the pooled analysis.

Furthermore, by aggregating data from a range of studies, a meta-analysis can identify patterns, disagreements, or other interesting relationships that may have been hidden in individual studies.

This helps to enhance the generalizability of findings, making the conclusions drawn from a meta-analysis particularly powerful and informative for policy and practice.

Title: Cholesterol and Alzheimer’s Disease Risk: A Meta-Meta-Analysis

Citation: Sáiz-Vazquez, O., Puente-Martínez, A., Ubillos-Landa, S., Pacheco-Bonrostro, J., & Santabárbara, J. (2020). Cholesterol and Alzheimer’s disease risk: a meta-meta-analysis. Brain sciences, 10(6), 386.

Source: https://doi.org/10.3390/brainsci10060386

O verview: This study examines the relationship between cholesterol and Alzheimer’s disease (AD). Researchers conducted a systematic search of meta-analyses and reviewed several databases, collecting 100 primary studies and five meta-analyses to analyze the connection between cholesterol and Alzheimer’s disease. They find that the literature compellingly demonstrates that low-density lipoprotein cholesterol (LDL-C) levels significantly influence the development of Alzheimer’s disease.

The power of feedback revisited: A meta-analysis of educational feedback research (Wisniewski, Zierer & Hattie, 2020) – read here
How Much Does Education Improve Intelligence? A Meta-Analysis (Ritchie & Tucker-Drob, 2018) – read here
A meta-analysis of factors related to recycling (Geiger et al., 2019) – read here
Stress management interventions for police officers and recruits (Patterson, Chung & Swan, 2014) – read here

Other Types of Reviews

Scoping Review: This type of review is used to map the key concepts underpinning a research area and the main sources and types of evidence available. It can be undertaken as stand-alone projects in their own right, or as a precursor to a systematic review.
Rapid Review: This type of review accelerates the systematic review process in order to produce information in a timely manner. This is achieved by simplifying or omitting stages of the systematic review process.
Integrative Review: This review method is more inclusive than others, allowing for the simultaneous inclusion of experimental and non-experimental research. The goal is to more comprehensively understand a particular phenomenon.
Critical Review: This is similar to a narrative review but requires a robust understanding of both the subject and the existing literature. In a critical review, the reviewer not only summarizes the existing literature, but also evaluates its strengths and weaknesses. This is common in the social sciences and humanities .
State-of-the-Art Review: This considers the current level of advancement in a field or topic and makes recommendations for future research directions. This type of review is common in technological and scientific fields but can be applied to any discipline.

How to Write a Narrative Review (Tips for Undergrad Students)

Most undergraduate students conducting a capstone research project will be writing narrative reviews. Below is a five-step process for conducting a simple review of the literature for your project.

Search for Relevant Literature: Use scholarly databases related to your field of study, provided by your university library, along with appropriate search terms to identify key scholarly articles that have been published on your topic.
Evaluate and Select Sources: Filter the source list by selecting studies that are directly relevant and of sufficient quality, considering factors like credibility , objectivity, accuracy, and validity.
Analyze and Synthesize: Review each source and summarize the main arguments in one paragraph (or more, for postgrad). Keep these summaries in a table.
Identify Themes: With all studies summarized, group studies that share common themes, such as studies that have similar findings or methodologies.
Write the Review: Write your review based upon the themes or subtopics you have identified. Give a thorough overview of each theme, integrating source data, and conclude with a summary of the current state of knowledge then suggestions for future research based upon your evaluation of what is lacking in the literature.

Literature reviews don’t have to be as scary as they seem. Yes, they are difficult and require a strong degree of comprehension of academic studies. But it can be feasibly done through following a structured approach to data collection and analysis. With my undergraduate research students (who tend to conduct small-scale qualitative studies ), I encourage them to conduct a narrative literature review whereby they can identify key themes in the literature. Within each theme, students can critique key studies and their strengths and limitations , in order to get a lay of the land and come to a point where they can identify ways to contribute new insights to the existing academic conversation on their topic.

Ankrah, S., & Omar, A. T. (2015). Universities–industry collaboration: A systematic review. Scandinavian Journal of Management, 31(3), 387-408.

Asghari, P., Rahmani, A. M., & Javadi, H. H. S. (2019). Internet of Things applications: A systematic review. Computer Networks , 148 , 241-261.

Dyrbye, L., & Shanafelt, T. (2016). A narrative review on burnout experienced by medical students and residents. Medical education , 50 (1), 132-149.

Geiger, J. L., Steg, L., Van Der Werff, E., & Ünal, A. B. (2019). A meta-analysis of factors related to recycling. Journal of environmental psychology , 64 , 78-97.

Martin, F., Sun, T., & Westine, C. D. (2020). A systematic review of research on online teaching and learning from 2009 to 2018. Computers & education , 159 , 104009.

Mavilidi, M. F., Ruiter, M., Schmidt, M., Okely, A. D., Loyens, S., Chandler, P., & Paas, F. (2018). A narrative review of school-based physical activity for enhancing cognition and learning: The importance of relevancy and integration. Frontiers in psychology , 2079.

Patterson, G. T., Chung, I. W., & Swan, P. W. (2014). Stress management interventions for police officers and recruits: A meta-analysis. Journal of experimental criminology , 10 , 487-513.

Reith, T. P. (2018). Burnout in United States healthcare professionals: a narrative review. Cureus , 10 (12).

Ritchie, S. J., & Tucker-Drob, E. M. (2018). How much does education improve intelligence? A meta-analysis. Psychological science , 29 (8), 1358-1369.

Roman, S., Sánchez-Siles, L. M., & Siegrist, M. (2017). The importance of food naturalness for consumers: Results of a systematic review. Trends in food science & technology , 67 , 44-57.

Sáiz-Vazquez, O., Puente-Martínez, A., Ubillos-Landa, S., Pacheco-Bonrostro, J., & Santabárbara, J. (2020). Cholesterol and Alzheimer’s disease risk: a meta-meta-analysis. Brain sciences, 10(6), 386.

Vermeir, P., Vandijck, D., Degroote, S., Peleman, R., Verhaeghe, R., Mortier, E., … & Vogelaers, D. (2015). Communication in healthcare: a narrative review of the literature and practical recommendations. International journal of clinical practice , 69 (11), 1257-1267.

Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology , 10 , 3087.

Yli-Huumo, J., Ko, D., Choi, S., Park, S., & Smolander, K. (2016). Where is current research on blockchain technology?—a systematic review. PloS one , 11 (10), e0163477.

Zestcott, C. A., Blair, I. V., & Stone, J. (2016). Examining the presence, consequences, and reduction of implicit bias in health care: a narrative review. Group Processes & Intergroup Relations , 19 (4), 528-542

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ Social-Emotional Learning (Definition, Examples, Pros & Cons)
Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ What is Educational Psychology?
Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ What is IQ? (Intelligence Quotient)
Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 5 Top Tips for Succeeding at University

Literature Review: What is a Literature Review?

Sample Searches
Examples of Published Literature Reviews
Researching Your Topic
Subject Searching
Google Scholar
Track Your Work
Citation Managers This link opens in a new window
Citation Guides This link opens in a new window
Tips on Writing Your Literature Review This link opens in a new window
Research Help

Ask a Librarian

Chat with a Librarian

Lisle: (630) 829-6057 Mesa: (480) 878-7514 Toll Free: (877) 575-6050 Email: [email protected]

Book a Research Consultation Library Hours

Not Just for Graduate Students!

A literature review is an in-depth critical analysis of published scholarly research related to a specific topic. Published scholarly research (the "literature") may include journal articles, books, book chapters, dissertations and thesis, or conference proceedings.

A solid lit review must:

be organized around and related directly to the thesis or research question you're developing
synthesize results into a summary of what is and is not known
identify areas of controversy in the literature
formulate questions that need further research

Why Conduct a Literature Review?

to distinguish what has been done from what needs to be done
to discover important variables relevant to the topic
to synthesize and gain new perspective
to identify relationships between ideas and practices
to establish the context of the topic
to rationalize the significance of the problem
to enhance and acquire subject vocabulary
to understand the structure of the subject
tp relate ideas and theory to applications
to identify main methodologies and research techniques that have been used
to place research in a historical context to show familiarity with state-of-art development

Questions to Consider

What is the overarching question or problem your literature review seeks to address?
How much familiarity do you already have with the field? Are you already familiar with common methodologies or professional vocabularies?
What types of strategies or questions have others in your field pursued?
How will you synthesize or summarize the information you gather?
What do you or others perceive to be lacking in your field?
Is your topic broad? How could it be narrowed?
Can you articulate why your topic is important in your field?
Next: Sample Searches >>
Last Updated: Apr 25, 2024 3:34 PM
URL: https://researchguides.ben.edu/lit-review

Kindlon Hall 5700 College Rd. Lisle, IL 60532 (630) 829-6050

Gillett Hall 225 E. Main St. Mesa, AZ 85201 (480) 878-7514

Study resources
Calendar - Graduate
Calendar - Undergraduate
Class schedules
Class cancellations
Course registration
Important academic dates
More academic resources
Campus services
IT services
Job opportunities
Mental health support
Student Service Centre (Birks)
Calendar of events
Latest news
Media Relations
Faculties, Schools & Colleges
Arts and Science
Gina Cody School of Engineering and Computer Science
John Molson School of Business
School of Graduate Studies
All Schools, Colleges & Departments
Directories

My Library Account (Sofia) View checkouts, fees, place requests and more
Interlibrary Loans Request books from external libraries
Zotero Manage your citations and create bibliographies
E-journals via BrowZine Browse & read journals through a friendly interface
Article/Chapter Scan & Deliver Request a PDF of an article/chapter we have in our physical collection
Course Reserves Online course readings
Spectrum Deposit a thesis or article
WebPrint Upload documents to print with DPrint
Sofia Discovery tool
Databases by subject
Course Reserves
E-journals via BrowZine
E-journals via Sofia
Article/Chapter Scan & Deliver
Intercampus Delivery of Bound Periodicals/Microforms
Interlibrary Loans
Spectrum Research Repository
Special Collections
Additional resources & services
Loans & Returns (Circulation)
Subject & course guides
Open Educational Resources Guide
General guides for users
Evaluating...
Ask a librarian
Research Skills Tutorial
Quick Things for Digital Knowledge
Bibliometrics & research impact guide
Concordia University Press
Copyright Guide
Copyright Guide for Thesis Preparation
Digital Scholarship
Digital Preservation
Open Access
ORCID at Concordia
Research data management guide
Scholarship of Teaching & Learning
Systematic Reviews
How to get published speaker series
Borrow (laptops, tablets, equipment)
Connect (netname, Wi-Fi, guest accounts)
Desktop computers, software & availability maps
Group study, presentation practice & classrooms
Printers, copiers & scanners
Technology Sandbox
Visualization Studio
Webster Library
Vanier Library
Grey Nuns Reading Room
Book a group study room/scanner
Study spaces
Floor plans
Room booking for academic events
Exhibitions
Librarians & staff
University Librarian
Memberships & collaborations
Indigenous Student Librarian program
Wikipedian in residence
Researcher-in-Residence
Feedback & improvement
Annual reports & fast facts
Annual Plan
Library Services Fund
Giving to the Library
Webster Transformation blog
Policies & Code of Conduct

The Campaign for Concordia

Library Research Skills Tutorial

Log into...

My Library account (Sofia)
Interlibrary loans
Article/chapter scan
Course reserves

Quick links

How to write a literature review

Examples of a published literature review.

Literature reviews are often published as scholarly articles, books, and reports. Here is an example of a recent literature review published as a scholarly journal article:

Ledesma, M. C., & Calderón, D. (2015). Critical race theory in education: A review of past literature and a look to the future. Qualitative Inquiry, 21(3), 206-222. Link to the article

Home » Literature Review – Types Writing Guide and Examples

Literature Review – Types Writing Guide and Examples

Table of Contents

Literature Review

Definition:

A literature review is a comprehensive and critical analysis of the existing literature on a particular topic or research question. It involves identifying, evaluating, and synthesizing relevant literature, including scholarly articles, books, and other sources, to provide a summary and critical assessment of what is known about the topic.

Types of Literature Review

Types of Literature Review are as follows:

Narrative literature review : This type of review involves a comprehensive summary and critical analysis of the available literature on a particular topic or research question. It is often used as an introductory section of a research paper.
Systematic literature review: This is a rigorous and structured review that follows a pre-defined protocol to identify, evaluate, and synthesize all relevant studies on a specific research question. It is often used in evidence-based practice and systematic reviews.
Meta-analysis: This is a quantitative review that uses statistical methods to combine data from multiple studies to derive a summary effect size. It provides a more precise estimate of the overall effect than any individual study.
Scoping review: This is a preliminary review that aims to map the existing literature on a broad topic area to identify research gaps and areas for further investigation.
Critical literature review : This type of review evaluates the strengths and weaknesses of the existing literature on a particular topic or research question. It aims to provide a critical analysis of the literature and identify areas where further research is needed.
Conceptual literature review: This review synthesizes and integrates theories and concepts from multiple sources to provide a new perspective on a particular topic. It aims to provide a theoretical framework for understanding a particular research question.
Rapid literature review: This is a quick review that provides a snapshot of the current state of knowledge on a specific research question or topic. It is often used when time and resources are limited.
Thematic literature review : This review identifies and analyzes common themes and patterns across a body of literature on a particular topic. It aims to provide a comprehensive overview of the literature and identify key themes and concepts.
Realist literature review: This review is often used in social science research and aims to identify how and why certain interventions work in certain contexts. It takes into account the context and complexities of real-world situations.
State-of-the-art literature review : This type of review provides an overview of the current state of knowledge in a particular field, highlighting the most recent and relevant research. It is often used in fields where knowledge is rapidly evolving, such as technology or medicine.
Integrative literature review: This type of review synthesizes and integrates findings from multiple studies on a particular topic to identify patterns, themes, and gaps in the literature. It aims to provide a comprehensive understanding of the current state of knowledge on a particular topic.
Umbrella literature review : This review is used to provide a broad overview of a large and diverse body of literature on a particular topic. It aims to identify common themes and patterns across different areas of research.
Historical literature review: This type of review examines the historical development of research on a particular topic or research question. It aims to provide a historical context for understanding the current state of knowledge on a particular topic.
Problem-oriented literature review : This review focuses on a specific problem or issue and examines the literature to identify potential solutions or interventions. It aims to provide practical recommendations for addressing a particular problem or issue.
Mixed-methods literature review : This type of review combines quantitative and qualitative methods to synthesize and analyze the available literature on a particular topic. It aims to provide a more comprehensive understanding of the research question by combining different types of evidence.

Parts of Literature Review

Parts of a literature review are as follows:

Introduction

The introduction of a literature review typically provides background information on the research topic and why it is important. It outlines the objectives of the review, the research question or hypothesis, and the scope of the review.

Literature Search

This section outlines the search strategy and databases used to identify relevant literature. The search terms used, inclusion and exclusion criteria, and any limitations of the search are described.

Literature Analysis

The literature analysis is the main body of the literature review. This section summarizes and synthesizes the literature that is relevant to the research question or hypothesis. The review should be organized thematically, chronologically, or by methodology, depending on the research objectives.

Critical Evaluation

Critical evaluation involves assessing the quality and validity of the literature. This includes evaluating the reliability and validity of the studies reviewed, the methodology used, and the strength of the evidence.

The conclusion of the literature review should summarize the main findings, identify any gaps in the literature, and suggest areas for future research. It should also reiterate the importance of the research question or hypothesis and the contribution of the literature review to the overall research project.

The references list includes all the sources cited in the literature review, and follows a specific referencing style (e.g., APA, MLA, Harvard).

How to write Literature Review

Here are some steps to follow when writing a literature review:

Define your research question or topic : Before starting your literature review, it is essential to define your research question or topic. This will help you identify relevant literature and determine the scope of your review.
Conduct a comprehensive search: Use databases and search engines to find relevant literature. Look for peer-reviewed articles, books, and other academic sources that are relevant to your research question or topic.
Evaluate the sources: Once you have found potential sources, evaluate them critically to determine their relevance, credibility, and quality. Look for recent publications, reputable authors, and reliable sources of data and evidence.
Organize your sources: Group the sources by theme, method, or research question. This will help you identify similarities and differences among the literature, and provide a structure for your literature review.
Analyze and synthesize the literature : Analyze each source in depth, identifying the key findings, methodologies, and conclusions. Then, synthesize the information from the sources, identifying patterns and themes in the literature.
Write the literature review : Start with an introduction that provides an overview of the topic and the purpose of the literature review. Then, organize the literature according to your chosen structure, and analyze and synthesize the sources. Finally, provide a conclusion that summarizes the key findings of the literature review, identifies gaps in knowledge, and suggests areas for future research.
Edit and proofread: Once you have written your literature review, edit and proofread it carefully to ensure that it is well-organized, clear, and concise.

Examples of Literature Review

Here’s an example of how a literature review can be conducted for a thesis on the topic of “ The Impact of Social Media on Teenagers’ Mental Health”:

Start by identifying the key terms related to your research topic. In this case, the key terms are “social media,” “teenagers,” and “mental health.”
Use academic databases like Google Scholar, JSTOR, or PubMed to search for relevant articles, books, and other publications. Use these keywords in your search to narrow down your results.
Evaluate the sources you find to determine if they are relevant to your research question. You may want to consider the publication date, author’s credentials, and the journal or book publisher.
Begin reading and taking notes on each source, paying attention to key findings, methodologies used, and any gaps in the research.
Organize your findings into themes or categories. For example, you might categorize your sources into those that examine the impact of social media on self-esteem, those that explore the effects of cyberbullying, and those that investigate the relationship between social media use and depression.
Synthesize your findings by summarizing the key themes and highlighting any gaps or inconsistencies in the research. Identify areas where further research is needed.
Use your literature review to inform your research questions and hypotheses for your thesis.

For example, after conducting a literature review on the impact of social media on teenagers’ mental health, a thesis might look like this:

“Using a mixed-methods approach, this study aims to investigate the relationship between social media use and mental health outcomes in teenagers. Specifically, the study will examine the effects of cyberbullying, social comparison, and excessive social media use on self-esteem, anxiety, and depression. Through an analysis of survey data and qualitative interviews with teenagers, the study will provide insight into the complex relationship between social media use and mental health outcomes, and identify strategies for promoting positive mental health outcomes in young people.”

Reference: Smith, J., Jones, M., & Lee, S. (2019). The effects of social media use on adolescent mental health: A systematic review. Journal of Adolescent Health, 65(2), 154-165. doi:10.1016/j.jadohealth.2019.03.024

Reference Example: Author, A. A., Author, B. B., & Author, C. C. (Year). Title of article. Title of Journal, volume number(issue number), page range. doi:0000000/000000000000 or URL

Applications of Literature Review

some applications of literature review in different fields:

Social Sciences: In social sciences, literature reviews are used to identify gaps in existing research, to develop research questions, and to provide a theoretical framework for research. Literature reviews are commonly used in fields such as sociology, psychology, anthropology, and political science.
Natural Sciences: In natural sciences, literature reviews are used to summarize and evaluate the current state of knowledge in a particular field or subfield. Literature reviews can help researchers identify areas where more research is needed and provide insights into the latest developments in a particular field. Fields such as biology, chemistry, and physics commonly use literature reviews.
Health Sciences: In health sciences, literature reviews are used to evaluate the effectiveness of treatments, identify best practices, and determine areas where more research is needed. Literature reviews are commonly used in fields such as medicine, nursing, and public health.
Humanities: In humanities, literature reviews are used to identify gaps in existing knowledge, develop new interpretations of texts or cultural artifacts, and provide a theoretical framework for research. Literature reviews are commonly used in fields such as history, literary studies, and philosophy.

Role of Literature Review in Research

Here are some applications of literature review in research:

Identifying Research Gaps : Literature review helps researchers identify gaps in existing research and literature related to their research question. This allows them to develop new research questions and hypotheses to fill those gaps.
Developing Theoretical Framework: Literature review helps researchers develop a theoretical framework for their research. By analyzing and synthesizing existing literature, researchers can identify the key concepts, theories, and models that are relevant to their research.
Selecting Research Methods : Literature review helps researchers select appropriate research methods and techniques based on previous research. It also helps researchers to identify potential biases or limitations of certain methods and techniques.
Data Collection and Analysis: Literature review helps researchers in data collection and analysis by providing a foundation for the development of data collection instruments and methods. It also helps researchers to identify relevant data sources and identify potential data analysis techniques.
Communicating Results: Literature review helps researchers to communicate their results effectively by providing a context for their research. It also helps to justify the significance of their findings in relation to existing research and literature.

Purpose of Literature Review

Some of the specific purposes of a literature review are as follows:

To provide context: A literature review helps to provide context for your research by situating it within the broader body of literature on the topic.
To identify gaps and inconsistencies: A literature review helps to identify areas where further research is needed or where there are inconsistencies in the existing literature.
To synthesize information: A literature review helps to synthesize the information from multiple sources and present a coherent and comprehensive picture of the current state of knowledge on the topic.
To identify key concepts and theories : A literature review helps to identify key concepts and theories that are relevant to your research question and provide a theoretical framework for your study.
To inform research design: A literature review can inform the design of your research study by identifying appropriate research methods, data sources, and research questions.

Characteristics of Literature Review

Some Characteristics of Literature Review are as follows:

Identifying gaps in knowledge: A literature review helps to identify gaps in the existing knowledge and research on a specific topic or research question. By analyzing and synthesizing the literature, you can identify areas where further research is needed and where new insights can be gained.
Establishing the significance of your research: A literature review helps to establish the significance of your own research by placing it in the context of existing research. By demonstrating the relevance of your research to the existing literature, you can establish its importance and value.
Informing research design and methodology : A literature review helps to inform research design and methodology by identifying the most appropriate research methods, techniques, and instruments. By reviewing the literature, you can identify the strengths and limitations of different research methods and techniques, and select the most appropriate ones for your own research.
Supporting arguments and claims: A literature review provides evidence to support arguments and claims made in academic writing. By citing and analyzing the literature, you can provide a solid foundation for your own arguments and claims.
I dentifying potential collaborators and mentors: A literature review can help identify potential collaborators and mentors by identifying researchers and practitioners who are working on related topics or using similar methods. By building relationships with these individuals, you can gain valuable insights and support for your own research and practice.
Keeping up-to-date with the latest research : A literature review helps to keep you up-to-date with the latest research on a specific topic or research question. By regularly reviewing the literature, you can stay informed about the latest findings and developments in your field.

Advantages of Literature Review

There are several advantages to conducting a literature review as part of a research project, including:

Establishing the significance of the research : A literature review helps to establish the significance of the research by demonstrating the gap or problem in the existing literature that the study aims to address.
Identifying key concepts and theories: A literature review can help to identify key concepts and theories that are relevant to the research question, and provide a theoretical framework for the study.
Supporting the research methodology : A literature review can inform the research methodology by identifying appropriate research methods, data sources, and research questions.
Providing a comprehensive overview of the literature : A literature review provides a comprehensive overview of the current state of knowledge on a topic, allowing the researcher to identify key themes, debates, and areas of agreement or disagreement.
Identifying potential research questions: A literature review can help to identify potential research questions and areas for further investigation.
Avoiding duplication of research: A literature review can help to avoid duplication of research by identifying what has already been done on a topic, and what remains to be done.
Enhancing the credibility of the research : A literature review helps to enhance the credibility of the research by demonstrating the researcher’s knowledge of the existing literature and their ability to situate their research within a broader context.

Limitations of Literature Review

Limitations of Literature Review are as follows:

Limited scope : Literature reviews can only cover the existing literature on a particular topic, which may be limited in scope or depth.
Publication bias : Literature reviews may be influenced by publication bias, which occurs when researchers are more likely to publish positive results than negative ones. This can lead to an incomplete or biased picture of the literature.
Quality of sources : The quality of the literature reviewed can vary widely, and not all sources may be reliable or valid.
Time-limited: Literature reviews can become quickly outdated as new research is published, making it difficult to keep up with the latest developments in a field.
Subjective interpretation : Literature reviews can be subjective, and the interpretation of the findings can vary depending on the researcher’s perspective or bias.
Lack of original data : Literature reviews do not generate new data, but rather rely on the analysis of existing studies.
Risk of plagiarism: It is important to ensure that literature reviews do not inadvertently contain plagiarism, which can occur when researchers use the work of others without proper attribution.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

Institutional Review Board – Application Sample...

Evaluating Research – Process, Examples and...

Log in using your username and password

Search More Search for this keyword Advanced search
Latest content
Current issue
Write for Us
BMJ Journals More You are viewing from: Google Indexer

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Implementing evidence into practice requires nurses to identify, critically appraise and synthesise research. This may require a comprehensive literature review: this article aims to outline the approaches and stages required and provides a working example of a published review.

Are there different approaches to undertaking a literature review?

What stages are required to undertake a literature review.

The rationale for the review should be established; consider why the review is important and relevant to patient care/safety or service delivery. For example, Noble et al 's 4 review sought to understand and make recommendations for practice and research in relation to dialysis refusal and withdrawal in patients with end-stage renal disease, an area of care previously poorly described. If appropriate, highlight relevant policies and theoretical perspectives that might guide the review. Once the key issues related to the topic, including the challenges encountered in clinical practice, have been identified formulate a clear question, and/or develop an aim and specific objectives. The type of review undertaken is influenced by the purpose of the review and resources available. However, the stages or methods used to undertake a review are similar across approaches and include:

Formulating clear inclusion and exclusion criteria, for example, patient groups, ages, conditions/treatments, sources of evidence/research designs;

Justifying data bases and years searched, and whether strategies including hand searching of journals, conference proceedings and research not indexed in data bases (grey literature) will be undertaken;

Developing search terms, the PICU (P: patient, problem or population; I: intervention; C: comparison; O: outcome) framework is a useful guide when developing search terms;

Developing search skills (eg, understanding Boolean Operators, in particular the use of AND/OR) and knowledge of how data bases index topics (eg, MeSH headings). Working with a librarian experienced in undertaking health searches is invaluable when developing a search.

Once studies are selected, the quality of the research/evidence requires evaluation. Using a quality appraisal tool, such as the Critical Appraisal Skills Programme (CASP) tools, 5 results in a structured approach to assessing the rigour of studies being reviewed. 3 Approaches to data synthesis for quantitative studies may include a meta-analysis (statistical analysis of data from multiple studies of similar designs that have addressed the same question), or findings can be reported descriptively. 6 Methods applicable for synthesising qualitative studies include meta-ethnography (themes and concepts from different studies are explored and brought together using approaches similar to qualitative data analysis methods), narrative summary, thematic analysis and content analysis. 7 Table 1 outlines the stages undertaken for a published review that summarised research about parents’ experiences of living with a child with a long-term condition. 8

View inline

An example of rapid evidence assessment review

In summary, the type of literature review depends on the review purpose. For the novice reviewer undertaking a review can be a daunting and complex process; by following the stages outlined and being systematic a robust review is achievable. The importance of literature reviews should not be underestimated—they help summarise and make sense of an increasingly vast body of research promoting best evidence-based practice.

↵ Centre for Reviews and Dissemination . Guidance for undertaking reviews in health care . 3rd edn . York : CRD, York University , 2009 .
↵ Canadian Best Practices Portal. http://cbpp-pcpe.phac-aspc.gc.ca/interventions/selected-systematic-review-sites / ( accessed 7.8.2015 ).
Bridges J , et al
↵ Critical Appraisal Skills Programme (CASP). http://www.casp-uk.net / ( accessed 7.8.2015 ).
Dixon-Woods M ,
Shaw R , et al
Agarwal S ,
Jones D , et al
Cheater F ,

Twitter Follow Joanna Smith at @josmith175

Competing interests None declared.

Read the full text or download the PDF:

Open access
Published: 17 August 2023

Data visualisation in scoping reviews and evidence maps on health topics: a cross-sectional analysis

Emily South ORCID: orcid.org/0000-0003-2187-4762 1 &
Mark Rodgers 1

Systematic Reviews volume 12 , Article number: 142 ( 2023 ) Cite this article

3614 Accesses

13 Altmetric

Metrics details

Scoping reviews and evidence maps are forms of evidence synthesis that aim to map the available literature on a topic and are well-suited to visual presentation of results. A range of data visualisation methods and interactive data visualisation tools exist that may make scoping reviews more useful to knowledge users. The aim of this study was to explore the use of data visualisation in a sample of recent scoping reviews and evidence maps on health topics, with a particular focus on interactive data visualisation.

Ovid MEDLINE ALL was searched for recent scoping reviews and evidence maps (June 2020-May 2021), and a sample of 300 papers that met basic selection criteria was taken. Data were extracted on the aim of each review and the use of data visualisation, including types of data visualisation used, variables presented and the use of interactivity. Descriptive data analysis was undertaken of the 238 reviews that aimed to map evidence.

Of the 238 scoping reviews or evidence maps in our analysis, around one-third (37.8%) included some form of data visualisation. Thirty-five different types of data visualisation were used across this sample, although most data visualisations identified were simple bar charts (standard, stacked or multi-set), pie charts or cross-tabulations (60.8%). Most data visualisations presented a single variable (64.4%) or two variables (26.1%). Almost a third of the reviews that used data visualisation did not use any colour (28.9%). Only two reviews presented interactive data visualisation, and few reported the software used to create visualisations.

Conclusions

Data visualisation is currently underused by scoping review authors. In particular, there is potential for much greater use of more innovative forms of data visualisation and interactive data visualisation. Where more innovative data visualisation is used, scoping reviews have made use of a wide range of different methods. Increased use of these more engaging visualisations may make scoping reviews more useful for a range of stakeholders.

Peer Review reports

Scoping reviews are “a type of evidence synthesis that aims to systematically identify and map the breadth of evidence available on a particular topic, field, concept, or issue” ([ 1 ], p. 950). While they include some of the same steps as a systematic review, such as systematic searches and the use of predetermined eligibility criteria, scoping reviews often address broader research questions and do not typically involve the quality appraisal of studies or synthesis of data [ 2 ]. Reasons for conducting a scoping review include the following: to map types of evidence available, to explore research design and conduct, to clarify concepts or definitions and to map characteristics or factors related to a concept [ 3 ]. Scoping reviews can also be undertaken to inform a future systematic review (e.g. to assure authors there will be adequate studies) or to identify knowledge gaps [ 3 ]. Other evidence synthesis approaches with similar aims have been described as evidence maps, mapping reviews or systematic maps [ 4 ]. While this terminology is used inconsistently, evidence maps can be used to identify evidence gaps and present them in a user-friendly (and often visual) way [ 5 ].

Scoping reviews are often targeted to an audience of healthcare professionals or policy-makers [ 6 ], suggesting that it is important to present results in a user-friendly and informative way. Until recently, there was little guidance on how to present the findings of scoping reviews. In recent literature, there has been some discussion of the importance of clearly presenting data for the intended audience of a scoping review, with creative and innovative use of visual methods if appropriate [ 7 , 8 , 9 ]. Lockwood et al. suggest that innovative visual presentation should be considered over dense sections of text or long tables in many cases [ 8 ]. Khalil et al. suggest that inspiration could be drawn from the field of data visualisation [ 7 ]. JBI guidance on scoping reviews recommends that reviewers carefully consider the best format for presenting data at the protocol development stage and provides a number of examples of possible methods [ 10 ].

Interactive resources are another option for presentation in scoping reviews [ 9 ]. Researchers without the relevant programming skills can now use several online platforms (such as Tableau [ 11 ] and Flourish [ 12 ]) to create interactive data visualisations. The benefits of using interactive visualisation in research include the ability to easily present more than two variables [ 13 ] and increased engagement of users [ 14 ]. Unlike static graphs, interactive visualisations can allow users to view hierarchical data at different levels, exploring both the “big picture” and looking in more detail ([ 15 ], p. 291). Interactive visualizations are often targeted at practitioners and decision-makers [ 13 ], and there is some evidence from qualitative research that they are valued by policy-makers [ 16 , 17 , 18 ].

Given their focus on mapping evidence, we believe that scoping reviews are particularly well-suited to visually presenting data and the use of interactive data visualisation tools. However, it is unknown how many recent scoping reviews visually map data or which types of data visualisation are used. The aim of this study was to explore the use of data visualisation methods in a large sample of recent scoping reviews and evidence maps on health topics. In particular, we were interested in the extent to which these forms of synthesis use any form of interactive data visualisation.

This study was a cross-sectional analysis of studies labelled as scoping reviews or evidence maps (or synonyms of these terms) in the title or abstract.

The search strategy was developed with help from an information specialist. Ovid MEDLINE® ALL was searched in June 2021 for studies added to the database in the previous 12 months. The search was limited to English language studies only.

The search strategy was as follows:

Ovid MEDLINE(R) ALL

(scoping review or evidence map or systematic map or mapping review or scoping study or scoping project or scoping exercise or literature mapping or evidence mapping or systematic mapping or literature scoping or evidence gap map).ab,ti.

limit 1 to english language

(202006* or 202007* or 202008* or 202009* or 202010* or 202011* or 202012* or 202101* or 202102* or 202103* or 202104* or 202105*).dt.

The search returned 3686 records. Records were de-duplicated in EndNote 20 software, leaving 3627 unique records.

A sample of these reviews was taken by screening the search results against basic selection criteria (Table 1 ). These criteria were piloted and refined after discussion between the two researchers. A single researcher (E.S.) screened the records in EPPI-Reviewer Web software using the machine-learning priority screening function. Where a second opinion was needed, decisions were checked by a second researcher (M.R.).

Our initial plan for sampling, informed by pilot searching, was to screen and data extract records in batches of 50 included reviews at a time. We planned to stop screening when a batch of 50 reviews had been extracted that included no new types of data visualisation or after screening time had reached 2 days. However, once data extraction was underway, we found the sample to be richer in terms of data visualisation than anticipated. After the inclusion of 300 reviews, we took the decision to end screening in order to ensure the study was manageable.

Data extraction

A data extraction form was developed in EPPI-Reviewer Web, piloted on 50 reviews and refined. Data were extracted by one researcher (E. S. or M. R.), with a second researcher (M. R. or E. S.) providing a second opinion when needed. The data items extracted were as follows: type of review (term used by authors), aim of review (mapping evidence vs. answering specific question vs. borderline), number of visualisations (if any), types of data visualisation used, variables/domains presented by each visualisation type, interactivity, use of colour and any software requirements.

When categorising review aims, we considered “mapping evidence” to incorporate all of the six purposes for conducting a scoping review proposed by Munn et al. [ 3 ]. Reviews were categorised as “answering a specific question” if they aimed to synthesise study findings to answer a particular question, for example on effectiveness of an intervention. We were inclusive with our definition of “mapping evidence” and included reviews with mixed aims in this category. However, some reviews were difficult to categorise (for example where aims were unclear or the stated aims did not match the actual focus of the paper) and were considered to be “borderline”. It became clear that a proportion of identified records that described themselves as “scoping” or “mapping” reviews were in fact pseudo-systematic reviews that failed to undertake key systematic review processes. Such reviews attempted to integrate the findings of included studies rather than map the evidence, and so reviews categorised as “answering a specific question” were excluded from the main analysis. Data visualisation methods for meta-analyses have been explored previously [ 19 ]. Figure 1 shows the flow of records from search results to final analysis sample.

Flow diagram of the sampling process

Data visualisation was defined as any graph or diagram that presented results data, including tables with a visual mapping element, such as cross-tabulations and heat maps. However, tables which displayed data at a study level (e.g. tables summarising key characteristics of each included study) were not included, even if they used symbols, shading or colour. Flow diagrams showing the study selection process were also excluded. Data visualisations in appendices or supplementary information were included, as well as any in publicly available dissemination products (e.g. visualisations hosted online) if mentioned in papers.

The typology used to categorise data visualisation methods was based on an existing online catalogue [ 20 ]. Specific types of data visualisation were categorised in five broad categories: graphs, diagrams, tables, maps/geographical and other. If a data visualisation appeared in our sample that did not feature in the original catalogue, we checked a second online catalogue [ 21 ] for an appropriate term, followed by wider Internet searches. These additional visualisation methods were added to the appropriate section of the typology. The final typology can be found in Additional file 1 .

We conducted descriptive data analysis in Microsoft Excel 2019 and present frequencies and percentages. Where appropriate, data are presented using graphs or other data visualisations created using Flourish. We also link to interactive versions of some of these visualisations.

Almost all of the 300 reviews in the total sample were labelled by review authors as “scoping reviews” ( n = 293, 97.7%). There were also four “mapping reviews”, one “scoping study”, one “evidence mapping” and one that was described as a “scoping review and evidence map”. Included reviews were all published in 2020 or 2021, with the exception of one review published in 2018. Just over one-third of these reviews ( n = 105, 35.0%) included some form of data visualisation. However, we excluded 62 reviews that did not focus on mapping evidence from the following analysis (see “ Methods ” section). Of the 238 remaining reviews (that either clearly aimed to map evidence or were judged to be “borderline”), 90 reviews (37.8%) included at least one data visualisation. The references for these reviews can be found in Additional file 2 .

Number of visualisations

Thirty-six (40.0%) of these 90 reviews included just one example of data visualisation (Fig. 2 ). Less than a third ( n = 28, 31.1%) included three or more visualisations. The greatest number of data visualisations in one review was 17 (all bar or pie charts). In total, 222 individual data visualisations were identified across the sample of 238 reviews.

Number of data visualisations per review

Categories of data visualisation

Graphs were the most frequently used category of data visualisation in the sample. Over half of the reviews with data visualisation included at least one graph ( n = 59, 65.6%). The least frequently used category was maps, with 15.6% ( n = 14) of these reviews including a map.

Of the total number of 222 individual data visualisations, 102 were graphs (45.9%), 34 were tables (15.3%), 23 were diagrams (10.4%), 15 were maps (6.8%) and 48 were classified as “other” in the typology (21.6%).

Types of data visualisation

All of the types of data visualisation identified in our sample are reported in Table 2 . In total, 35 different types were used across the sample of reviews.

The most frequently used data visualisation type was a bar chart. Of 222 total data visualisations, 78 (35.1%) were a variation on a bar chart (either standard bar chart, stacked bar chart or multi-set bar chart). There were also 33 pie charts (14.9% of data visualisations) and 24 cross-tabulations (10.8% of data visualisations). In total, these five types of data visualisation accounted for 60.8% ( n = 135) of all data visualisations. Figure 3 shows the frequency of each data visualisation category and type; an interactive online version of this treemap is also available ( https://public.flourish.studio/visualisation/9396133/ ). Figure 4 shows how users can further explore the data using the interactive treemap.

Data visualisation categories and types. An interactive version of this treemap is available online: https://public.flourish.studio/visualisation/9396133/ . Through the interactive version, users can further explore the data (see Fig. 4 ). The unit of this treemap is the individual data visualisation, so multiple data visualisations within the same scoping review are represented in this map. Created with flourish.studio ( https://flourish.studio )

Screenshots showing how users of the interactive treemap can explore the data further. Users can explore each level of the hierarchical treemap ( A Visualisation category > B Visualisation subcategory > C Variables presented in visualisation > D Individual references reporting this category/subcategory/variable permutation). Created with flourish.studio ( https://flourish.studio )

Data presented

Around two-thirds of data visualisations in the sample presented a single variable ( n = 143, 64.4%). The most frequently presented single variables were themes ( n = 22, 9.9% of data visualisations), population ( n = 21, 9.5%), country or region ( n = 21, 9.5%) and year ( n = 20, 9.0%). There were 58 visualisations (26.1%) that presented two different variables. The remaining 21 data visualisations (9.5%) presented three or more variables. Figure 5 shows the variables presented by each different type of data visualisation (an interactive version of this figure is available online).

Variables presented by each data visualisation type. Darker cells indicate a larger number of reviews. An interactive version of this heat map is available online: https://public.flourish.studio/visualisation/10632665/ . Users can hover over each cell to see the number of data visualisations for that combination of data visualisation type and variable. The unit of this heat map is the individual data visualisation, so multiple data visualisations within a single scoping review are represented in this map. Created with flourish.studio ( https://flourish.studio )

Most reviews presented at least one data visualisation in colour ( n = 64, 71.1%). However, almost a third ( n = 26, 28.9%) used only black and white or greyscale.

Interactivity

Only two of the reviews included data visualisations with any level of interactivity. One scoping review on music and serious mental illness [ 22 ] linked to an interactive bubble chart hosted online on Tableau. Functionality included the ability to filter the studies displayed by various attributes.

The other review was an example of evidence mapping from the environmental health field [ 23 ]. All four of the data visualisations included in the paper were available in an interactive format hosted either by the review management software or on Tableau. The interactive versions linked to the relevant references so users could directly explore the evidence base. This was the only review that provided this feature.

Software requirements

Nine reviews clearly reported the software used to create data visualisations. Three reviews used Tableau (one of them also used review management software as discussed above) [ 22 , 23 , 24 ]. Two reviews generated maps using ArcGIS [ 25 ] or ArcMap [ 26 ]. One review used Leximancer for a lexical analysis [ 27 ]. One review undertook a bibliometric analysis using VOSviewer [ 28 ], and another explored citation patterns using CitNetExplorer [ 29 ]. Other reviews used Excel [ 30 ] or R [ 26 ].

To our knowledge, this is the first systematic and in-depth exploration of the use of data visualisation techniques in scoping reviews. Our findings suggest that the majority of scoping reviews do not use any data visualisation at all, and, in particular, more innovative examples of data visualisation are rare. Around 60% of data visualisations in our sample were simple bar charts, pie charts or cross-tabulations. There appears to be very limited use of interactive online visualisation, despite the potential this has for communicating results to a range of stakeholders. While it is not always appropriate to use data visualisation (or a simple bar chart may be the most user-friendly way of presenting the data), these findings suggest that data visualisation is being underused in scoping reviews. In a large minority of reviews, visualisations were not published in colour, potentially limiting how user-friendly and attractive papers are to decision-makers and other stakeholders. Also, very few reviews clearly reported the software used to create data visualisations. However, 35 different types of data visualisation were used across the sample, highlighting the wide range of methods that are potentially available to scoping review authors.

Our results build on the limited research that has previously been undertaken in this area. Two previous publications also found limited use of graphs in scoping reviews. Results were “mapped graphically” in 29% of scoping reviews in any field in one 2014 publication [ 31 ] and 17% of healthcare scoping reviews in a 2016 article [ 6 ]. Our results suggest that the use of data visualisation has increased somewhat since these reviews were conducted. Scoping review methods have also evolved in the last 10 years; formal guidance on scoping review conduct was published in 2014 [ 32 ], and an extension of the PRISMA checklist for scoping reviews was published in 2018 [ 33 ]. It is possible that an overall increase in use of data visualisation reflects increased quality of published scoping reviews. There is also some literature supporting our findings on the wide range of data visualisation methods that are used in evidence synthesis. An investigation of methods to identify, prioritise or display health research gaps (25/139 included studies were scoping reviews; 6/139 were evidence maps) identified 14 different methods used to display gaps or priorities, with half being “more advanced” (e.g. treemaps, radial bar plots) ([ 34 ], p. 107). A review of data visualisation methods used in papers reporting meta-analyses found over 200 different ways of displaying data [ 19 ].

Only two reviews in our sample used interactive data visualisation, and one of these was an example of systematic evidence mapping from the environmental health field rather than a scoping review (in environmental health, systematic evidence mapping explicitly involves producing a searchable database [ 35 ]). A scoping review of papers on the use of interactive data visualisation in population health or health services research found a range of examples but still limited use overall [ 13 ]. For example, the authors noted the currently underdeveloped potential for using interactive visualisation in research on health inequalities. It is possible that the use of interactive data visualisation in academic papers is restricted by academic publishing requirements; for example, it is currently difficult to incorporate an interactive figure into a journal article without linking to an external host or platform. However, we believe that there is a lot of potential to add value to future scoping reviews by using interactive data visualisation software. Few reviews in our sample presented three or more variables in a single visualisation, something which can easily be achieved using interactive data visualisation tools. We have previously used EPPI-Mapper [ 36 ] to present results of a scoping review of systematic reviews on behaviour change in disadvantaged groups, with links to the maps provided in the paper [ 37 ]. These interactive maps allowed policy-makers to explore the evidence on different behaviours and disadvantaged groups and access full publications of the included studies directly from the map.

We acknowledge there are barriers to use for some of the data visualisation software available. EPPI-Mapper and some of the software used by reviews in our sample incur a cost. Some software requires a certain level of knowledge and skill in its use. However numerous online free data visualisation tools and resources exist. We have used Flourish to present data for this review, a basic version of which is currently freely available and easy to use. Previous health research has been found to have used a range of different interactive data visualisation software, much of which does not required advanced knowledge or skills to use [ 13 ].

There are likely to be other barriers to the use of data visualisation in scoping reviews. Journal guidelines and policies may present barriers for using innovative data visualisation. For example, some journals charge a fee for publication of figures in colour. As previously mentioned, there are limited options for incorporating interactive data visualisation into journal articles. Authors may also be unaware of the data visualisation methods and tools that are available. Producing data visualisations can be time-consuming, particularly if authors lack experience and skills in this. It is possible that many authors prioritise speed of publication over spending time producing innovative data visualisations, particularly in a context where there is pressure to achieve publications.

Limitations

A limitation of this study was that we did not assess how appropriate the use of data visualisation was in our sample as this would have been highly subjective. Simple descriptive or tabular presentation of results may be the most appropriate approach for some scoping review objectives [ 7 , 8 , 10 ], and the scoping review literature cautions against “over-using” different visual presentation methods [ 7 , 8 ]. It cannot be assumed that all of the reviews that did not include data visualisation should have done so. Likewise, we do not know how many reviews used methods of data visualisation that were not well suited to their data.

We initially relied on authors’ own use of the term “scoping review” (or equivalent) to sample reviews but identified a relatively large number of papers labelled as scoping reviews that did not meet the basic definition, despite the availability of guidance and reporting guidelines [ 10 , 33 ]. It has previously been noted that scoping reviews may be undertaken inappropriately because they are seen as “easier” to conduct than a systematic review ([ 3 ], p.6), and that reviews are often labelled as “scoping reviews” while not appearing to follow any established framework or guidance [ 2 ]. We therefore took the decision to remove these reviews from our main analysis. However, decisions on how to classify review aims were subjective, and we did include some reviews that were of borderline relevance.

A further limitation is that this was a sample of published reviews, rather than a comprehensive systematic scoping review as have previously been undertaken [ 6 , 31 ]. The number of scoping reviews that are published has increased rapidly, and this would now be difficult to undertake. As this was a sample, not all relevant scoping reviews or evidence maps that would have met our criteria were included. We used machine learning to screen our search results for pragmatic reasons (to reduce screening time), but we do not see any reason that our sample would not be broadly reflective of the wider literature.

Data visualisation, and in particular more innovative examples of it, is currently underused in published scoping reviews on health topics. The examples that we have found highlight the wide range of methods that scoping review authors could draw upon to present their data in an engaging way. In particular, we believe that interactive data visualisation has significant potential for mapping the available literature on a topic. Appropriate use of data visualisation may increase the usefulness, and thus uptake, of scoping reviews as a way of identifying existing evidence or research gaps by decision-makers, researchers and commissioners of research. We recommend that scoping review authors explore the extensive free resources and online tools available for data visualisation. However, we also think that it would be useful for publishers to explore allowing easier integration of interactive tools into academic publishing, given the fact that papers are now predominantly accessed online. Future research may be helpful to explore which methods are particularly useful to scoping review users.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Organisation formerly known as Joanna Briggs Institute

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Munn Z, Pollock D, Khalil H, Alexander L, McLnerney P, Godfrey CM, Peters M, Tricco AC. What are scoping reviews? Providing a formal definition of scoping reviews as a type of evidence synthesis. JBI Evid Synth. 2022;20:950–952.

Peters MDJ, Marnie C, Colquhoun H, Garritty CM, Hempel S, Horsley T, Langlois EV, Lillie E, O’Brien KK, Tunçalp Ӧ, et al. Scoping reviews: reinforcing and advancing the methodology and application. Syst Rev. 2021;10:263.

Article PubMed PubMed Central Google Scholar

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18:143.

Sutton A, Clowes M, Preston L, Booth A. Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J. 2019;36:202–22.

Article PubMed Google Scholar

Miake-Lye IM, Hempel S, Shanman R, Shekelle PG. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016;5:28.

Tricco AC, Lillie E, Zarin W, O’Brien K, Colquhoun H, Kastner M, Levac D, Ng C, Sharpe JP, Wilson K, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16:15.

Khalil H, Peters MDJ, Tricco AC, Pollock D, Alexander L, McInerney P, Godfrey CM, Munn Z. Conducting high quality scoping reviews-challenges and solutions. J Clin Epidemiol. 2021;130:156–60.

Lockwood C, dos Santos KB, Pap R. Practical guidance for knowledge synthesis: scoping review methods. Asian Nurs Res. 2019;13:287–94.

Article Google Scholar

Pollock D, Peters MDJ, Khalil H, McInerney P, Alexander L, Tricco AC, Evans C, de Moraes ÉB, Godfrey CM, Pieper D, et al. Recommendations for the extraction, analysis, and presentation of results in scoping reviews. JBI Evidence Synthesis. 2022;10:11124.

Google Scholar

Peters MDJ GC, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping reviews (2020 version). In: Aromataris E MZ, editor. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . Accessed 1 Feb 2023.

Tableau Public. https://www.tableau.com/en-gb/products/public . Accessed 24 January 2023.

flourish.studio. https://flourish.studio/ . Accessed 24 January 2023.

Chishtie J, Bielska IA, Barrera A, Marchand J-S, Imran M, Tirmizi SFA, Turcotte LA, Munce S, Shepherd J, Senthinathan A, et al. Interactive visualization applications in population health and health services research: systematic scoping review. J Med Internet Res. 2022;24: e27534.

Isett KR, Hicks DM. Providing public servants what they need: revealing the “unseen” through data visualization. Public Adm Rev. 2018;78:479–85.

Carroll LN, Au AP, Detwiler LT, Fu T-c, Painter IS, Abernethy NF. Visualization and analytics tools for infectious disease epidemiology: a systematic review. J Biomed Inform. 2014;51:287–298.

Lundkvist A, El-Khatib Z, Kalra N, Pantoja T, Leach-Kemon K, Gapp C, Kuchenmüller T. Policy-makers’ views on translating burden of disease estimates in health policies: bridging the gap through data visualization. Arch Public Health. 2021;79:17.

Zakkar M, Sedig K. Interactive visualization of public health indicators to support policymaking: an exploratory study. Online J Public Health Inform. 2017;9:e190–e190.

Park S, Bekemeier B, Flaxman AD. Understanding data use and preference of data visualization for public health professionals: a qualitative study. Public Health Nurs. 2021;38:531–41.

Kossmeier M, Tran US, Voracek M. Charting the landscape of graphical displays for meta-analysis and systematic reviews: a comprehensive review, taxonomy, and feature analysis. BMC Med Res Methodol. 2020;20:26.

Ribecca, S. The Data Visualisation Catalogue. https://datavizcatalogue.com/index.html . Accessed 23 November 2021.

Ferdio. Data Viz Project. https://datavizproject.com/ . Accessed 23 November 2021.

Golden TL, Springs S, Kimmel HJ, Gupta S, Tiedemann A, Sandu CC, Magsamen S. The use of music in the treatment and management of serious mental illness: a global scoping review of the literature. Front Psychol. 2021;12: 649840.

Keshava C, Davis JA, Stanek J, Thayer KA, Galizia A, Keshava N, Gift J, Vulimiri SV, Woodall G, Gigot C, et al. Application of systematic evidence mapping to assess the impact of new research when updating health reference values: a case example using acrolein. Environ Int. 2020;143: 105956.

Article CAS PubMed PubMed Central Google Scholar

Jayakumar P, Lin E, Galea V, Mathew AJ, Panda N, Vetter I, Haynes AB. Digital phenotyping and patient-generated health data for outcome measurement in surgical care: a scoping review. J Pers Med. 2020;10:282.

Qu LG, Perera M, Lawrentschuk N, Umbas R, Klotz L. Scoping review: hotspots for COVID-19 urological research: what is being published and from where? World J Urol. 2021;39:3151–60.

Article CAS PubMed Google Scholar

Rossa-Roccor V, Acheson ES, Andrade-Rivas F, Coombe M, Ogura S, Super L, Hong A. Scoping review and bibliometric analysis of the term “planetary health” in the peer-reviewed literature. Front Public Health. 2020;8:343.

Hewitt L, Dahlen HG, Hartz DL, Dadich A. Leadership and management in midwifery-led continuity of care models: a thematic and lexical analysis of a scoping review. Midwifery. 2021;98: 102986.

Xia H, Tan S, Huang S, Gan P, Zhong C, Lu M, Peng Y, Zhou X, Tang X. Scoping review and bibliometric analysis of the most influential publications in achalasia research from 1995 to 2020. Biomed Res Int. 2021;2021:8836395.

Vigliotti V, Taggart T, Walker M, Kusmastuti S, Ransome Y. Religion, faith, and spirituality influences on HIV prevention activities: a scoping review. PLoS ONE. 2020;15: e0234720.

van Heemskerken P, Broekhuizen H, Gajewski J, Brugha R, Bijlmakers L. Barriers to surgery performed by non-physician clinicians in sub-Saharan Africa-a scoping review. Hum Resour Health. 2020;18:51.

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5:371–85.

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, McInerney P, Godfrey CM, Khalil H. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18:2119–26.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73.

Nyanchoka L, Tudur-Smith C, Thu VN, Iversen V, Tricco AC, Porcher R. A scoping review describes methods used to identify, prioritize and display gaps in health research. J Clin Epidemiol. 2019;109:99–110.

Wolffe TAM, Whaley P, Halsall C, Rooney AA, Walker VR. Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environ Int. 2019;130:104871.

Digital Solution Foundry and EPPI-Centre. EPPI-Mapper, Version 2.0.1. EPPI-Centre, UCL Social Research Institute, University College London. 2020. https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=3790 .

South E, Rodgers M, Wright K, Whitehead M, Sowden A. Reducing lifestyle risk behaviours in disadvantaged groups in high-income countries: a scoping review of systematic reviews. Prev Med. 2022;154: 106916.

Download references

Acknowledgements

We would like to thank Melissa Harden, Senior Information Specialist, Centre for Reviews and Dissemination, for advice on developing the search strategy.

This work received no external funding.

Author information

Authors and affiliations.

Centre for Reviews and Dissemination, University of York, York, YO10 5DD, UK

Emily South & Mark Rodgers

You can also search for this author in PubMed Google Scholar

Contributions

Both authors conceptualised and designed the study and contributed to screening, data extraction and the interpretation of results. ES undertook the literature searches, analysed data, produced the data visualisations and drafted the manuscript. MR contributed to revising the manuscript, and both authors read and approved the final version.

Corresponding author

Correspondence to Emily South .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Typology of data visualisation methods.

Additional file 2.

References of scoping reviews included in main dataset.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

South, E., Rodgers, M. Data visualisation in scoping reviews and evidence maps on health topics: a cross-sectional analysis. Syst Rev 12 , 142 (2023). https://doi.org/10.1186/s13643-023-02309-y

Download citation

Received : 21 February 2023

Accepted : 07 August 2023

Published : 17 August 2023

DOI : https://doi.org/10.1186/s13643-023-02309-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Scoping review
Evidence map
Data visualisation

Systematic Reviews

ISSN: 2046-4053

Submission enquiries: Access here and click Contact Us
General enquiries: [email protected]

Open access
Published: 01 May 2024

The effectiveness of virtual reality training on knowledge, skills and attitudes of health care professionals and students in assessing and treating mental health disorders: a systematic review

Cathrine W. Steen 1 , 2 ,
Kerstin Söderström 1 , 2 ,
Bjørn Stensrud 3 ,
Inger Beate Nylund 2 &
Johan Siqveland 4 , 5

BMC Medical Education volume 24 , Article number: 480 ( 2024 ) Cite this article

439 Accesses

1 Altmetric

Metrics details

Virtual reality (VR) training can enhance health professionals’ learning. However, there are ambiguous findings on the effectiveness of VR as an educational tool in mental health. We therefore reviewed the existing literature on the effectiveness of VR training on health professionals’ knowledge, skills, and attitudes in assessing and treating patients with mental health disorders.

We searched MEDLINE, PsycINFO (via Ovid), the Cochrane Library, ERIC, CINAHL (on EBSCOhost), Web of Science Core Collection, and the Scopus database for studies published from January 1985 to July 2023. We included all studies evaluating the effect of VR training interventions on attitudes, knowledge, and skills pertinent to the assessment and treatment of mental health disorders and published in English or Scandinavian languages. The quality of the evidence in randomized controlled trials was assessed with the Cochrane Risk of Bias Tool 2.0. For non-randomized studies, we assessed the quality of the studies with the ROBINS-I tool.

Of 4170 unique records identified, eight studies were eligible. The four randomized controlled trials were assessed as having some concern or a high risk of overall bias. The four non-randomized studies were assessed as having a moderate to serious overall risk of bias. Of the eight included studies, four used a virtual standardized patient design to simulate training situations, two studies used interactive patient scenario training designs, while two studies used a virtual patient game design. The results suggest that VR training interventions can promote knowledge and skills acquisition.

Conclusions

The findings indicate that VR interventions can effectively train health care personnel to acquire knowledge and skills in the assessment and treatment of mental health disorders. However, study heterogeneity, prevalence of small sample sizes, and many studies with a high or serious risk of bias suggest an uncertain evidence base. Future research on the effectiveness of VR training should include assessment of immersive VR training designs and a focus on more robust studies with larger sample sizes.

Trial registration

This review was pre-registered in the Open Science Framework register with the ID-number Z8EDK.

Peer Review reports

A robustly trained health care workforce is pivotal to forging a resilient health care system [ 1 ], and there is an urgent need to develop innovative methods and emerging technologies for health care workforce education [ 2 ]. Virtual reality technology designs for clinical training have emerged as a promising avenue for increasing the competence of health care professionals, reflecting their potential to provide effective training [ 3 ].

Virtual reality (VR) is a dynamic and diverse field, and can be described as a computer-generated environment that simulates sensory experiences, where user interactions play a role in shaping the course of events within that environment [ 4 ]. When optimally designed, VR gives users the feeling that they are physically within this simulated space, unlocking its potential as a dynamic and immersive learning tool [ 5 ]. The cornerstone of the allure of VR is its capacity for creating artificial settings via sensory deceptions, encapsulated by the term ‘immersion’. Immersion conveys the sensation of being deeply engrossed or enveloped in an alternate world, akin to absorption in a video game. Some VR systems will be more immersive than others, based on the technology used to influence the senses. However, the degree of immersion does not necessarily determine the user’s level of engagement with the application [ 6 ].

A common approach to categorizing VR systems is based on the design of the technology used, allowing them to be classified into: 1) non-immersive desktop systems, where users experience virtual environments through a computer screen, 2) immersive CAVE systems with large projected images and motion trackers to adjust the image to the user, and 3) fully immersive head-mounted display systems that involve users wearing a headset that fully covers their eyes and ears, thus entirely immersing them in the virtual environment [ 7 ]. Advances in VR technology have enabled a wide range of VR experiences. The possibility for health care professionals to repeatedly practice clinical skills with virtual patients in a risk-free environment offers an invaluable learning platform for health care education.

The impact of VR training on health care professionals’ learning has predominantly been researched in terms of the enhancement of technical surgical abilities. This includes refining procedural planning, familiarizing oneself with medical instruments, and practicing psychomotor skills such as dexterity, accuracy, and speed [ 8 , 9 ]. In contrast, the exploration of VR training in fostering non-technical or ‘soft’ skills, such as communication and teamwork, appears to be less prevalent [ 10 ]. A recent systematic review evaluates the outcomes of VR training in non-technical skills across various medical specialties [ 11 ], focusing on vital cognitive abilities (e.g., situation awareness, decision-making) and interprofessional social competencies (e.g., teamwork, conflict resolution, leadership). These skills are pivotal in promoting collaboration among colleagues and ensuring a safe health care environment. At the same time, they are not sufficiently comprehensive for encounters with patients with mental health disorders.

For health care professionals providing care to patients with mental health disorders, acquiring specific skills, knowledge, and empathic attitudes is of utmost importance. Many individuals experiencing mental health challenges may find it difficult to communicate their thoughts and feelings, and it is therefore essential for health care providers to cultivate an environment where patients feel safe and encouraged to share feelings and thoughts. Beyond fostering trust, health care professionals must also possess in-depth knowledge about the nature and treatment of various mental health disorders. Moreover, they must actively practice and internalize the skills necessary to translate their knowledge into clinical practice. While the conventional approach to training mental health clinical skills has been through simulation or role-playing with peers under expert supervision and practicing with real patients, the emergence of VR applications presents a compelling alternative. This technology promises a potentially transformative way to train mental health professionals. Our review identifies specific outcomes in knowledge, skills, and attitudes, covering areas from theoretical understanding to practical application and patient interaction. By focusing on these measurable concepts, which are in line with current healthcare education guidelines [ 12 ], we aim to contribute to the knowledge base and provide a detailed analysis of the complexities in mental health care training. This approach is designed to highlight the VR training’s practical relevance alongside its contribution to academic discourse.

A recent systematic review evaluated the effects of virtual patient (VP) interventions on knowledge, skills, and attitudes in undergraduate psychiatry education [ 13 ]. This review’s scope is limited to assessing VP interventions and does not cover other types of VR training interventions. Furthermore, it adopts a classification of VP different from our review, rendering their findings and conclusions not directly comparable to ours.

To the best of our knowledge, no systematic review has assessed and summarized the effectiveness of VR training interventions for health professionals in the assessment and treatment of mental health disorders. This systematic review addresses the gap by exploring the effectiveness of virtual reality in the training of knowledge, skills, and attitudes health professionals need to master in the assessment and treatment of mental health disorders.

This systematic review follows the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analysis [ 14 ]. The protocol of the systematic review was registered in the Open Science Framework register with the registration ID Z8EDK.

We included randomized controlled trials, cohort studies, and pretest–posttest studies, which met the following criteria: a) a population of health care professionals or health care professional students, b) assessed the effectiveness of a VR application in assessing and treating mental health disorders, and c) reported changes in knowledge, skills, or attitudes. We excluded studies evaluating VR interventions not designed for training in assessing and treating mental health disorders (e.g., training of surgical skills), studies evaluating VR training from the first-person perspective, studies that used VR interventions for non-educational purposes and studies where VR interventions trained patients with mental health problems (e.g., social skills training). We also excluded studies not published in English or Scandinavian languages.

Search strategy

The literature search reporting was guided by relevant items in PRISMA-S [ 15 ]. In collaboration with a senior academic librarian (IBN), we developed the search strategy for the systematic review. Inspired by the ‘pearl harvesting’ information retrieval approach [ 16 ], we anticipated a broad spectrum of terms related to our interdisciplinary query. Recognizing that various terminologies could encapsulate our central ideas, we harvested an array of terms for each of the four elements ‘health care professionals and health care students’, ‘VR’, ‘training’, and ‘mental health’. The pearl harvesting framework [ 16 ] consists of four steps which we followed with some minor adaptions. Step 1: We searched for and sampled a set of relevant research articles, a book chapter, and literature reviews. Step 2: The librarian scrutinized titles, abstracts, and author keywords, as well as subject headings used in databases, and collected relevant terms. Step 3: The librarian refined the lists of terms. Step 4: The review group, in collaboration with a VR consultant from KildeGruppen AS (a Norwegian media company), validated the refined lists of terms to ensure they included all relevant VR search terms. This process for the element VR resulted in the inclusion of search terms such as ‘3D simulated environment’, ‘second life simulation’, ‘virtual patient’, and ‘virtual world’. We were given a peer review of the search strategy by an academic librarian at Inland Norway University of Applied Sciences.

In June and July 2021, we performed comprehensive searches for publications dating from January 1985 to the present. This period for the inclusion of studies was chosen since VR systems designed for training in health care first emerged in the early 1990s. The searches were carried out in seven databases: MEDLINE and PsycInfo (on Ovid), ERIC and CINAHL (on EBSCOhost), the Cochrane Library, Web of Science Core Collection, and Scopus. Detailed search strategies from each database are available for public access at DataverseNO [ 17 ]. On July 2, 2021, a search in CINAHL yielded 993 hits. However, when attempting to transfer these records to EndNote using the ‘Folder View’—a feature designed for organizing and managing selected records before export—only 982 records were successfully transferred. This discrepancy indicates that 11 records could not be transferred through Folder View, for reasons not specified. The process was repeated twice, consistently yielding the same discrepancy. The missing 11 records pose a risk of failing to capture relevant studies in the initial search. In July 2023, to make sure that we included the latest publications, we updated our initial searches, focusing on entries since January 1, 2021. This ensured that we did not miss any new references recently added to these databases. Due to a lack of access to the Cochrane Library in July 2023, we used EBMR (Evidence Based Medicine Reviews) on the Ovid platform instead, including the databases Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, and Cochrane Clinical Answers. All references were exported to Endnote and duplicates were removed. The number of records from each database can be observed in the PRISMA diagram [ 14 ], Fig. 1 .

PRISMA flow chart of the records and study selection process

Study selection and data collection

Two reviewers (JS, CWS) independently assessed the titles and abstracts of studies retrieved from the literature search based on the eligibility criteria. We employed the Rayyan website for the screening process [ 18 ]. The same reviewers (JS, CWS) assessed the full-text articles selected after the initial screening. Articles meeting the eligibility criteria were incorporated into the review. Any disagreements were resolved through discussion.

Data extracted from the studies by the first author (CWS) and cross-checked by another reviewer (JS) included: authors of the study, publication year, country, study design, participant details (education, setting), interventions (VR system, class label), comparison types, outcomes, and main findings. This data is summarized in Table 1 and Additional file 1 . In the process of reviewing the VR interventions utilized within the included studies, we sought expertise from advisers associated with VRINN, a Norwegian immersive learning cluster, and SIMInnlandet, a center dedicated to simulation in mental health care at Innlandet Hospital Trust. This collaboration ensured a thorough examination and accurate categorization of the VR technologies applied. Furthermore, the classification of the learning designs employed in the VP interventions was conducted under the guidance of an experienced VP scholar at Paracelcus Medical University in Salzburg.

Data analysis

We initially intended to perform a meta-analysis with knowledge, skills, and attitudes as primary outcomes, planning separate analyses for each. However, due to significant heterogeneity observed among the included studies, it was not feasible to carry out a meta-analysis. Consequently, we opted for a narrative synthesis based on these pre-determined outcomes of knowledge, skills, and attitudes. This approach allowed for an analysis of the relationships both within and between the studies. The effect sizes were calculated using a web-based effect size calculator [ 27 ]. We have interpreted effect sizes based on commonly used descriptions for Cohen’s d: small = 0.2, moderate = 0.5, and large = 0.8, and for Cramer’s V: small = 0.10, medium = 0.30, and large = 0.50.

Risk of bias assessment

JS and CWS independently evaluated the risk of bias for all studies using two distinct assessment tools. We used the Cochrane risk of bias tool RoB 2 [ 28 ] to assess the risk of bias in the RCTs. With the RoB 2 tool, the bias was assessed as high, some concerns or low for five domains: randomization process, deviations from the intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result [ 28 ].

We used the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool [ 29 ] to assess the risk of bias in the cohort and single-group studies. By using ROBINS-I for the non-randomized trials, the risk of bias was assessed using the categories low, moderate, serious, critical or no information for seven domains: confounding, selection of participants, classification of interventions, deviations from intended interventions, missing data, measurement of outcomes, and selection of the reported result [ 29 ].

We included eight studies in the review (Fig. 1 ). An overview of the included studies is presented in detail in Table 1 .

Four studies were RCTs [ 19 , 20 , 21 , 22 ], two were single group pretest–posttest studies [ 23 , 26 ], one was a controlled before and after study [ 25 ], and one was a cohort study [ 24 ]. The studies included health professionals from diverse educational backgrounds, including some from mental health and medical services, as well as students in medicine, social work, and nursing. All studies, published from 2009 to 2021, utilized non-immersive VR desktop system interventions featuring various forms of VP designs. Based on an updated classification of VP interventions by Kononowicz et al. [ 30 ] developed from a model proposed by Talbot et al. [ 31 ], we have described the characteristics of the interventions in Table 1 . Four of the studies utilized a virtual standardized patient (VSP) intervention [ 20 , 21 , 22 , 23 ], a conversational agent that simulates clinical presentations for training purposes. Two studies employed an interactive patient scenario (IPS) design [ 25 , 26 ], an approach that primarily uses text-based multimedia, enhanced with images and case histories through text or voice narratives, to simulate clinical scenarios. Lastly, two studies used a virtual patient game (VP game) intervention [ 19 , 24 ]. These interventions feature training scenarios using 3D avatars, specifically designed to improve clinical reasoning and team training skills. It should be noted that the interventions classified as VSPs in this review, being a few years old, do not encompass artificial intelligence (AI) as we interpret it today. However, since the interventions include some kind of algorithm that provides answers to questions, we consider them as conversational agents, and therefore as VSPs. As the eight included studies varied significantly in terms of design, interventions, and outcome measures, we could not incorporate them into a meta-analysis.

The overall risk of bias for the four RCTs was high [ 19 , 20 , 22 ] or of some concern [ 21 ] (Fig. 2 ). They were all assessed as low or of some concern in the domains of randomization. Three studies were assessed with a high risk of bias in one [ 19 , 20 ] or two domains [ 22 ]; one study had a high risk of bias in the domain of selection of the reported result [ 19 ], one in the domain of measurement of outcome [ 20 ], and one in the domains of deviation from the intended interventions and missing outcome data [ 22 ]. One study was not assessed as having a high risk of bias in any domain [ 21 ].

Risk of bias summary: review authors assessments of each risk of bias item in the included RCT studies

For the four non-randomized studies, the overall risk of bias was judged to be moderate [ 26 ] or serious [ 23 , 24 , 25 ] (Fig. 3 ). One study had a serious risk of bias in two domains: confounding and measurement of outcomes [ 23 ]. Two studies had a serious risk of bias in one domain, namely confounding [ 24 , 25 ], while one study was judged not to have a serious risk of bias in any domain [ 26 ].

Risk of bias summary: review authors assessments of each risk of bias item in the included non-randomized studies

Three studies investigated the impact of virtual reality training on mental health knowledge [ 24 , 25 , 26 ]. One study with 32 resident psychiatrists in a single group pretest–posttest design assessed the effect of a VR training intervention on knowledge of posttraumatic stress disorder (PTSD) symptomatology, clinical management, and communication skills [ 26 ]. The intervention consisted of an IPS. The assessment of the outcome was conducted using a knowledge test with 11 multiple-choice questions and was administered before and after the intervention. This study reported a significant improvement on the knowledge test after the VR training intervention.

The second study examined the effect of a VR training intervention on knowledge of dementia [ 25 ], employing a controlled before and after design. Seventy-nine medical students in clinical training were divided into two groups, following a traditional learning program. The experimental group received an IPS intervention. The outcome was evaluated with a knowledge test administered before and after the intervention with significantly higher posttest scores in the experimental group than in the control group, with a moderate effects size observed between the groups.

A third study evaluated the effect of a VR training intervention on 299 undergraduate nursing students’ diagnostic recognition of depression and schizophrenia (classified as knowledge) [ 24 ]. In a prospective cohort design, the VR intervention was the only difference in the mental health related educational content provided to the two cohorts, and consisted of a VP game design, developed to simulate training situations with virtual patient case scenarios, including depression and schizophrenia. The outcome was assessed by determining the accuracy of diagnoses made after reviewing case vignettes of depression and schizophrenia. The study found no statistically significant effect of VR training on diagnostic accuracy between the simulation and the non-simulation cohort.

Summary: All three studies assessing the effect of a VR intervention on knowledge were non-randomized studies with different study designs using different outcome measures. Two studies used an IPS design, while one study used a VP game design. Two of the studies found a significant effect of VR training on knowledge. Of these, one study had a moderate overall risk of bias [ 26 ], while the other was assessed as having a serious overall risk of bias [ 25 ]. The third study, which did not find any effect of the virtual reality intervention on knowledge, was assessed to have a serious risk of bias [ 24 ].

Three RCTs assessed the effectiveness of VR training on skills [ 20 , 21 , 22 ]. One of them evaluated the effect of VR training on clinical skills in alcohol screening and intervention [ 20 ]. In this study, 102 health care professionals were randomly allocated to either a group receiving no training or a group receiving a VSP intervention. To evaluate the outcome, three standardized patients rated each participant using a checklist based on clinical criteria. The VSP intervention group demonstrated significantly improved posttest skills in alcohol screening and brief intervention compared to the control group, with moderate and small effect sizes, respectively.

Another RCT, including 67 medical college students, evaluated the effect of VR training on clinical skills by comparing the frequency of questions asked about suicide in a VSP intervention group and a video module group [ 21 ]. The assessment of the outcome was a psychiatric interview with a standardized patient. The primary outcome was the frequency with which the students asked the standardized patient five questions about suicide risk. Minimal to small effect sizes were noted in favor of the VSP intervention, though they did not achieve statistical significance for any outcomes.

One posttest only RCT evaluated the effect of three training programs on skills in detecting and diagnosing major depressive disorder and posttraumatic stress disorder (PTSD) [ 22 ]. The study included 30 family physicians, and featured interventions that consisted of two different VSPs designed to simulate training situations, and one text-based program. A diagnostic form filled in by the participants after the intervention was used to assess the outcome. The results revealed a significant effect on diagnostic accuracy for major depressive disorder for both groups receiving VR training, compared to the text-based program, with large effect sizes observed. For PTSD, the intervention using a fixed avatar significantly improved diagnostic accuracy with a large effect size, whereas the intervention with a choice avatar demonstrated a moderate to large effect size compared to the text-based program.

Summary: Three RCTs assessed the effectiveness of VR training on clinical skills [ 20 , 21 , 22 ], all of which used a VSP design. To evaluate the effect of training, two of the studies utilized standardized patients with checklists. The third study measured the effect on skills using a diagnostic form completed by the participants. Two of the studies found a significant effect on skills [ 20 , 22 ], both were assessed to have a high risk of bias. The third study, which did not find any effect of VR training on skills, had some concern for risk of bias [ 21 ].

Knowledge and skills

One RCT study with 227 health care professionals assessed knowledge and skills as a combined outcome compared to a waitlist control group, using a self-report survey before and after the VR training [ 19 ]. The training intervention was a VP game designed to practice knowledge and skills related to mental health and substance abuse disorders. To assess effect of the training, participants completed a self-report scale measuring perceived knowledge and skills. Changes between presimulation and postsimulation scores were reported only for the within treatment group ( n = 117), where the composite postsimulation score was significantly higher than the presimulation score, with a large effect size observed. The study was judged to have a high risk of bias in the domain of selection of the reported result.

One single group pretest–posttest study with 100 social work and nursing students assessed the effect of VSP training on attitudes towards individuals with substance abuse disorders [ 23 ]. To assess the effect of the training, participants completed an online pretest and posttest survey including questions from a substance abuse attitudes survey. This study found no significant effect of VR training on attitudes and was assessed as having a serious risk of bias.

Perceived competence

The same single group pretest–posttest study also assessed the effect of a VSP training intervention on perceived competence in screening, brief intervention, and referral to treatment in encounters with patients with substance abuse disorders [ 23 ]. A commonly accepted definition of competence is that it comprises integrated components of knowledge, skills, and attitudes that enable the successful execution of a professional task [ 32 ]. To assess the effect of the training, participants completed an online pretest and posttest survey including questions on perceived competence. The study findings demonstrated a significant increase in perceived competence following the VSP intervention. The risk of bias in this study was judged as serious.

This systematic review aimed to investigate the effectiveness of VR training on knowledge, skills, and attitudes that health professionals need to master in the assessment and treatment of mental health disorders. A narrative synthesis of eight included studies identified VR training interventions that varied in design and educational content. Although mixed results emerged, most studies reported improvements in knowledge and skills after VR training.

We found that all interventions utilized some type of VP design, predominantly VSP interventions. Although our review includes a limited number of studies, it is noteworthy that the distribution of interventions contrasts with a literature review on the use of ‘virtual patient’ in health care education from 2015 [ 30 ], which identified IPS as the most frequent intervention. This variation may stem from our review’s focus on the mental health field, suggesting a different intervention need and distribution than that observed in general medical education. A fundamental aspect of mental health education involves training skills needed for interpersonal communication, clinical interviews, and symptom assessment, which makes VSPs particularly appropriate. While VP games may be suitable for clinical reasoning in medical fields, offering the opportunity to perform technical medical procedures in a virtual environment, these designs may present some limitations for skills training in mental health education. Notably, avatars in a VP game do not comprehend natural language and are incapable of engaging in conversations. Therefore, the continued advancement of conversational agents like VSPs is particularly compelling and considered by scholars to hold the greatest potential for clinical skills training in mental health education [ 3 ]. VSPs, equipped with AI dialogue capabilities, are particularly valuable for repetitive practice in key skills such as interviewing and counseling [ 31 ], which are crucial in the assessment and treatment of mental health disorders. VSPs could also be a valuable tool for the implementation of training methods in mental health education, such as deliberate practice, a method that has gained attention in psychotherapy training in recent years [ 33 ] for its effectiveness in refining specific performance areas through consistent repetition [ 34 ]. Within this evolving landscape, AI system-based large language models (LLMs) like ChatGPT stand out as a promising innovation. Developed from extensive datasets that include billions of words from a variety of sources, these models possess the ability to generate and understand text in a manner akin to human interaction [ 35 ]. The integration of LLMs into educational contexts shows promise, yet careful consideration and thorough evaluation of their limitations are essential [ 36 ]. One concern regarding LLMs is the possibility of generating inaccurate information, which represents a challenge in healthcare education where precision is crucial [ 37 ]. Furthermore, the use of generative AI raises ethical questions, notably because of potential biases in the training datasets, including content from books and the internet that may not have been verified, thereby risking the perpetuation of these biases [ 38 ]. Developing strategies to mitigate these challenges is imperative, ensuring LLMs are utilized safely in healthcare education.

All interventions in our review were based on non-immersive desktop VR systems, which is somewhat surprising considering the growing body of literature highlighting the impact of immersive VR technology in education, as exemplified by reviews such as that of Radianti et al. [ 39 ]. Furthermore, given the recent accessibility of affordable, high-quality head-mounted displays, this observation is noteworthy. Research has indicated that immersive learning based on head-mounted displays generally yields better learning outcomes than non-immersive approaches [ 40 ], making it an interesting research area in mental health care training and education. Studies using immersive interventions were excluded in the present review because of methodological concerns, paralleling findings described in a systematic review on immersive VR in education [ 41 ], suggesting the potential early stage of research within this field. Moreover, the integration of immersive VR technology into mental health care education may encounter challenges associated with complex ethical and regulatory frameworks, including data privacy concerns exemplified by the Oculus VR headset-Facebook integration, which could restrict the implementation of this technology in healthcare setting. Prioritizing specific training methodologies for enhancing skills may also affect the utilization of immersive VR in mental health education. For example, integrating interactive VSPs into a fully immersive VR environment remains a costly endeavor, potentially limiting the widespread adoption of immersive VR in mental health care. Meanwhile, the use of 360-degree videos in immersive VR environments for training purposes [ 42 ] can be realized with a significantly lower budget. Immersive VR offers promising opportunities for innovative training, but realizing its full potential in mental health care education requires broader research validation and the resolution of existing obstacles.

This review bears some resemblance to the systematic review by Jensen et al. on virtual patients in undergraduate psychiatry education [ 13 ] from 2024, which found that virtual patients improved learning outcomes compared to traditional methods. However, these authors’ expansion of the commonly used definition of virtual patient makes their results difficult to compare with the findings in the present review. A recognized challenge in understanding VR application in health care training arises from the literature on VR training for health care personnel, where ‘virtual patient’ is a term broadly used to describe a diverse range of VR interventions, which vary significantly in technology and educational design [ 3 , 30 ]. For instance, reviews might group different interventions using various VR systems and designs under a single label (virtual patient), or primary studies may use misleading or inadequately defined classifications for the virtual patient interventions evaluated. Clarifying the similarities and differences among these interventions is vital to inform development and enhance communication and understanding in educational contexts [ 43 ].

Strengths and limitations

To the best of our knowledge, this is the first systematic review to evaluate the effectiveness of VR training on knowledge, skills, and attitudes in health care professionals and students in assessing and treating mental health disorders. This review therefore provides valuable insights into the use of VR technology in training and education for mental health care. Another strength of this review is the comprehensive search strategy developed by a senior academic librarian at Inland Norway University of Applied Sciences (HINN) and the authors in collaboration with an adviser from KildeGruppen AS (a Norwegian media company). The search strategy was peer-reviewed by an academic librarian at HINN. Advisers from VRINN (an immersive learning cluster in Norway) and SIMInnlandet (a center for simulation in mental health care at Innlandet Hospital Trust) provided assistance in reviewing the VR systems of the studies, while the classification of the learning designs was conducted under the guidance of a VP scholar. This systematic review relies on an established and recognized classification of VR interventions for training health care personnel and may enhance understanding of the effectiveness of VR interventions designed for the training of mental health care personnel.

This review has some limitations. As we aimed to measure the effect of the VR intervention alone and not the effect of a blended training design, the selection of included studies was limited. Studies not covered in this review might have offered different insights. Given the understanding that blended learning designs, where technology is combined with other forms of learning, have significant positive effects on learning outcomes [ 44 ], we were unable to evaluate interventions that may be more effective in clinical settings. Further, by limiting the outcomes to knowledge, skills, and attitudes, we might have missed insights into other outcomes that are pivotal to competence acquisition.

Limitations in many of the included studies necessitate cautious interpretation of the review’s findings. Small sample sizes and weak designs in several studies, coupled with the use of non-validated outcome measures in some studies, diminish the robustness of the findings. Furthermore, the risk of bias assessment in this review indicates a predominantly high or serious risk of bias across most of the studies, regardless of their design. In addition, the heterogeneity of the studies in terms of study design, interventions, and outcome measures prevented us from conducting a meta-analysis.

Further research

Future research on the effectiveness of VR training for specific learning outcomes in assessing and treating mental health disorders should encompass more rigorous experimental studies with larger sample sizes. These studies should include verifiable descriptions of the VR interventions and employ validated tools to measure outcomes. Moreover, considering that much professional learning involves interactive and reflective practice, research on VR training would probably be enhanced by developing more in-depth study designs that evaluate not only the immediate learning outcomes of VR training but also the broader learning processes associated with it. Future research should also concentrate on utilizing immersive VR training applications, while additionally exploring the integration of large language models to augment interactive learning in mental health care. Finally, this review underscores the necessity in health education research involving VR to communicate research findings using agreed terms and classifications, with the aim of providing a clearer and more comprehensive understanding of the research.

This systematic review investigated the effect of VR training interventions on knowledge, skills, and attitudes in the assessment and treatment of mental health disorders. The results suggest that VR training interventions can promote knowledge and skills acquisition. Further studies are needed to evaluate VR training interventions as a learning tool for mental health care providers. This review emphasizes the necessity to improve future study designs. Additionally, intervention studies of immersive VR applications are lacking in current research and should be a future area of focus.

Availability of data and materials

Detailed search strategies from each database is available in the DataverseNO repository, https://doi.org/10.18710/TI1E0O .

Abbreviations

Virtual Reality

Cave Automatic Virtual Environment

Randomized Controlled Trial

Non-Randomized study

Virtual Standardized Patient

Interactive Patient Scenario

Virtual Patient

Post Traumatic Stress Disorder

Standardized Patient

Artificial intelligence

Inland Norway University of Applied Sciences

Doctor of Philosophy

Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010;376(9756):1923–58.

Article Google Scholar

World Health Organization. eLearning for undergraduate health professional education: a systematic review informing a radical transformation of health workforce development. Geneva: World Health Organization; 2015.

Google Scholar

Talbot T, Rizzo AS. Virtual human standardized patients for clinical training. In: Rizzo AS, Bouchard S, editors. Virtual reality for psychological and neurocognitive interventions. New York: Springer; 2019. p. 387–405.

Chapter Google Scholar

Merriam-Webster dictionary. Springfield: Merriam-Webster Incorporated; c2024. Virtual reality. Available from: https://www.merriam-webster.com/dictionary/virtual%20reality . [cited 2024 Mar 24].

Winn W. A conceptual basis for educational applications of virtual reality. Technical Publication R-93–9. Seattle: Human Interface Technology Laboratory, University of Washington; 1993.

Bouchard S, Rizzo AS. Applications of virtual reality in clinical psychology and clinical cognitive neuroscience–an introduction. In: Rizzo AS, Bouchard S, editors. Virtual reality for psychological and neurocognitive interventions. New York: Springer; 2019. p. 1–13.

Waller D, Hodgson E. Sensory contributions to spatial knowledge of real and virtual environments. In: Steinicke F, Visell Y, Campos J, Lécuyer A, editors. Human walking in virtual environments: perception, technology, and applications. New York: Springer New York; 2013. p. 3–26. https://doi.org/10.1007/978-1-4419-8432-6_1 .

Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R. Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg. 2013;80(5):e9–19. https://doi.org/10.1016/j.wneu.2012.08.022 .

Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, et al. Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg. 2005;241(2):364–72. https://doi.org/10.1097/01.sla.0000151982.85062.80 .

Kyaw BM, Saxena N, Posadzki P, Vseteckova J, Nikolaou CK, George PP, et al. Virtual reality for health professions education: systematic review and meta-analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(1):e12959. https://doi.org/10.2196/12959 .

Bracq M-S, Michinov E, Jannin P. Virtual reality simulation in nontechnical skills training for healthcare professionals: a systematic review. Simul Healthc. 2019;14(3):188–94. https://doi.org/10.1097/sih.0000000000000347 .

World Health Organization. Transforming and scaling up health professionals’ education and training: World Health Organization guidelines 2013. Geneva: World Health Organization; 2013. Available from: https://www.who.int/publications/i/item/transforming-and-scaling-up-health-professionals%E2%80%99-education-and-training . Accessed 15 Jan 2024.

Jensen RAA, Musaeus P, Pedersen K. Virtual patients in undergraduate psychiatry education: a systematic review and synthesis. Adv Health Sci Educ. 2024;29(1):329–47. https://doi.org/10.1007/s10459-023-10247-6 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev. 2021;10(1):89. https://doi.org/10.1186/s13643-021-01626-4 .

Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, et al. PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews. Syst Rev. 2021;10(1):39. https://doi.org/10.1186/s13643-020-01542-z .

Sandieson RW, Kirkpatrick LC, Sandieson RM, Zimmerman W. Harnessing the power of education research databases with the pearl-harvesting methodological framework for information retrieval. J Spec Educ. 2010;44(3):161–75. https://doi.org/10.1177/0022466909349144 .

Steen CW, Söderström K, Stensrud B, Nylund IB, Siqveland J. Replication data for: the effectiveness of virtual reality training on knowledge, skills and attitudes of health care professionals and students in assessing and treating mental health disorders: a systematic review. In: Inland Norway University of Applied S, editor. V1 ed: DataverseNO; 2024. https://doi.org/10.18710/TI1E0O .

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. https://doi.org/10.1186/s13643-016-0384-4 .

Albright G, Bryan C, Adam C, McMillan J, Shockley K. Using virtual patient simulations to prepare primary health care professionals to conduct substance use and mental health screening and brief intervention. J Am Psych Nurses Assoc. 2018;24(3):247–59. https://doi.org/10.1177/1078390317719321 .

Fleming M, Olsen D, Stathes H, Boteler L, Grossberg P, Pfeifer J, et al. Virtual reality skills training for health care professionals in alcohol screening and brief intervention. J Am Board Fam Med. 2009;22(4):387–98. https://doi.org/10.3122/jabfm.2009.04.080208 .

Foster A, Chaudhary N, Murphy J, Lok B, Waller J, Buckley PF. The use of simulation to teach suicide risk assessment to health profession trainees—rationale, methodology, and a proof of concept demonstration with a virtual patient. Acad Psych. 2015;39:620–9. https://doi.org/10.1007/s40596-014-0185-9 .

Satter R. Diagnosing mental health disorders in primary care: evaluation of a new training tool [dissertation]. Tempe (AZ): Arizona State University; 2012.

Hitchcock LI, King DM, Johnson K, Cohen H, McPherson TL. Learning outcomes for adolescent SBIRT simulation training in social work and nursing education. J Soc Work Pract Addict. 2019;19(1/2):47–56. https://doi.org/10.1080/1533256X.2019.1591781 .

Liu W. Virtual simulation in undergraduate nursing education: effects on students’ correct recognition of and causative beliefs about mental disorders. Comput Inform Nurs. 2021;39(11):616–26. https://doi.org/10.1097/CIN.0000000000000745 .

Matsumura Y, Shinno H, Mori T, Nakamura Y. Simulating clinical psychiatry for medical students: a comprehensive clinic simulator with virtual patients and an electronic medical record system. Acad Psych. 2018;42(5):613–21. https://doi.org/10.1007/s40596-017-0860-8 .

Pantziaras I, Fors U, Ekblad S. Training with virtual patients in transcultural psychiatry: Do the learners actually learn? J Med Internet Res. 2015;17(2):e46. https://doi.org/10.2196/jmir.3497 .

Wilson DB. Practical meta-analysis effect size calculator [Online calculator]. n.d. https://campbellcollaboration.org/research-resources/effect-size-calculator.html . Accessed 08 March 2024.

Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. Br Med J. 2019;366:l4898.

Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355:i4919. https://doi.org/10.1136/bmj.i4919 .

Kononowicz AA, Zary N, Edelbring S, Corral J, Hege I. Virtual patients - what are we talking about? A framework to classify the meanings of the term in healthcare education. BMC Med Educ. 2015;15(1):11. https://doi.org/10.1186/s12909-015-0296-3 .

Talbot TB, Sagae K, John B, Rizzo AA. Sorting out the virtual patient: how to exploit artificial intelligence, game technology and sound educational practices to create engaging role-playing simulations. Int J Gaming Comput-Mediat Simul. 2012;4(3):1–19.

Baartman LKJ, de Bruijn E. Integrating knowledge, skills and attitudes: conceptualising learning processes towards vocational competence. Educ Res Rev. 2011;6(2):125–34. https://doi.org/10.1016/j.edurev.2011.03.001 .

Mahon D. A scoping review of deliberate practice in the acquisition of therapeutic skills and practices. Couns Psychother Res. 2023;23(4):965–81. https://doi.org/10.1002/capr.12601 .

Ericsson KA, Lehmann AC. Expert and exceptional performance: evidence of maximal adaptation to task constraints. Annu Rev Psychol. 1996;47(1):273–305.

Roumeliotis KI, Tselikas ND. ChatGPT and Open-AI models: a preliminary review. Future Internet. 2023;15(6):192. https://doi.org/10.3390/fi15060192 .

Kasneci E, Sessler K, Küchemann S, Bannert M, Dementieva D, Fischer F, et al. ChatGPT for good? On opportunities and challenges of large language models for education. Learn Individ Differ. 2023;103:102274. https://doi.org/10.1016/j.lindif.2023.102274 .

Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29(8):1930–40. https://doi.org/10.1038/s41591-023-02448-8 .

Touvron H, Lavril T, Gautier I, Martinet X, Marie-Anne L, Lacroix T, et al. LLaMA: open and efficient foundation language models. arXivorg. 2023;2302.13971. https://doi.org/10.48550/arxiv.2302.13971 .

Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I. A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput Educ. 2020;147:103778. https://doi.org/10.1016/j.compedu.2019.103778 .

Wu B, Yu X, Gu X. Effectiveness of immersive virtual reality using head-mounted displays on learning performance: a meta-analysis. Br J Educ Technol. 2020;51(6):1991–2005. https://doi.org/10.1111/bjet.13023 .

Di Natale AF, Repetto C, Riva G, Villani D. Immersive virtual reality in K-12 and higher education: a 10-year systematic review of empirical research. Br J Educ Technol. 2020;51(6):2006–33. https://doi.org/10.1111/bjet.13030 .

Haugan S, Kværnø E, Sandaker J, Hustad JL, Thordarson GO. Playful learning with VR-SIMI model: the use of 360-video as a learning tool for nursing students in a psychiatric simulation setting. In: Akselbo I, Aune I, editors. How can we use simulation to improve competencies in nursing? Cham: Springer International Publishing; 2023. p. 103–16. https://doi.org/10.1007/978-3-031-10399-5_9 .

Huwendiek S, De leng BA, Zary N, Fischer MR, Ruiz JG, Ellaway R. Towards a typology of virtual patients. Med Teach. 2009;31(8):743–8. https://doi.org/10.1080/01421590903124708 .

Ødegaard NB, Myrhaug HT, Dahl-Michelsen T, Røe Y. Digital learning designs in physiotherapy education: a systematic review and meta-analysis. BMC Med Educ. 2021;21(1):48. https://doi.org/10.1186/s12909-020-02483-w .

Download references

Acknowledgements

The authors thank Mole Meyer, adviser at SIMInnlandet, Innlandet Hospital Trust, and Keith Mellingen, manager at VRINN, for their assistance with the categorization and classification of VR interventions, and Associate Professor Inga Hege at the Paracelcus Medical University in Salzburg for valuable contributions to the final classification of the interventions. The authors would also like to thank Håvard Røste from the media company KildeGruppen AS, for assistance with the search strategy; Academic Librarian Elin Opheim at the Inland Norway University of Applied Sciences for valuable peer review of the search strategy; and the Library at the Inland Norway University of Applied Sciences for their support. Additionally, we acknowledge the assistance provided by OpenAI’s ChatGPT for support with translations and language refinement.

Open access funding provided by Inland Norway University Of Applied Sciences The study forms a part of a collaborative PhD project funded by South-Eastern Norway Regional Health Authority through Innlandet Hospital Trust and the Inland University of Applied Sciences.

Author information

Authors and affiliations.

Mental Health Department, Innlandet Hospital Trust, P.B 104, Brumunddal, 2381, Norway

Cathrine W. Steen & Kerstin Söderström

Inland Norway University of Applied Sciences, P.B. 400, Elverum, 2418, Norway

Cathrine W. Steen, Kerstin Söderström & Inger Beate Nylund

Norwegian National Advisory Unit On Concurrent Substance Abuse and Mental Health Disorders, Innlandet Hospital Trust, P.B 104, Brumunddal, 2381, Norway

Bjørn Stensrud

Akershus University Hospital, P.B 1000, Lørenskog, 1478, Norway

Johan Siqveland

National Centre for Suicide Research and Prevention, Oslo, 0372, Norway

You can also search for this author in PubMed Google Scholar

Contributions

CWS, KS, BS, and JS collaboratively designed the study. CWS and JS collected and analysed the data and were primarily responsible for writing the manuscript text. All authors contributed to the development of the search strategy. IBN conducted the literature searches and authored the chapter on the search strategy in the manuscript. All authors reviewed, gave feedback, and granted their final approval of the manuscript.

Corresponding author

Correspondence to Cathrine W. Steen .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Not applicable .

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: table 2..

Effects of VR training in the included studies: Randomized controlled trials (RCTs) and non-randomized studies (NRSs).

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Steen, C.W., Söderström, K., Stensrud, B. et al. The effectiveness of virtual reality training on knowledge, skills and attitudes of health care professionals and students in assessing and treating mental health disorders: a systematic review. BMC Med Educ 24 , 480 (2024). https://doi.org/10.1186/s12909-024-05423-0

Download citation

Received : 19 January 2024

Accepted : 12 April 2024

Published : 01 May 2024

DOI : https://doi.org/10.1186/s12909-024-05423-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Health care professionals
Health care students
Virtual reality
Mental health
Clinical skills
Systematic review

BMC Medical Education

ISSN: 1472-6920

Submission enquiries: [email protected]
General enquiries: [email protected]

Open access
Published: 01 May 2024

Hospital performance evaluation indicators: a scoping review

Shirin Alsadat Hadian ORCID: orcid.org/0000-0002-1443-1990 1 ,
Reza Rezayatmand ORCID: orcid.org/0000-0002-9907-3597 2 ,
Nasrin Shaarbafchizadeh ORCID: orcid.org/0000-0001-7104-2214 3 ,
Saeedeh Ketabi ORCID: orcid.org/0000-0002-6778-5645 4 &
Ahmad Reza Pourghaderi ORCID: orcid.org/0000-0003-2682-2160 5

BMC Health Services Research volume 24 , Article number: 561 ( 2024 ) Cite this article

336 Accesses

Metrics details

Hospitals are the biggest consumers of health system budgets and hence measuring hospital performance by quantitative or qualitative accessible and reliable indicators is crucial. This review aimed to categorize and present a set of indicators for evaluating overall hospital performance.

We conducted a literature search across three databases, i.e., PubMed, Scopus, and Web of Science, using possible keyword combinations. We included studies that explored hospital performance evaluation indicators from different dimensions.

We included 91 English language studies published in the past 10 years. In total, 1161 indicators were extracted from the included studies. We classified the extracted indicators into 3 categories, 14 subcategories, 21 performance dimensions, and 110 main indicators. Finally, we presented a comprehensive set of indicators with regard to different performance dimensions and classified them based on what they indicate in the production process, i.e., input, process, output, outcome and impact.

The findings provide a comprehensive set of indicators at different levels that can be used for hospital performance evaluation. Future studies can be conducted to validate and apply these indicators in different contexts. It seems that, depending on the specific conditions of each country, an appropriate set of indicators can be selected from this comprehensive list of indicators for use in the performance evaluation of hospitals in different settings.

Peer Review reports

Healthcare is complex [ 1 ] and a key sector [ 2 ] that is now globally faced with problems of rising costs, lack of service efficiency, competition, and equity as well as responsiveness to users [ 3 ]. One estimate by the WHO has shown a yearly waste of approximately 20–40% of total healthcare resources because of inefficiency [ 4 ]. European countries have spent on average 9.6% of their gross domestic product (GDP) on healthcare in 2017 and 9.92% in 2019. Germany, France, and Sweden reported the highest healthcare expenditures in Europe in 2018 (between 10.9% and 11.5% of GDP) [ 5 ]. In the U.S., healthcare spending consumes 18% of the GDP, which is likely to eclipse $6 trillion by 2027 [ 6 ].

Hospitals, as the biggest consumers of health system budgets [ 7 ], are the major part of the health system [ 8 ]. In many countries 50–80% of the health sector budget is dedicated to hospitals [ 8 , 9 ]. As a result, hospital performance analysis is becoming a routine task for every hospital manager. On the one hand, hospital managers worldwide are faced with difficult decisions regarding cost reduction, increasing service efficiency, and equity [ 10 ]. On the other hand, measuring hospital efficiency is an issue of interest among researchers because patients demand high-quality care at lower expenses [ 11 ].

To address the above mentioned need to measure hospital performance, implementing an appropriate hospital performance evaluation system is crucial in any hospital. In doing so, hospital administrators use various tools to analyse and monitor hospital activities [ 1 ], which need well-defined objectives, standards and quantitative indicators [ 12 ]. The latter are used to evaluate care provided to patients both quantitatively and qualitatively and are often related to input, output, processes, and outcomes. These indicators can be used for continuous quality improvement by monitoring, benchmarking, and prioritizing activities [ 13 ]. These parameters are developed to improve health outcomes and to provide comparative information for monitoring and managing and formulating policy objectives within and across health services [ 12 ]. Studies thus far have used their own set of indicators while evaluating hospital performance, which could be context dependent. In addition, those studies have mostly used a limited set of indicators that focus on few dimensions (2–6 dimensions) of hospital performance [ 14 , 15 , 16 , 17 , 18 ].

Therefore, comprehensive knowledge of potential indicators that can be used for hospital performance evaluation is necessary. It would help choose appropriate indicators when evaluating hospital performance in different contexts. It would also help researchers extend the range of analysis to evaluate performance from a wider perspective by considering more dimensions of performance. Although performance is a very commonly used term, it has several definitions [ 19 , 20 ], yet, it is often misunderstood [ 21 ]. Therefore, some researchers have expressed confusion about the related terms and considered them interchangeable. These terms are effectiveness, efficiency, productivity, quality, flexibility, creativity, sustainability, evaluation, and piloting [ 21 , 22 , 23 ]. Thus, this scoping review aimed to categorize and present a comprehensive set of indicators that can be used as a suitable set for hospital performance evaluation at any needed level of analysis, i.e., clinical, para-clinical, logistical, or departmental, and relate those indicators to the appropriate performance dimensions. The uniqueness of this paper is that it provides its readers with a comprehensive collection of indicators that have been used in different performance analysis studies.

Materials and methods

We conducted a scoping review of a body of literature. The scoping review can be of particular use when the topic has not yet been extensively reviewed or has a complex or heterogeneous nature. This type of review is commonly undertaken to examine the extent, range, and nature of research activity in a topic area; determine the value and potential scope and cost of undertaking a full systematic review; summarize and disseminate research findings; and identify research gaps in the existing literature. As a scoping review provides a rigorous and transparent method for mapping areas of research, it can be used as a standalone project or as a preliminary step to a systematic review [ 24 ]. While a systematic review (qualitative or quantitative) usually addresses a narrow topic/scope and is a method for integrating or comparing findings from previous studies [ 25 ].

In our study, we used the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist following the methods outlined by Arksey and O’Malley [ 26 ] and Tricco [ 27 ]. A systematic search for published and English-language literature on hospital performance evaluation models was conducted, using three databases, i.e., PubMed, Scopus, and Web of Science, from 2013 to January 2023. Initially, the identified keywords were refined and validated by a team of experts. Then, a combination of vocabularies was identified by the authors through a brainstorming process. The search strategy was formulated using Boolean operators. The title and abstract of the formulas were searched in the online databases. The search query for each database is presented in Table 1 .

In the screening process, relevant references related to hospital performance evaluation were screened and abstracted into researcher-developed Microsoft® Excel forms by dual independent reviewers and conflicting information was provided by other reviewers.

The inclusion criteria were as follows: focused only on the hospital setting, available full text and written in English. We excluded studies that focused on health organization indicators, not specifically on hospital indicators; articles without appropriate data (only focused on models and not indicators; or qualitative checklist questionnaires); and articles that focused only on clinical or disease-related indicators, not hospital performance dimensions, and provided very general items as indicators, not the domains of the indicators themselves. Then, a PRISMA-ScR Checklist was used to improve transparency in our review [ 28 ].

To extract the data, researcher-developed Microsoft® Excel forms (data tables) were designed. The following data were subsequently extracted into Microsoft®Excel for synthesis and evaluation: title, author, article year, country, indicator category, study environment (number of hospitals studied), study time frame, indicator name, number of indicators, indicator level (hospital level, department level), evaluation perspective (performance, productivity, efficiency, effectiveness, quality, cost, safety, satisfaction, etc. ) , study type (quantitative or qualitative), indicator subtype (input (structure), process, output (result), outcome and impact), and other explanations. To create a descriptive summary of the results that address the objectives of this scoping review, numerical summarization was also used.

The purpose of creating the main category and the evaluation perspective section was to develop them and create new categories, which focused on the type of indicators related to the performance term. For example, in the “Category” section, the names of the departments or wards of the hospital (such as hospital laboratories, pharmacies, clinical departments, and warehouses) and in the “Evaluation perspective” section, various terms related to the evaluation of hospital performance were extracted. These two types were used after extracting their information under the title “performance dimension”.

The indicators’ levels were collected to determine the level of performance evaluation with the relevant index. Some indicators were used to evaluate the performance of the entire hospital, some were used to evaluate the performance of hospital departments, and some were used to evaluate the performance at the level of a specific project. For example, several indicators (such as bed occupancy ratio, length of stay, and waiting time) were used to evaluate the performance of the entire hospital, and other indicators (such as laboratory department indicators, energy consumption indicators, and neonatal department indicators) were used only to measure the performance of specific departments. This sections were used under the title “category”. The “category” and “indicator’s name” sections were defined according to the results of the “subcategory” section.

The subtypes of indicators (input (structure), process, output(result), outcome and impact) were defined based on the chain model, and each of the selected indicators was linked to it (Appendix 1 ). As a result of the chain model, inputs were used to carry out activities, activities led to the delivery of services or products (outputs). The outputs started to bring about change (outcomes), and eventually, this (hopefully) contributed to the impact [ 29 ]. The classification of the set of input, process, output, outcome and impact indicators was such that readers could access these categories if necessary according to their chosen evaluation models. The term was used under the title “Indicators by types”.

The type of study was considered quantitative or qualitative for determining whether an indicator was able to perform calculations. In this way, readers can choose articles that use quantitative or qualitative indicators to evaluate hospital performance.

We included 91 full-text studies (out of 7475) in English published between 2013 and January 2023 (Fig. 1 ), approximately 40% of which were published between 2020 and 2023. More than 20% of the retrieved studies were conducted in Iran and USA.

Study selection and data abstraction

Study characteristic

As shown in Table 2 , in 85% of the reviewed studies, a number of hospitals (1 to 3828 hospitals, 13,221 hospitals in total) were evaluated. More than 90% of the studies used a quantitative approach. In more than 70% of the studies, hospital evaluation occurred at the department level, which can also be divided into three levels: administrative, clinical ward, and paramedical department. In addition, the administrative departments consist of 13 departments, including financial management [ 48 , 55 , 61 , 67 , 68 , 80 , 83 , 109 , 113 ], supply chain management and warehouse [ 15 , 43 , 84 ], value-based purchasing [ 33 , 85 ], human resource management [ 97 , 101 ], medical equipment [ 32 , 87 ], health information management department [ 90 ], information systems [ 106 ], nutritional assessment [ 93 ], energy management [ 30 , 45 , 92 ], facility management [ 52 , 53 ], building sustainability and resilience [ 35 ], research activities [ 44 ], and education [ 107 ].

The clinical wards consisted of 8 wards, namely, emergency departments (EDs) [ 16 , 39 , 56 , 57 , 69 , 70 , 89 ], surgery departments [ 58 , 62 , 63 , 91 , 102 ], intensive care units (ICUs) [ 47 , 64 , 65 ], operating rooms (ORs) [ 38 , 88 , 108 ], surgical intensive care units (SICUs) [ 111 ], obstetrics and gynecology department [ 59 ], neonatal intensive care units (NICUs) [ 74 , 103 ] and quality of care [ 18 , 31 , 40 , 50 , 72 , 92 , 95 , 112 ] indicators. The paramedical departments consisted of 3 departments, pharmacy [ 60 , 76 , 98 ], laboratory and blood bank [ 37 , 42 , 43 , 49 ], and outpatient assessment [ 86 ] indicators.

With regard to data categorization, firstly, a total of 1204 indicators in 91 studies were extracted and after detailed examination, 43 indices (such as hospital ownership, level of care, admission process, and personal discipline) were removed due to their generality and impossibility of calculation in the hospital environment. Then, 1161 performance indicators were entered in this research and were categorized based on the performance criteria (more details about the indicators can be found in Appendix 1 ). Secondly, 145 functional dimensions, including divisions based on different departments and units of the hospital, were defined according to several focus group discussions with 5 health experts. Then, re-categorization and functional summarization were performed, after which 21 performance dimensions were finalized.

As shown in Table 4 , the 21 performance dimensions were divided into three parts: category, subcategory, and related indicators. Additionally, according to the hospital levels, there were three categories: ‘organizational management’, ‘clinical management’, and ‘administrative management’. Then, according to the type of indicators, fifteen subcategories were defined for the 110 selected main indicators.

Performance dimensions

The ‘productivity’ dimension focuses on indicators reflecting the macro-performance of the hospital, considering that this index is more effective and efficient. The ‘efficiency’ dimension focuses on general performance indicators for the optimal use of resources to create optimal output in the hospital. The ‘effectiveness’ dimension is a general performance indicator with an outcome view. The ‘speed’ dimension focuses on the indicators that show attention to the service delivery time and the speed of the procedures. The ‘development’ dimension focuses on matters related to employees’ and students’ training and related training courses. In terms of ‘safety’ dimension, there were issues related to patient safety, unwanted and harmful events, and hospital infections.

The “quality of work life” dimension emphasizes matters related to personnel volume and work conditions. The ‘quality’ dimension is related to the quality of service provided in different parts of the hospital and possible complications in improving the quality of services. The ‘satisfaction’ dimension focuses on the satisfaction of patients, employees, and their complaints. The ‘innovation’ dimension relates to the research process and its output. The ‘appropriateness’ dimension involves proper service from clinical departments, pharmaceutical services, and patient treatment. The ‘evaluation’ dimension focuses on the indicators related to the assessment scores of the para-clinical departments of the hospital.

The ‘profitability’ dimension focuses on the overall output indicators for income and profitability. The ‘cost’ dimension focuses on indicators related to general expenditures and the average cost per bed and patient and budgeting. The ‘economy’ dimension is related to financial rates and their indicators. The ‘coherence’ dimension emphasizes the indicators related to the continuity of the service delivery process. The ‘patient-centeredness’ dimension focuses on the indicators related to the patient’s experience of the facility, environment, treatment processes, communications, and relevant support for the patient. The ‘equity’ dimension studies indicators related to social and financial justice and life expectancy. The ‘relationship’ dimension evaluates the process of consultations and discussions required during the patients’ care provided by the treatment team. The ‘sustainability’ dimension focuses on indicators related to energy standards. The ‘flexibility’ dimension focuses on the hospital’s response to the crisis.

According to Table 4 , most studies focused on ‘efficiency’, ‘productivity’, ‘safety’ and ‘effectiveness’ as performance dimensions in 54, 53, 38 and 37 studies, respectively (40–70% of studies). In the ‘efficiency’ subcategory, resource management, supportive unit assessment, and human resource management indicators were the first to third most common indicators used in 26, 23 and 22 studies, respectively (approximately 25% of the studies).

In addition, for the ‘efficiency’ dimension, ‘medical staff numbers’, ‘emergency department bed numbers’, and ‘nonmedical staff numbers’ were reported in 16, 13, and 11 studies, respectively (between 20 and 30% of the studies). For the ‘productivity’ subcategory, ‘bed utilization rate’ and ‘service delivery and treatment’ were reported in 50% and 20% of the studies, respectively (46 and 19 out of 91).

Additionally, for the ‘productivity’ dimension, the ‘length of stay’ indicator was used more than others and reported in approximately 80% of the studies (43 out of 53), followed by the ‘bed occupancy rate’ in approximately 40% of the studies (21 out of 53). The ‘bed turnover ratio’ and ‘hospitalization rate’ were also reported in 12 studies. Furthermore, for ‘safety’ dimensions, all indicators were in the ‘patient safety’ subcategory, which has been reported in 38 studies, and ‘complications’, ‘accidents or adverse events’, and ‘incidents or errors rates’ were the most concentrated indicators by researchers in 13, 12, and 11 studies, respectively. The performance dimension of ‘effectiveness’ was presented in 37 studies (40%), with only two indicators, ‘mortality rate’ in 29 studies and ‘readmission rate’ in 23 studies.

Performance categories

Considering the three categories shown in Table 4 , ‘organizational management’ indicators were more commonly used among the other two categories (‘clinical’ and ‘administrative’) and were present in more than 85% of the studies (78 out of 91). Two categories, ‘clinical management’ and ‘administrative management’, were reported in 62 and 51 studies, respectively.

Performance subcategories

Considering the 14 subcategories shown in Table 4 , both the ‘bed utilization rate’ and ‘patient safety’ indicators were mentioned in 46 studies and were more common among the other subcategories. The second most common indicator of the ‘financial management’ subcategory was reported in 38 studies. At the third level, both the ‘human resource management’ and ‘time management’ indicators were presented in 31 studies. The ‘paramedical’ subcategory indicators were presented in less than 10% of the studies [ 60 , 96 , 97 , 98 , 106 , 113 ].

Performance indicators

According to the indicator columns in Table 3 , the most used indicators in reviewed studies were the length of stay, mortality rate, and readmission rate in 47%, 32%, and 25% of studies, respectively. Bed occupancy rate and non-personnel costs were reported in 23% of studies. Additionally, among the 110 indicators, 16 indicators, namely, the lab cancellation rate, exam-physician ratios, number of coded diagnoses, number of medical records, laboratory sample/report intervals, medical information request time, safety standards in the archives, nutritional risk screening, imaging quality control failures, errors in medical reports, average impact factor, nutritional measures, laboratory scoring, imaging inspection, discharge process and emergency response rate, were reported in less than 1% of the studies.

The classification of the indicators in Table 4 was performed based on the chain model, which included the input, process, output, outcome and impact. The assignment of the indicators to each category was performed according to the experts’ opinions. For instance, the number of publications by academic member of an academic hospital and the average impact factor of those publications were considered outcome indicators. As depicted in the Table 4 , most studies (80%) focused more on output indicators. Additionally, fifteen studies focused on introducing and extracting some of the input, process, output, outcome and impact indicators; among those, only one study [ 96 ] has examined the input, process, output and impact indicators simultaneously.

Additionally, in approximately 42% (36 out of 91) of the studies, the indicators’ definitions, formulas, or descriptions have been illustrated, while less than 10% of the studies have defined measuring units, standard or benchmark units for all studied indicators [ 15 , 43 , 45 , 51 , 52 , 57 , 67 ].

Overall, nine studies related to hospital performance evaluation were conducted using systematic review methodologies (five systematic reviews [ 16 , 29 , 30 , 56 , 113 ], two literature reviews [ 79 , 80 ], one narrative review [ 98 ] and one brief review [ 92 ]). Most of these studies focused on extracting performance indicators from one or more hospital departments (e.g., the emergency department) [ 16 , 56 ], hospital laboratory and radiology information systems [ 106 ], supply chain performance [ 29 ], resources and financial results and activity [ 113 ], hospital water consumption [ 30 ], and the pharmaceutical sector [ 98 ]. Other reviews included a three-step process to review, evaluate and rank these hospital indicators in a systematic approach [ 16 ], or to evaluate performance indicator models to create an interactive network and visualize the causal relationships between performance indicators [ 79 ]; moreover, some have focused on the importance of indicators to ensure adequate coverage of the relevant areas of health care services to be evaluated [ 92 ].

Only one scoping review aimed to identify current assessments of hospital performance and compared quality measures from each method in the context of the six qualitative domains of STEEEP (safety, timeliness, effectiveness, efficiency, equity, and patient-centeredness) of the Institute of Medicine (IOM) in accordance with Donabedian’s framework and formulating policy recommendations [ 115 ].

In addition, 21 studies divided performance indicators into 2 to 6 dimensions of performance. Also, the reviewed studies included 2–40 indicators in zero [ 29 , 30 , 98 ] to 6 domains [ 34 ]. Moreover, none of the studies have tried to comprehensively summarize and categorize the performance indicators in several categories, focusing on all the indicators reflecting the performance of the entire hospital organization, or the indicators of administrative units or clinical departments.

In this scoping review, a unique set of hospital performance evaluation indicators related to the various performance dimensions was categorized from 91 studies over the past ten years.

Similarly, in a study, 19 performance dimensions, 32 sub-dimensions, and 138 indicators were extracted from only six studies. Those dimensions were described by all studies included in the review, but only three studies specified the relevant indicators, and the list provided for all possible indicators was not comprehensive. Also, despite current review, there was no classification of indicators based on the hospital levels: managerial, clinical, or organizational levels [ 116 ]. Another study has similarly investigated the performance evaluation indicators of the hospital in such a way that among 42 studies, 111 indicators were presented in the four categories: input, output, outcome, and impact. But, there was no classification of indicators based on performance dimensions and hospital levels [ 117 ].

In this study, the importance of categorized indicators, for the first time to our knowledge, was determined based on their frequency of use in the published literature (Appendix 2 ). The ‘Organizational management’ indicators were the most common compared with the other two categories (‘clinical’ and ‘administrative’). It could be because of the fact that the indicators such as ‘bed occupancy rate’, ‘average length of stay’, ‘mortality rate’, ‘hospital infection rate’, and ‘patient safety’ are easier to be registered in hospital software compared to other indicators, and also they better reflect the overall performance of hospital. Thus, researchers are more interested in using these indicators.

Considering 14 subcategories, indicators related to three subcategories i.e. bed utilization, patient safety and financial management are the most frequent used indicators for hospital performance evaluation. It reflects the need of hospital managers to increase the profitability of hospital in one hand, and to control cost on the other hand. As a results, researchers have paid special attention to ‘cost income’, ‘profitability’, ‘economic’, etc., as indicators for evaluating hospital performance.

When considering indicators by type, more studies have focused on output indicators, while input indicators were the least common used. This might be because of the fact that at hospital level, it is difficult for managers to change those inputs such as ‘beds’, ‘human resources’, ‘equipment and facilities’. In addition, due to the complexity of interdepartmental relationships in hospitals, process indicators seemed to provide more variety for analysis than input indicators, so they were more often used. As mentioned above, output indicators were the most used indicators for hospital performance evaluation due to their ease of calculation and interpretation.

The main purpose of this paper was to identify a comprehensive set of indicators that can be used to evaluate hospital performance in various hospital settings by being distilled into a smaller and more related set of indicators for every hospital or department setting. future studies could be designed to validate each set of indicators in any specific context. In addition, they could investigate the relationship between the indicators and their outcomes of interest and the performance dimension each could address. This will enable hospital managers to build their own set of indicators for performance evaluation both at organization or at department level. Also it should be mentioned that.

Although some previous studies have provided definitions for each indicator and determined the standard criteria for them, this was not done in this study because the focus of this study was to provide a collection of all the indicators used in hospital performance evaluation, which resulted in the identification of more than a thousand indicators without limiting to specific country or context. So while preparing a smaller set of indicators, specific conditions of each country, such as the type of health system and its policy, the type of financing system, and the structure of services, should be taken into account to select appropriate indicators.

In addition, although it is important to examine the scope of each article to compare the list of indicators and the relationships between the dimensions of the hospital in terms of size and type and between the number and type of selected indicators, this was considered beyond the scope of this review due to the high number of indicators, which made the abovementioned investigations impossible. Future studies could do that while working with a smaller set of indicators.

This review aimed to categorize and present a comprehensive set of indicators for evaluating overall hospital performance in a systematic way. 1161 hospital performance indicators were drawn from 91 studies over the past ten years. They then were summarized into 110 main indicators, and categorized into three categories: 14 subcategories, and 21 performance dimensions This scoping review also highlighted the most frequent used indicators in performance evaluation studies which could reflect their importance for that purpose. The results of this review help hospital managers to build their own set of indicators for performance evaluation both at organization or at department level with regard to various performance dimensions.

As the results of this review was not limited to any specific country or context, specific conditions of each country, such as the type of health system and its policy, the type of financing system, and the structure of services, should be taken into account while selecting appropriate indicators as a smaller set of indicators for hospital performance evaluation in specific context.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Gross domestic product

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews

Emergency departments

Intensive care unit

Operating room

Surgical intensive care unit

Neonatal intensive care unit

Readmission rate

Quality Control

Medication use evaluation

safety, timeliness, effectiveness, efficiency, equity, and patient-centeredness

Institute of Medicine

Abdullah A, Ahmad S, Athar MA, Rajpoot N, Talib F. Healthcare performance management using integrated FUCOM-MARCOS approach: the case of India. Int J Health Plann Manage. 2022;37(5):2635–68.

Article PubMed Google Scholar

Pestana M, Pereira R, Moro S. Improving health care management in hospitals through a productivity dashboard. J Med Syst. 2020;44(4):87.

Amos D. A practical framework for performance measurement of facilities management services in developing countries’ public hospitals. J Facil Manag. 2022;20(5):713–31.

Article Google Scholar

Ahmed S, Hasan MZ, MacLennan M, Dorin F, Ahmed MW, Hasan MM, et al. Measuring the efficiency of health systems in Asia: a data envelopment analysis. BMJ Open. 2019;9(3):e022155.

Article PubMed PubMed Central Google Scholar

Mitkova Z, Doneva M, Gerasimov N, Tachkov K, Dimitrova M, Kamusheva M, et al. Analysis of healthcare expenditures in Bulgaria. Healthc. 2022;10(2):274.

Patrinely JR, Walker SH, Glassman GE, Davis MJ, Abu-Ghname A, Khan U, et al. The importance of financial metrics in physician funding and performance evaluation. Plast Reconstr Surg. 2021;147:1213–8.

Article CAS PubMed Google Scholar

Buathong S, Bangchokdee S. The use of the performance measures in Thai public hospitals. ASIAN Rev Acc. 2017;25(4):472–85.

Google Scholar

Imani A, Alibabayee R, Golestani M, Dalal K. Key indicators affecting hospital efficiency: a systematic review. Front Public Heal. 2022;10:830102.

Mahdiyan S, Dehghani A, Tafti AD, Pakdaman M, Askari R. Hospitals’ efficiency in Iran: a systematic review and meta-analysis. J Educ Health Promot. 2019;8(1):126.

PubMed PubMed Central Google Scholar

Amos D, Musa ZN, Au-Yong CP. Performance measurement of facilities management services in Ghana’s public hospitals. Build Res Inf. 2020;48(2):218–38.

Feibert DC, Andersen B, Jacobsen P. Benchmarking healthcare logistics processes–a comparative case study of Danish and US hospitals. Total Qual Manag Bus Excell. 2019;30(1–2):108–34.

Gün I, Yilmaz F, Şenel IK. Efficiency analysis of health systems in world bank countries. Arch Heal Sci Res. 2021;8(2):147–52.

Breyer JZ, Giacomazzi J, Kuhmmer R, Lima KM, Hammes LS, Ribeiro RA, et al. Hospital quality indicators: a systematic review. Int J Health Care Qual Assur. 2019;32(2):474–87.

Regragui H, Sefiani N, Azzouzi H. Improving performance through measurement: the application of BSC and AHP in healthcare organization. In: Equipe De Recherche, Ingénierie, Innovation Et Management Des Systèmes Industriels, Université Abdelmalek Saadi. Tanger, Morocco: Institute of Electrical and Electronics Engineers Inc; 2018. p. 51–6.

Ghozali MT, Latifah DN, Darayani A. Analysis of Drug Supply Management of the Pharmacy Warehouse of Prof. Dr. Soerojo Mental Health Hospital, Magelang, Indonesia. Clin Schizophr Relat Psychoses. 2021;15:1–6.

Etu EE, Monplaisir L, Aguwa C, Arslanturk S, Masoud S, Markevych I, et al. Identifying indicators influencing emergency department performance during a medical surge: a consensus-based modified fuzzy Delphi approach. PLoS ONE. 2022;17(4 April):e0265101.

Article CAS PubMed PubMed Central Google Scholar

Lin C-Y, Shih F-C, Ho Y-H. Applying the balanced scorecard to build service performance measurements of medical institutions: An AHP-DEMATEL approach. Int J Environ Res Public Health. 2023;20(2):1022.

Backman C, Vanderloo S, Forster AJ. Measuring and improving quality in university hospitals in Canada: the collaborative for excellence in healthcare quality. Health Policy (New York). 2016;120(9):982–6.

Ghalem Â, Okar C, Chroqui R, Semma E. Performance: A concept to define. In: Performance: A concept to define. LOGISTIQUA 2016; 2016. p. 1–13.

Sonnentag S, Frese M. Performance Concepts and Performance Theory. In 2005. p. 1–25.

Tangen S. Demystifying productivity and performance. Int J Prod Perform Manag. 2005;54:34–46.

Elena-Iuliana I, Maria C. Organizational Performance – A Concept That Self-Seeks To Find Itself. Ann - Econ Ser Constantin Brancusi Univ Fac Econ. 2016;4(4):179–83.

Riratanaphong C, Van der Voordt T, Sarasoja A. Performance Measurement in the context of CREM and FM. In: Per Anker Jensen, Theo Van der Voordt CC, editor. The added value of facilities management: concepts, findings and perspectives. Lyngby Denmark: Polyteknisk Forlag; 2012. p. 1–21.

Pham M, Rajić A, Greig J, Sargeant J, Papadopoulos A, Mcewen S. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5:371–85.

Chaney M. So you want to write a narrative review article? J Cardiothorac Vasc Anesth. 2021;35:3045–9.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Tricco A, Lillie E, Zarin W, O’Brien K, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Dolatabad AH, Mahdiraji HA, Babgohari AZ, Garza-Reyes JA, Ai A. Analyzing the key performance indicators of circular supply chains by hybrid fuzzy cognitive mapping and Fuzzy DEMATEL: evidence from healthcare sector. Environ Dev Sustain. 2022;1–27.

Batista KJM, da Silva SR, Rabbani ERK, Zlatar T. Systematic review of indicators for the assessment of water consumption rates at hospitals. Water Supply. 2020;20(2):373–82.

Beta G, Role D, Berloviene D, Balkena Z. PATIENTS’ SATISFICATION AS THE QUALITY INDICATOR OF NURSING. In: Lubkina V, Kaupuzs A, Znotina D, editors. SOCIETY INTEGRATION EDUCATION, VOL VI: PUBLIC HEALTH AND SPORT, RESEARCHES IN ECONOMICS AND MANAGEMENT FOR SUSTAINABLE EDUCATION. 2020. p. 79–88.

Bhardwaj P, Joshi NK, Singh P, Suthar P, Joshi V, Jain YK, et al. Competence-based assessment of biomedical equipment management and maintenance system (e-Upkaran) using benefit evaluation framework. CUREUS J Med Sci. 2022;14(10):e30579.

Cheon O, Song M, Mccrea AM, Meier KJ. Health care in America: the relationship between subjective and objective assessments of hospitals. Int PUBLIC Manag J. 2021;24(5):596–622.

Craig KJT, McKillop MM, Huang HT, George J, Punwani ES, Rhee KB. US hospital performance methodologies: a scoping review to identify opportunities for crossing the quality chasm. BMC Health Serv Res. 2020;20(1):640.

Cristiano S, Ulgiati S, Gonella F. Systemic sustainability and resilience assessment of health systems, addressing global societal priorities: Learnings from a top nonprofit hospital in a bioclimatic building in Africa. Renew Sustain ENERGY Rev. 2021;141:110765.

Dadi D, Introna V, Santolamazza A, Salvio M, Martini C, Pastura T, et al. Private Hospital Energy Performance Benchmarking Using Energy Audit Data: An Italian Case Study. Energies. 2022;15(3):1–16.

Dawande PP, Wankhade RS, Akhtar FI, Noman O. Turnaround time: an efficacy measure for medical laboratories. CUREUS J Med Sci. 2022;14(9):e28824.

De Sousa LR, Mazzo A, De Almeida ACF, Tonello C, Lourençone LFM. Evaluation of quality indicators in the management of an operating room at a tertiary-level hospital. Med. 2022;55(1):1–8.

Drynda S, Schindler W, Slagman A, Pollmanns J, Horenkamp-Sonntag D, Schirrmeister W, et al. Evaluation of outcome relevance of quality indicators in the emergency department (ENQuIRE): study protocol for a prospective multicentre cohort study. BMJ Open. 2020;10(9):e038776.

Fekri O, Manukyan E, Klazinga N. Appropriateness, effectiveness and safety of care delivered in Canadian hospitals: a longitudinal assessment on the utility of publicly reported performance trend data between 2012–2013 and 2016–2017. BMJ Open. 2020;10(6):e035447.

Galloa AJO, Ramírez CA. Evaluating Colombian public hospitals productivity during 2004–2015. A luenberger-indicator approach. Rev Gerenc Y Polit Salud. 2020;19:1–23.

Gebreyes M, Sisay A, Tegen D, Asnake A, Wolde M. Evaluation of laboratory performance, associated factors and staff awareness towards achieving turnaround time in tertiary hospitals, Ethiopia. Ethiop J Health Sci. 2020;30(5):767–76.

Gnanaraj J, Kulkarni RG, Sahoo D, Abhishekh B. Assessment of the Key Performance Indicator Proposed by NABH in the Blood Centre of a Tertiary Health Care Hospital in Southern India. Indian J Hematol Blood Transfus. 2022;39:308–16.

Horenberg F, Lungu DA, Nuti S. Measuring research in the big data era: the evolution of performance measurement systems in the Italian teaching hospitals. Health Policy (New York). 2020;124(12):1387–94.

Hwang DK, Cho J, Moon J. Feasibility study on energy audit and data driven analysis procedure for building energy efficiency: bench-marking in Korean hospital buildings. Energies. 2019;14(15):3006.

Jaskova D. Efficiency of management, processes in a private hospital. Entrep Sustain Issues. 2021;9(1):436–46.

Jebraeily M, Valizadeh MA, Rahimi B, Saeidi S. The development of a clinical dashboard for monitoring of key performance indicators in ICU. J Iran Med Counc. 2022;5(2):308–17.

Kang Y, Kim M, Jung K. The equity of health care spending in South Korea: testing the impact of publicness. Int J Environ Res Public Health. 2020;17(5):1775.

Abou Tarieh RR, Zayyat R, Naoufal RN, Samaha HR. A case study exploring the impact of JCI standards implementation on staff productivity and motivation at the laboratory and blood bank. Heal Sci Rep. 2022;5(1):e497.

Kadoic N, Simic D, Mesaric J, Redep NB. Measuring quality of public hospitals in croatia using a multi-criteria Approach. Int J Environ Res Public Health. 2021;18:19.

Khalilabad T, Amir N, Asl P, Raeissi Shali M, Niknam N. Assessment of clinical and paraclinical departments of military hospitals based on the Pabon Lasso Model. J Educ Health Promot. 2020;9:1–6.

Lai JHK, Hou H, Edwards DJ, Yuen PL. An analytic network process model for hospital facilities management performance evaluation. Facilities. 2022;40(5–6):333–52.

Lai J, Yuen PL. Identification, classification and shortlisting of performance indicators for hospital facilities management. Facilities. 2021;39(1–2):4–18.

Lin CS, Chiu CM, Huang YC, Lang HC, Chen MS. Evaluating the operational efficiency and quality of Tertiary hospitals in Taiwan: the application of the EBITDA Indicator to the DEA Method and TOBIT Regression. Healthcare. 2022;10(1):58.

Matos R, Ferreira D, Pedro MI. Economic analysis of portuguese public hospitals through the construction of quality, efficiency, access, and financial related composite indicators. Soc Indic Res. 2021;157(1):361–92.

Morisod K, Luta X, Marti J, Spycher J, Malebranche M, Bodenmann P. Measuring health equity in emergency care using routinely collected data: a systematic review. Heal Equity. 2021;5(1):801–17.

Nik Hisamuddin R, Tuan Hairulnizam TK. Developing key performance indicators for emergency department of teaching hospitals: a mixed fuzzy Delphi and nominal group technique approach. Malays J Med Sci. 2022;29(2):114–25.

Ramírez Calazans A, Paredes Esteban RM, Grijalva Estrada OB, Ibarra Rodríguez MR. Assessment of quality indicators in pediatric major outpatient surgery. Influence of the COVID-19 pandemic. Cir Pediatr. 2023;36(1):17–21.

PubMed Google Scholar

Shaqura II, Gholami M, Akbari Sari A. Assessment of public hospitals performance in Gaza governorates using the Pabón Lasso Model. Int J Health Plann Manage. 2021;36(4):1223–35.

Al-Jazairi AS, Alnakhli AO. Quantifying clinical pharmacist activities in a tertiary care hospital using key performance indicators. Hosp Pharm. 2021;56(4):321–7.

Aloh HE, Onwujekwe OE, Aloh OG, Nweke CJ. Is bed turnover rate a good metric for hospital scale efficiency? A measure of resource utilization rate for hospitals in Southeast Nigeria. Cost Eff Resour Alloc. 2020;18(1):1–8.

Bari S, Incorvia J, Ahearn O, Dara L, Sharma S, Varallo J, et al. Building safe surgery knowledge and capacity in Cambodia: a mixed-methods evaluation of an innovative training and mentorship intervention. Glob Health Action. 2021;14(1):1998996.

Bari S, Incorvia J, Iverson KR, Bekele A, Garringer K, Ahearn O, et al. Surgical data strengthening in Ethiopia: results of a Kirkpatrick framework evaluation of a data quality intervention. Glob Health Action. 2021;14(1):1–11.

Bastos LSL, Hamacher S, Zampieri FG, Cavalcanti AB, Salluh JIF, Bozza FA. Structure and process associated with the efficiency of intensive care units in low-resource settings: an analysis of the CHECKLIST-ICU trial database. J Crit Care. 2020;59:118–23.

Bastos LSL, Wortel SA, de Keizer NF, Bakhshi-Raiez F, Salluh JIF, Dongelmans DA, et al. Comparing continuous versus categorical measures to assess and benchmark intensive care unit performance. J Crit Care. 2022;70:154063.

Kocisova K, Hass-Symotiuk M, Kludacz-Alessandri M. Use of the dea method to verify the performance model for hospitals. E M Ekon A Manag. 2018;21(4):125–40.

Lee D, Yu S, Yoon SN. Analysis of hospital management based on the characteristics of hospitals: focusing on financial indicators. Glob Bus Financ Rev. 2019;24(3):1–13.

Mirzaei A, Tabibi SJ, Nasiripour AA, Riahi L. Evaluating the feasibility of financial variables of health: A hospital administrator’s viewpoint. Galen Med J. 2016;5(1):25–30.

Middleton S, Gardner G, Gardner A, Considine J, FitzGerald G, Christofis L, et al. Are service and patient indicators different in the presence or absence of nurse practitioners? The EDPRAC cohort study of Australian emergency departments. BMJ Open. 2019;9(7):e024529.

Nobakht S, Jahangiri K, Hajinabi K. Correlation of performance indicators and productivity: A cross sectional study of emergency departments in Tehran, Iran during year 2016. Trauma Mon. 2018;23(5):1–6.

Nuti S, Grillo Ruggieri T, Podetti S. Do university hospitals perform better than general hospitals? A comparative analysis among Italian regions. BMJ Open. 2016;6(8):e011426.

Petrovic GM, Vukovic M, Vranes AJ. The impact of accreditation on health care quality in hospitals. Vojnosanit Pregl. 2018;75(8):803–8.

Pirani N, Zahiri M, Engali KA, Torabipour A. Hospital efficiency measurement before and after health sector evolution plan in Southwest of Iran: a DEA-panel data study. Acta Inf Med. 2018;26(2):106–10.

Profit J, Gould JB, Bennett M, Goldstein BA, Draper D, Phibbs CS, et al. The association of level of care with NICU quality. Pediatrics. 2016;137(3):44–51.

Rahimi H, Bahmaei J, Shojaei P, Kavosi Z, Khavasi M. Developing a strategy map to improve public hospitals performance with balanced scorecard and dematel approach. Shiraz E Med J. 2018;19(7):1–12.

Ahmed S, Hasan MZ, Laokri S, Jannat Z, Ahmed MW, Dorin F, et al. Technical efficiency of public district hospitals in Bangladesh: a data envelopment analysis. COST Eff Resour Alloc. 2019;17:17.

Rahman MH, Tumpa TJ, Ali SM, Paul SK. A grey approach to predicting healthcare performance. Meas J Int Meas Confed. 2019;134:307–25.

Sajadi HS, Sajadi ZS, Sajadi FA, Hadi M, Zahmatkesh M. The comparison of hospitals’ performance indicators before and after the Iran’s hospital care transformations plan. J Educ Health Promot. 2017;6:89.

Si S-L, You X-Y, Liu H-C, Huang J. Identifying key performance indicators for holistic hospital management with a modified DEMATEL approach. Int J Environ Res Public Health. 2017;14(8): 934.

Váchová L, Hajdíková T. Evaluation of Czech hospitals performance using MCDM methods. In: A SI, G WS, C D, editors. Department of exact methods, faculty of management, university of economics, Prague, Jarošovská 1117, Jindřichuv Hradec, vol. 37701. Czech Republic: Newswood Limited; 2017. p. 732–5.

Xenos P, Yfantopoulos J, Nektarios M, Polyzos N, Tinios P, Constantopoulos A. Efficiency and productivity assessment of public hospitals in Greece during the crisis period 2009–2012. Cost Eff Resour Alloc. 2017;15(1):6.

Zhang L, Liu R, Jiang S, Luo G, Liu H-C. Identification of key performance indicators for hospital management using an extended hesitant linguistic DEMATEL Approach. Healthc (Basel Switzerland). 2019;8(1):7.

Aksezer CS. A nonparametric approach for optimal reliability allocation in health services. Int J Qual Reliab Manag. 2016;33(2):284–94.

Cagliano AC, Grimaldi S, Rafele C. Assessing warehouse centralization and outsourcing in the healthcare sector: an Italian case study. In: Department of Management and Production Engineering, Politecnico Di Torino, Corso Duca Degli Abruzzi 24, Torino, 10129. Italy: AIDI - Italian Association of Industrial Operations Professors; 2017. p. 244–50.

Cefalu MS, Elliott MN, Setodji CM, Cleary PD, Hays RD. Hospital quality indicators are not unidimensional: a reanalysis of Lieberthal and Comer. Health Serv Res. 2019;54(2):502–8.

Gao H, Chen H, Feng J, Qin X, Wang X, Liang S, et al. Balanced scorecard-based performance evaluation of Chinese county hospitals in underdeveloped areas. J Int Med Res. 2018;46(5):1947–62.

Gonnelli V, Satta F, Frosini F, Iadanza E. Evidence-based approach to medical equipment maintenance monitoring. In: V HEO, V J, editors. University of Florence, Dept. of Information Engineering. Florence, Italy: Springer; 2017. p. 258–61.

Helkio P, Aantaa R, Virolainen P, Tuominen R. Productivity benchmarks for operative service units. ACTA Anaesthesiol Scand. 2016;60(4):450–6.

Khalifa M, Zabani I. Developing emergency room key performance indicators: What to measure and why should we measure it? J. M, A. H, P. G, A. K, M.S. H, editors. Vol. 226. King Faisal Specialist Hospital and Research Center, Jeddah, Saudi Arabia: IOS Press BV; 2016. p. 179–182.

Ajami S, Ebadsichani A, Tofighi S, Tavakoli N. Medical records department and balanced scorecard approach. J Educ Health Promot. 2013;2:7.

Bosse G, Mtatifikolo F, Abels W, Strosing C, Breuer J-P, Spies C. Immediate outcome indicators in perioperative care: a controlled intervention study on quality improvement in hospitals in Tanzania. PLoS One. 2013;8(6):e65428.

Hung K-Y, Jerng J-S. Time to have a paradigm shift in health care quality measurement. J Formos Med Assoc. 2014;113(10):673–9.

Jeejeebhoy KN, Keller H, Gramlich L, Allard JP, Laporte M, Duerksen DR, et al. Nutritional assessment: comparison of clinical assessment and objective variables for the prediction of length of hospital stay and readmission. Am J Clin Nutr. 2015;101(5):956–65.

Kittelsen SAC, Anthun KS, Goude F, Huitfeldt IMS, Häkkinen U, Kruse M, et al. Costs and quality at the hospital level in the nordic countries. Heal Econ (United Kingdom). 2015;24:140–63.

Koné Péfoyo AJ, Wodchis WP. Organizational performance impacting patient satisfaction in Ontario hospitals: a multilevel analysis. BMC Res Notes. 2013;6: 509.

Li CH, Yu CH. Performance evaluation of public non-profit hospitals using a BP Artificial neural network: the case of Hubei Province in China. Int J Environ Res Public Health. 2013;10(8):3619–33.

Liu K, Jain S, Shi J. Physician performance assessment using a composite quality index. Stat Med. 2013;32(15):2661–80.

Lloyd GF, Bajorek B, Barclay P, Goh S. Narrative review: Status of key performance indicators in contemporary hospital pharmacy practice. J Pharm Pract Res. 2015;45(4):396–403.

Mehrtak M, Yusefzadeh H, Jaafaripooyan E. Pabon Lasso and data envelopment analysis: a complementary approach to hospital performance measurement. Glob J Health Sci. 2014;6(4):107–16.

Mohammadi M, Ziapoor A, Mahboubi M, Faroukhi A, Amani N, Pour FH, et al. Performance evaluation of hospitals under supervision of Kermanshah medical sciences using pabonlasoty diagram of a five-year period (2008–2012). Life Sci J. 2014;11:77–81 ( 1 SPECL. ISSUE) ).

Niaksu O, Zaptorius J. Applying operational research and data mining to performance based medical personnel motivation system. In: Vilnius University, Institute of Mathematics and Informatics. Lithuania: IOS; 2014. p. 63–70.

Córdoba S, Caballero I, Navalón R, Martínez-Sánchez D, Martínez-Morán C, Borbujo J. Analysis of the surgical activity in the dermatology department of Fuenlabrada University Hospital, Madrid, Spain, between 2005 and 2010: determination of the standard operating times. Actas Dermosifiliogr. 2013;104(2):141–7.

Profit J, Kowalkowski MA, Zupancic JAF, Pietz K, Richardson P, Draper D, et al. Baby-MONITOR: a composite indicator of NICU Quality. Pediatrics. 2014;134(1):74–82.

Rabar D, Pap N. Evaluation of crotia’s regional hospital effiency: an application of data envelopment analysis . Bacher U, Barkovic D, Dernoscheg KH, LamzaMaronic M, Matic B, Runzheimer B, editors. Interdisciplinary Management Research IX. 2013;9:649–59.

Ramos MCA, da Cruz LP, Kishima VC, Pollara WM, de Lira ACO, Couttolenc BF. Performance evaluation of hospitals that provide care in the public health system, Brazil. Rev Saude Publica. 2015;49:1–9.

Schuers M, Joulakian MB, Griffon N, Pachéco J, Périgard C, Lepage E, et al. In: S IN, de PM AM, editors. Quality indicators from laboratory and radiology information systems. A. G. Volume 216. France: IOS; 2015. pp. 212–6. Department of Biomedical Informatics, Rouen University Hospital, Rouen Cedex, 76031,.

Tabrizi JS, Saadati M, Sadeghi-Bazargani H, Ebadi A, Golzari SEJ. Developing indicators to improve educational governance in hospitals. Clin Gov. 2014;19(2):117–25.

Costa A Jr, aS., Leão LE, Novais MA, Zucchi P. An assessment of the quality indicators of operative and non-operative times in a public university hospital. Einstein (Sao Paulo). 2015;13(4):594–9.

Coyne JS, Helton J. How prepared are US hospitals for the affordable care act? A financial condition analysis of US hospitals in 2011. J Health Care Finance. 2015;41(3).

Davis P, Milne B, Parker K, Hider P, Lay-Yee R, Cumming J, et al. Efficiency, effectiveness, equity (E-3). Evaluating hospital performance in three dimensions. Health Policy (New York). 2013;112(1–2):19–27.

Flatow VH, Ibragimova N, Divino CM, Eshak DSA, Twohig BC, Bassily-Marcus AM, et al. Quality outcomes in the surgical intensive care unit after electronic health record implementation. Appl Clin Inf. 2015;6(4):611–8.

Article CAS Google Scholar

Fonseca JRS, Ramos RMP, Santos AMP, Fonseca APSS. Policy effects on the quality of public health care: evaluating Portuguese public hospitals’ quality through customers’ views. Cent Eur J Public Policy. 2015;9(2):122–40.

Hadji B, Meyer R, Melikeche S, Escalon S, Degoulet P. Assessing the Relationships Between Hospital Resources and Activities: A Systematic Review. J Med Syst. 2014;38(10):1–21.

Hajduová Z, Herbrik G, Beslerová S. Application of DEA in the environment of Slovak hospitals. Invest Manag Financ Innov. 2015;12(4):148–53.

Thomas Craig KJ, McKillop MM, Huang HT, George J, Punwani ES, Rhee KB. U.S. hospital performance methodologies: a scoping review to identify opportunities for crossing the quality chasm. BMC Health Serv Res. 2020;20(1):640.

Carini E, Gabutti I, Frisicale EM, Di Pilla A, Pezzullo AM, de Waure C, et al. Assessing hospital performance indicators. What dimensions? Evidence from an umbrella review. BMC Health Serv Res. 2020;20(1):1038.

Rasi V, Delgoshaee B, Maleki M. Identification of common indicators of hospital performance evaluation models: a scoping review. J Educ Health Promot. 2020;9(1):63.

Shaqura II, Gholami M, Sari AA. Evaluation of performance at Palestinian public hospitals using Pabon Lasso model. Int J Health Plann Manage. 2021;36(3):896–910.

Li J, Seale H, Ray P, Wang Q, Yang P, Li S, et al. E-Health preparedness assessment in the context of an influenza pandemic: a qualitative study in China. BMJ Open. 2013;3(3):e002293.

Huang C-Y, Lai C-H. Effects of internal branding management in a hospital context. Serv Ind J. 2021;41(15–16):985–1006.

Download references

Acknowledgements

The authors are grateful for the support of the Vice Chancellor for Research of Isfahan University of Medical Sciences.

The present article is part of the result of a doctoral thesis approved by Isfahan University of Medical Sciences with code 55657 (IR.MUI.NUREMA.REC.1401.005), without financial source.

Author information

Authors and affiliations.

Student Research Committee, School of Management and Medical Information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran

Shirin Alsadat Hadian

Health Management and Economics Research Center, Isfahan University of Medical Sciences, Isfahan, Iran

Reza Rezayatmand

Hospital Management Research Center, Health Management Research Institute, Iran University of Medical Sciences, Tehran, Iran

Nasrin Shaarbafchizadeh

Department of Management, Faculty of Administrative Sciences and Economics, University of Isfahan, Isfahan, Iran

Saeedeh Ketabi

School of Public Health and Preventive Medicine, Monash University, Victoria, Australia

Ahmad Reza Pourghaderi

You can also search for this author in PubMed Google Scholar

Contributions

Shirin Alsadat Hadian and Reza Rezayatmans and Saeedeh Ketabi: Study conceptualization and design. Acquisition of data: Shirin Alsadat Hadian, Reza Rezayatmand. Analysis and interpretation of data: Shirin Alsadat Hadian, Reza Rezayatmand, Nasrin Shaarbafchizadeh, Saeedeh Ketabi. Drafting of the manuscript: Shirin Alsadat Hadian, Reza Rezayatmand. Critical revision of the manuscript for important intellectual content: Reza Rezayatmand, Nasrin Shaarbafchizadeh, Saeedeh Ketabi, Ahmad Reza Pourghaderi.

Corresponding author

Correspondence to Reza Rezayatmand .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Reprints and permissions

About this article

Cite this article.

Hadian, S.A., Rezayatmand, R., Shaarbafchizadeh, N. et al. Hospital performance evaluation indicators: a scoping review. BMC Health Serv Res 24 , 561 (2024). https://doi.org/10.1186/s12913-024-10940-1

Download citation

Received : 03 January 2024

Accepted : 02 April 2024

Published : 01 May 2024

DOI : https://doi.org/10.1186/s12913-024-10940-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Performance evaluation

BMC Health Services Research

ISSN: 1472-6963

General enquiries: [email protected]

Open access
Published: 05 May 2024

The learning curve in endoscopic transsphenoidal skull-base surgery: a systematic review

Abdulraheem Alomari 1 ,
Mazin Alsarraj 2 &
Sarah Alqarni 3

BMC Surgery volume 24 , Article number: 135 ( 2024 ) Cite this article

156 Accesses

Metrics details

The endoscopic endonasal transsphenoidal approach (EETA) has revolutionized skull-base surgery; however, it is associated with a steep learning curve (LC), necessitating additional attention from surgeons to ensure patient safety and surgical efficacy. The current literature is constrained by the small sample sizes of studies and their observational nature. This systematic review aims to evaluate the literature and identify strengths and weaknesses related to the assessment of EETA-LC.

A systematic review was conducted following the PRISMA guidelines. PubMed and Google Scholar were searched for clinical studies on EETA-LC using detailed search strategies, including pertinent keywords and Medical Subject Headings. The selection criteria included studies comparing the outcomes of skull-base surgeries involving pure EETA in the early and late stages of surgeons’ experience, studies that assessed the learning curve of at least one surgical parameter, and articles published in English.

The systematic review identified 34 studies encompassing 5,648 patients published between 2002 and 2022, focusing on the EETA learning curve. Most studies were retrospective cohort designs (88%). Various patient assortment methods were noted, including group-based and case-based analyses. Statistical analyses included descriptive and comparative methods, along with regression analyses and curve modeling techniques. Pituitary adenoma (PA) being the most studied pathology (82%). Among the evaluated variables, improvements in outcomes across variables like EC, OT, postoperative CSF leak, and GTR. Overcoming the initial EETA learning curve was associated with sustained outcome improvements, with a median estimated case requirement of 32, ranging from 9 to 120 cases. These findings underscore the complexity of EETA-LC assessment and the importance of sustained outcome improvement as a marker of proficiency.

Conclusions

The review highlights the complexity of assessing the learning curve in EETA and underscores the need for standardized reporting and prospective studies to enhance the reliability of findings and guide clinical practice effectively.

Peer Review reports

With the advent of endoscopic techniques, skull-base surgery has significantly advanced. The modern history of neuro-endoscopy began in the early 1900s with an innovation by Lespinasse and Dandy, involving intraventricular endoscopy to coagulate the choroid plexus for treating communicating hydrocephalus [ 1 ]. In 1963, Guiot first reported an endoscopic approach via the transsphenoidal route as an adjunct to procedures performed under microscopy [ 2 , 3 ]. In 1992, Jankowski et al. described a purely endoscopic approach for pituitary adenoma resection [ 1 ].

The advantages of endoscopy have encouraged skull-base surgeons to adopt this technique, which provides a panoramic view of critical anatomical landmarks and improved access to the corners and deep surgical areas while inducing only minor trauma to the nasal structures, thereby enhancing postoperative patient comfort [ 4 ]. Compared with procedures involving microscopy, the endoscopic approach results in a shorter operating time (OT), a reduced hospitalization period, a lower rate of complications, and a higher endocrinological cure rate [ 5 , 6 ]. Despite these benefits, the endoscopic approach is hindered by a two-dimensional view, instrument interference, difficulties in achieving homeostasis, and a steep learning curve (LC) [ 4 ].

Since its inception, pioneers in the field have recognized the steep LC associated with the endoscopic technique [ 7 ]. The safety and efficacy of the endoscopic endonasal transsphenoidal approach (EETA), as an alternative to the gold-standard microscopic technique, have been established. However, the steep LC associated with the endoscopic approach may affect short-term outcomes post-procedure [ 5 , 6 ]. Additionally, as the skull-base endoscopic technique constantly evolves and expands, a thorough understanding of the associated LC is critical.

The results of existing publications on the EETA-LC are challenging to interpret due to small sample sizes, observational study designs, and a lack of standardization in assessment methodologies. In this systematic review aims to elucidate the EETA-LC from the literature by addressing the following questions: How was EETA LC evaluated? Which set of variables was used to assess the LC? What is the influence of the LC on the examined variables?

A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines [ 8 ]. The review was registered on PROSPERO (CRD42023494731). We searched different databases for articles that assessed the learning curve of EETA without date restriction (PubMed, and Google Scholar). We used a particular equation for each database using a combination of the following keywords and Medical Subject Headings: (Endoscopy OR endoscopic skull base OR endoscopic endonasal transsphenoidal approach) AND (Skull Base Neoplasms OR Pituitary OR pituitary adenoma) AND (Learning Curve OR endoscopic learning curve OR surgical learning curve).

First, two authors (AA, MA) independently screened the titles and abstracts of articles in the databases for learning curve analysis of EETA, either for a single surgeon or a team, by directly comparing outcomes between early and late cases performed. The full texts of the relevant articles were reviewed. When there was a disagreement, the articles were thoroughly discussed before their inclusion in the review. The bibliographies of the selected studies were also screened for relevant citations, which turned up studies that were already selected from the database search.

Studies were included according to the following inclusion criteria: 1) Comparison of outcomes between initial and advanced experiences with the endonasal endoscopic transsphenoidal approach to treat skull-base pathology, defined as "early experience" and "late experience," respectively; 2) Assessment of at least one parameter based on early and late experiences; 3) Randomized controlled trials, prospective cohort studies, retrospective cohort studies, case–control studies, and case series studies were included; and 4) English-language publications.

The study’s exclusion criteria included the following: 1) Studies not performing learning curve analysis; 2) Studies comparing the outcomes of microscopic and endoscopic transsphenoidal approaches without providing separate data for the endoscopic approach; 3) Studies comparing the learning curve between two EETA techniques, using simulated models or questionnaire-based analysis; 4) Studies comparing the microscopic vs. endoscopic approach without separate data available specifically for the endoscopic arm. Additionally, case reports, reviews, animal studies, technical notes, comments, and correspondence were excluded.

Data collection and analysis

The following data were extracted directly from the articles: 1) author names; 2) the year of publication; 3) Time interval of performed procedures; 4) study design; 5) the sample size; 6) techniques used for learning curve analysis (methods used to assort the patients for the analysis); (conducting statistical analysis vs. simple comparison of outcomes); 7) the sample size in each study arm when group splitting performed (early experience vs. late experience); 8) detailed information about surgeon experience at the time of LC assessment (including or omitting the first few EETA cases); 9) single vs. multiple pathologies; 10) team vs. single-surgeon experiences; 11) evaluated set of variables; 12) Variables that improved with experience; and 13) the number of cases required to overcome the initial LC or other methods to identify overcoming the learning curve.

Study quality assessment and risk of bias

Two reviewers conducted a quality assessment and evaluated the risk of bias in the included articles. We utilized the Newcastle–Ottawa Scale (NOS) [ 9 ] and the GRADE system [ 10 ].

Heterogeneity Analysis: Due to substantial heterogeneity observed among the included studies, which encompassed variations in study design, included pathologies, and outcome measures, a formal meta-analysis was not feasible. Therefore, we opted for a qualitative synthesis instead of a formal meta-analysis. Heterogeneity analysis and sensitivity analyses were not explicitly conducted.

Based on the inclusion and exclusion criteria, a total of 34 studies were identified (6 articles excluded after reviewing the full articles), including 5,648 patients [ 7 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ] (Fig. 1 ). The included studies were published between 2002 and 2022, and the evaluated procedures were performed between 1990 and 2018. The majority of the included articles comprised retrospective cohort studies (88%), with two being prospective studies, and two articles presenting data from both prospective and retrospective study designs. Assessing a surgical learning curve involves various methods and techniques documented within the included articles. We observed various methods for patient assortment in conducting learning curve analyses across the literature, with group-based learning curve analysis noticeable in a significant proportion of articles (68%). Within these studies, there was an unclear rationale behind patient grouping. Nonetheless, patients were categorized into either equal group, segmented based on arbitrary time periods, or separated based on improvements in outcomes observed retrospectively after data analysis. Eleven articles (32%) utilize case-based analysis, where individual surgical cases serve as distinct data points, and their outcomes are monitored over time.

PRISMA flow diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses

* The bibliographies of the selected studies were also screened for relevant citations which turned up studies already included from databases search

Our systematic review encompasses a wide range of statistical tests employed in the included studies to analyze various data types and address multifaceted research inquiries. The primary statistical methodologies utilized encompass descriptive statistical analysis, which includes metrics such as mean, median, frequency, and standard deviation, along with comparative statistical analysis, which includes techniques such as Chi-square analysis, analysis of variance (ANOVA), and t-tests. Descriptive statistical analysis alone was evident in 10 articles (29%), whereas comparative statistical analysis was present in 24 articles (71%). Noteworthy examples include Leach et al. [ 16 ], who conducted analysis of variance (ANOVA) with post hoc Bonferroni tests for parametric data, Chi-Square Test, or Mann–Whitney tests for nonparametric data, and regression analysis to explore the relationship between surgical duration and relevant factors. Smeth et al. [ 17 ] undertook analyses using chi-square, Fisher exact, Student t-test, Mann–Whitney U test, and analysis of variance, aligning with their examination of categorical and continuous variables across distinct groups. Similarly, Sonnenburg et al. [ 12 ] applied a one-way ANOVA to discern variations between groups, highlighting the importance of understanding differences in means across categorical variables or treatment cohorts.

Regression analyses, scatterplots, McNemar tests, ROC curve analysis, and logistic regression models were integral across various studies, serving multiple purposes. Regression analyses, such as linear regression models, facilitated the exploration of intricate relationships among variables like age, tumor size, and surgical duration, identifying potential risk factors in surgical contexts [ 22 ]. Scatterplots visually depicted these relationships, offering intuitive insights into temporal variations, notably in the examination of surgery date versus duration [ 22 ]. McNemar tests were instrumental in evaluating changes in hormone levels, crucial for understanding postoperative outcomes and hormonal dynamics [ 37 ]. Additionally, ROC curve analysis provided a robust method for determining the level of surgical experience necessary to achieve gross total resection (GTR), offering actionable insights into surgical proficiency and patient outcomes [ 37 ]. Binary logistic regression models were utilized to identify prognostic factors contributing to the attainment of Gross Total Resection (GTR), hormonal recuperation, and visual restoration. For instance, variables such as surgical experience (≤ 100 vs. > 100 cases) were examined within this analytical framework [ 37 ].

In our examination of the included articles, we noted a lack of thorough description regarding the experience of surgeons or surgical teams with the endoscopic endonasal transsphenoidal approach (EETA), the extent of the approach undertaken, and the level of involvement of individual surgeons or surgical teams during procedures. Thirteen articles (38%) reported including the initial cases of EETA, which may indicate a lack of prior experience with the approach. Additionally, seven articles (21%) detailed the experience of a single surgeon, while the majority (79%) evaluated team experiences. There was a wide range of pathologies included in all the studies. Twenty articles (59%) focused on a single pathology, while fourteen studies (41%) examined multiple pathologies. Pituitary adenoma (PA) was the most frequently reported pathology (82%), followed by craniopharyngioma (CP) (44%). Three studies assessed the learning curve of cerebrospinal fluid (CSF) leak repair following treatment of multiple pathologies. Descriptions of the surgical approach, particularly distinguishing between simple and extended techniques, were notably lacking across all articles. However, seventeen articles (50%) did mention pathologies that often require an extended approach, such as meningioma, chordoma, and CP. A number of studies have investigated the variations in tumor type and size among the examined groups, particularly between early and late groups. Notably, findings from studies such as [ 7 , 16 , 17 , 22 , 23 , 26 , 38 ] indicated that no statistical differences were observed between these groups. The characteristics of the included studies [ 7 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ] are summarized in Table 1 .

The EETA-LC was evaluated based on a diverse set of variables. The most frequently analyzed variables were postoperative cerebrospinal fluid (CSF) leak in 28 articles (82%) [ 7 , 12 , 13 , 15 , 16 , 17 , 19 , 20 , 21 , 22 , 23 , 25 , 27 , 28 , 29 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ], gross total resection (GTR) in 21 articles (62%) [ 7 , 13 , 14 , 16 , 19 , 21 , 22 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 36 , 37 , 38 , 39 , 40 ], post operative diabetes insipidus (DI) in 15 articles (44%) [ 12 , 13 , 16 , 17 , 19 , 21 , 22 , 29 , 30 , 31 , 32 , 34 , 36 , 37 , 41 ], operative time (OT) in 12 articles (35%) [ 7 , 13 , 14 , 16 , 17 , 22 , 29 , 32 , 34 , 35 , 36 , 38 ] and visual improvement in 12 articles (35%) [ 13 , 14 , 16 , 21 , 22 , 28 , 31 , 32 , 34 , 36 , 37 , 41 ]. (Fig. 2 ).

Frequency at which certain variables were evaluated in the literature to assess the EETA learning curve. EETA, endoscopic endonasal transsphenoidal approach; post-op, postoperative; CSF, cerebrospinal fluid; GTR, gross total resection; DI, diabetes insipidus; LOS, length of stay; IOP, intraoperative; ICA, internal carotid artery; SIADH, syndrome of inappropriate antidiuretic hormone secretion; LD, lumbar drain; CNS, central nervous system; CN, cranial nerve; EBL, estimated blood loss; DVT, deep vein thrombosis

In all the studies included, improvements were observed between early and late-experience stages [ 7 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ]. Among the evaluated variables, the following improvements were noted: the endocrinological cure rate (EC) showed improvement in all 7 articles out of 7 evaluated [ 13 , 16 , 18 , 21 , 24 , 30 , 33 ], operative time (OT) improved in 11 out of 12 articles (91%) [ 13 , 14 , 16 , 17 , 22 , 29 , 32 , 34 , 35 , 36 , 38 ], postoperative cerebrospinal fluid leak (CSF) improved in 23 out of 28 articles (82%) [ 12 , 15 , 17 , 19 , 20 , 22 , 23 , 25 , 27 , 28 , 29 , 31 , 32 , 33 , 34 , 35 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ], visual improvement was observed in 9 out of 12 articles (75%) [ 13 , 14 , 16 , 22 , 28 , 31 , 34 , 37 , 41 ], gross total resection (GTR) improved in 14 out of 21 articles (67%) [ 7 , 13 , 14 , 19 , 21 , 22 , 26 , 27 , 28 , 29 , 30 , 38 , 39 , 40 ], hospital length of stay (LOS) decreased in five out of 10 studies (50%) [ 11 , 12 , 16 , 17 , 22 ], and postoperative diabetes insipidus (DI) decreased in 7 out of 15 articles (47%) [ 3 , 14 , 16 , 17 , 21 , 22 , 33 ] (Fig. 3 ).

Proportion of main improved variables with experiences. EC, Endocrinological cure; OT, Operative time; post-op: postoperative; CSF, cerebrospinal fluid; GTR, gross total resection; hLOS, hospital length of stay; DI, diabetes insipidus

Moreover, 12 articles (35%) reported both significant and non-significant improvements in outcomes [ 7 , 13 , 14 , 16 , 17 , 21 , 22 , 31 , 32 , 34 , 38 , 41 ]. In 10 studies (29%), solely a trend of improvement was observed [ 11 , 15 , 19 , 20 , 23 , 26 , 27 , 29 , 30 , 40 ], while 8 articles (23%) reported solely significant improvements [ 18 , 24 , 25 , 35 , 36 , 37 , 42 , 43 ]. However, in four studies, despite observing a tendency towards better outcomes, no statistical disparities were identified among all assessed variables [ 12 , 28 , 33 , 39 ]. None of the included studies reported a deterioration in any of the assessed outcomes over time, except for one study where a significant decline in GTR was observed in the late group [ 33 ]. This decline was attributed to the inclusion of more invasive and complex tumors in the late group. Nevertheless, Younus et al. documented ongoing improvement in GTR even after surpassing the initial learning curve [ 7 ].

In this systematic review, the primary technique employed to determine the transition point indicating the overcoming of the initial learning curve involved observing sustained and consistent improvement in outcomes over time. In almost half of the included articles, overcoming the initial learning curve (observing improvement of outcomes) was linked to the number of cases performed. Out of the 34 analyzed studies, 16 (47%) estimated the number of cases needed to overcome the initial learning curve of EETA. Reported cases ranged widely from 9 to 120, with a mode of 50. Considering both the median and the Interquartile Range (IQR) provides a comprehensive understanding of the reported case distribution and central tendency for overcoming the initial EETA learning curve. The median number of cases needed is 32, with an IQR of 20. These numbers are estimates and require careful interpretation [ 16 , 17 , 20 , 21 , 22 , 23 , 24 , 25 , 29 , 31 , 32 , 33 , 35 , 36 , 37 , 38 , 42 ].

Regarding the quality of included studies, the NOS quality assessment scale was used. 21 studies graded as fair quality while the remaining 13 articles rated as poor quality [ 9 ]. The risk of bias was evaluated according to the GRADE system. All included studies are observational cohort study and graded either as low or very low grade [ 10 ]. This reflects the great heterogeneity and high risk of bias due to the study design of the current EETA-LC literature.

Endoscopic techniques have drastically improved skull-base surgery. Unlike procedures involving a microscope, many neurosurgeons have acquired experience in endoscopic techniques later in their careers, and the level of exposure to these techniques during training years has varied among surgeons. The LC is a critical factor in the acquisition of new surgical skills. Understanding the link between the EETA-LC and surgical outcomes will enable surgeons to better understand what to expect and what measures to apply as those surgical skills develop. Many studies in other surgical domains have reported on the LC during the acquisition of new surgical techniques [ 44 , 45 , 46 , 47 ]. Most minimally invasive surgeries are associated with a challenging LC, and EETA is no exception [ 7 , 46 ].

The concept of the LC was first established in the field of aircraft manufacturing and refers to an improvement in performance over time [ 48 ]. Smith et al. [ 17 ] have defined it as the number of procedures that must be performed for the outcomes to approach a long-term mean rate. Typically, an LC is characterized by an S-shaped curve with three stages: an early phase, during which new skill sets are acquired; a middle phase, in which the speed of learning rapidly increases; and an expert phase in which the performance reaches a plateau [ 49 ]. However, other curves have been proposed that involve a dip in the LC following the initial acceleration of the learning rate; this occurs especially with handling more challenging cases. Another potential decline may emerge after a long period of experience. Despite having reached a plateau in the learning curve after an extended period, declines in manual dexterity, eyesight, memory, and cognition may overshadow the benefits of accumulated experience, leading to diminished performance levels [ 50 ].

The absence of consensus on the best applicable methods to describe and assess the learning curve may explain the diversity of analysis methods observed in this systematic review. In their large systematic review regarding learning curve assessment in healthcare technologies, Ramsay et al. [ 51 ] reported that group splitting was the most frequent method. They defined group splitting as dividing the data by experience levels and conducting testing on discrete groups, often halves or thirds. The statistical methods applied included t-tests, chi-squared tests, Mann–Whitney U tests, and simple ANOVA.

In our review, we reached a similar conclusion. We observed that a substantial portion of articles (68%) utilized group-based learning curve analysis [ 7 , 11 , 12 , 13 , 16 , 17 , 19 , 21 , 22 , 23 , 26 , 27 , 28 , 30 , 31 , 32 , 33 , 34 , 36 , 37 , 38 , 42 , 43 ]. Additionally, we similarly noted that papers frequently lacked explanations for the selection of cut points, raising concerns about potential bias resulting from data-dependent splitting. It is important to acknowledge that this method of group categorization has inherent drawbacks, including challenges related to small sample sizes, the use of arbitrary cutoff points, and the inability to eliminate all potential confounding variables [ 52 ].

Descriptive analysis was found in 10 articles (29%) within this review [ 11 , 15 , 19 , 20 , 23 , 26 , 27 , 29 , 30 , 40 ]. While providing an initial grasp of data distribution and characteristics, descriptive analysis may fall short in capturing the intricate dynamics of the learning curve over time or the factors affecting its impact [ 51 ]. Alternatively, conducting rigorous statistical analyses afterward offers better insight and interpretation of the results. This approach aims to mitigate the influence of confounding factors on outcome assessments over time [ 51 , 52 ].

In our review, 24 articles (71%) conducted a wide variety of statistical analyses [ 7 , 12 , 13 , 14 , 16 , 17 , 18 , 21 , 22 , 24 , 25 , 28 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 41 , 42 , 43 ], including but not limited to the following tests: Chi-square Test, Fischer exact test, Student's t-test, Analysis of Variance (ANOVA), Mann–Whitney U Test, McNemar tests, Multivariate linear regression model, Cumulative Sum (CUSUM), and ROC Curve Analysis [ 13 , 16 , 22 , 32 , 37 , 38 , 39 ]. Four studies indicated that there was no statistically significant difference observed among the variables under evaluation. The lack of significance was attributed to several factors including small sample sizes, meticulous case selection, involvement of an otolaryngology team throughout the procedure, an increase in the number of invasive tumors in the late-experience study group, previous surgical experience, intensive training, level of supervision, and gradual inclusion of residents [ 12 , 28 , 33 , 39 ]. These efforts should be regarded as beneficial strategies aimed at reducing the steepness of the EETA learning curve.

To obtain more accurate results, it is crucial to eliminate confounding factors, such as the level of supervision, prior experience, the heterogeneity of cases being treated, and their complexity when evaluating the LC. Thus, it is essential to incorporate multivariate logistic regression analysis to mitigate the impact of these potential confounding factors [ 51 ]. Chi et al. [ 22 ] divided their patients into equal groups of 40 cases each. They then compared potential confounding variables to minimize their influence on learning curve assessment. This comparison includes demographic and clinical factors between the two groups, such as sex distribution, mean age, tumor size (microadenomas vs. macroadenomas), visual field defects, and tumor types (non-functioning, functioning adenomas, etc.). By conducting these comparisons, the researchers sought to identify discrepancies in demographic and clinical features between the groups.

The description of a surgeon's extensive prior experience is crucial for accurately quantifying the assessment of the learning curve, a point reported to be neglected during the assessment in various types of learning assessments related to healthcare procedures [ 49 ]. In our review, we observed the same conclusion in all included studies. However, the inclusion of the initial first few cases was mentioned in 13 (38%) articles, which might be used as a surrogate for no prior experience with EETA. Furthermore, five articles did not include the initial few cases. Among these, four studies examined the learning curve of more complex cases such as meningioma, craniopharyngioma, and growth hormone pituitary adenoma, employing an extended approach. Conversely, Younus et al. [ 7 ] deliberately excluded these cases to assess various stages of the learning curve.

Assessing multiple pathologies with varying complexities could significantly impact learning curve assessments. In our review, 59% of articles focused on a single pathology, while 41% explored multiple pathologies. Pituitary adenoma (PA) was the most evaluated (82%), followed by craniopharyngioma (CP) (44%). Controlling confounding variables like tumor type and size may yield more reliable results. Some studies used statistical analyses to compare early and late cases, while others relied on descriptive analyses. Shou et al. noted a drop in GTR over time due to late involvement of complex cases [ 33 ]. Conversely, studies analyzing tumor size and type found GTR improvement with experience [ 7 , 23 ]. Thorough multivariable analysis of confounding factors is crucial for representative LC analysis.

The LC is often assessed based on two main categories of variables: those related to the surgical procedure (OT, estimated blood loss, and extent of resection) and those related to patient outcomes (duration of hospitalization, the incidence of complications, and the mortality rate) [ 50 ]. In this systematic review, OT was one of the most frequent parameters that significantly reduced as one gained experience. Although OT is commonly utilized as an outcome measure, it is only a surrogate means of evaluating the LC and may not always accurately represent patient outcomes [ 52 ]. Another point to consider is the lack of standardized variables for assessing the LC, and the included studies evaluated more than 45 distinct variables. Khan et al. highlighted the importance of using consistent variable definitions across studies to derive accurate conclusions from aggregated LC data [ 52 ].

A dynamic relationship exists between surgical outcomes and the LC, and each phase of the LC influences a distinct set of variables differently. One study, which included data from 1,000 EETA cases after purposely eliminating the first 200 cases, showed that variables such as GTR and the endocrinological cure rate continued to improve after the first 200 cases, whereas other parameters remained unchanged. Authors concluded that some variables will continue to improve after passing the initial LC phase [ 7 ]. Determining the precise number of cases needed to surpass the initial learning curve (LC) has proven challenging. Shikary et al. observed a notable decrease in postoperative CSF leaks after 100 surgeries, while a reduction in operative time was evident after 120 cases [ 35 ]. However, specifying a definitive number to overcome the learning curve of the Endoscopic Endonasal Transsphenoidal Approach (EETA) remains challenging due to individual variability, diverse pathologies, and evolving surgical techniques.

Assessing the learning curve of the Endoscopic Endonasal Transsphenoidal Approach (EETA-LC) faces notable challenges due to its intricate techniques and the wide array of pathologies it addresses. The diversity across specialties makes standardizing studies difficult. To understand the dynamic learning process in EETA-LC, influenced by individual surgeon skill, patient nuances, and procedural complexities, longitudinal studies and advanced analytical methods are essential. Moreover, the complexity of statistical analysis adds another layer of challenge, highlighting the necessity for interdisciplinary collaboration and innovative methodologies.

To address the current limitations in the literature regarding the EETA LC, we propose several key strategies for future studies. Firstly, we advocate for multicenter collaboration, coupled with standardized processes, to comprehensively assess the EETA LC. This collaborative approach will facilitate the aggregation of data from diverse surgical settings, enhancing the generalizability of findings and minimizing bias. Furthermore, rigorous documentation of the previous and current experience of involved surgeons is paramount. We suggest categorizing surgeons based on their levels of experience to accurately elucidate the impact of proficiency on surgical outcomes. Secondly, given the wide variety of complexities of skull base pathologies encountered, we recommend further categorization of cases based on their levels of complexity. This stratification will enable a more nuanced analysis of the learning curve across different levels of surgical challenge. Thirdly, standardization of outcome measures used to assess the learning curve is imperative, with specific definitions provided for each outcome. This ensures consistency and comparability across studies, facilitating meaningful interpretation of results. Finally, conducting prospective study designs with sufficient follow-up periods, along with rigorous multivariate statistical analyses among these categorized groups, is essential to mitigate the influence of confounding variables and strengthen the validity of findings. Implementing these strategies will help future studies to overcome the current limitations in the literature, leading to a deeper understanding of the EETA learning curve and ultimately improving patient outcomes.

This systematic review identified 34 studies that reported a relationship between improvements in surgical outcomes and a surgeon’s level of experience with EETA. There is notable significant heterogeneity in the current literature on EETA-LC regarding the techniques used to assess the LC, variables assessed, types of pathology included, and insufficient reporting of the surgeon or team's current and previous experience with EETA. The main variables improved with experience were EC, postoperative CSF leak, OT, GTR visual improvement, and hospital LOS. Future studies with multicenter collaboration and standardized processes for assessing the EETA LC will enhance generalizability and minimize bias. Rigorous documentation of surgeons' experience levels, categorization of cases by complexity, and standardized outcome measures are essential. Additionally, rigorous statistical analyses will strengthen validity and mitigate confounding variables. Implementing these strategies will deepen our understanding of the EETA learning curve, ultimately leading to improved patient outcomes.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Endoscopic endonasal transsphenoidal approach

Learning curve

Cerebrospinal fluid

Diabetes insipidus

Gross total resection

Cappabianca P, de Divitiis E. Endoscopy and transsphenoidal surgery. Neurosurgery. 2004;54:1043–50.

Article PubMed Google Scholar

Gandhi CD, Post KD. Historical movements in transsphenoidal surgery. Neurosurg Focus. 2001;11:1–4.

Article Google Scholar

Gandhi CD, Christiano LD, Eloy JA, Prestigiacomo CJ, Post KD. The historical evolution of transsphenoidal surgery: facilitation by technological advances. Neurosurg Focus. 2009;27:E8.

de Divitiis E. Endoscopic transsphenoidal surgery: stone-in-the-pond effect. Neurosurgery. 2006;59:512–20.

Rotenberg B, Tam S, Ryu WHA, Duggal N. Microscopic versus endoscopic pituitary surgery: a systematic review. Laryngoscope. 2010;120:1292–7.

Tabaee A, Anand VK, Barrón Y, Hiltzik DH, Brown SM, Kacker A, et al. Endoscopic pituitary surgery: a systematic review and meta-analysis. J Neurosurg. 2009;111:545–54.

Younus I, Gerges MM, Uribe-Cardenas R, Morgenstern PF, Eljalby M, Tabaee A, et al. How long is the tail end of the learning curve? Results from 1000 consecutive endoscopic endonasal skull base cases following the initial 200 cases. J Neurosurg. 2020;134:750–60.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906.

Wells G, Shea B, O'Connell D, Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2013. Retrieved from http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp .

Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–6. https://doi.org/10.1016/j.jclinepi.2010.07.015 .

Cappabianca P, Cavallo L, Colao A, Del Basso De Caro M, Esposito F, Cirillo S, et al. Endoscopic endonasal transsphenoidal approach: outcome analysis of 100 consecutive procedures. Minim Invasive Neurosurg. 2002;45:193–200.

Article CAS PubMed Google Scholar

Sonnenburg RE, White D, Ewend MG, Senior B. The learning curve in minimally invasive pituitary surgery. Am J Rhinol. 2004;18:259–63.

Kenan K, İhsan A, Dilek O, Burak C, Gurkan K, Savas C. The learning curve in endoscopic pituitary surgery and our experience. Neurosurg Rev. 2006;29:298–305.

Yano S, Kawano T, Kudo M, Makino K, Nakamura H, Kai Y, et al. Endoscopic endonasal transsphenoidal approach through the bilateral nostrils for pituitary adenomas. Neurol Med Chir (Tokyo). 2009;49:1–7.

Gondim JA, Schops M, de Almeida JPC, de Albuquerque LAF, Gomes E, Ferraz T, et al. Endoscopic endonasal transsphenoidal surgery: surgical results of 228 pituitary adenomas treated in a pituitary center. Pituitary. 2010;13:68–77.

Leach P, Abou-Zeid AH, Kearney T, Davis J, Trainer PJ, Gnanalingham KK. Endoscopic transsphenoidal pituitary surgery: evidence of an operative learning curve. Neurosurgery. 2010;67:1205–12.

Smith SJ, Eralil G, Woon K, Sama A, Dow G, Robertson I. Light at the end of the tunnel: the learning curve associated with endoscopic transsphenoidal skull base surgery. Skull Base. 2010;20:69–74.

Article PubMed PubMed Central Google Scholar

Wagenmakers MAE, Netea-Maier RT, van Lindert EJ, Pieters GF, Grotenhuis AJ, Hermus AR. Results of endoscopic transsphenoidal pituitary surgery in 40 patients with a growth hormone-secreting macroadenoma. Acta Neurochir (Wien). 2011;153:1391–9.

Kumar S, Darr A, Hobbs C, Carlin W. Endoscopic, endonasal, trans-sphenoidal hypophysectomy: retrospective analysis of 171 procedures. J Laryngol Otol. 2012;126:1033–40.

Snyderman CH, Pant H, Kassam AB, Carrau RL, Prevedello DM, Gardner PA. The learning curve for endonasal surgery of the cranial base: A systematic approach to training. In: Kassam AB, Gardner PA, editors. Endoscopic approaches to the skull base. Ettlingen: Karger Publishers; 2012. p. 222–31.

Chapter Google Scholar

Bokhari AR, Davies MA, Diamond T. Endoscopic transsphenoidal pituitary surgery: a single surgeon experience and the learning curve. Br J Neurosurg. 2013;27:449.

Chi F, Wang Y, Lin Y, Ge J, Qiu Y, Guo L. A learning curve of endoscopic transsphenoidal surgery for pituitary adenoma. J Craniofac Surg. 2013;24:2064–7.

de los Santos G, Fragola C, Del Castillo R, Rodríguez V, D’oleo C, Reyes P. Endoscopic approaches to pituitary lesions: difficulties and challenges. Acta Otorrinolaringol Esp. 2013;64(258):64.

Google Scholar

Hazer DB, Işık S, Berker D, Güler S, Gürlek A, Yücel T, et al. Treatment of acromegaly by endoscopic transsphenoidal surgery: surgical experience in 214 cases and cure rates according to current consensus criteria. J Neurosurg. 2013;119:1467–77.

Jakimovski D, Bonci G, Attia M, Shao H, Hofstetter C, Tsiouris AJ, et al. Incidence and significance of intraoperative cerebrospinal fluid leak in endoscopic pituitary surgery using intrathecal fluorescein. World Neurosurg. 2014;82:e513–23.

Koutourousiou M, Fernandez-Miranda JC, Wang EW, Snyderman CH, Gardner PA. Endoscopic endonasal surgery for olfactory groove meningiomas: outcomes and limitations in 50 patients. Neurosurg Focus. 2014;37:E8.

Mascarenhas L, Moshel YA, Bayad F, Szentirmai O, Salek AA, Leng LZ, et al. The transplanum transtuberculum approaches for suprasellar and sellar-suprasellar lesions: avoidance of cerebrospinal fluid leak and lessons learned. World Neurosurg. 2014;82:186–95.

Ottenhausen M, Banu MA, Placantonakis DG, Tsiouris AJ, Khan OH, Anand VK, et al. Endoscopic endonasal resection of suprasellar meningiomas: the importance of case selection and experience in determining extent of resection, visual improvement, and complications. World Neurosurg. 2014;82:442–9.

Ananth G, Hosmath AV, Varadaraju DN, Patil SR, Usman MM, Patil RP, et al. Learning curve in endoscopic transnasal sellar region surgery. J Evid Based Med Healthc. 2016;3:3166–72.

Jang JH, Kim KH, Lee YM, Kim JS, Kim YZ. Surgical results of pure endoscopic endonasal transsphenoidal surgery for 331 pituitary adenomas: a 15-year experience from a single institution. World Neurosurg. 2016;96:545–55.

Kshettry VR, Do H, Elshazly K, Farrell CJ, Nyquist G, Rosen M, et al. The learning curve in endoscopic endonasal resection of craniopharyngiomas. Neurosurg Focus. 2016;41:E9.

Qureshi T, Chaus F, Fogg L, Dasgupta M, Straus D, Byrne RW. Learning curve for the transsphenoidal endoscopic endonasal approach to pituitary tumors. Br J Neurosurg. 2016;30:637–42.

Shou X, Shen M, Zhang Q, Zhang Y, He W, Ma Z, et al. Endoscopic endonasal pituitary adenomas surgery: the surgical experience of 178 consecutive patients and learning curve of two neurosurgeons. BMC Neurol. 2016;16:1–8.

Ding H, Gu Y, Zhang X, Xie T, Liu T, Hu F, et al. Learning curve for the endoscopic endonasal approach for suprasellar craniopharyngiomas. J Clin Neurosci. 2017;42:209–16.

Shikary T, Andaluz N, Meinzen-Derr J, Edwards C, Theodosopoulos P, Zimmer LA. Operative learning curve after transition to endoscopic transsphenoidal pituitary surgery. World Neurosurg. 2017;102:608–12.

Eseonu CI, ReFaey K, Pamias-Portalatin E, Asensio J, Garcia O, Boahene KD, et al. Three-hand endoscopic endonasal transsphenoidal surgery: experience with an anatomy-preserving mononostril approach technique. Oper Neurosurg (Hagerstown). 2018;14:158–65.

Kim JH, Lee JH, Lee JH, Hong AR, Kim YJ, Kim YH. Endoscopic transsphenoidal surgery outcomes in 331 nonfunctioning pituitary adenoma cases after a single surgeon learning curve. World Neurosurg. 2018;109:e409–16.

Lofrese G, Vigo V, Rigante M, Grieco DL, Maresca M, Anile C, et al. Learning curve of endoscopic pituitary surgery: experience of a neurosurgery/ENT collaboration. J Clin Neurosci. 2018;47:299–303.

Robins JM, Alavi SA, Tyagi AK, Nix PA, Wilson TM, Phillips NI. The learning curve for endoscopic trans-sphenoidal resection of pituitary macroadenomas. A single institution experience, Leeds, UK. Acta Neurochir (Wien). 2018;160:39–47.

Algattas H, Setty P, Goldschmidt E, Wang EW, Tyler-Kabara EC, Snyderman CH, et al. Endoscopic endonasal approach for craniopharyngiomas with intraventricular extension: case series, long-term outcomes, and review. World Neurosurg. 2020;144:e447–59.

Soliman MA, Eaton S, Quint E, Alkhamees AF, Shahab S, O’Connor A, et al. Challenges, learning curve, and safety of endoscopic endonasal surgery of sellar-suprasellar lesions in a community hospital. World Neurosurg. 2020;138:e940–54.

Nix P, Alavi SA, Tyagi A, Phillips N. Endoscopic repair of the anterior skull base-is there a learning curve? Br J Neurosurg. 2018;32:407–11.

Park W, Nam D-H, Kong D-S, Lee KE, Park SI, Kim HY, et al. Learning curve and technical nuances of endoscopic skull base reconstruction with nasoseptal flap to control high-flow cerebrospinal fluid leakage: reconstruction after endoscopic skull base surgery other than pituitary surgery. Eur Arch Otorhinolaryngol. 2022;279:1335–40.

Lubowitz JH, Sahasrabudhe A, Appleby D. Minimally invasive surgery in total knee arthroplasty: the learning curve. Orthopedics. 2007;30:80.

PubMed Google Scholar

Hoppe DJ, Simunovic N, Bhandari M, Safran MR, Larson CM, Ayeni OR. The learning curve for hip arthroscopy: a systematic review. Arthroscopy. 2014;30:389–97.

Sclafani JA, Kim CW. Complications associated with the initial learning curve of minimally invasive spine surgery: a systematic review. Clin Orthop Relat Res. 2014;472:1711–7.

Pernar LI, Robertson FC, Tavakkoli A, Sheu EG, Brooks DC, Smink DS. An appraisal of the learning curve in robotic general surgery. Surg Endosc. 2017;31:4583–96.

Wright TP. Factors affecting the cost of airplanes. J Aeronaut Sci. 1936;3:122–8.

Cook JA, Ramsay CR, Fayers P. Using the literature to quantify the learning curve: a case study. Int J Technol Assess Health Care. 2007;23:255–60.

Hopper A, Jamison M, Lewis W. Learning curves in surgical practice. Postgrad Med J. 2007;83:777–9.

Article CAS PubMed PubMed Central Google Scholar

Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT. Assessment of the learning curve in health technologies: a systematic review. Int J Technol Assess Health Care. 2000;16:1095–108.

Khan N, Abboudi H, Khan MS, Dasgupta P, Ahmed K. Measuring the surgical ‘learning curve’: methods, variables and competency. BJU Int. 2014;113:504–8.

Download references

Acknowledgements

Not applicable

No funding was received for this study.

Author information

Authors and affiliations.

Neurosurgery Department, East Jeddah Hospital, 2277 King Abdullah Rd, Al Sulaymaniyah, 22253, Jeddah, Saudi Arabia

Abdulraheem Alomari

Otolaryngology and Head and Neck Surgery Department, King Abdullah Medical Complex, Prince Nayef Street, Northern Abhor, 23816, Jeddah, Saudi Arabia

Mazin Alsarraj

Neurosurgery Department, King Abdulaziz Medical City, 21423, Jeddah, Saudi Arabia

Sarah Alqarni

You can also search for this author in PubMed Google Scholar

Contributions

AA: Acquisition of Data, Analysis and Interpretation of Data, Drafting the Article, Critically Revising the Article, Drafting the Article, Reviewed submitted version of manuscript, Approved the final version of the manuscript on behalf of all authors. Study supervision; MA: Conception and Design, Analysis and Interpretation of Data, Reviewed submitted version of manuscript, Analysis and Interpretation of Data; SA: Acquisition of Data, Conception and Design, Critically Revising the Article, Analysis and Interpretation of Data, Administrative / technical / material support.

Corresponding author

Correspondence to Abdulraheem Alomari .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Alomari, A., Alsarraj, M. & Alqarni, S. The learning curve in endoscopic transsphenoidal skull-base surgery: a systematic review. BMC Surg 24 , 135 (2024). https://doi.org/10.1186/s12893-024-02418-y

Download citation

Received : 21 September 2023

Accepted : 20 April 2024

Published : 05 May 2024

DOI : https://doi.org/10.1186/s12893-024-02418-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Endoscopic skull base
Transsphenoidal surgery

BMC Surgery

ISSN: 1471-2482

Submission enquiries: [email protected]
General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
My Account Login
Explore content
About the journal
Publish with us
Sign up for alerts
Review Article
Open access
Published: 09 May 2024

Domestic violence against women during the COVID19 pandemic in Jordan: a systematic review

Maissa N. Alrawashdeh 1 ,
Rula Odeh Alsawalqa ORCID: orcid.org/0000-0002-5605-5444 2 ,
Rami Aljbour 3 ,
Ann Alnajdawi 4 &
Fawzi Khalid AlTwahya 5

Humanities and Social Sciences Communications volume 11 , Article number: 598 ( 2024 ) Cite this article

107 Accesses

Metrics details

This study aimed to explore the forms and causes of domestic violence against women in Jordan during the COVID-19 pandemic through a systematic literature review. The review yielded eight articles published between April 2020 and November 2022 in the final sample, all of which met the inclusion criteria. The results revealed 11 forms of domestic violence against women in Jordan during and after the full and partial lockdowns due to the pandemic. Physical violence was the most prevalent form of domestic violence, followed by economic, psychological, emotional, verbal, and sexual forms, as well as control and humiliation, bullying, online abuse, harassment and neglect-related violence. The causes were a combination of economic, socio-cultural, and psychological factors emerging because of the pandemic and lockdowns (e.g., poverty, job loss, low wages, gender discrimination, double burden on women [monotonous roles, paid work], male dominance, reduced income, high cost of living). Additionally, effects of the pandemic included psychological, mental, and emotional negative consequences (e.g., anxiety, fear, stress, depression, loneliness, failure, status frustration). Individuals in Jordanian societies employed the norms, ideas, and values of the patriarchal culture to negatively adapt to the economic and psychological effects of the pandemic, which contributed to more domestic violence cases.

Loneliness trajectories over three decades are associated with conspiracist worldviews in midlife

Determinants of behaviour and their efficacy as targets of behavioural change interventions

MDMA-assisted therapy for moderate to severe PTSD: a randomized, placebo-controlled phase 3 trial

Introduction.

Domestic violence (DV) is a significant global and social problem. United Nations ( 2022 ) defined DV as a pattern of behavior in any relationship that is used to gain or maintain power and control over an intimate partner, including physical, sexual, psychological, emotional, and financial acts or threats of action that influence another person. DV is also called “domestic abuse,” “family violence” or “intimate partner violence (IPV)” (Xue et al. 2020 ; Alsawalqa 2021a ). “DV is reaching across national boundaries as well as socio-economic, cultural, racial and class distinctions… its incidence is also extensive, making it a typical and accepted behavior… Its continued existence is morally indefensible…” (Kaur & Garg, 2008 : p. 73). DV is the most common form of violence against women (Xue et al. 2020 ). According to UN Women ( 2021 : p. 4), the experiences of violence against women can be classified into several categories: physical abuse (i.e., been slapped, hit, kicked, had things thrown at them, or other physical harm); verbal abuse (i.e., being yelled at, called names, humiliated); denied basic needs (i.e., health care, money, food, water, shelter); denied communication (i.e., with other people, including being forced to stay alone for long periods of time); and sexual harassment (i.e., being subjected to inappropriate jokes, suggestive comments, leering or unwelcome touch/kisses). UN Women used this definition for the purpose of the measuring the impact of COVID-19 on violence against women.

DV as a “shadow pandemic” grew and intensified during the COVID-19 pandemic, particularly against women and girls (Women UN 2021 ). UN women ( 2021 ) confirmed there was an increase in calls to DV helplines in many countries since the outbreak of COVID-19. Xue et al. ( 2020 ) indicated that during the COVID-19 lockdown, homes became an unsafe environment for victims of DV; women and children were disproportionately affected and made vulnerable during the crisis, which prevented them from seeking help, thus increasing their vulnerability and suffering. The lockdown measures due to the pandemic imposed social distancing, leading to increased family isolation and limited access to legal and social services (Xue et al. 2020 ; Leslie and Wilson 2020 ). These socio-economic stressors led to negative emotional, behavioral, and psychological consequences including depression, anxiety, panic, obsessive-compulsive behaviors, and paranoia (Pedrosa et al. 2020 ), and thus facilitated DV, child abuse, and elder abuse (Al-Tammemi 2020 ; Xue et al. 2020 ).

Since the first registered case of COVID-19 on March 2, 2020, the Jordanian government took several protective measures to prevent the rapid spread of the virus by implementing the National Defense Law on March 14. Under this law, the roads, schools, universities, air and land border crossings, all private businesses, and non-essential public services were closed. Additionally, this law suspended air traffic, and public gatherings as well as religious practices were banned. These rigorous measures had negative multidimensional effects on the economic and social conditions of Jordanian citizens, including human rights violations, such as labor rights, freedom of expression (Alsawalqa et al. 2022a ), loss of human resources, decreased income levels because of declining economic growth, drop in productivity, dismissal of employees from their work, and the inability of some organizations to pay employees’ salaries (Abufaraj et al. 2021 ; Al-Tammemi 2020 ). These poor conditions contributed to an increase in DV cases, in particular against women and children. Jordanian Juvenile and Family Protection Department ( 2020 ) stated that the number of reports of DV increased from 41,221 in 2018, to 54,743 in 2020; of these cases, 58.7% were physical violence, and 34% sexual violence, and most of the victims were female (Higher Population Council, 2021 ; National Council for Family Affairs 2022 ; 2013 ). According to the United Nations Population Fund, approximately 69% of Jordanian women were victims of some type of gender-based DV during the COVID-19 pandemic (Anderson 2020 ).

Jordanian women suffering multiple socio-economic constraints was directly related to the economic, social, and health impacts of the COVID-19 pandemic, which imposed new obligations on women, consumed more of their time and effort, and increased their stress and responsibility owing to the long period for which they had to stay at home. Homes became increasingly crowded with all members of the family present, thus increasing the household chores and caregiving burden faced by women and also the burden in meeting family needs in terms of food and supervising their children’s online education and recreation activities. Moreover, women also had to ensure that health and safety precautions to protect their family were performed diligently. Additionally, they experienced increasing financial obligations, declining income, and accumulated debt owing to the pandemic (AbuTayeh 2021 ). Similarly, Sisterhood is Global Institute-Jordan (SIGI) ( 2021a , 2021b ) mentioned that the most important effects of the COVID-19 pandemic on women included the burden of unpaid family care; the high risk of women and children being exposed to DV, and the decline in protection, prevention and response services provided to them; the exacerbation of a lack of women’s economic participation, the increase in the requirement of women in the healthcare sector, and the weak representation of women in leadership positions and response teams.

The Jordanian government implemented several measures to address cases of DV and violence against women and girls. These included the establishment of the Family Protection Department in 1997, which was merged with the Juvenile Department in 2021 to become the Juveniles and Family Protection Department (Public Security Directorate 2022 ). For the first time in Jordan, the principle of promoting gender equality was adopted as an organizing factor in the economic and social development plan (1999–2003), by integrating a gender perspective (The Jordanian National Commission for Women 1998 ). In addition, the National Council for Family Affairs was established in 2001 as a civil society organization to contribute to formulating and analyzing legislation for senior citizens, family counseling, and for early childhood and family protection against violence; it included the national team for family protection against violence to address the gaps in the protection system for family members and stop incidents of DV. The government also approved the Protection from Domestic Violence Law No. 15 in 2017, in comparison with the Protection from Domestic Violence Law No. 6 of 2008 that can be regarded as a protective law without mention of gender-based violence, Law No. 15 states the importance of adapting legal texts to address the needs of Jordanian families, and to expand the umbrella of family preservation by introducing alternative measures to punishment that could reform the family and enable it to overcome the obstacles it might face (EuroMed Rights, 2018 ; National Council for Family Affairs 2022 ; 2013 ). According to the Secretary-General of the National Council for Family Affairs, this new law gave a broader concept of the place where family members usually reside and expanded the concept of family by adding relatives up to the fourth degree and in-laws from the third and fourth degrees to the list of those covered by the protection law (Protection from domestic violence law, 2017; Al-Nimri 2019 ). Within the women’s protection system in Jordan, there are five main care homes affiliated with the Ministry of Social Development regarding family harmony for women whose lives are at risk: Dar Al-Wefaq Amman, Dar Al-Wefaq Irbid, Dar Al-Karama, Rusaifa Girls’ Care Home, and Amna. These care homes welcome women whose lives are threatened by honor killings; they wish to end “preventive detention,” in which women whose lives are at risk are referred to prisons in order to preserve their lives (Ministry of Social Development 2022 ). Regarding government priorities in combating violence against women in light of the COVID-19 pandemic, in March 2020, the Council of Ministers approved the national strategy for women’s empowerment (2020–2025) for a society free from discrimination and gender-based violence, where women and girls can enjoy full human rights and equal opportunities to achieve comprehensive and sustainable development (The Jordanian National Commission For Women 2020 ).

In spite of the official Jordanian statements about the increase in the number of DV cases against women and girls during the COVID-19 lockdown, the growing momentum of Jordanian media reports on issues of DV against women, and the publication of individual stories of women and girls who have been subjected to physical and psychological violence, murders, or underage marriage of girls (Darwish 2020 ; Ziyadat 2021 ; Zoud 2021 ), the data on DV during the COVID-19 pandemic still remains scarce in Jordan. The reliance on the press and media reports to understand the reasons for this has increased. Therefore, this review aimed to explore the link between the economic, social, and health-related effects of the pandemic and the high number of DV cases against women in Jordan. For this purpose, we explored the forms and causes of DV against women in Jordan, during and after the full and partial lockdowns imposed due to the pandemic, as discussed in the relevant literature.

Sample selection process and search criteria

All studies that revealed the forms and causes of DV against women in Jordan during the COVID-19 pandemic, that were published between April 2020 and November 2022, were searched using the University of Jordan Library, PubMed, Google Scholar, and SCOPUS database. We used the following six sets of terms: “gender-based violence” + “Jordan” + “COVID-19,” “partner violence” + “Jordan” + “COVID-19,” “domestic violence” + “Jordan” + “COVID-19,” “domestic abuse” + “Jordan” + “COVID-19,” “women abuse” + “Jordan” + “COVID-19,” and “wife abuse” + “Jordan” + “COVID-19.” The search comprised studies published in Arabic and English in academic journals, which used quantitative, qualitative, and mixed methods research designs. Primary search results yielded 231 studies. Of these studies, 66 studies were excluded as they did not meet the inclusion criteria. The systematic review was developed according to the PRISMA (see the flowchart [Fig. 1 ]).

PRISMA Preferred Reporting Items for Systematic reviews and Meta-Analyses. A PRISMA flow diagram visually summarises the screening process. It records the total number of articles found from the initial searches, and then makes the selection process transparent by reporting on inclusion or exclusion decisions made at various stages of the systematic review (Celegence, 2022 ; Macquarie University Library, 2023 ).

Data inclusion and exclusion criteria

The primary inclusion criteria for the papers include (a) the presence of DV against women construct, (b) Jordanian society, (c) use of Arabic and English language, (d) published scientific research, and (e) the time range during and after the COVID-19 pandemic between April 2020 and November 2022. This period was identified because it witnessed an increased momentum in scientific research regarding the consequences of the pandemic globally. Papers on DV, especially those about DV in Jordan, experienced stagnation toward the end of 2022. It was observed by the researchers that scientific journals, especially local Jordanian journals, no longer include the repercussions of the pandemic on DV among their priorities and interests in publishing. This was also confirmed through communication with other researchers and publishers.

We also included both qualitative and quantitative articles focusing on constructs connected to DV against women, such as women abuse, wife abuse, partner violence, gender-based violence, domestic abuse, family abuse, and family violence. Studies that examine DV against women in Jordan before the COVID-19 pandemic, and DV against the elderly before the COVID-19 pandemic, or children before or during and after the COVID-19 pandemic were excluded. Additionally, unpublished studies, national and international reports, duplicate articles, master’s and Ph.D. theses, conference abstracts, and studies that included refugee DV against women (e.g. families or Syrian women) in Jordan were also excluded.

After assessing the quality of the primary studies and refining the included studies in the final sample, all eight remaining studies that met the inclusion criteria were exported to EndNote. The data were extracted and sorted through a structured table that contained seven variables: (a) author and publication year, (b) aim, (c) study design, (d) sample details, (e) occurred sample location, (f) definition of DV, (g) forms of DV, (h) causes of DV, (i) abuser, and (j) study limitations (see Table 1 ).

Quality assessment

The search in the literature via a comprehensive search of bibliographic databases, manual searches, and exploration of grey literature searches contribute to reducing bias. Moreover, the inclusion of studies has been thoroughly reviewed and their eligibility was independently assessed by two sociology professors, A.A and K.B with expertise in the fields of violence and abuse within the Jordanian context. They screened all study titles and abstracts to ensure that the study objectives were adhered to, in collaboration with both researchers M.A and R.A. They then examined the texts of the studies following the systematic review protocol as a guide (PRISMA 2020 ); R.A and A.A and reviewed the data, and the whole team discussed and agreed on the results of the data. Quantitative studies were assessed using the Quality Assessment Tool for quantitative studies, which includes eight categories: selection bias, study design, confounders, blinding, data collection method, withdrawals and dropouts, intervention integrity, and analysis appropriate to questions. Each examined practice receives a mark ranging between “strong,” “moderate,” and “weak” (Thomas 2003 ). For qualitative studies, the Critical Appraisal Skills Program (CASP Qualitative Studies Checklist) was used, which includes ten questions, answerable with three options (“yes”, “no” or “can’t tell”), to evaluate the study aims, validity of methodology, data analysis and results, research design validity and novelty. The CASP is divided into three sections — Section A: Are the results of the study valid? Section B: What are the results? Section C: Will the results help locally? (CASP website 2022 ).

Sample description

The eight articles included in the final sample were published between April 2020 and November 2022 in eight separate English-language journals on interpersonal violence, clinical practice, humanities and social sciences, and psychology. Of these super specialty journals and their scope in issues related to violence, only one journal specialized in violence—the Journal of Interpersonal Violence . Most of these studies had a quantitative descriptive design via online survey, and included cross-sectional (four articles), descriptive statistical (two articles), and descriptive comparative (one article) studies. The remaining study adopted qualitative exploratory descriptive analysis through a semi-structured interview guide (one article). Most of the participants in these studies were married women, between 18 and 55 years of age, who had experienced violence from their husbands; a few of them had experienced it from their father, brother, mother, or work colleagues. Only one study addressed DV among pregnant women (Abujilban et al. 2022 ). Two articles included samples of men: one of them included only married men and explored the causes and forms of DV against their wives from the perspective of the husbands. Most of these studies considered the rural and urban classification in selecting the population samples from the north, central, and southern regions of Jordan (five articles), while three articles selected samples from Amman, the capital of Jordan. No article defined or presented a clear concept of DV, except three articles, one of which was based on the World Health Organization’s definition (Abujilban et al. 2022 ). The other articles defined DV as any abusive or violent behavior that occurred between spouses. Two articles used the term “intimate partner” to refer to the husband, and some used IPV, “spousal violence,” “women abuse,” and “male violence against women” synonymously with DV (Table 1 ).

Limitations of the study

The present study is one of the first studies to address DV against women during and after the COVID-19 pandemic in Jordan, that is, within an Arab Muslim social- cultural context. The strengths of this review are the focus on the concept of DV used in studies as well as the identity of the abuser. in Jordan (rural, urban/ North, Central, South), with differently ages (18-60 years) and their educational, professional, and marital status, as well as their health status (e.g. pregnant women).

This study enriches the field of family sociology studies and women’s studies in Jordan, considering that the data on DV Against Women during the COVID-19 pandemic are still scarce in Jordan and the reliance on the press and media reports to understand the reasons for this has increased. Nevertheless, the present study also had some limitations. First, it excludes women of other nationalities, such as those who hold Syrian and Iraqi nationality and lived for a period in Jordan and were involved in its cultural context and experienced the same economic conditions, especially during and after the pandemic. Without this exclusion, the study sample could have enhanced the results and presented clearer details on women’s DV experiences in Jordan in general. Second, this review was limited to publications in English and Arabic, and in international and local academic journals only, which means that studies that may have been published in other languages were not considered, and scientific studies conducted by organizations and educational institutions that could have added to causes or forms of violence were not considered. Finally, this study does not consider studies on DV consequences and intervention strategies, which may hinder a complete understanding of this complex phenomenon.

The findings of our review were divided according to the main research goals: exploring forms and causes of DV against Jordanian women during and after the full and partial lockdowns imposed due to the pandemic.

Forms of DV against women during and after the COVID-19 pandemic

The results of our systematic review revealed 11 forms of DV experienced by women and girls in Jordan during and after the full and partial lockdowns due to the pandemic: physical (Abujilban et al. 2022 ; Abuhammad, 2021 ; Alsawalqa, 2021a ; Alsawalqa, 2021b ; Alsawalqa et al. 2021 ; Kataybeh 2021 ; Qudsieh et al. 2022 ), economic (Alsawalqa 2021a ; Alsawalqa, 2021b ; Qudsieh et al. 2022 ), psychological (Abujilban et al. 2022 ; Abuhammad, 2021 ; Alsawalqa 2021a ; Alsawalqa 2021b ; Kataybeh 2021 ), emotional (Alsawalqa, 2021a ; Alsawalqa 2021b ) verbal (Kataybeh 2021 ; Qudsieh et al. 2022 ; Alsawalqa et al. 2021 ), sexual (Abujilban et al. 2022 ; Kataybeh 2021 ), control and humiliation (Abujilban et al. 2022 ), bullying (Alsawalqa et al. 2021 ), online abuse (Alsawalqa et al. 2021 ), harassment (Alsawalqa 2021b ), and neglect (Alsawalqa 2021a ). Physical violence was the most prevalent among the samples of these studies. Notably, some of these articles dealt with psychological and emotional abuse as belonging to the same category, or as a separate concept [e.g., Qudsieh et al. 2022 ; Kataybeh 2021 ]. By contrast, some researchers defined the concepts of verbal, psychological, and emotional abuse, and dealt with them separately [e.g., Alsawalqa 2021b ; Kataybeh 2021 ]. One article addressed DV in general without clarifying its types and measurement; its results simply indicated that “20.5% of the participants suffered from increased domestic abuse during the COVID-19 pandemic” (Aolymat, 2020 : 520).

Causes of DV against women during and after the COVID-19 pandemic

We found that the causes of DV Against Women during the pandemic were the result of a combination of the economic, socio-cultural, and psychological effects of the COVID-19 pandemic and curfew. Six articles confirmed that the economic factors included poverty; job loss; low, insufficient, or reduced income; and high cost of living and healthcare facilities, which led to the spread and rise of DV (Abuhammad, 2021 ; Qudsieh et al. 2022 ; Alsawalqa, 2021a ; Alsawalqa 2021b ; Aolymat 2020 ; Kataybeh 2021 ). Moreover, the lockdown increased the time spent with partners and family members and caused negative psychological responses, such as stress and tension (Abuhammad, 2021 ). Five articles emphasized that the main socio-cultural factor that encouraged violence against women was the hegemonic masculinity and patriarchy that normalized violence, making women accept, tolerate, and even justify it (Abuhammad, 2021 ; Alsawalqa 2021a ; Alsawalqa 2021b ; Alsawalqa et al. 2021 ; Kataybeh 2021 ). Furthermore, the studies showed that the wives’ families often interfered in their marital life (Alsawalqa 2021a ; Kataybeh 2021 ; Alsawalqa et al. 2021 ). The results of the study Alsawalqa ( 2021a ) also confirmed that the wives’ long preoccupation with social media, and neglect of the house, children, and their personal hygiene, in addition to the husband’s unfulfilled sexual needs, were among the main reasons why they were violent to their wives, before and during the pandemic. All studies indicated that these economic, social, and cultural factors and their negative effects on the psychological, emotional, and social well-being of individuals had long-term effects which continued after the end of the lockdown.

Despite the abundance of publications on DV in Jordan, particularly during the COVID-19 pandemic, there was no clear or explicit definition of the concept of DV or an accurate distinction of its forms (Alsawalqa et al. 2022b ). Our study attempted to shed light on published scientific research that addressed DV in Jordan during and after the COVID-19 pandemic, to gain knowledge and enhance our understanding of the link between the pandemic’s ramifications and high rates of DV in Jordan.

Globally, before the COVID-19 pandemic began, one in every three women experienced physical or sexual violence mostly by an intimate partner; 245 million women and girls aged 15 years or over have been subjected to sexual or physical violence. The violence against women and girls has intensified since the outbreak of COVID-19 (Women UN 2021 ).

According to World Health Organization ( 2020 : p1) the Eastern Mediterranean Region has the second highest prevalence of violence against women (37%) worldwide. This is due to structural systems that maintain gender inequalities at different levels of society, compounded by political crises and socioeconomic instability in the region. Based on a Women UN ( 2021 ) study, in collaboration with Ipsos, and with support from national statistical offices, national women’s machineries and a technical advisory group of experts, the pooled data from 13 countries covering more than 16,000 women respondents, found that 1 in 4 women say that household conflicts have become more frequent, and they feel more unsafe in their home, and (58%) women have experienced or know a woman who has experienced violence since COVID-19. The most common form is verbal abuse (50%), followed by sexual harassment (40%), physical abuse (36%), denial of basic needs (35%) and denial of means of communication (30%). Additionally, (56%) women felt less safe at home since the COVID-19 pandemic. (44%) women living in rural areas, were more likely to report feeling more unsafe while walking alone at night since the pandemic, compared to women living in urban areas (39%). (62%) were also more likely to think that sexual harassment in public spaces has worsened, compared to (55%) of women living in urban areas. Moreover, younger women aged 18–49 years were the more vulnerable group, with nearly 1 in 2 of them affected, and more than 3 in 10 women (34%) aged 60+ and more than 4 in 10 women aged 50–59 years (42%) reported having experienced violence or knowing someone who has since the pandemic began. Women living with children were more likely to report having experienced violence or to know someone who has experienced it since the pandemic, whether they were partnered (47%) or not (48%). Conversely, nearly 4 in 10 women living without children, partnered (37%) or not (41%), reported such experiences. Women who were not employed during the pandemic were also particularly affected, with an estimated (52%) reporting such experiences, compared to (43%) of employed women. Additionally, exposure was highest among women in Kenya (80%), Morocco (69%), Jordan (49%) and Nigeria (48%). Those in Paraguay were the least likely to report such experiences, at (25%) (p. 5-6, 8).

In Jordan, the spread of the COVID-19 pandemic and the strict lockdown measures to prevent its spread had negative economic, social, and psychological effects on the citizens. Regarding economic repercussions, according to Raouf et al. ( 2020 ), the Jordanian gross domestic product decreased by (23%) during the lockdown. The service sector was the hardest hit, seeing an estimated drop in output of approximately (30%), and food systems experienced a reduction in output of approximately (40%). Moreover, employment losses during the lockdown were estimated at over (20%), mainly driven by job losses in the service and agriculture sectors. Household incomes decreased on average by approximately one-fifth owing to the lockdown, mainly driven by contraction in the service sector, slowdown in manufacturing, and lower remittances from abroad. As a result of this lockdown and company exploitation of the National Defense Law’s decisions imposed in emergency situations to ensure protect public safety, many workers were dismissed, several others did not receive their salaries, and some workers’ incomes decreased by (30–50%). Additionally, some companies and organizations deducted the period of lockdown imposed by the government from workers’ salaries (Jordan Labor Watch 2020 ; Jordan Economic Forum 2020 ).

The unemployment rate during the fourth quarter of 2019 reached (19%), an increase of (0.3%) from the fourth quarter of 2018. Unemployment rates reached (24.7%) in 2020 and (25%) during the first quarter of 2021, reflecting an increase of (5.7%) from the first quarter of 2020. The rate in the fourth quarter of 2021 reached (23.3%), an increase of (0.1%) from the third quarter of the same year, and a decrease of (1.4%) from the fourth quarter of 2020 (Department of Statistics 2021 ; 2020 ). Additionally, self-isolation measures and lockdown policies led to a lack of access to adequate healthcare services (Alijla 2021 ), particularly restricting women’s access to healthcare (Jordanian Economic and Social Council 2020a , 2020b ). There were also increases in food prices and people getting into debt (UNDP 2020 ).

According to the latest survey on household income and expenditures conducted by the Department of Statistics (2017-2018) (Department of Statistics, Jordan [DOS], 2018 ), poverty in Jordan was relatively high. It reached (15.7%), representing 1.069 million Jordanians, while the rate of (extreme) hunger poverty in Jordan reached (0.12%), which is equivalent to 7993 Jordanian individuals. Poverty in Jordan increased significantly during the COVID-19 pandemic; the estimates of the World Bank showed that the potential increase in the short-term poverty rates in Jordan may increase by an additional (11%) over the official rate declared before the COVID-19 pandemic (15.7%), to approximately (27%) (Baybars 2021 ). A UNICEF study ( 2020 ), which covered both Jordanian and Syrian families, also found that the number of households with a monthly income of less than 100 JD (140 USD) had doubled since before the COVID-19 pandemic, and only (28%) of households had adequate finances to sustain themselves for a two-week period. Four out of ten families were unable to purchase the hygiene products they need; children went to bed hungry in (28%) of the homes during lockdown, decreasing to (15%) post-lockdown. As a result of the pandemic, employment was disrupted in (68%) of households. Furthermore, (17%) of the children under five years did not receive basic vaccinations, (23%) of children who were sick during the pandemic did not receive medical attention (largely owing to fear of the virus and lack of funds), eight out of 10 households adopted negative coping strategies, and (89%) of young women performed household duties (including caring) compared to (49%) of young men.

Notably, The Jordan Economic Monitor report issued by the World Bank showed that economic growth in Jordan in 2021 was strong at (2.2%), owing to the significant expansions in the service, industry, and travel and tourism sectors. However, some sectors, such as the service sector that deals directly with the public (through restaurants, hotels, and so on) were still experiencing low levels of economic growth before the COVID-19 pandemic in 2020 (Refaqat et al. 2022 ). The termination of worker services in affected organizations and businesses had a significant impact on the economic circumstances of many families, particularly women-led households, causing health and psychological harm due to the inability to ensure general well-being and access to effective healthcare. Additionally, women working in low-wage, informal, temporary, or short-term sectors, such as seasonal jobs or small-scale businesses, were disproportionately affected by the pandemic. Furthermore, (35.4%) of Jordanian women who worked in the education sector had to switch to remote teaching because of the pandemic, which added to their domestic work. By contrast, (13.4%) of women in Jordan who worked in the health and social services sectors continued to provide their services along with the burdens of social responsibilities imposed on them (Jordanian Economic and Social Council 2020a , 2020b ).

Regarding social and psychological repercussions, the negative economic conditions, in addition to the lockdowns, social distancing behaviors, and the lack of in-person social interaction, created enormous pressures that led to high levels of anxiety, stress, and depression among the Jordanian people, particularly among those aged 18–39 years. The stress was greater in men than in women, and anxiety and depression levels were higher in women than in men (Abuhammad et al. 2022 ). People with poor social support, who were younger and female, were more likely to experience lockdown-related anxiety (Massad et al. 2022 ). Moreover, the COVID-19 pandemic had a notable effect on the mental health of the Jordanians who had low monthly income (<500 JD) or were unemployed, as well as diabetes patients (Suleiman et al. 2022 ), who felt neglected or lonely. Married couples with higher income were less likely to feel lonely than others (Jordanian Economic and Social Council, 2020a , 2020b ). Gresham et al. ( 2021 ) found that COVID-19-related stressors (financial anxiety, social disconnection, health anxiety, COVID-19-specific stress, and so on) were associated with greater IPV during the pandemic. Additionally, IPV was associated with movement outside of the home (leaving the residence); greater movement outside the home may act as a way for victims to physically distance themselves from their partners, thereby reducing stress and avoiding further abuse.

These negative socio-economic effects caused by the COVID-19 pandemic led to exacerbating DV among Jordanian families. During the mandatory curfew, (35%) of Jordanians were subjected to at least one form of domestic abuse (10% total increase), (58%) of which were victims of abuse by a male family member (25% father, 16.5% husband, and 16.5% brother), (33%) by a female family member (25% mother, 8% sister), and (9%) by others. The most prevalent forms of DV reported during the lockdown were verbal violence (48%), psychological violence (26%), neglect (17%), and physical abuse (9%). These violent acts (between March 21st to April 26th, 2020) occurred 1–3 times among (75%) of the COVID-19 domestic abuse victims, 4–6 among (19%), and (7+) times among (7%) (Center for Strategic Studies-Jordan 2020 ). Additionally, since the beginning of 2021 until November 23, 2021, 15 family murders were reported (Sisterhood is Global Institute-Jordan 2021b ).

Jordanian husbands confirmed that poverty, insufficient salary, wives’ money spending habits and not considering the negative economic effects of COVID-19 on their work led to their abusive behavior and violent response (Alsawalqa 2021a ). The patriarchal structure in the Jordanian society requires men to adhere to masculine standards and ideals of “true manhood,” which require men to be strong, independent, emotionally restrained, tough, and assume responsibility for leadership, family care, and financial support, given that they occupy a higher rank than women. Masculinity has social advantages and entitlements, the most prominent of which are male control, ownership, and dominance. If men are unable to achieve these standards, society can confront them with humiliation, marginalization, stigmatization, and blame (particularly from their wives). This can make them feel shame, disgrace, sadness, depression, failure, status frustration, anxiety, fear of the future, and low self-esteem. Couples express these negative feelings through aggressive and violent behaviors (Alsawalqa et al. 2021 ; Alsawalqa and Alrawashdeh 2022 ).

Additionally, some wives (working and non-working) were unable to repay their loans, especially their loans from microfinance institutions. These financial institutions provide loans and financial facilities to specific economic sectors (such as agriculture) and groups of the population (such as women, the poor, and artisans). These institutions have been excessive in lending to women for consumption purposes at the expense of productive projects and have been unable to lift women out of poverty and empower them economically. Owing to the existence of a major defect in the legislative system, lending conditions, and the economic repercussions of the COVID-19 pandemic, women borrowers (known as “female debtors,” or Algharimat in Arabic) were subjected to legal accountability and imprisonment if they failed to pay back the debt (Jordanian Economic and Social Council 2020b ). Notably, some husbands or fathers forced their wives to borrow because of poverty by dominating or exercising coercive control over the women; some women borrowed voluntarily with the aim of improving the family income. If women refuse to contribute to the household income or borrow money, they can face physical, verbal, psychological, and economic abuse.

Moreover, if women borrow money (which subjects them to legal accountability) without the knowledge of her husband or male guardian, it can expose them to more violence, because men can consider this act as a violation of their control which can cause a scandal or encourage stigma (Sa’deh, 2022 ). Jordanian working women who are married and live in rural and urban areas have encountered spousal economic abuse through control of their economic resources, management of their financial decisions, and exploitation of their economic resources. Moreover, they have endured emotional, psychological, and physical abuse, and harassment as tactics by husbands to reinforce their economic abuse and maintain control over them (Alsawalqa 2021b ). Peterman et al. ( 2020 ) indicated that the increased violence against women and children during the pandemic was associated with economic insecurity and poverty-related stress, lockdowns, social isolation, exploitative relationships, and reduced health and domestic support options.

The Information and Research Center - King Hussein Foundation (IRCKHF) and Hivos report ( 2020 : p. 4, 14) on the double burden of women in Jordan during COVID-19, showed that in times of disease outbreaks, most women take up greater responsibilities and are often overburdened with paid and unpaid work, which impacts their overall well-being. Unpaid work within the home, sometimes referred to as reproductive work, includes a variety of tasks such as childbearing and caring, preparing meals, cleaning, doing laundry, maintaining the house, and taking care of older adults or disabled family members, among others. Although paid work is tied to a monetary value, unpaid work is usually not recognized as “real work”. Moreover, the new e-learning systems resulted in parents, especially mothers, spending a lot of time ensuring that their children were learning and following up on their schoolwork. Some parents worried about the future of their children’s education; the closure of schools and nurseries created an additional burden for working parents who had to find childcare solutions while they went to work. These additional burdens created new psychological pressures and increased the problem of violence among family members. Married Jordanian women, particularly those who work and are educated, often face conflicts in the economic, political, and socio-cultural aspects (Alsawalqa 2016 ). Moreover, they experience Marriage and emotional burnout that worsens with the increase in the number of children, which negatively affects their health and leads to headaches, eating disorders, irregular heart rate, stomach pain, and so on (Alsawalqa, 2019 ; Alsawalqa, 2017 ). These problems contribute to a higher marriage burnout rate among spouses, particularly among those who work full-time jobs, have been married for ≥ 10 years, and have children. Marriage burnout is a painful state of emotional exhaustion, with physical and emotional depletion experienced by spouses. This state results from emotional exhaustion, work exhaustion, and failure to fulfill the requirements (particularly emotional requirements) of the marriage (Alsawalqa 2019 ).

Conclusions

The COVID-19 pandemic did not create new motives for DV in Jordan but contributed to its negative economic and psychological repercussions in exacerbating and confirming the pre-existing motives. The patriarchal structure and gender stereotypes in Jordanian society that establish the relationships between men and women based on coercive control produced prejudices that led to social injustice and gender inequality and made both sexes victims of DV. Individuals in Jordanian societies employed the norms, ideas, and values of the patriarchal culture to negatively adapt to the adverse economic and psychological effects of the pandemic, leading to more cases of DV.

The COVID-19 pandemic revealed the weak economic and social structures in place before the crisis and their negative impact during the pandemic, such as women’s poor economic participation, the gender gap in the labor market, high income tax, high cost of living, administrative and financial corruption, a weak labor market and infrastructure, and high rates of unemployment and poverty. This study highlighted the need for serious participatory work from the government and civil society organizations based on scientific research and approaches to change the cultural and social norms that reinforce patriarchal domination and stereotypes that perpetuate the gender gap. One way of achieving this is through the educational curricula in schools and universities; educating individuals on how to deal with life pressures; choosing a life partner; understanding the motives for marriage and the foundations for successful and healthy relationships, and raising children; and positive social self-development. We recommend supporting the efforts of the Jordanian government to continue the process of development and diligently follow up on the implementation of the directives and vision of His Majesty Abdullah II bin Al-Hussein, King of Jordan, on economic, political, and administrative reform, activating the role of youth and empowering women. We realize that social and cultural change, and the reforming of sectors is not an easy task and requires tremendous efforts over a long period.

Implications of the study

Considering the current study results, a follow-up study to highlight the lack of conceptual clarity on DV Against Women and its various forms is recommended. Future research must carefully study the behaviors involved in each form of DV and identify the abuser of women as this will help in developing effective policies and practices to reduce or end DV against women. In addition, it would contribute to directing researchers via a correct scientific methodology as they study women’s resistance to violence, and educating women to understand forms of violence and seek the most appropriate resistance strategies to reduce or end it, and avoid involvement in the cycle of violence. Moreover, we recommend paying attention to conducting more studies on DV against males by their wives, mothers, or intimate partners, and DV against the elderly, which will contribute to understanding the factors and forms of domestic violence in more detail. We believe that these aforementioned suggestions will aid in formulating the best intervention strategies.

To reduce DV in Jordan, it is necessary to implementing courses and workshops for women, girls, and males, especially in rural areas, aiming to increase their awareness about the motives for marriage and the foundations of choosing a suitable partner, how to manage emotions, and resist and reduce violence. Additionally, promoting anti-gender discrimination thinking in educational curricula in universities and schools. Moreover, remove obstacles in implementing the Law on Protection from DV, and not to waive personal rights in crimes of DV. In addition, rehabilitation for victims and perpetrators of violence.

Data availability

All data relevant to the study are included in the article.

AbuTayeh AM (2021) Socio-economic constraints Jordanian women had encountered as a result of COVID-19 pandemic, and coping mechanisms. Asian Soc Sci 17:63–76. https://doi.org/10.5539/ass.v17n10p63

Article Google Scholar

Abufaraj M, Eyadat Z, Al-Sabbagh MQ, Nimer A, Moonesar IA, Yang L et al. (2021) Gender-based disparities on health indices during COVID-19 crisis: A nationwide cross-sectional study in Jordan. Int J Equity Health 20:91. https://doi.org/10.1186/s12939-021-01435-0

Article PubMed PubMed Central Google Scholar

Abuhammad S (2021) Violence against Jordanian women during COVID-19 outbreak. Int J Clin Pract 75:e13824. https://doi.org/10.1111/ijcp.13824

Article CAS PubMed Google Scholar

Abuhammad S, Khabour OF, Alomari MA, Alzoubi KH (2022) Depression, stress, anxiety among Jordanian people during COVID-19 pandemic: A survey-based study. Inf Med Unlocked 30:100936. https://doi.org/10.1016/j.imu.2022.100936

Abujilban S, Mrayan L, Hamaideh S, Obeisat S, Damra J (2022) Intimate partner violence against pregnant Jordanian women at the time of COVID-19 pandemic’s quarantine. J Interpers Violence 37:NP2442–64. https://doi.org/10.1177/0886260520984259

Alijla A (2021) Possibilities and challenges: Social protection and COVID-19 crisis in Jordan. Civ, 1. https://doi.org/10.28943/CSKC.002.90003

Al-Nimri N (2019). Adoption of national legislation to strengthen the family and child protection system: The amended Domestic Violence Protection Law is among the most prominent national social achievements. Al Ghad. [Accessed 10 February 2022]. https://alghad.com/

Alsawalqa RO (2021a) A qualitative study to investigate male victims’ experiences of female-perpetrated domestic abuse in Jordan Curr Psychol 1–16. https://doi.org/10.1007/s12144-021-01905-2

Alsawalqa RO, Al Qaralleh AS, Al-Asasfeh AM (2022a) The Threat of the COVID-19 Pandemic to Human Rights: Jordan as a Model. J Hum Rights Soc Work 7:265–276. https://doi.org/10.1007/s41134-021-00203-y

Alsawalqa RO (2021b) Women’s abuse experiences in Jordan: A comparative study using rural and urban classifications. Humanit Soc Sci Commun 8:186. https://doi.org/10.1057/s41599-021-00853-3

Alsawalqa RO, Alrawashdeh MN, Sa’deh YAR, Abuanzeh A (2022b) Exploring Jordanian women’s resistance strategies to domestic violence: A scoping review. Front Socio. 7:1026408. https://doi.org/10.3389/fsoc.2022.1026408

Alsawalqa RO, Alrawashdeh MN, Hasan S (2021) Understanding the Man Box: The link between gender socialization and domestic violence in Jordan. Heliyon 7:e08264. https://doi.org/10.1016/j.heliyon.2021.e08264

Alsawalqa RO, Alrawashdeh MN (2022) The role of patriarchal structure and gender stereotypes in cyber dating abuse: A qualitative examination of male perpetrators experiences. Br J Socio 73:587–606. https://doi.org/10.1111/1468-4446.12946

Alsawalqa R (2016) Social change and conflict of values among educated women in Jordanian society: A comparative study. Dirasat Hum Soc Sci 43:2067–93

Alsawalqa RO (2017) Emotional burnout among working wives: Dimensions and effect. Can Soc Sci 13:58–69

Google Scholar

Alsawalqa RO(2019) Marriage burnout: When the emotions exhausted quietly quantitative research Iran J Psychiatry Behav Sci 13:e68833. https://doi.org/10.5812/ijpbs.68833

Anderson K (2020) Daring to Ask, Listen, and Act: A Snapshot of the Impacts of COVID-19 on Women and Girls’ Rights and Sexual and Reproductive Health [Report]. United Nations Fund for Population Activities, Geneva, https://reliefweb.int/sites/reliefweb.int/files/resources/20200511_Daring%20to%20ask%20Rapid%20Assessment%20Report_FINAL.pdf

Aolymat I (2020) A cross-sectional study of the impact of COVID-19 on domestic violence, menstruation, genital tract health, and contraception use among women in Jordan. Am J Trop Med Hyg 104:519–25. https://doi.org/10.4269/ajtmh.20-1269

Article CAS PubMed PubMed Central Google Scholar

Al-Tammemi AB (2020) The battle against COVID-19 in Jordan: An early overview of the Jordanian experience. Front Public Health 8:188. https://doi.org/10.3389/fpubh.2020.00188

Baybars S (2021) The World Bank: Corona’s Repercussions Have Raised the Poverty Rate in Jordan. https://alghad.com . [Accessed 10 February 2022]

Celegence (2022) PRISMA flow diagram – CAPTIS™ features. https://www.celegence.com/prisma-flow-diagram/ . Accessed May 2022

Center for Strategic Studies-Jordan (2020). Coronavirus lockdown exacerbating domestic violence in Jordan. Jordan’s barometer: The pulse of the Jordanian street– (20). Hivos [Study]. https://www.cawtarclearinghouse.org/

Darwish R (2020) 2020 Was Not a Perfect Year for Jordanian Women. https://www.bbc.com/arabic/middleeast-55468564 . [Accessed 10 February 2022]

Department of Statistics, Jordan [DOS] (2018) Jordan population and family and health survey 2017–18: key indicators. DOS and ICF, Amman, Jordan, and Rockville, MD, USA, http://www.dos.gov.jo/dos_home_a/main/linked-html/DHS2017_en.pdf

Department of Statistics (DoS) (2020) Unemployment Rate During the First Quarter of 2020 – Official Report. http://dosweb.dos.gov.jo/unemp_q12020

Department of Statistics (DoS). (2021). Unemployment Rate During the First Quarter of 2021 –official Report. http://dos.gov.jo/dos_home_a/main/archive/unemp/2021/Emp_Q12021.pdf

EuroMed Rights (2018) Situation Report on Violence against Women: Legislative Framework. Report Online, Retrievedِ April 2020. https://euromedrights.org

CASP website (2022).CASP Checklist: 10 questions to help you make sense of a Qualitative research. Retrieved May 2022. https://casp-uk.net/casp-tools-checklists/

Gresham AM, Peters BJ, Karantzas G, Cameron LD, Simpson JA (2021) Examining associations between COVID-19 stressors, intimate partner violence, health, and health behaviors. J Soc Personal Relat 38:2291–307. https://doi.org/10.1177/02654075211012098

Higher Population Council (HPC) (2021) Jordan joins the world in commemorating the International Day of Non-Violence. https://www.hpc.org.jo/en/content/jordan-joins-world-commemorating-international-day-non-violence . [Accessed 10 February 2022]

IRCKHF and Hivos (2020) COVID-19 and the Double Burden on Women in Jordan. Ministry of Foreign Affairs of the Netherlands [Report], Hivos, https://irckhf.org/projects/covid-19-and-the-double-burden-on-women-in-jordan/

Jordan Economic Forum (2020) Unemployment in Jordan: Reality, expectations and proposals. https://soundcloud.com/jordaneconomicforum-jef/5xedpqmblc51

Jordan Labor Watch J (2020) Unprecedented Challenges for Workers in Jordan: A Report on the Occasion of International Labor Day. http://phenixcenter.net/

Jordanian Economic & Social Council (2020a) Impact of COVID-19 on Gender Roles and Violence Against Women -Results from Jordan [Study]. Amman, Hashemite Kingdom of Jordan. https://jordan.un.org/

Jordanian Economic & Social Council (2020b). Gender-related impacts of Coronavirus pandemic in the areas of health, domestic violence and the economy in Jordan. Hashemite Kingdom of Jordan: Amman. https://jordan.unwomen.org/

Jordanian Juvenile and Family Protection Department (2020). Digital statistics. https://www.psd.gov.jo/en-us/content/digital-statistics/ . [Accessed 10 February 2022]

Kataybeh Y (2021) Male Violence Against Women: An Exploratory Study of Its Manifestations, Causes, and Discrepancies over Jordanian Women under Corona Pandemic. Preprints 1. https://pesquisa.bvsalud.org/global-literature-on-novel-coronavirus-2019-ncov/resource/pt/ppzbmed-10.20944.preprints202104.0695.v1?lang=en

Kaur R, Garg S (2008) Addressing domestic violence against women: an unfinished agenda. Indian J Community Med: official publication of Indian Association of Preventive & Social Medicine 33(2):73–76. https://doi.org/10.4103/0970-0218.40871

Leslie E, Wilson R (2020) Sheltering in place and domestic violence: Evidence from calls for service during COVID-19. J Public Econ (2020) 189:104241. https://doi.org/10.1016/j.jpubeco.2020.104241

Macquarie University Library (2023) Systematic reviews: PRISMA flow diagram & diagram generator tool. https://libguides.mq.edu.au/systematic_reviews/prisma_screen . Accessed May 2024

Massad I, Al-Taher R, Massad F, Al-Sabbagh MQ, Haddad M, Abufaraj M (2022) The impact of the COVID-19 pandemic on mental health: Early quarantine-related anxiety and its correlates among Jordanians. East Mediterr Health J 26:1165–72. https://doi.org/10.26719/emhj.20.115

Ministry of Social Development (2022) Care homes for family harmony for women whose lives are at risk. http://mosd.gov.jo

National Council for Family Affairs (2013). Socioeconomic characteristics of domestic violence cases [Study]. [Accessed 10 February 2022]. https://ncfa.org.jo/uploads/2020/07/ab0d5083-3d7e-5f1feb76ebd1.pdf

National Council for Family Affairs. (2022). Domestic violence in Jordan: Knowledge, reality. Trends. [Accessed 10 February 2022]. https://ncfa.org.jo/uploads/2020/07/38ceffed-d7a4-5f1ffdb3f907.pdf

Pedrosa AL, Bitencourt L, Fróes ACF, Cazumbá MLB, Campos RGB, de Brito SBCS et al. (2020) Emotional, behavioral, and psychological impact of the COVID-19 pandemic. Front Psychol 11:566212. https://doi.org/10.3389/fpsyg.2020.566212

Peterman A, Potts A, O’Donnell M, Thompson K, Shah N, OerteltPrigione S, et al. (2020). Pandemics and violence against women and children. Center for Global Development Working Paper 528. http://iawmh.org/wp-content/uploads/2020/04/pandemics-and-vawg-april2.pdf . [Accessed 24 September, 2022]

PRISMA (2020) The NEW Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) website. https://www.prisma-statement.org . Retrievedِ february 2021

Public Security Directorate (2022) Juvenile and Family Protection Department. https://www.psd.gov.jo/en-us/psd-department-s/family-and-juvenile-protection-department/ . Accessed 10 February 2022

Qudsieh S, Mahfouz IA, Qudsieh H, Barbarawi LA, Asali F, Al-Zubi M et al. (2022) The impact of the coronavirus pandemic curfew on the psychosocial lives of pregnant women in Jordan. Midwifery 109:103317. https://doi.org/10.1016/j.midw.2022.103317

Raouf M, Elsabbagh D, Wiebelt M (2020) Impact of COVID-19 on the Jordanian Economy: Economic Sectors, Food Systems, and Households (Projest Paper). The International Food Policy Research Institute. https://doi.org/10.2499/p15738coll2.134132

Refaqat S, Janzer-Araji A, Mahmood A, Kim J (2022) Jordan economic Monitor. Spring. DC: World Bank Group. (2022): Global Turbulence Dampens Recovery and Job Creation (English).Jordan Economic Monitor, Washington, http://documents.worldbank.org/curated/en/099410007122222740/IDU05823c2b70646004a400b9fa0477cea7736a4

Sa’deh Y (2022) Economic abuse from a gender perspective: A qualitative study on female debtors “Algharimat” in Jordan [unpublished Master’s thesis]. Department of Sociology and Anthropology, Doha Institute for Graduate Studies, Qatar

Sisterhood is global institute-Jordan (SIGI) (2021a). What do the Numbers tell us about Injuries and Deaths: An Analytical Reading from a Gender Perspective (Facts Paper). [Accessed 12 March 2022]. https://www.sigi-jordan.org/?p=10613

Sisterhood is global institute-Jordan (SIGI). (2021b). 47% of Women in Jordan Have Their Voices Unheard and Their Sufferings Unseen with Regard to Domestic Violence [Report] https://www.unicef.org/jordan/media/3041/file/Socio%20Economic%20Assessment.pdf

Suleiman YA, Abdel-Qader DH, Suleiman BA, Suleiman AH, Hamadi S, Al Meslamani AZ (2022) Evaluating the impact of COVID-19 on mental health of the public in Jordan: A cross-sectional study. J Pharm Pharmacogn Res 10:196–205. https://doi.org/10.56499/jppres21.1191_10.2.196

Article CAS Google Scholar

The Jordanian National Commission for Women (2020). National strategy for women in Jordan (2020–2025). https://digitallibrary.un.org/record/1309642?ln=en

The Jordanian National Commission for Women, (1998). A preliminary report on the implementation of the Beijing Approach. https://digitallibrary.un.org/record/1309642?ln=en

Thomas, H (2003). Quality assessment tool for quantitative studies. Effective public health practice project. Hamilton, ON, Canada: McMaster University. https://www.ephpp.ca/PDF/Quality%20Assessment%20Tool_2010_2.pdf

UNDP (2020). COVID-19 Impact on Households in Jordan: A rapid assessment. https://www.undp.org/sites/g/files/zskgke326/files/migration/arabstates/UNDP-Impact-of-COVID-19-on-Households-General-21-FINAL-21-May.pdf . [Accessed 10 February 2022]

UNICEF (2020). Socio-Economic Assessment of Children and Youth in the time of COVID-19. Jordan Publishing [Study]. https://www.unicef.org/jordan/media/3041/file/Socio%20Economic%20Assessment.pdf

United Nations. (2022). What is domestic abuse? https://www.un.org/en/coronavirus/what-is-domestic-abuse [Accessed 12 March 2022]

Women UN (2021). Measuring the Shadow Pandemic: Violence Against Women During COVID-19 [Report]. https://data.unwomen.org/sites/default/files/documents/Publications/Measuring-shadow-pandemic.pdf . [Accessed 12 March 2022]

World Health Organization (2020) COVID-19 and violence against women in the Eastern Mediterranean Region. Report Online, Retrieved February 2022. https://pmnch.who.int/docs/librariesprovider9/meeting-reports/covid-19-and-violence-against-women-in-emro.pdf?sfvrsn=970ca82e_3

Xue J, Chen J, Chen C, Hu R, Zhu T (2020) The hidden pandemic of family violence during COVID-19: Unsupervised learning of tweets. J. Med Internet Res 22:e24361. https://doi.org/10.2196/24361

Ziyadat A (2021) Domestic violence: Murders rise in Jordan during Corona. The New Press, Arab, https://www.alaraby.co.uk/society/

Zoud B (2021). How did Corona raise the rate of Child marriage in Jordan? Ammannet. https://tinyurl.com/yz5549an . [Accessed 12 March 2022]

Download references

Acknowledgements

We would like to express our gratitude to Editage ( www.editage.com ) for their help with English language editing, and proofreading.

Author information

Authors and affiliations.

Department of Sociology, School of Arts, The University of Jordan, Amman, Hashemite Kingdom of Jordan

Maissa N. Alrawashdeh

Department of Sociology, The University of Jordan, Amman, Hashemite Kingdom of Jordan

Rula Odeh Alsawalqa

Al- Balqa’ Applied University, Amman, Jordan

Rami Aljbour

Department of Social Work, School of Arts, The University of Jordan, Amman, Hashemite Kingdom of Jordan

Ann Alnajdawi

School of Arts, The University of Jordan, Amman, Hashemite Kingdom of Jordan

Fawzi Khalid AlTwahya

You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, MA, ROA; Methodology, MA, ROA, AA, RA; Resources, MA, ROA, AA, RA, FA; Writing—original draft preparation, MA, RA, AA; Writing—review and editing, All authors; Project administration MA, ROA. Correspondence to ROA. All authors have read and agreed to the submitted version of the manuscript.

Corresponding author

Correspondence to Rula Odeh Alsawalqa .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

Ethical approval was not required as the study did not involve human participants.

Informed consent

Additional information.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Alrawashdeh, M.N., Alsawalqa, R.O., Aljbour, R. et al. Domestic violence against women during the COVID19 pandemic in Jordan: a systematic review. Humanit Soc Sci Commun 11 , 598 (2024). https://doi.org/10.1057/s41599-024-03117-y

Download citation

Received : 12 June 2023

Accepted : 24 April 2024

Published : 09 May 2024

DOI : https://doi.org/10.1057/s41599-024-03117-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

Explore articles by subject
Guide to authors
Editorial policies

CASE REPORT article

Toxic epidermal necrolysis caused by viral hepatitis a: a case report and literature review provisionally accepted.

1 Department of Gastroenterology and Hepatology, Zhenjiang Third People's Hospital, China
2 Jiangsu University, China

The final, formatted version of the article will be published soon.

Toxic epidermal necrolysis (TEN) is a rare but serious immune-mediated life-threatening skin and mucous membrane reaction that is mainly caused by drugs, infections, vaccines, and malignant tumors. A 74-year-old woman presented with a moderate fever of unknown cause, which was relieved after two days, but with weakness and decreased appetite. Red maculopapules appeared successively on the neck, trunk, and limbs, expanding gradually, forming herpes and fusion, containing a yellow turbidous liquid and rupturing to reveal a bright red erosive surface spreading around the eyes and mouth. The affected body surface area was >90%. The severity of illness score for toxic epidermal necrolysis was 2 points. The drug eruption area and severity index score was 77. She was diagnosed with TEN caused by hepatitis A virus and treated with 160 mg/day methylprednisolone, 300 mg/day cyclosporine, and 20 g/day gammaglobulin. Her skin showed improvements after 3 days of treatment and returned to nearly normal after 1 month, and liver function was completely normal after 2 months.

Keywords: toxic epidermal necrolysis, Hepatitis A virus, treatments, Skin, immune-mediated

Received: 03 Mar 2024; Accepted: 13 May 2024.

Copyright: © 2024 Ye, Zhang and Tan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Dr. Youwe Tan, Jiangsu University, Zhenjiang, China

People also looked at

IMAGES

39 Best Literature Review Examples (Guide & Samples)
39 Best Literature Review Examples (Guide & Samples)
50 Smart Literature Review Templates (APA) ᐅ TemplateLab
Literature Review: Short Writing Guidelines & 4 Examples
39 Best Literature Review Examples (Guide & Samples)
50 Smart Literature Review Templates (APA) ᐅ TemplateLab

VIDEO

3_session2 Importance of literature review, types of literature review, Reference management tool
How to write literature review #literaturereview #review #research #researcheverything #researchtips
Literature Review, Systematic Literature Review, Meta
How to write a Literature review
Exploring Forms of Literature: Easy Explanation, Types, and Examples
Funny Glitch with OpenAI's new Sora Video Generator#sora #openai #ai

COMMENTS

What is a Literature Review? How to Write It (with Examples)
A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship ...
Literature Review: Examples of Published Literature Reviews
Literature Review (Historiographic Essay): Making sense of what has been written on your topic. To find examples of published literature reviews in your field or niche, try searching ProQuest Dissertations and Theses by keyword, advisor, or subject.
Writing a Literature Review
A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays).
Literature Review Guide: Examples of Literature Reviews
Sample Literature Reviews as part of a articles or Theses Building Customer Loyalty: A Customer Experience Based Approach in a Tourism Context Detailed one for Masters see chapters two and three Sample Literature Review on Critical Thinking (Gwendolyn Reece, American University Library)
Writing a literature review
Writing a literature review requires a range of skills to gather, sort, evaluate and summarise peer-reviewed published data into a relevant and informative unbiased narrative. Digital access to research papers, academic texts, review articles, reference databases and public data sets are all sources of information that are available to enrich ...
PDF How to Write a Literature Review
• To critically analyze a segment of a published body of knowledge through summary, classification, and comparison of prior research studies, reviews of literature, and ... (sample, site, problem, purpose, question) • sub-questions ... literature review and a larger area of study such as a discipline, a scientific endeavor, or a ...
How to write a superb literature review
The best proposals are timely and clearly explain why readers should pay attention to the proposed topic. It is not enough for a review to be a summary of the latest growth in the literature: the ...
How To Write A Literature Review (+ Free Template)
As mentioned above, writing your literature review is a process, which I'll break down into three steps: Finding the most suitable literature. Understanding, distilling and organising the literature. Planning and writing up your literature review chapter. Importantly, you must complete steps one and two before you start writing up your chapter.
Literature Review Example (PDF + Template)
We start off by discussing the five core sections of a literature review chapter by unpacking our free literature review template. This includes: The literature review opening/ introduction section. The theoretical framework (or foundation of theory) The empirical research. The research gap.
Writing a literature review
Writing a literature review requires a range of skills to gather, sort, evaluate and summarise peer-reviewed published data into a relevant and informative unbiased narrative. Digital access to research papers, academic texts, review articles, reference databases and public data sets are all sources of information that are available to enrich ...
Ten Simple Rules for Writing a Literature Review
Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications .For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively .Given such mountains of papers, scientists cannot be expected to examine in detail every ...
How To Write A Literature Review
1. Outline and identify the purpose of a literature review. As a first step on how to write a literature review, you must know what the research question or topic is and what shape you want your literature review to take. Ensure you understand the research topic inside out, or else seek clarifications.
Writing a literature review
How to write a literature review in 6 steps. How do you write a good literature review? This step-by-step guide on how to write an excellent literature review covers all aspects of planning and writing literature reviews for academic papers and theses.
PDF How to Write a Literature Review
A literature review is a review or discussion of the current published material available on a particular topic. It attempts to synthesizeand evaluatethe material and information according to the research question(s), thesis, and central theme(s). In other words, instead of supporting an argument, or simply making a list of summarized research ...
15 Literature Review Examples (2024)
15 Literature Review Examples. By Chris Drew (PhD) / December 6, 2023. Literature reviews are a necessary step in a research process and often required when writing your research proposal. They involve gathering, analyzing, and evaluating existing knowledge about a topic in order to find gaps in the literature where future studies will be needed.
PDF Writing an Effective Literature Review
examples along the way. In the end, I hope you will have a simple, practical strategy ... A literature review is a survey of published work relevant to a particular issue, field of research, topic or theory. ... literature review in academia, at this point it might be useful to state what a literature
Comprehensive Literature Review: A Guide
Literature Reviews that are organized methodologically consist of paragraphs/sections that are based on the methods used in the literature found.This approach is most appropriate when you are using new methods on a research question that has already been explored.Since literature review structures are not mutually exclusive, you can organize the use of these methods in chronological order.
Literature Review: What is a Literature Review?
A literature review is an in-depth critical analysis of published scholarly research related to a specific topic.Published scholarly research (the "literature") may include journal articles, books, book chapters, dissertations and thesis, or conference proceedings. A solid lit review must:
A practical guide to data analysis in general literature reviews
This article is a practical guide to conducting data analysis in general literature reviews. The general literature review is a synthesis and analysis of published research on a relevant clinical issue, and is a common format for academic theses at the bachelor's and master's levels in nursing, physiotherapy, occupational therapy, public health and other related fields.
Examples of a published literature review
Literature reviews are often published as scholarly articles, books, and reports. Here is an example of a recent literature review published as a scholarly journal article: Ledesma, M. C., & Calderón, D. (2015). Critical race theory in education: A review of past literature and a look to the future. Qualitative Inquiry, 21 (3), 206-222.
Literature review as a research methodology: An ...
As mentioned previously, there are a number of existing guidelines for literature reviews. Depending on the methodology needed to achieve the purpose of the review, all types can be helpful and appropriate to reach a specific goal (for examples, please see Table 1).These approaches can be qualitative, quantitative, or have a mixed design depending on the phase of the review.
Literature Review
Types of Literature Review are as follows: Narrative literature review: This type of review involves a comprehensive summary and critical analysis of the available literature on a particular topic or research question. It is often used as an introductory section of a research paper. Systematic literature review: This is a rigorous and ...
Reviewing the literature
Implementing evidence into practice requires nurses to identify, critically appraise and synthesise research. This may require a comprehensive literature review: this article aims to outline the approaches and stages required and provides a working example of a published review. Literature reviews aim to answer focused questions to: inform professionals and patients of the best available ...
Structuring a literature review
Structuring a literature review. In general, literature reviews are structured in a similar way to a standard essay, with an introduction, a body and a conclusion. These are key structural elements. Additionally, a stand-alone extended literature review has an abstract. Throughout, headings and subheadings are used to divide up the literature ...
Data visualisation in scoping reviews and evidence maps on health
Scoping reviews and evidence maps are forms of evidence synthesis that aim to map the available literature on a topic and are well-suited to visual presentation of results. A range of data visualisation methods and interactive data visualisation tools exist that may make scoping reviews more useful to knowledge users. The aim of this study was to explore the use of data visualisation in a ...
The effectiveness of virtual reality training on knowledge, skills and
Virtual reality (VR) training can enhance health professionals' learning. However, there are ambiguous findings on the effectiveness of VR as an educational tool in mental health. We therefore reviewed the existing literature on the effectiveness of VR training on health professionals' knowledge, skills, and attitudes in assessing and treating patients with mental health disorders.
Hospital performance evaluation indicators: a scoping review
Hospitals are the biggest consumers of health system budgets and hence measuring hospital performance by quantitative or qualitative accessible and reliable indicators is crucial. This review aimed to categorize and present a set of indicators for evaluating overall hospital performance. We conducted a literature search across three databases, i.e., PubMed, Scopus, and Web of Science, using ...
The learning curve in endoscopic transsphenoidal skull-base surgery: a
The current literature is constrained by the small sample sizes of studies and their observational nature. This systematic review aims to evaluate the literature and identify strengths and weaknesses related to the assessment of EETA-LC. ... and articles published in English. The systematic review identified 34 studies encompassing 5,648 ...
Domestic violence against women during the COVID19 pandemic in ...
The review yielded eight articles published between April 2020 and November 2022 in the final sample, all of which met the inclusion criteria. ... 19 pandemic through a systematic literature ...
Frontiers
Toxic epidermal necrolysis (TEN) is a rare but serious immune-mediated life-threatening skin and mucous membrane reaction that is mainly caused by drugs, infections, vaccines, and malignant tumors. A 74-year-old woman presented with a moderate fever of unknown cause, which was relieved after two days, but with weakness and decreased appetite. Red maculopapules appeared successively on the neck ...

What is a Literature Review? How to Write It (with Examples)

Table of Contents

What is a literature review?

What is the purpose of literature review?

Literature review example

Literature Review on Climate Change Impacts on Biodiversity:

a. Habitat Loss and Species Extinction:

b. Range Shifts and Phenological Changes:

c. Ocean Acidification and Coral Reefs:

d. Adaptive Strategies and Conservation Efforts:

How to write a good literature review

Conducting a literature review

Choose a Topic and Define the Research Question:

Decide on the Scope of Your Review:

Select Databases for Searches:

Conduct Searches and Keep Track:

Review the Literature:

Organize and Write Your Literature Review:

Frequently asked questions

Related Reads:

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Writing a Literature Review

Welcome to the Purdue OWL

Where, when, and why would I write a lit review?

What are the parts of a lit review?

How should I organize my lit review?

What are some strategies or tips I can use while writing my lit review?

Literature Review Guide: Examples of Literature Reviews

Sample Literature Reviews as part of a articles or Theses

Links to sample Literature Reviews from other libraries

Standalone Literature Reviews

Irish Theses

Literature Review Example/Sample

FAQ: Literature Review Example

Can I replicate this literature review for my dissertation?

Where can I find more examples of literature reviews?

How do I get the literature review template?

Is the template really free?

Psst… there’s more!

You Might Also Like:

Submit a Comment Cancel reply

Literature review

How to write a literature review in 6 steps

How to write a systematic literature review [9 steps]

What is a literature review? [with examples]

15 Literature Review Examples

Literature Review Examples

1. Narrative Review Examples

2. Systematic Review Examples

3. Meta-analysis

Other Types of Reviews

How to Write a Narrative Review (Tips for Undergrad Students)

Leave a Comment Cancel Reply

Literature Review: What is a Literature Review?

Ask a Librarian

Not Just for Graduate Students!

Why Conduct a Literature Review?

Questions to Consider

How to write a literature review

Literature Review – Types Writing Guide and Examples

Literature Review

Types of Literature Review

Parts of Literature Review

Introduction

Literature Search

Literature Analysis

Critical Evaluation

How to write Literature Review

Examples of Literature Review

Applications of Literature Review

Role of Literature Review in Research

Purpose of Literature Review

Characteristics of Literature Review

Advantages of Literature Review

Limitations of Literature Review

About the author

Muhammad Hassan

You may also like

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...