critical review of the literature

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

A faster, more affordable way to improve your paper

Scribbr’s new AI Proofreader checks your document and corrects spelling, grammar, and punctuation mistakes with near-human accuracy and the efficiency of AI!

Proofread my paper

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved November 28, 2023, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

Learn how to write a review of literature

What is a review of literature.

The format of a review of literature may vary from discipline to discipline and from assignment to assignment.

A review may be a self-contained unit — an end in itself — or a preface to and rationale for engaging in primary research. A review is a required part of grant and research proposals and often a chapter in theses and dissertations.

Generally, the purpose of a review is to analyze critically a segment of a published body of knowledge through summary, classification, and comparison of prior research studies, reviews of literature, and theoretical articles.

Writing the introduction

In the introduction, you should:

Define or identify the general topic, issue, or area of concern, thus providing an appropriate context for reviewing the literature.

Point out overall trends in what has been published about the topic; or conflicts in theory, methodology, evidence, and conclusions; or gaps in research and scholarship; or a single problem or new perspective of immediate interest.

Establish the writer’s reason (point of view) for reviewing the literature; explain the criteria to be used in analyzing and comparing literature and the organization of the review (sequence); and, when necessary, state why certain literature is or is not included (scope).

Writing the body

In the body, you should:

Group research studies and other types of literature (reviews, theoretical articles, case studies, etc.) according to common denominators such as qualitative versus quantitative approaches, conclusions of authors, specific purpose or objective, chronology, etc.

Summarize individual studies or articles with as much or as little detail as each merits according to its comparative importance in the literature, remembering that space (length) denotes significance.

Provide the reader with strong “umbrella” sentences at beginnings of paragraphs, “signposts” throughout, and brief “so what” summary sentences at intermediate points in the review to aid in understanding comparisons and analyses.

Writing the conclusion

In the conclusion, you should:

Summarize major contributions of significant studies and articles to the body of knowledge under review, maintaining the focus established in the introduction.

Evaluate the current “state of the art” for the body of knowledge reviewed, pointing out major methodological flaws or gaps in research, inconsistencies in theory and findings, and areas or issues pertinent to future study.

Conclude by providing some insight into the relationship between the central topic of the literature review and a larger area of study such as a discipline, a scientific endeavor, or a profession.

For further information see our handouts on Writing a Critical Review of a Nonfiction Book or Article or Reading a Book to Review It .

To learn more about literature reviews, take a look at our workshop on Writing Literature Reviews of Published Research.

Sample Literature Reviews

An important strategy for learning how to compose literature reviews in your field or within a specific genre is to locate and analyze representative examples. The following collection of annotated sample literature reviews written and co-written by colleagues associated with UW-Madison showcases how these reviews can do different kind of work for different purposes. Use these successful examples as a starting point for understanding how other writers have approached the challenging and important task of situating their idea in the context of established research.

• Sample 1 (PDF) A brief literature review within a political scientists’  National Science Foundation Graduate Research Fellowship  grant
• Sample 2 (PDF) A several-page literature review at the beginning of a published, academic article about philosophy
• Sample 3 (PDF) A brief literature review at the beginning of a published, academic article about photochemistry

Academic and Professional Writing

This is an accordion element with a series of buttons that open and close related content panels.

Analysis Papers

Reading Poetry

A Short Guide to Close Reading for Literary Analysis

Using Literary Quotations

Play Reviews

Writing a Rhetorical Précis to Analyze Nonfiction Texts

Incorporating Interview Data

Grant Proposals

Planning and Writing a Grant Proposal: The Basics

Additional Resources for Grants and Proposal Writing

Job Materials and Application Essays

Writing Personal Statements for Ph.D. Programs

• Before you begin: useful tips for writing your essay
• Guided brainstorming exercises
• Get more help with your essay
• Frequently Asked Questions

Resume Writing Tips

CV Writing Tips

Cover Letters

Business Letters

Proposals and Dissertations

Resources for Proposal Writers

Resources for Dissertators

Research Papers

Planning and Writing Research Papers

Quoting and Paraphrasing

Writing Annotated Bibliographies

Creating Poster Presentations

Writing an Abstract for Your Research Paper

Thank-You Notes

Advice for Students Writing Thank-You Notes to Donors

Reading for a Review

Critical Reviews

Writing a Review of Literature

Scientific Reports

Scientific Report Format

Sample Lab Assignment

Writing for the Web

Writing an Effective Blog Post

Writing for Social Media: A Guide for Academics

Library Guides

Literature reviews: criticality.

• Criticality

Express Critical Analysis

The literature review of a dissertation should include critical analysis. You cannot simply juxtapose the literature you find: you have to  evaluate and draw conclusions from it.  

Paragraph level  

Try expressing your voice in each paragraph of your literature review. Write strong paragraphs. In strong paragraphs your voice can be heard in the topic sentence, development (where you analyse and compare/contrast the sources, sometimes as individual pieces, sometimes in a synthesis) and, even more easily, in the concluding sentence, where you present the "therefore" of the paragraph. 

How to express criticality at the paragraph level:  

Identify the significance of the sources, and why the points they are making are relevant  

Make connections between the sources 

Compare and contrast sources, literatures  

Accept/adopt points made by the sources, with reasons  

Reject the points made by the sources, with reasons (e.g., limitations in the methodology; out of date; limited scope; geographical delimitation) 

Indicate the position you are taking in your own work on the theories and concepts presented by the sources 

Show how limitations in the existing literature create a research gap for you 

Organise the materials, synthesising them in an original way, that sheds new light on the topic.  

To find out more about paragraph writing, check out the Assignment Writing Guides.

Literature review level 

Try to take ownership of the literature review. Remember the purposes of the review (providing background on the subject you are researching and identifying a gap in the existing literature on this subject). Thus, throughout the review:   

Identify the key themes relevant to your subject matter  

Identify the most logical and effective order for your themes 

Relate the sources back to the dissertation's research question 

Shed new light on the topic 

Draw conclusions on the existing literature  

Identify gaps in the literature  

Your literature review should present an argument (which you can recap in the concluding paragraph of the literature review). For instance, 

"The literature says/illustrates/reveals that... there are debates in the literature as of... it can be understood from the literature that... however, there are gaps in the literature... the literature does not specifically address (specific sector/location/population)... there is a lack of independent/recent studies on...  therefore in order to answer the research question(s) (you can repeat the question) this dissertation uses method xyz, as illustrated in the next section (if applicable)". 

Manchester University’s  academic phrase bank  is a great resource for learning new words and phrases. 

Extra Resources

For extra help with all aspects of study skills including how to undertake literature reviews, appointments are available with learning advisors on Engage. 

Appointments are also available with an Academic Engagement Librarian to discuss any issues you might be having with research.

• << Previous: Synthesis
• Last Updated: Nov 18, 2023 10:56 PM
• URL: https://libguides.westminster.ac.uk/literature-reviews

CONNECT WITH US

• Locations and Hours
• UCLA Library
• Research Guides
• Biomedical Library Guides

Systematic Reviews

• Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

• Planning Your Systematic Review
• Database Searching
• Creating the Search
• Search Filters & Hedges
• Grey Literature
• Managing & Appraising Results
• Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Home
• Next: Planning Your Systematic Review >>
• Last Updated: Nov 15, 2023 3:04 PM
• URL: https://guides.library.ucla.edu/systematicreviews

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Oct 31, 2023 3:00 PM
• URL: https://guides.library.yale.edu/YSNDoctoral

Which review is that? A guide to review types.

• Which review is that?
• Review Comparison Chart
• Decision Tool
• Critical Review
• Integrative Review
• Narrative Review
• State of the Art Review
• Narrative Summary
• Systematic Review
• Meta-analysis
• Comparative Effectiveness Review
• Diagnostic Systematic Review
• Network Meta-analysis
• Prognostic Review
• Psychometric Review
• Review of Economic Evaluations
• Systematic Review of Epidemiology Studies
• Umbrella Review
• Review of Reviews
• Rapid Review
• Rapid Evidence Assessment
• Rapid Realist Review
• Qualitative Evidence Synthesis
• Qualitative Interpretive Meta-synthesis
• Qualitative Meta-synthesis
• Qualitative Research Synthesis
• Framework Synthesis - Best-fit Framework Synthesis
• Meta-aggregation
• Meta-ethnography
• Meta-interpretation
• Meta-narrative Review
• Meta-summary
• Thematic Synthesis
• Mixed Methods Synthesis
• Narrative Synthesis
• Bayesian Meta-analysis
• EPPI-Centre Review
• Critical Interpretive Synthesis
• Realist Synthesis - Realist Review
• Scoping Review
• Mapping Review
• Systematised Review
• Concept Synthesis
• Expert Opinion - Policy Review
• Technology Assessment Review
• Methodological Review
• Systematic Search and Review

"A critical review aims to demonstrate that the writer has extensively researched the literature and critically evaluated its quality. It goes beyond mere description of identified articles and includes a degree of analysis and conceptual innovation" and "an effective critical review presents, analyses and synthesizes material from diverse sources". "There is no formal requirement to present methods of the search, synthesis and analysis explicitly" (Grant & Booth 2009).  

Cooper, Harris M & Cooper, Harris M. Synthesizing research (2017). Research synthesis and meta-analysis : a step-by-step approach (Fifth edition). SAGE Publications, Los Angeles (Available soon) Kirkevold, M. (1997). Integrative nursing research—an important strategy to further the development of nursing science and nursing practice.  Journal of advanced nursing ,  25 (5), 977-984. Full Text Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.  Full text

Younas, A., & Maddigan, J. (2019). Proposing a policy framework for nursing education for fostering compassion in nursing students: A critical review.  Journal of advanced nursing ,  75 (8), 1621–1636. Full Text Rew, L., Young, C. C., Monge, M., & Bogucka, R. (2021). Review: Puberty blockers for transgender and gender diverse youth-a critical review of the literature.  Child and adolescent mental health ,  26 (1), 3–14. Full Text

References Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health information & libraries journal , 26 (2), 91-108. Full Text

• << Previous: Traditional review family
• Next: Integrative Review >>
• Last Updated: Nov 29, 2023 1:36 PM
• URL: https://unimelb.libguides.com/whichreview

IOE Writing Centre

• Writing a Critical Review

Writing a Critique

A critique (or critical review) is not to be mistaken for a literature review. A 'critical review', or 'critique', is a complete type of text (or genre), discussing one particular article or book in detail.  In some instances, you may be asked to write a critique of two or three articles (e.g. a comparative critical review). In contrast, a 'literature review', which also needs to be 'critical', is a part of a larger type of text, such as a chapter of your dissertation.

Most importantly: Read your article / book as many times as possible, as this will make the critical review much easier.

1. Read and take notes 2. Organising your writing 3. Summary 4. Evaluation 5. Linguistic features of a critical review 6. Summary language 7. Evaluation language 8. Conclusion language 9. Example extracts from a critical review 10. Further resources

Read and Take Notes

To improve your reading confidence and efficiency, visit our pages on reading.

Further reading: Read Confidently

After you are familiar with the text, make notes on some of the following questions. Choose the questions which seem suitable:

• What kind of article is it (for example does it present data or does it present purely theoretical arguments)?
• What is the main area under discussion?
• What are the main findings?
• What are the stated limitations?
• Where does the author's data and evidence come from? Are they appropriate / sufficient?
• What are the main issues raised by the author?
• What questions are raised?
• How well are these questions addressed?
• What are the major points/interpretations made by the author in terms of the issues raised?
• Is the text balanced? Is it fair / biased?
• Does the author contradict herself?
• How does all this relate to other literature on this topic?
• How does all this relate to your own experience, ideas and views?
• What else has this author written? Do these build / complement this text?
• (Optional) Has anyone else reviewed this article? What did they say? Do I agree with them?

^ Back to top

Organising your writing

You first need to summarise the text that you have read. One reason to summarise the text is that the reader may not have read the text. In your summary, you will

• focus on points within the article that you think are interesting
• summarise the author(s) main ideas or argument
• explain how these ideas / argument have been constructed. (For example, is the author basing her arguments on data that they have collected? Are the main ideas / argument purely theoretical?)

In your summary you might answer the following questions:     Why is this topic important?     Where can this text be located? For example, does it address policy studies?     What other prominent authors also write about this?

Evaluation is the most important part in a critical review.

Use the literature to support your views. You may also use your knowledge of conducting research, and your own experience. Evaluation can be explicit or implicit.

Explicit evaluation

Explicit evaluation involves stating directly (explicitly) how you intend to evaluate the text. e.g. "I will review this article by focusing on the following questions. First, I will examine the extent to which the authors contribute to current thought on Second Language Acquisition (SLA) pedagogy. After that, I will analyse whether the authors' propositions are feasible within overseas SLA classrooms."

Implicit evaluation

Implicit evaluation is less direct. The following section on Linguistic Features of Writing a Critical Review contains language that evaluates the text. A difficult part of evaluation of a published text (and a professional author) is how to do this as a student. There is nothing wrong with making your position as a student explicit and incorporating it into your evaluation. Examples of how you might do this can be found in the section on Linguistic Features of Writing a Critical Review. You need to remember to locate and analyse the author's argument when you are writing your critical review. For example, you need to locate the authors' view of classroom pedagogy as presented in the book / article and not present a critique of views of classroom pedagogy in general.

Linguistic features of a critical review

The following examples come from published critical reviews. Some of them have been adapted for student use.

Summary language

•     This article / book is divided into two / three parts. First...
•     While the title might suggest...
•     The tone appears to be...
•     Title is the first / second volume in the series Title, edited by...The books / articles in this series address...
•     The second / third claim is based on...
•     The author challenges the notion that...
•     The author tries to find a more middle ground / make more modest claims...
•     The article / book begins with a short historical overview of...
•     Numerous authors have recently suggested that...(see Author, Year; Author, Year). Author would also be once such author. With his / her argument that...
•     To refer to title as a...is not to say that it is...
•     This book / article is aimed at... This intended readership...
•     The author's book / article examines the...To do this, the author first...
•     The author develops / suggests a theoretical / pedagogical model to…
•     This book / article positions itself firmly within the field of...
•     The author in a series of subtle arguments, indicates that he / she...
•     The argument is therefore...
•     The author asks "..."
•     With a purely critical / postmodern take on...
•     Topic, as the author points out, can be viewed as...
•     In this recent contribution to the field of...this British author...
•     As a leading author in the field of...
•     This book / article nicely contributes to the field of...and complements other work by this author...
•     The second / third part of...provides / questions / asks the reader...
•     Title is intended to encourage students / researchers to...
•     The approach taken by the author provides the opportunity to examine...in a qualitative / quantitative research framework that nicely complements...
•     The author notes / claims that state support / a focus on pedagogy / the adoption of...remains vital if...
•     According to Author (Year) teaching towards examinations is not as effective as it is in other areas of the curriculum. This is because, as Author (Year) claims that examinations have undue status within the curriculum.
•     According to Author (Year)…is not as effective in some areas of the curriculum / syllabus as others. Therefore the author believes that this is a reason for some school's…

Evaluation language

•     This argument is not entirely convincing, as...furthermore it commodifies / rationalises the...
•     Over the last five / ten years the view of...has increasingly been viewed as 'complicated' (see Author, Year; Author, Year).
•     However, through trying to integrate...with...the author...
•     There are difficulties with such a position.
•     Inevitably, several crucial questions are left unanswered / glossed over by this insightful / timely / interesting / stimulating book / article. Why should...
•     It might have been more relevant for the author to have written this book / article as...
•     This article / book is not without disappointment from those who would view...as...
•     This chosen framework enlightens / clouds...
•     This analysis intends to be...but falls a little short as...
•     The authors rightly conclude that if...
•     A detailed, well-written and rigorous account of...
•     As a Korean student I feel that this article / book very clearly illustrates...
•     The beginning of...provides an informative overview into...
•     The tables / figures do little to help / greatly help the reader...
•     The reaction by scholars who take a...approach might not be so favourable (e.g. Author, Year).
•     This explanation has a few weaknesses that other researchers have pointed out (see Author, Year; Author, Year). The first is...
•     On the other hand, the author wisely suggests / proposes that...By combining these two dimensions...
•     The author's brief introduction to...may leave the intended reader confused as it fails to properly...
•     Despite my inability to...I was greatly interested in...
•     Even where this reader / I disagree(s), the author's effort to...
•     The author thus combines...with...to argue...which seems quite improbable for a number of reasons. First...
•     Perhaps this aversion to...would explain the author's reluctance to...
•     As a second language student from ...I find it slightly ironic that such an anglo-centric view is...
•     The reader is rewarded with...
•     Less convincing is the broad-sweeping generalisation that...
•     There is no denying the author's subject knowledge nor his / her...
•     The author's prose is dense and littered with unnecessary jargon...
•     The author's critique of...might seem harsh but is well supported within the literature (see Author, Year; Author, Year; Author, Year). Aligning herself with the author, Author (Year) states that...
•     As it stands, the central focus of Title is well / poorly supported by its empirical findings...
•     Given the hesitation to generalise to...the limitation of...does not seem problematic...
•     For instance, the term...is never properly defined and the reader left to guess as to whether...
•     Furthermore, to label...as...inadvertently misguides...
•     In addition, this research proves to be timely / especially significant to... as recent government policy / proposals has / have been enacted to...
•     On this well researched / documented basis the author emphasises / proposes that...
•     Nonetheless, other research / scholarship / data tend to counter / contradict this possible trend / assumption...(see Author, Year; Author, Year).
•     Without entering into detail of the..., it should be stated that Title should be read by...others will see little value in...
•     As experimental conditions were not used in the study the word 'significant' misleads the reader.
•     The article / book becomes repetitious in its assertion that...
•     The thread of the author's argument becomes lost in an overuse of empirical data...
•     Almost every argument presented in the final section is largely derivative, providing little to say about...
•     She / he does not seem to take into consideration; however, that there are fundamental differences in the conditions of…
•     As Author (Year) points out, however, it seems to be necessary to look at…
•     This suggest that having low…does not necessarily indicate that…is ineffective.
•     Therefore, the suggestion made by Author (Year)…is difficult to support.
•     When considering all the data presented…it is not clear that the low scores of some students, indeed, reflects…

Conclusion language

•     Overall this article / book is an analytical look at...which within the field of...is often overlooked.
•     Despite its problems, Title offers valuable theoretical insights / interesting examples / a contribution to pedagogy and a starting point for students / researchers of...with an interest in...
•     This detailed and rigorously argued...
•     This first / second volume / book / article by...with an interest in...is highly informative...

Example extracts from a critical review

Writing critically.

If you have been told your writing is not critical enough, it probably means that your writing treats the knowledge claims as if they are true, well supported, and applicable in the context you are writing about. This may not always be the case.

In these two examples, the extracts refer to the same section of text. In each example, the section that refers to a source has been highlighted in bold. The note below the example then explains how the writer has used the source material.    

There is a strong positive effect on students, both educationally and emotionally, when the instructors try to learn to say students' names without making pronunciation errors (Kiang, 2004).

Use of source material in example a: 

This is a simple paraphrase with no critical comment. It looks like the writer agrees with Kiang. (This is not a good example for critical writing, as the writer has not made any critical comment).        

Kiang (2004) gives various examples to support his claim that "the positive emotional and educational impact on students is clear" (p.210) when instructors try to pronounce students' names in the correct way. He quotes one student, Nguyet, as saying that he "felt surprised and happy" (p.211) when the tutor said his name clearly . The emotional effect claimed by Kiang is illustrated in quotes such as these, although the educational impact is supported more indirectly through the chapter. Overall, he provides more examples of students being negatively affected by incorrect pronunciation, and it is difficult to find examples within the text of a positive educational impact as such.

Use of source material in example b: 

The writer describes Kiang's (2004) claim and the examples which he uses to try to support it. The writer then comments that the examples do not seem balanced and may not be enough to support the claims fully. This is a better example of writing which expresses criticality.

^Back to top

Further resources

You may also be interested in our page on criticality, which covers criticality in general, and includes more critical reading questions.

Further reading: Read and Write Critically

We recommend that you do not search for other university guidelines on critical reviews. This is because the expectations may be different at other institutions. Ask your tutor for more guidance or examples if you have further questions.

IOE Writing Centre Online

Self-access resources from the Academic Writing Centre at the UCL Institute of Education.

Anonymous Suggestions Box

Information for Staff

Academic Writing Centre

Academic Writing Centre, UCL Institute of Education [email protected] Twitter:   @AWC_IOE Skype:   awc.ioe

• Schools & departments

Literature review

A general guide on how to conduct and write a literature review.

Please check course or programme information and materials provided by teaching staff , including your project supervisor, for subject-specific guidance.

What is a literature review?

A literature review is a piece of academic writing demonstrating knowledge and understanding of the academic literature on a specific topic placed in context.  A literature review also includes a critical evaluation of the material; this is why it is called a literature review rather than a literature report. It is a process of reviewing the literature, as well as a form of writing.

To illustrate the difference between reporting and reviewing, think about television or film review articles.  These articles include content such as a brief synopsis or the key points of the film or programme plus the critic’s own evaluation.  Similarly the two main objectives of a literature review are firstly the content covering existing research, theories and evidence, and secondly your own critical evaluation and discussion of this content. 

Usually a literature review forms a section or part of a dissertation, research project or long essay.  However, it can also be set and assessed as a standalone piece of work.

What is the purpose of a literature review?

…your task is to build an argument, not a library. Rudestam, K.E. and Newton, R.R. (1992) Surviving your dissertation: A comprehensive guide to content and process. California: Sage, p49.

In a larger piece of written work, such as a dissertation or project, a literature review is usually one of the first tasks carried out after deciding on a topic.  Reading combined with critical analysis can help to refine a topic and frame research questions.  Conducting a literature review establishes your familiarity with and understanding of current research in a particular field before carrying out a new investigation.  After doing a literature review, you should know what research has already been done and be able to identify what is unknown within your topic.

When doing and writing a literature review, it is good practice to:

• summarise and analyse previous research and theories;
• identify areas of controversy and contested claims;
• highlight any gaps that may exist in research to date.

Conducting a literature review

Focusing on different aspects of your literature review can be useful to help plan, develop, refine and write it.  You can use and adapt the prompt questions in our worksheet below at different points in the process of researching and writing your review.  These are suggestions to get you thinking and writing.

Developing and refining your literature review (pdf)

Developing and refining your literature review (Word)

Developing and refining your literature review (Word rtf)

Writing a literature review has a lot in common with other assignment tasks.  There is advice on our other pages about thinking critically, reading strategies and academic writing.  Our literature review top tips suggest some specific things you can do to help you submit a successful review.

Literature review top tips (pdf)

Literature review top tips (Word rtf)

Our reading page includes strategies and advice on using books and articles and a notes record sheet grid you can use.

Reading at university

The Academic writing page suggests ways to organise and structure information from a range of sources and how you can develop your argument as you read and write.

Academic writing

The Critical thinking page has advice on how to be a more critical researcher and a form you can use to help you think and break down the stages of developing your argument.

Critical thinking

As with other forms of academic writing, your literature review needs to demonstrate good academic practice by following the Code of Student Conduct and acknowledging the work of others through citing and referencing your sources.  

Good academic practice

As with any writing task, you will need to review, edit and rewrite sections of your literature review.  The Editing and proofreading page includes tips on how to do this and strategies for standing back and thinking about your structure and checking the flow of your argument.

Editing and proofreading

Guidance on literature searching from the University Library

The Academic Support Librarians have developed LibSmart I and II, Learn courses to help you develop and enhance your digital research skills and capabilities; from getting started with the Library to managing data for your dissertation.

Searching using the library’s DiscoverEd tool: DiscoverEd

Finding resources in your subject: Subject guides

The Academic Support Librarians also provide one-to-one appointments to help you develop your research strategies.

1 to 1 support for literature searching and systematic reviews

Advice to help you optimise use of Google Scholar, Google Books and Google for your research and study: Using Google

Managing and curating your references

A referencing management tool can help you to collect and organise and your source material to produce a bibliography or reference list. 

Referencing and reference management

Information Services provide access to Cite them right online which is a guide to the main referencing systems and tells you how to reference just about any source (EASE log-in may be required).

Cite them right

Published study guides

There are a number of scholarship skills books and guides available which can help with writing a literature review.  Our Resource List of study skills guides includes sections on Referencing, Dissertation and project writing and Literature reviews.

Study skills guides

• UWF Libraries

Literature Review: Conducting & Writing

• Sample Literature Reviews
• Steps for Conducting a Lit Review
• Finding "The Literature"
• Organizing/Writing
• Chicago: Notes Bibliography

Sample Lit Reviews from Communication Arts

Have an exemplary literature review.

• Literature Review Sample 1
• Literature Review Sample 2
• Literature Review Sample 3

Have you written a stellar literature review you care to share for teaching purposes?

Are you an instructor who has received an exemplary literature review and have permission from the student to post?

Please contact Britt McGowan at [email protected] for inclusion in this guide. All disciplines welcome and encouraged.

• << Previous: MLA Style
• Next: Get Help! >>
• Last Updated: Aug 24, 2023 9:59 AM
• URL: https://libguides.uwf.edu/litreview

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Advanced Search
• Journal List
• Clinics (Sao Paulo)

Approaching literature review for academic purposes: The Literature Review Checklist

Debora f.b. leite.

I Departamento de Ginecologia e Obstetricia, Faculdade de Ciencias Medicas, Universidade Estadual de Campinas, Campinas, SP, BR

II Universidade Federal de Pernambuco, Pernambuco, PE, BR

III Hospital das Clinicas, Universidade Federal de Pernambuco, Pernambuco, PE, BR

Maria Auxiliadora Soares Padilha

Jose g. cecatti.

A sophisticated literature review (LR) can result in a robust dissertation/thesis by scrutinizing the main problem examined by the academic study; anticipating research hypotheses, methods and results; and maintaining the interest of the audience in how the dissertation/thesis will provide solutions for the current gaps in a particular field. Unfortunately, little guidance is available on elaborating LRs, and writing an LR chapter is not a linear process. An LR translates students’ abilities in information literacy, the language domain, and critical writing. Students in postgraduate programs should be systematically trained in these skills. Therefore, this paper discusses the purposes of LRs in dissertations and theses. Second, the paper considers five steps for developing a review: defining the main topic, searching the literature, analyzing the results, writing the review and reflecting on the writing. Ultimately, this study proposes a twelve-item LR checklist. By clearly stating the desired achievements, this checklist allows Masters and Ph.D. students to continuously assess their own progress in elaborating an LR. Institutions aiming to strengthen students’ necessary skills in critical academic writing should also use this tool.

INTRODUCTION

Writing the literature review (LR) is often viewed as a difficult task that can be a point of writer’s block and procrastination ( 1 ) in postgraduate life. Disagreements on the definitions or classifications of LRs ( 2 ) may confuse students about their purpose and scope, as well as how to perform an LR. Interestingly, at many universities, the LR is still an important element in any academic work, despite the more recent trend of producing scientific articles rather than classical theses.

The LR is not an isolated section of the thesis/dissertation or a copy of the background section of a research proposal. It identifies the state-of-the-art knowledge in a particular field, clarifies information that is already known, elucidates implications of the problem being analyzed, links theory and practice ( 3 - 5 ), highlights gaps in the current literature, and places the dissertation/thesis within the research agenda of that field. Additionally, by writing the LR, postgraduate students will comprehend the structure of the subject and elaborate on their cognitive connections ( 3 ) while analyzing and synthesizing data with increasing maturity.

At the same time, the LR transforms the student and hints at the contents of other chapters for the reader. First, the LR explains the research question; second, it supports the hypothesis, objectives, and methods of the research project; and finally, it facilitates a description of the student’s interpretation of the results and his/her conclusions. For scholars, the LR is an introductory chapter ( 6 ). If it is well written, it demonstrates the student’s understanding of and maturity in a particular topic. A sound and sophisticated LR can indicate a robust dissertation/thesis.

A consensus on the best method to elaborate a dissertation/thesis has not been achieved. The LR can be a distinct chapter or included in different sections; it can be part of the introduction chapter, part of each research topic, or part of each published paper ( 7 ). However, scholars view the LR as an integral part of the main body of an academic work because it is intrinsically connected to other sections ( Figure 1 ) and is frequently present. The structure of the LR depends on the conventions of a particular discipline, the rules of the department, and the student’s and supervisor’s areas of expertise, needs and interests.

Interestingly, many postgraduate students choose to submit their LR to peer-reviewed journals. As LRs are critical evaluations of current knowledge, they are indeed publishable material, even in the form of narrative or systematic reviews. However, systematic reviews have specific patterns 1 ( 8 ) that may not entirely fit with the questions posed in the dissertation/thesis. Additionally, the scope of a systematic review may be too narrow, and the strict criteria for study inclusion may omit important information from the dissertation/thesis. Therefore, this essay discusses the definition of an LR is and methods to develop an LR in the context of an academic dissertation/thesis. Finally, we suggest a checklist to evaluate an LR.

WHAT IS A LITERATURE REVIEW IN A THESIS?

Conducting research and writing a dissertation/thesis translates rational thinking and enthusiasm ( 9 ). While a strong body of literature that instructs students on research methodology, data analysis and writing scientific papers exists, little guidance on performing LRs is available. The LR is a unique opportunity to assess and contrast various arguments and theories, not just summarize them. The research results should not be discussed within the LR, but the postgraduate student tends to write a comprehensive LR while reflecting on his or her own findings ( 10 ).

Many people believe that writing an LR is a lonely and linear process. Supervisors or the institutions assume that the Ph.D. student has mastered the relevant techniques and vocabulary associated with his/her subject and conducts a self-reflection about previously published findings. Indeed, while elaborating the LR, the student should aggregate diverse skills, which mainly rely on his/her own commitment to mastering them. Thus, less supervision should be required ( 11 ). However, the parameters described above might not currently be the case for many students ( 11 , 12 ), and the lack of formal and systematic training on writing LRs is an important concern ( 11 ).

An institutional environment devoted to active learning will provide students the opportunity to continuously reflect on LRs, which will form a dialogue between the postgraduate student and the current literature in a particular field ( 13 ). Postgraduate students will be interpreting studies by other researchers, and, according to Hart (1998) ( 3 ), the outcomes of the LR in a dissertation/thesis include the following:

• To identify what research has been performed and what topics require further investigation in a particular field of knowledge;
• To determine the context of the problem;
• To recognize the main methodologies and techniques that have been used in the past;
• To place the current research project within the historical, methodological and theoretical context of a particular field;
• To identify significant aspects of the topic;
• To elucidate the implications of the topic;
• To offer an alternative perspective;
• To discern how the studied subject is structured;
• To improve the student’s subject vocabulary in a particular field; and
• To characterize the links between theory and practice.

A sound LR translates the postgraduate student’s expertise in academic and scientific writing: it expresses his/her level of comfort with synthesizing ideas ( 11 ). The LR reveals how well the postgraduate student has proceeded in three domains: an effective literature search, the language domain, and critical writing.

Effective literature search

All students should be trained in gathering appropriate data for specific purposes, and information literacy skills are a cornerstone. These skills are defined as “an individual’s ability to know when they need information, to identify information that can help them address the issue or problem at hand, and to locate, evaluate, and use that information effectively” ( 14 ). Librarian support is of vital importance in coaching the appropriate use of Boolean logic (AND, OR, NOT) and other tools for highly efficient literature searches (e.g., quotation marks and truncation), as is the appropriate management of electronic databases.

Language domain

Academic writing must be concise and precise: unnecessary words distract the reader from the essential content ( 15 ). In this context, reading about issues distant from the research topic ( 16 ) may increase students’ general vocabulary and familiarity with grammar. Ultimately, reading diverse materials facilitates and encourages the writing process itself.

Critical writing

Critical judgment includes critical reading, thinking and writing. It supposes a student’s analytical reflection about what he/she has read. The student should delineate the basic elements of the topic, characterize the most relevant claims, identify relationships, and finally contrast those relationships ( 17 ). Each scientific document highlights the perspective of the author, and students will become more confident in judging the supporting evidence and underlying premises of a study and constructing their own counterargument as they read more articles. A paucity of integration or contradictory perspectives indicates lower levels of cognitive complexity ( 12 ).

Thus, while elaborating an LR, the postgraduate student should achieve the highest category of Bloom’s cognitive skills: evaluation ( 12 ). The writer should not only summarize data and understand each topic but also be able to make judgments based on objective criteria, compare resources and findings, identify discrepancies due to methodology, and construct his/her own argument ( 12 ). As a result, the student will be sufficiently confident to show his/her own voice .

Writing a consistent LR is an intense and complex activity that reveals the training and long-lasting academic skills of a writer. It is not a lonely or linear process. However, students are unlikely to be prepared to write an LR if they have not mastered the aforementioned domains ( 10 ). An institutional environment that supports student learning is crucial.

Different institutions employ distinct methods to promote students’ learning processes. First, many universities propose modules to develop behind the scenes activities that enhance self-reflection about general skills (e.g., the skills we have mastered and the skills we need to develop further), behaviors that should be incorporated (e.g., self-criticism about one’s own thoughts), and each student’s role in the advancement of his/her field. Lectures or workshops about LRs themselves are useful because they describe the purposes of the LR and how it fits into the whole picture of a student’s work. These activities may explain what type of discussion an LR must involve, the importance of defining the correct scope, the reasons to include a particular resource, and the main role of critical reading.

Some pedagogic services that promote a continuous improvement in study and academic skills are equally important. Examples include workshops about time management, the accomplishment of personal objectives, active learning, and foreign languages for nonnative speakers. Additionally, opportunities to converse with other students promotes an awareness of others’ experiences and difficulties. Ultimately, the supervisor’s role in providing feedback and setting deadlines is crucial in developing students’ abilities and in strengthening students’ writing quality ( 12 ).

HOW SHOULD A LITERATURE REVIEW BE DEVELOPED?

A consensus on the appropriate method for elaborating an LR is not available, but four main steps are generally accepted: defining the main topic, searching the literature, analyzing the results, and writing ( 6 ). We suggest a fifth step: reflecting on the information that has been written in previous publications ( Figure 2 ).

First step: Defining the main topic

Planning an LR is directly linked to the research main question of the thesis and occurs in parallel to students’ training in the three domains discussed above. The planning stage helps organize ideas, delimit the scope of the LR ( 11 ), and avoid the wasting of time in the process. Planning includes the following steps:

• Reflecting on the scope of the LR: postgraduate students will have assumptions about what material must be addressed and what information is not essential to an LR ( 13 , 18 ). Cooper’s Taxonomy of Literature Reviews 2 systematizes the writing process through six characteristics and nonmutually exclusive categories. The focus refers to the reviewer’s most important points of interest, while the goals concern what students want to achieve with the LR. The perspective assumes answers to the student’s own view of the LR and how he/she presents a particular issue. The coverage defines how comprehensive the student is in presenting the literature, and the organization determines the sequence of arguments. The audience is defined as the group for whom the LR is written.
• Designating sections and subsections: Headings and subheadings should be specific, explanatory and have a coherent sequence throughout the text ( 4 ). They simulate an inverted pyramid, with an increasing level of reflection and depth of argument.
• Identifying keywords: The relevant keywords for each LR section should be listed to guide the literature search. This list should mirror what Hart (1998) ( 3 ) advocates as subject vocabulary . The keywords will also be useful when the student is writing the LR since they guide the reader through the text.
• Delineating the time interval and language of documents to be retrieved in the second step. The most recently published documents should be considered, but relevant texts published before a predefined cutoff year can be included if they are classic documents in that field. Extra care should be employed when translating documents.

Second step: Searching the literature

The ability to gather adequate information from the literature must be addressed in postgraduate programs. Librarian support is important, particularly for accessing difficult texts. This step comprises the following components:

• Searching the literature itself: This process consists of defining which databases (electronic or dissertation/thesis repositories), official documents, and books will be searched and then actively conducting the search. Information literacy skills have a central role in this stage. While searching electronic databases, controlled vocabulary (e.g., Medical Subject Headings, or MeSH, for the PubMed database) or specific standardized syntax rules may need to be applied.

In addition, two other approaches are suggested. First, a review of the reference list of each document might be useful for identifying relevant publications to be included and important opinions to be assessed. This step is also relevant for referencing the original studies and leading authors in that field. Moreover, students can directly contact the experts on a particular topic to consult with them regarding their experience or use them as a source of additional unpublished documents.

Before submitting a dissertation/thesis, the electronic search strategy should be repeated. This process will ensure that the most recently published papers will be considered in the LR.

• Selecting documents for inclusion: Generally, the most recent literature will be included in the form of published peer-reviewed papers. Assess books and unpublished material, such as conference abstracts, academic texts and government reports, are also important to assess since the gray literature also offers valuable information. However, since these materials are not peer-reviewed, we recommend that they are carefully added to the LR.

This task is an important exercise in time management. First, students should read the title and abstract to understand whether that document suits their purposes, addresses the research question, and helps develop the topic of interest. Then, they should scan the full text, determine how it is structured, group it with similar documents, and verify whether other arguments might be considered ( 5 ).

Third step: Analyzing the results

Critical reading and thinking skills are important in this step. This step consists of the following components:

• Reading documents: The student may read various texts in depth according to LR sections and subsections ( defining the main topic ), which is not a passive activity ( 1 ). Some questions should be asked to practice critical analysis skills, as listed below. Is the research question evident and articulated with previous knowledge? What are the authors’ research goals and theoretical orientations, and how do they interact? Are the authors’ claims related to other scholars’ research? Do the authors consider different perspectives? Was the research project designed and conducted properly? Are the results and discussion plausible, and are they consistent with the research objectives and methodology? What are the strengths and limitations of this work? How do the authors support their findings? How does this work contribute to the current research topic? ( 1 , 19 )
• Taking notes: Students who systematically take notes on each document are more readily able to establish similarities or differences with other documents and to highlight personal observations. This approach reinforces the student’s ideas about the next step and helps develop his/her own academic voice ( 1 , 13 ). Voice recognition software ( 16 ), mind maps ( 5 ), flowcharts, tables, spreadsheets, personal comments on the referenced texts, and note-taking apps are all available tools for managing these observations, and the student him/herself should use the tool that best improves his/her learning. Additionally, when a student is considering submitting an LR to a peer-reviewed journal, notes should be taken on the activities performed in all five steps to ensure that they are able to be replicated.

Fourth step: Writing

The recognition of when a student is able and ready to write after a sufficient period of reading and thinking is likely a difficult task. Some students can produce a review in a single long work session. However, as discussed above, writing is not a linear process, and students do not need to write LRs according to a specific sequence of sections. Writing an LR is a time-consuming task, and some scholars believe that a period of at least six months is sufficient ( 6 ). An LR, and academic writing in general, expresses the writer’s proper thoughts, conclusions about others’ work ( 6 , 10 , 13 , 16 ), and decisions about methods to progress in the chosen field of knowledge. Thus, each student is expected to present a different learning and writing trajectory.

In this step, writing methods should be considered; then, editing, citing and correct referencing should complete this stage, at least temporarily. Freewriting techniques may be a good starting point for brainstorming ideas and improving the understanding of the information that has been read ( 1 ). Students should consider the following parameters when creating an agenda for writing the LR: two-hour writing blocks (at minimum), with prespecified tasks that are possible to complete in one section; short (minutes) and long breaks (days or weeks) to allow sufficient time for mental rest and reflection; and short- and long-term goals to motivate the writing itself ( 20 ). With increasing experience, this scheme can vary widely, and it is not a straightforward rule. Importantly, each discipline has a different way of writing ( 1 ), and each department has its own preferred styles for citations and references.

Fifth step: Reflecting on the writing

In this step, the postgraduate student should ask him/herself the same questions as in the analyzing the results step, which can take more time than anticipated. Ambiguities, repeated ideas, and a lack of coherence may not be noted when the student is immersed in the writing task for long periods. The whole effort will likely be a work in progress, and continuous refinements in the written material will occur once the writing process has begun.

LITERATURE REVIEW CHECKLIST

In contrast to review papers, the LR of a dissertation/thesis should not be a standalone piece or work. Instead, it should present the student as a scholar and should maintain the interest of the audience in how that dissertation/thesis will provide solutions for the current gaps in a particular field.

A checklist for evaluating an LR is convenient for students’ continuous academic development and research transparency: it clearly states the desired achievements for the LR of a dissertation/thesis. Here, we present an LR checklist developed from an LR scoring rubric ( 11 ). For a critical analysis of an LR, we maintain the five categories but offer twelve criteria that are not scaled ( Figure 3 ). The criteria all have the same importance and are not mutually exclusive.

First category: Coverage

1. justified criteria exist for the inclusion and exclusion of literature in the review.

This criterion builds on the main topic and areas covered by the LR ( 18 ). While experts may be confident in retrieving and selecting literature, postgraduate students must convince their audience about the adequacy of their search strategy and their reasons for intentionally selecting what material to cover ( 11 ). References from different fields of knowledge provide distinct perspective, but narrowing the scope of coverage may be important in areas with a large body of existing knowledge.

Second category: Synthesis

2. a critical examination of the state of the field exists.

A critical examination is an assessment of distinct aspects in the field ( 1 ) along with a constructive argument. It is not a negative critique but an expression of the student’s understanding of how other scholars have added to the topic ( 1 ), and the student should analyze and contextualize contradictory statements. A writer’s personal bias (beliefs or political involvement) have been shown to influence the structure and writing of a document; therefore, the cultural and paradigmatic background guide how the theories are revised and presented ( 13 ). However, an honest judgment is important when considering different perspectives.

3. The topic or problem is clearly placed in the context of the broader scholarly literature

The broader scholarly literature should be related to the chosen main topic for the LR ( how to develop the literature review section). The LR can cover the literature from one or more disciplines, depending on its scope, but it should always offer a new perspective. In addition, students should be careful in citing and referencing previous publications. As a rule, original studies and primary references should generally be included. Systematic and narrative reviews present summarized data, and it may be important to cite them, particularly for issues that should be understood but do not require a detailed description. Similarly, quotations highlight the exact statement from another publication. However, excessive referencing may disclose lower levels of analysis and synthesis by the student.

4. The LR is critically placed in the historical context of the field

Situating the LR in its historical context shows the level of comfort of the student in addressing a particular topic. Instead of only presenting statements and theories in a temporal approach, which occasionally follows a linear timeline, the LR should authentically characterize the student’s academic work in the state-of-art techniques in their particular field of knowledge. Thus, the LR should reinforce why the dissertation/thesis represents original work in the chosen research field.

5. Ambiguities in definitions are considered and resolved

Distinct theories on the same topic may exist in different disciplines, and one discipline may consider multiple concepts to explain one topic. These misunderstandings should be addressed and contemplated. The LR should not synthesize all theories or concepts at the same time. Although this approach might demonstrate in-depth reading on a particular topic, it can reveal a student’s inability to comprehend and synthesize his/her research problem.

6. Important variables and phenomena relevant to the topic are articulated

The LR is a unique opportunity to articulate ideas and arguments and to purpose new relationships between them ( 10 , 11 ). More importantly, a sound LR will outline to the audience how these important variables and phenomena will be addressed in the current academic work. Indeed, the LR should build a bidirectional link with the remaining sections and ground the connections between all of the sections ( Figure 1 ).

7. A synthesized new perspective on the literature has been established

The LR is a ‘creative inquiry’ ( 13 ) in which the student elaborates his/her own discourse, builds on previous knowledge in the field, and describes his/her own perspective while interpreting others’ work ( 13 , 17 ). Thus, students should articulate the current knowledge, not accept the results at face value ( 11 , 13 , 17 ), and improve their own cognitive abilities ( 12 ).

Third category: Methodology

8. the main methodologies and research techniques that have been used in the field are identified and their advantages and disadvantages are discussed.

The LR is expected to distinguish the research that has been completed from investigations that remain to be performed, address the benefits and limitations of the main methods applied to date, and consider the strategies for addressing the expected limitations described above. While placing his/her research within the methodological context of a particular topic, the LR will justify the methodology of the study and substantiate the student’s interpretations.

9. Ideas and theories in the field are related to research methodologies

The audience expects the writer to analyze and synthesize methodological approaches in the field. The findings should be explained according to the strengths and limitations of previous research methods, and students must avoid interpretations that are not supported by the analyzed literature. This criterion translates to the student’s comprehension of the applicability and types of answers provided by different research methodologies, even those using a quantitative or qualitative research approach.

Fourth category: Significance

10. the scholarly significance of the research problem is rationalized.

The LR is an introductory section of a dissertation/thesis and will present the postgraduate student as a scholar in a particular field ( 11 ). Therefore, the LR should discuss how the research problem is currently addressed in the discipline being investigated or in different disciplines, depending on the scope of the LR. The LR explains the academic paradigms in the topic of interest ( 13 ) and methods to advance the field from these starting points. However, an excess number of personal citations—whether referencing the student’s research or studies by his/her research team—may reflect a narrow literature search and a lack of comprehensive synthesis of ideas and arguments.

11. The practical significance of the research problem is rationalized

The practical significance indicates a student’s comprehensive understanding of research terminology (e.g., risk versus associated factor), methodology (e.g., efficacy versus effectiveness) and plausible interpretations in the context of the field. Notably, the academic argument about a topic may not always reflect the debate in real life terms. For example, using a quantitative approach in epidemiology, statistically significant differences between groups do not explain all of the factors involved in a particular problem ( 21 ). Therefore, excessive faith in p -values may reflect lower levels of critical evaluation of the context and implications of a research problem by the student.

Fifth category: Rhetoric

12. the lr was written with a coherent, clear structure that supported the review.

This category strictly relates to the language domain: the text should be coherent and presented in a logical sequence, regardless of which organizational ( 18 ) approach is chosen. The beginning of each section/subsection should state what themes will be addressed, paragraphs should be carefully linked to each other ( 10 ), and the first sentence of each paragraph should generally summarize the content. Additionally, the student’s statements are clear, sound, and linked to other scholars’ works, and precise and concise language that follows standardized writing conventions (e.g., in terms of active/passive voice and verb tenses) is used. Attention to grammar, such as orthography and punctuation, indicates prudence and supports a robust dissertation/thesis. Ultimately, all of these strategies provide fluency and consistency for the text.

Although the scoring rubric was initially proposed for postgraduate programs in education research, we are convinced that this checklist is a valuable tool for all academic areas. It enables the monitoring of students’ learning curves and a concentrated effort on any criteria that are not yet achieved. For institutions, the checklist is a guide to support supervisors’ feedback, improve students’ writing skills, and highlight the learning goals of each program. These criteria do not form a linear sequence, but ideally, all twelve achievements should be perceived in the LR.

CONCLUSIONS

A single correct method to classify, evaluate and guide the elaboration of an LR has not been established. In this essay, we have suggested directions for planning, structuring and critically evaluating an LR. The planning of the scope of an LR and approaches to complete it is a valuable effort, and the five steps represent a rational starting point. An institutional environment devoted to active learning will support students in continuously reflecting on LRs, which will form a dialogue between the writer and the current literature in a particular field ( 13 ).

The completion of an LR is a challenging and necessary process for understanding one’s own field of expertise. Knowledge is always transitory, but our responsibility as scholars is to provide a critical contribution to our field, allowing others to think through our work. Good researchers are grounded in sophisticated LRs, which reveal a writer’s training and long-lasting academic skills. We recommend using the LR checklist as a tool for strengthening the skills necessary for critical academic writing.

AUTHOR CONTRIBUTIONS

Leite DFB has initially conceived the idea and has written the first draft of this review. Padilha MAS and Cecatti JG have supervised data interpretation and critically reviewed the manuscript. All authors have read the draft and agreed with this submission. Authors are responsible for all aspects of this academic piece.

ACKNOWLEDGMENTS

We are grateful to all of the professors of the ‘Getting Started with Graduate Research and Generic Skills’ module at University College Cork, Cork, Ireland, for suggesting and supporting this article. Funding: DFBL has granted scholarship from Brazilian Federal Agency for Support and Evaluation of Graduate Education (CAPES) to take part of her Ph.D. studies in Ireland (process number 88881.134512/2016-01). There is no participation from sponsors on authors’ decision to write or to submit this manuscript.

No potential conflict of interest was reported.

1 The questions posed in systematic reviews usually follow the ‘PICOS’ acronym: Population, Intervention, Comparison, Outcomes, Study design.

2 In 1988, Cooper proposed a taxonomy that aims to facilitate students’ and institutions’ understanding of literature reviews. Six characteristics with specific categories are briefly described: Focus: research outcomes, research methodologies, theories, or practices and applications; Goals: integration (generalization, conflict resolution, and linguistic bridge-building), criticism, or identification of central issues; Perspective: neutral representation or espousal of a position; Coverage: exhaustive, exhaustive with selective citations, representative, central or pivotal; Organization: historical, conceptual, or methodological; and Audience: specialized scholars, general scholars, practitioners or policymakers, or the general public.

We use cookies on this site to enhance your experience

By clicking any link on this page you are giving your consent for us to set cookies.

A link to reset your password has been sent to your email.

Back to login

We need additional information from you. Please complete your profile first before placing your order.

Thank you. payment completed., you will receive an email from us to confirm your registration, please click the link in the email to activate your account., there was error during payment, orcid profile found in public registry, download history, difference between a literature review and a critical review.

• Charlesworth Author Services
• 08 October, 2021

As you read research papers, you may notice that there are two very different kinds of review of prior studies. Sometimes, this section of a paper is called a literature review, and at other times, it is referred to as a critical review or a critical context . These differences may be more commonly seen across different fields. Although both these sections are about reviewing prior and existing studies, this article aims to clarify the differences between the two.

Literature review

A literature review is a summary of prior or existing studies that are related to your own research paper . A literature review can be a part of a research paper or can form a paper in itself . For the former, the literature review is designed as a basis upon which your own current study is designed and built. The latter forms a synthesis of prior studies and is a way to highlight future research agendas or a framework.

Writing a literature review

In a literature review, you should attempt to discuss the arguments and findings in prior studies and then work to build on these studies as you develop your own research. You can also highlight the connection between existing and prior literature to demonstrate how the current study you are presenting can advance your knowledge in the field .

When performing a literature review, you should aim to summarise your discussions using a specific aspect of the literature, such as by topic, time, methodology/ design and findings . By doing so, you should be able to establish an effective way to present the relevant literature and demonstrate the connection between prior studies and your research.

Do note that a literature review does not include a presentation or discussion of any results or findings – this should come at a later point in the paper or study. You should also not impose your subjective viewpoints or opinions on the literature you discuss. 

Critical review

A critical review is also a popular way of reviewing prior and existing studies. It can cover and discuss the main ideas or arguments in a book or an article, or it can review a specific concept, theme, theoretical perspective or key construct found in the existing literature .

However, the key feature that distinguishes a critical review from a literature review is that the former is more than just a summary of different topics or methodologies. It offers more of a reflection and critique of the concept in question, and is engaged by authors to more clearly contextualise their own research within the existing literature and to present their opinions, perspectives and approaches .

Given that a critical review is not just a summary of prior literature, it is generally not considered acceptable to follow the same strategy as for a literature review. Instead, aim to organise and structure your critical review in a way that would enable you to discuss the key concepts, assert your perspectives and locate your arguments and research within the existing body of work. 

Structuring a critical review

A critical review would generally begin with an introduction to the concepts you would like to discuss. Depending on how broad the topics are, this can simply be a brief overview or it could set up a more complex framework. The discussion that follows through the rest of the review will then address and discuss your chosen themes or topics in more depth. 

Writing a critical review

The discussion within a critical review will not only present and summarise themes but also critically engage with the varying arguments, writings and perspectives within those themes. One important thing to note is that, similar to a literature review , you should keep your personal opinions, likes and dislikes out of a review. Whether you personally agree with a study or argument – and whether you like it or not – is immaterial. Instead, you should focus upon the effectiveness and relevance of the arguments , considering such elements as the evidence provided, the interpretations and analysis of the data, whether or not a study may be biased in any way, what further questions or problems it raises or what outstanding gaps and issues need to be addressed.

In conclusion

Although a review of previous and existing literature can be performed and presented in different ways, in essence, any literature or critical review requires a solid understanding of the most prominent work in the field as it relates to your own study. Such an understanding is crucial and significant for you to build upon and synthesise the existing knowledge, and to create and contribute new knowledge to advance the field .

Read previous (fourth) in series: How to refer to other studies or literature in the different sections of a research paper

Maximise your publication success with Charlesworth Author Services .

Charlesworth Author Services, a trusted brand supporting the world’s leading academic publishers, institutions and authors since 1928. 

To know more about our services, visit:  Our Services

Share with your colleagues

Related articles.

Conducting a Literature Review

Charlesworth Author Services 10/03/2021 00:00:00

Important factors to consider as you Start to Plan your Literature Review

Charlesworth Author Services 06/10/2021 00:00:00

How to Structure and Write your Literature Review

Charlesworth Author Services 07/10/2021 00:00:00

Related webinars

Bitesize Webinar: How to write and structure your academic article for publication- Module 3: Understand the structure of an academic paper

Charlesworth Author Services 04/03/2021 00:00:00

Bitesize Webinar: How to write and structure your academic article for publication: Module 5: Conduct a Literature Review

Bitesize Webinar: How to write and structure your academic article for publication: Module 7: Write a strong theoretical framework section

Charlesworth Author Services 05/03/2021 00:00:00

Bitesize Webinar: How to write and structure your academic article for publication: Module 11: Know when your article is ready for submission

Literature search.

Why and How to do a literature search

Charlesworth Author Services 17/08/2020 00:00:00

Best tips to do a PubMed search

Charlesworth Author Services 26/08/2021 00:00:00

Best tips to do a Scopus search

A Critical Literature Review on Anorexia Nervosa

Victoria he.

This scientific literature review, written for a First Year Writing Seminar on the psychology of eating disorders, analyzes the characteristics, diagnosis, epidemiology, and complications of anorexia nervosa. It highlights the most substantiated research from peer-reviewed journal articles to provide recommendations for prevention and treatment, as well as necessary areas of further research. Anorexia nervosa is a dangerous, multifaceted mental disorder that disrupts numerous areas of life, including mental, emotional, and physical realms. It can significantly reduce lifespan and be fatal if left untreated. It is characterized by deliberate self-starvation to reach an excessively low weight, along with other practices like overexercising. These behaviors stem from an intense fear of gaining weight and a distorted body perception. The prevalence of anorexia has overall increased since the 1960s, particularly throughout adolescent girls in modern Western cultures due to the heightened ideal of thinness. In addition to environmental risk factors, genetics can also increase the likelihood of anorexia, such as a mother with a history of eating disorders. Comorbid complications often arise with anorexia, such as anxiety disorders, obsessive-compulsive disorders, and social phobia. Physical ailments also occur, including cardiovascular diseases and osteoporosis. Although some health abnormalities linked at heart disease return to normal after a full recovery from anorexia, other diseases like osteoporosis can remain for a lifetime. People diagnosed with anorexia receive either inpatient, partial, or outpatient treatment, in combination with psychotherapy. There should be a greater movement toward body positivity in the media, and stigma around mental disorders should be eliminated.

For additional context, listen to the author's reflection on the piece below.

Author Biography

Victoria He is a member of the class of 2025 from Tampa, Florida. She is double majoring in Psychology and Human & Organizational Development and double minoring in Data Science and Business. Victoria intends to pursue a career in law or business, although she is also very passionate about the field of psychology. On campus, she is involved in Mock Trial, the Undergraduate Honor Council, and the BRAINS Lab where she conducts research on anxiety and depressive disorders. In her free time, Victoria enjoys singing, attending concerts with friends, running, and watching movies.

Author's Reflection

How to Cite

• Endnote/Zotero/Mendeley (RIS)

Scaffold: A Showcase of Vanderbilt First-Year Writing  is a digital collection of first-year writing curated by the Vanderbilt Writing Studio  in connection with the annual Undergraduate Writing Symposium .

• Open access
• Published: 27 November 2023

Conceptualising the value of simulation modelling for public engagement with policy: a critical literature review

• Victoria Loblay   ORCID: orcid.org/0000-0003-4094-9619 1 , 2 ,
• Louise Freebairn 1 , 2 , 3 &
• Jo-An Occhipinti 2 , 4  

Health Research Policy and Systems volume  21 , Article number:  123 ( 2023 ) Cite this article

32 Accesses

1 Altmetric

Metrics details

As we face complex and dynamically changing public health and environmental challenges, simulation modelling has come to occupy an increasingly central role in public engagements with policy. Shifts are occurring not only in terms of wider public understandings of modelling, but also in how the value of modelling is conceptualised within scientific modelling communities. We undertook a critical literature review to synthesise the underlying epistemic, theoretical and methodological assumptions about the role and value of simulation modelling within the literature across a range of fields (e.g., health, social science and environmental management) that engage with participatory modelling approaches. We identified four cross-cutting narrative conceptualisations of the value of modelling across different research traditions: (1) models simulate and help solve complex problems; (2) models as tools for community engagement; (3) models as tools for consensus building; (4) models as volatile technologies that generate social effects. Exploring how these ideas of ‘value’ overlap and what they offer one another has implications for how participatory simulation modelling approaches are designed, evaluated and communicated to diverse audiences. Deeper appreciation of the conditions under which simulation modelling can catalyse multiple social effects is recommended.

Peer Review reports

The COVID-19 pandemic has placed simulation models and modelling practice in the spotlight. Diverse publics are increasingly engaging with simulation modelling on issues ranging from health to the environment, and demands for transparency and access to model results are growing. The purpose of models and how they are made useful are now pressing topics of both scientific discussion [ 1 , 2 ] and public conversation [ 3 ]. This has also prompted calls for greater transparency and scrutiny of models [ 4 ] and burgeoning commentary on ‘best practices’ for responsible, transparent modelling to ensure that these tools serve society’s best interests [ 5 , 6 , 7 ].

Subsequently, simulation models and their communication have been transforming. In the early days of the COVID pandemic, as governments and publics tried to make sense of the new virus, ‘Flatten the Curve’ was introduced alongside hand-washing as a key public health message. Flattening or bending the curve became a familiar visual, representing modelled projections of the effects of slowing the spread of an infectious disease outbreak. By April 2020, Forbes magazine declared ‘Flatten the curve’ an historic disease image [ 8 ]. A form of societal activism coalesced around modelling, for example, citizen scientists developed models as decision aids for office managers wondering whether and when they should send employees home [ 9 ]. With the availability of COVID vaccines in 2021, public interest in modelling turned to the question of how to safely open up alongside vaccine rollouts. Modelling practice responded to the escalating public attention by exploring new avenues for making model results publicly accessible [ 10 ]. Meanwhile, questions began to emerge around whether we ought to have other ways of framing and narrating the COVID crisis beyond the ‘obscure predicates and designs’ of modelling [ 11 ]. In the midst of such questions and efforts toward transparency, however, broader public understandings and interpretations of models and modelling practice remained somewhat unclear.

The terms 'model' and 'modelling' are used to refer to a wide range of practices including statistical modelling, systems modelling, microsimulation and economic modelling, and modelling techniques of data science. At times, the term ‘model’ may simply be used as shorthand for something that is perceived as an ideal example of a phenomenon, for example, a policy or desirable behaviour [ 12 ]. In this article, we do not focus on any particular type of model or modelling practice, however, we are interested in the broad category of simulation models aimed at forecasting future trajectories and exploring potential impacts of policy options. Simulation modelling in this context includes analytic methods of complex systems science such as system dynamics modelling, agent-based modelling, and discrete event simulation. These methods have been widely applied to sectors such as engineering, economics, defence, ecology, and business since the mid-1950s, and more recently to healthcare and population health [ 13 , 14 , 15 , 16 , 17 , 18 ]. Particularly, we are interested in how these models are conceptualised in relation to networks of social relations that are harnessed to respond to future national and global challenges.

The social dynamics of modelling are explored in detail in the field of participatory modelling. Participatory approaches to modelling practice are varied, but research in this area tends to be oriented toward the engagement of stakeholders and community members in modelling practice [ 19 ]. Emphasis is placed on effective communication between modellers and stakeholders, as well as among diverse stakeholder groups [ 20 ]. Understanding how core participants—as the ‘first audience’ of a model—engage with and understand models may provide clues as to how models are communicated and used with broader audiences. Policymakers engaged in participatory modelling, for example, may require models to provide concise advice accompanied by “a plausible narrative” that is relevant to areas of immediate policy concern [ 7 ]. The value of modelling is often located in participatory engagements with the model: participants are able to appreciate complexity of issues, understand the crucial role of data availability, recognise the range of influences at play when contemplating policy and planning decisions [ 21 ], or confront sensitive issues with other policy experts [ 22 ]. Much of this research investigating the value of participatory modelling for stakeholders tends to be based on case studies of modelling in particular sectors such as public health, or environmental science [ 23 , 24 ]. As illustrated by the multi-dimensional environmental, socio-psychological, economic and political impacts of the COVID pandemic, however, crises, and potential solutions, are rarely confined to single disciplines or sectors [ 7 ].

Recent critical approaches to participatory modelling within science and technology studies have begun to consider how health modelling could be informed by approaches in other sectors which explore how scientists and lay publics can more effectively collaborate [ 25 , 26 ]. Yet there is generally limited guidance about what learnings from different research traditions have to offer one another, or how issues of transparency and accessibility can travel beyond the core participants involved in modelling projects. In cases where models circulate with lay publics or decision-makers who have not been intimately involved in participatory modelling—as happened with COVID models and continues to happen with climate change modelling—the ramifications of these broader engagements and the value models hold in wider societal contexts remains unclear. To generate a deeper understanding of how models and model findings are made valuable through their circulation and socialisation with diverse publics, we undertook a critical literature review. The review covers applications of simulation modelling across systems science, environmental, biological, health systems, as well as the social sciences where critical perspectives and discussions on public engagements with modelling have been most developed. The review explores (1) How the role and value of models is conceptualised in relation to knowledge sharing, community participation and public engagement (2) The epistemic, theoretical and methodological traditions and assumptions that underpin these approaches to modelling practice. We go on to consider how critical reflections on ‘value’ from different traditions might inform the development of research on modelling practice and the judicious use of models going forward.

As we were interested in exploring underlying assumptions within the empirical stories of the literature [ 27 ], we adopted hermeneutics as the philosophy and methodology for conducting the review [ 28 ]. Many reviews of participatory modelling focus on the question of ‘what works?’ or synthesise learnings from a particular field (i.e., health policy or environmental science). In our critical literature review, the emphasis is on induction, interpretation and critique [ 29 ], with the goal of deepening understanding of how ‘value’ is understood in different approaches to participatory modelling. Conventional systematic review methodology—testing theories around ‘what works’ through exhaustive searches, or determining averages through quantitative data—would be inappropriate for our aims [ 29 ]. Although numerous texts in the simulation modelling literature draw on insights derived from quantitative models, our critical analysis centres on unpacking the implicit assumptions conveyed through the narrative descriptions of the purpose of models and why they are valuable.

Greenhalgh and colleagues’ [ 30 ] emphasis on mapping storylines of different research traditions influenced our approach. However, we decided it was more appropriate and informative to pursue a “dynamic, recursive and reflexive” synthesis [ 27 ] of how the modelling literature conceptualised the value of models in relation to knowledge sharing and public engagement, rather than a highly systematic, meta-narrative analysis. Drawing on our collective expertise in participatory modelling and social studies of science, we prioritised texts that were relevant to highlight distinctive contributions and major themes, instead of presenting an exhaustive overview of the entire body of literature and creating strict boundaries around inclusion and exclusion criteria.

Through an iterative approach, a search strategy began with a preliminary overview of a broad range of modelling literature across different sectors to help identify key search terms and refine our research questions. After trialling searches in a number of electronic databases including Google Scholar, Web of Science, PubMed, Science Direct, and Proquest we found that Scopus enabled us to locate the most relevant papers in relation to our areas of focus. The review was conducted in two phases between August and December 2021. Sources included in the search were scholarly documents (peer reviewed journals, conference proceedings and PhD dissertations) published in English between 2006 and 2021. Initially review search terms included ‘systems model’ OR ‘systems modeling’ OR ‘systems modelling’ OR ‘systems models’ AND engagement OR engaging OR ‘policy maker’ OR ‘policy makers’ OR ‘policy makes’ yielding 890 documents. Based on themes and terms identified in the initial search, a second search was conducted including the following search terms: ‘group model building’ OR ‘participatory dynamic simulation model’ OR ‘participatory dynamic simulation modeling’ OR ‘participatory model’ OR ‘participatory modeling’ yielding 1235 documents. VL screened and sorted records, initially by reviewing titles and abstracts and then by reviewing full papers. Foundational papers in the literature were supplemented with a focus on more recent publications between 2017 and 2021, to capture current thinking and innovations around participatory methods. This preliminary search was aided by citation tracking and snowballing, as well as recommendations from LF and JO, both with extensive experience in participatory modelling.

Reflecting on these searches, the authors iteratively tailored the search strategy toward literature with a specific emphasis on:

Models aimed at forecasting future trajectories and engagement with policy questions and decisions

Transparency, socialisation and communication of models

Novel methods for integrating data/knowledge and bringing diverse groups of people together around the development and use of models

This search led to a total of 53 texts being included for review, which are summarised below in Table 1 .

The texts were read closely for key concepts describing the purpose of models or engagements with stakeholders. We focused on instances in the texts where the logic of value was invoked and where value was described in relation to the development and communication of simulation models. The epistemologies, methods and focus of the texts are summarised in Table 1 . As the table indicates, these papers comprised a mixture of case studies, review articles and theoretical papers. In keeping with a hermeneutic approach, the analysis involved a dialectical tacking back and forth between descriptive detail and the broader themes in the literature [ 28 ]. This allowed for new ways of linking concepts and synthesising theories. Themes were identified by VL and presented to LF and JO for feedback where we addressed contradictions and refined the descriptive labels and statements. This approach conforms to Standards for Reporting Qualitative Research [ 31 ] (Additional file 1 ).

Results are presented as four over-arching narrative conceptions of the role and value of models (see Fig. 1 ). We identified cross-cutting themes around approaches to participation, knowledge sharing and public engagement. These narrative conceptions are not mutually exclusive thematic domains, but are in some cases overlapping and in conversation with one another.

Summary of narrative conceptions

Narrative conception #1: models simulate and help solve complex problems

Models in this research tradition are often referred to as ‘decision-support’ tools that seek to provide predictive planning frameworks to inform and improve policy decisions [ 32 ]. In this conceptualisation, models are designed to help policymakers gain better understandings of system behaviour and the multiple challenges and goals in a complex system [ 33 ]. The kinds of decisions that models in this tradition are supportive of include service planning, investment strategies, and policy reform. Models act to facilitate sophisticated and intricate understandings of a problem and how it will respond to alternative courses of action, allowing decision-makers to identify more effective intervention strategies [ 34 ]. Through forecasting implications of particular investment strategies models have the potential to both optimise resource allocation and minimise unanticipated harmful consequences of decisions.

This conceptualisation of models often draws on the modelling methods of complex systems science (including system dynamics modelling), which examine interrelationships, feedback loops and the presence of equilibria in the system under investigation. The aim of such modelling is to move beyond simple ‘rational’ choice decisions, based on straightforward inputs and outputs, in order to identify opportunities for intervention or action that might have been overlooked by traditional ‘evidence-based’ or cost/benefit approaches. The inclusion of local and practice-based knowledge alongside research data is considered to challenge accepted conventions of evidence ‘hierarchies’ in health [ 35 ]. Further, concepts of evidence and data are not understood in terms of isolated individual studies, but rather as a ‘complex puzzle’ which takes account of the additive effects of combining different intervention approaches over time [ 36 ]. Modelling can be undertaken at different levels of abstraction (i.e., micro-, meso-, macro-) to arrive at a testable hypothesis and explanation for why a system behaves the way it does [ 37 ]. Systems models therefore allow decision-makers to consider how competing priorities and agendas across, for example, different levels of government may undermine the impact of desired policy outcomes [ 38 ]. Agent Based Modelling in particular is favoured by those interested in the activities and/or heterogeneity of individual agents (such as people, equipment, services or vehicles) [ 39 ]. These models strive toward a depiction of detailed reality through the inclusion of individual behaviour, social interactions and dynamics, and environmental variations [ 40 ].

Approaches to participation of stakeholders and public engagement in this tradition are varied. In the participatory development of Agent Based Models, experts are described as being able to easily relate to the depiction of the model at an individual level as they can draw on experiential knowledge of patient case histories and deploy professional judgement to validate and help calibrate the model [ 41 , 42 ]. This and other approaches are underpinned by an ideal of ‘transparency,’ which eschews the ‘black box’ model in favour of making the model structure, assumptions, and limitations available for outside appraisal to assist in interpreting model results [ 43 ]. Others draw on the concept of ‘knowledge mobilisation’ aiming to include ‘end users’ (i.e. decision-makers) of the model in the model development process [ 32 , 44 ]. Stakeholders are typically policymakers, researchers and practice experts (e.g. clinicians) whose participation is said to increase trust in the model outputs [ 45 ], ensure consideration of practical issues with policy implementation or evaluation [ 46 ] as well as the translatability of the model findings [ 35 ].

Benefits of participation in this tradition include opportunities for capacity building in terms of knowledge and expertise in systems science and modelling methods [ 37 , 47 ]. This expertise is theorised to improve literacy in interpreting modelling results and empower end-users make better appraisals of future models they encounter in their policy practice [ 37 ]. Participatory modelling can be used to aid planning and policy choices in terms of organisational management, to enhance learning and promote shared understanding of complex problems among different actors in the system [ 33 , 48 , 49 ].

Questions surrounding language, accessibility, cultural adaptation, and power dynamics in the modelling process and translation of model results are rarely explored in depth in this approach. Some have pointed out that recruiting a diversity of participants and stakeholders in this tradition tends to favour experts such as health practitioners or policymakers. By contrast, small numbers of community participants are chosen to represent a variety of personal experiences within the system as a whole [ 50 ].

Narrative conception #2: models as tools for community engagement

Drawing on principles of community-based participatory research, community development and equity, the emphasis in this conceptualisation is on modelling practice as a ‘powerful learning process’ [ 23 ]. Research in this tradition is in keeping with efforts to engage in dialogues and enhance communication between experts and lay publics [ 51 ]. Drawing on trends in citizen science, it responds to demands by citizens to be included as stakeholders in planning decisions that affect them [ 52 ].

Central to this narrative construct is a focus on harnessing the capacity of multiple publics to engage in coordinated efforts to address the problem/s at hand. Goals of social learning and building ‘social capital’ among participants take priority over collectively producing model results [ 53 ]. This orientation is often underscored by a logic that the ‘problems’ tackled through modelling disproportionately affect marginalised groups, therefore meaningful participation of such groups is critical to the success of modelling efforts. Community-based participants are treated as experts of local and historical context that bring crucial knowledge, skills and labour to the model building [ 52 , 53 ]. Such groups may be identified by virtue of their experience—for example people with personal experience of substance abuse and domestic violence [ 50 ], or by virtue of their relationship to place and experience of historic initiatives (e.g. First Nations people). Concepts of evidence and data are construed as contingent in this tradition and there is a recognition that vast diversity of participant experiences exists in relation to modelling issues of interest [ 50 ]. Knowledge produced through modelling exercises is therefore fluid and mediated by an “interactive and iterative” learning process [ 54 ].

Commitment to engaging a broad range of community participants is not only practical, but also ideological [ 55 ], encouraging modelling practice to strive toward engaging participants in ways that empower them to take ownership of the products of the modelling [ 52 ]. Power dynamics—and who may be left out of participatory modelling—are of key concern within this research tradition, and authors emphasise how powerful stakeholders can encourage, or conversely prevent, other actors from fully participating [ 53 ]. Attending to unequal power dynamics is seen as essential for encouraging engagement of marginalised community members, and methodological approaches carefully consider how to meaningfully engage all participants and ‘amplify’ the voices of previously marginalised stakeholders in the modelling process [ 54 , 56 ]. Many approaches to modelling in this tradition experiment with forms of knowledge production that go beyond scientific publication, often using arts and performance-based mediums such as storylines, blogs, and alternative forms of visualisation [ 53 , 54 ]. In addition to experimenting with communication mediums, many authors also draw attention to the issue that some marginalised participant groups may lack the necessary resources (time, financial, psychological, transportation) to take part in projects requiring intensive commitment [ 50 ], calling for compensation or remuneration to remedy this. Many authors stress, however, that communication of model results should not be confined to the model builders and decision makers. An enduring question in this tradition is how stakeholders who participated in the model building can take their new understandings and communicate them to others who were not involved in the model construction [ 57 ].

Narrative conception #3: models as tools for consensus building

This conceptualisation of models builds on and overlaps with narrative conceptions #1 and #2, but is distinguished by its emphasis on the value of modelling processes for facilitating communication and providing opportunities for negotiating conflicts and building consensus among stakeholder groups. Much of the research on participatory modelling practice acknowledges the value of modelling practice for promoting intellectual exchange and advancing contentious debates. Models are described as platforms for strengthening relationships between different knowledge communities (e.g. academics and policymakers [ 58 ] or historically antagonistic institutions [ 59 ]). Some texts take this concept further, delving deeply into the role of models and the specific qualities inherent in modelling that lends this practice to consensus building.

Approaches to participation in this tradition are centred on the negotiation of conflict, or reconciling different viewpoints among participants. A major impetus for modelling in this conception is the development of a ‘shared language’ among participants [ 60 ]. This is important because modelling is often used to address complex and messy problems and participants bring with them rich, but often partial, prior knowledge of the situation [ 61 ]. Participants developing a ‘shared understanding’ of the issue being modelled—be it policy implementation or the overview of a system—is seen as an inherently desirable outcome of the modelling process [ 46 ].

Issues of bias and values are also of central concern. Tools and methodologies are deployed to elicit unconscious bias [ 62 ] and recognise beliefs and values [ 23 ]. Voinov and Gaddis [ 59 ] have been particularly influential in arguing that modellers ought to acknowledge and embrace how their work is driven by ‘values’. Others have drawn on psychological theories of persuasion and mental model change to explore how model building processes influence beliefs, attitudes and perceptions of norms [ 61 ]. This research identifies “counter-intuitive insights”—where model simulations or views of other participants run counter to originally held positions on an issue—as a key strength in shifting mental models [ 63 ]. Mental modelling techniques and tools have been popularised by the work of Gray and colleagues [ 64 ] to facilitate engagement and communication between participants [ 65 ] and track changes in mental models during model building [ 66 ].

‘Boundary object’ theory is often invoked in this research tradition. Many of these texts draw on Star and Greisemer’s [ 67 ] theoretical synthesis of Boundary Objects in Science and Technology Studies. Models are depicted as ‘adaptable’ objects whose flexible representation enables people with different backgrounds and expertise to communicate more effectively and engage in co-ordinated activities [ 65 ]. Texts emphasise and analyse models’ ability to facilitate communication on contentious issues through developing a shared language and a shared understanding between different groups [ 68 ]. From this perspective, what matters is less a model’s precision than its capacity to provide a platform that can mitigate conflict and friction between different groups.

Narrative conception #4: models as volatile technologies that generate social effects

This conceptualisation of models is inspired by efforts from the social sciences to critique dominant approaches to modelling. In contrast to the other three narrative conceptions, this approach is less concerned with developing ‘best practice’ approaches (themselves premised on another kind of model) than with rethinking taken-for-granted assumptions within modelling.

Science and Technology studies, and the work of Bruno Latour [ 69 ] has a strong influence on studies in this tradition. In this sense, models are not simply neutral technologies that describe or represent activities and issues of policy interest, they are “generative” [ 26 ] and “transformational” [ 70 ]. Models—like other products of science in Latour’s theorisation—are considered for their ability to generate social effects. They are dynamic entities or “assemblages,” clusters of actors, practices, discourses and material objects which circulate across social fields and intersect with policy practice. Models and modelled evidence are multiple in their meanings and are perpetually emerging. This conceptualisation of models may also be framed within broader critical approaches to anticipatory governance [ 71 ], New Public Management [ 12 ] and practices of ‘projection’ or ‘simulation’ [ 72 , 73 ].

Callon and Law’s [ 74 ] conceptual reconfiguration of the concept of calculation to include notions of judgement and passion has spawned a number of texts exploring the ‘affective’ and emotional components of modelling practice [ 72 ]. The affective qualities of models, and their ability to generate and be shaped by emotional responses, have led some authors to note the “virality” of models and their virus-like capacity for contagion [ 12 ]. Some suggest that the ‘seductive simulations’ tempt modellers to ‘oversell’ their products as ‘truth machines’ [ 73 ]. Such studies emphasise the messiness and uncertainties involved in modelling practice.

Knowledge and ‘evidence’ are characterised in this tradition as tentative, evolving, contingent and emergent, highlighting the limits of conceptualisations of models based on ‘evidence-based’ approaches. Some, such as Pawson [ 1 ], draw on complexity science and evaluation theory to question the evidence that informs health modelling. This critique further extends to dominant conceptions of ‘interventions’ as the evidence upon which model assumptions are based, stressing “the impossibility, of trying to capture a complex, self-transforming process as a model ‘parameter’” [ 1 ]. Texts highlight the dangers of building models based on evidence from interventions which are regarded as ‘fixed’ both in terms of implementation and effect. The focus here invokes questions around the role of “context” in modelling practice. Whilst acknowledging that some models do make generalised assumptions while paying insufficient attention to local conditions, this perspectival view (unlike Narrative conception #2) does not pursue a line of reasoning whereby better models depend on increasingly attuning them to local conditions [ 2 , 75 ]. Models, after all, will always be “simplifications based on abstraction” [ 2 ]. Instead, both interventions and models are thought of as fluid and continually adapting as elements of ‘evidence-making interventions’ [ 76 ]. A model may function as an ‘interested amateur,’ adopting the role of an outsider that aids the interaction of policy experts, freeing up participants to openly voice criticisms and confront sensitive issues [ 22 ]. In this way, a measure of a good model is not simply how well it incorporates local data within its calculations, but how well models are mobilised as part of an adaptive science in which they can be treated as a pathway to dialogue, and deeper appreciation of interventions and their effects.

Other reviews of participatory modelling have explored the value of stakeholder engagement in a particular field (e.g. social and environmental sciences or health). This review, however, examines the narrative conceptualisations of the value of models across a range of fields that engage with participatory modelling approaches. Examining these conceptions side-by-side allows us to consider how learnings from different fields might inform one another. In each of these narrative conceptions, value is constituted through the process of building models with stakeholders. In narrative conception 1, models employ quantitative methods but are embedded in social processes that are pivotal to their success or failure. The process of participatory modelling enhances the quality of model results by integrating new forms of knowledge and building trust with decision-makers. In other conceptions, the process is less about refining model accuracy and more about empowering marginalised stakeholders (narrative conception 2) or negotiating friction and developing a shared language among model participants from different knowledge communities (narrative conception 3). In narrative conception 4, the process is characterised as less goal-oriented, instead describing the unpredictability, multiplicity and self-organisational properties of model building with stakeholders.

An important point of difference in these conceptualisations lies in whether models are considered to be analytic tools to support decision-making, or whether modelling is itself a form of intervention capable of generating social change. Different framings of the role and value of modelling have implications for the evaluation of modelling and understanding the effects of a modelling project. Evaluations of participatory modelling are often underdeveloped and there is a need to develop clearer appreciation of the critical elements entailed in the purpose and associated value of modelling tools [ 77 , 78 ]. If models are framed as tools for supporting decision-making (narrative conception 1), evaluation would focus on investigating the experience of participant stakeholders including their understanding and perception of the problem, or their trust in the legitimacy of model results [ 60 , 79 ]. If models are deployed as aides in collaborative problem-solving, or as conflict resolution tools (narrative conception 3) evaluation might focus on how model representations are received among the group, along with their capacity to de-personalise conflicts and allow participants to negotiate less threatening paths to develop shared language [ 63 ] and shape consensus [ 68 ]. If models are avenues for building social capital or empowering traditionally marginalised communities (narrative conception #2), then evaluation of conflict resolution, or assessments of the development of new knowledge among participants must also attune to attenuating power dynamics and the foregrounding of marginalised or non-scientific forms of knowledge. Importantly, the goal of empowerment ought not to focus on individual participants, but facilitating circumstances so that publics are capable of producing new knowledge [ 25 ]. In narrative conception #4, modelling is framed as a starting point or a trigger for generating change processes. This has ramifications for evaluation in that the role of the model can be considered at project initiation, however flexibility is needed as the role may change over time and with different audiences and contexts.

These narrative conceptions may alternatively be used as a package to aid modellers in communicating the role and value of modelling tools to stakeholders including policy experts, funders, media or community representatives. Drawing on all four narrative conceptions of value can help illustrate how models serve multiple purposes and broaden understandings of their role in change processes. If the value of models is too narrowly defined, if their predictive value is overstated, or based on notions of infallible evidence to predict outcomes of policy decisions, there is a risk that the impact of modelling tools may be overlooked.

Conclusions

Crises such as the COVID pandemic and environmental catastrophes offer opportunities to transform the dynamics of knowledge sharing and engagement with models across diverse community and policymaking contexts. Bringing together the different narrative conceptions highlights the multiple ways that simulation modelling can be of value for public policy engagement. Linking ideas in these narrative conceptions—for example, considering how model representations can sit alongside and speak to non-scientific ways of knowing in informing policy decisions—offers new possibilities for harnessing the value of modelling. It may also prompt critical reflection on taken-for-granted ideas such as the value of consensus. The development of consensus or shared language must be weighed against opportunities for building social capital or encouraging participants to better articulate and communicate alternative perspectives and ways of understanding issues. In this sense, modelling may be considered valuable for its potential to generate and legitimate multiplicity.

At the same time, if the value of modelling is also located in the process of model-building or collaborative interaction with models, more work is needed to understand the evolutionary dynamics and systemic transformations that are potentially triggered by modelling beyond the participants directly engaged in model development. Future research tracking the value of models and their ripple effects as they circulate in wider public and policy spheres is needed. These different narrative conceptions may be used as a starting point for understanding the conditions under which simulation modelling can generate proactive pathways for change, or alternatively risk perpetuating the status quo.

Availability of data and materials

Not applicable.

Pawson R. The coronavirus response: boxed in by models. Evaluation. 2021;27(2):149–67.

Article   Google Scholar  

Kaehne A. From abstract to ideal—the limits of models: a reply to Pawson’s ‘boxed in by models.’ Evaluation. 2021. https://doi.org/10.1177/13563890211007505 .

Tufekci Z. Don’t believe the COVID-19 models: that’s not what they’re for. Washington: The Atlantic; 2020.

Google Scholar  

Barton CM, Alberti M, Ames D, Atkinson J-A, Bales J, Burke E, et al. Call for transparency of COVID-19 models. Science. 2020;368(6490):482–3.

Article   PubMed   Google Scholar  

Jalali MS, DiGennaro C, Sridhar D. Transparency assessment of COVID-19 models. Lancet Glob Health. 2020;8(12):e1459–60.

Article   PubMed   PubMed Central   Google Scholar  

Saltelli A, Bammer G, Bruno I, Charters E, Di Fiore M, Didier E, et al. Five ways to ensure that models serve society: a manifesto. Nature. 2020;582(7813):482–4.

Article   CAS   PubMed   Google Scholar  

Squazzoni F, Polhill JG, Edmonds B, Ahrweiler P, Antosz P, Scholz G, et al. Computational models that matter during a global pandemic outbreak: a call to action. J Artif Soc Soc Simul. 2020. https://doi.org/10.18564/jasss.4298 .

Amsen E. How ‘Flatten the Curve’ is on its way to become a historic disease image. Jersey City: Forbes; 2020.

Rhodes T, Lancaster K, Rosengarten M. A model society: maths, models and expertise in viral outbreaks. Crit Public Health. 2020;30(3):253–6.

Karvelas P. Policy makers, public able to compare COVID-19 modelling on new site. 2021.

Anderson W. The model crisis, or how to have critical promiscuity in the time of Covid-19. Soc Stud Sci. 2021;51(2):167–88.

Østebø MT. Village gone viral: understanding the spread of policy models in a digital age. Redwood City: Stanford University Press; 2021.

Book   Google Scholar  

Sterman J. Business dynamics. New York: McGraw-Hill, Inc.; 2000.

Sterman JD. Learning from evidence in a complex world. Am J Public Health. 2006;96(3):505–14.

Mobus GE. Principles of systems science. In: Mobus GE, editor. Systems science: theory, analysis, modeling, and design. Berlin: Springer; 2022. p. 41–87.

Chapter   Google Scholar  

Homer JB, Hirsch GB. System dynamics modeling for public health: background and opportunities. Am J Public Health. 2006;96(3):452–8.

Marshall DA, Burgos-Liz L, IJzerman MJ, Crown W, Padula WV, Wong PK, et al. Selecting a dynamic simulation modeling method for health care delivery research—Part 2: report of the ISPOR Dynamic Simulation Modeling Emerging Good Practices Task Force. Val Health. 2015;18(2):147–60.

Marshall DA, Burgos-Liz L, IJzerman MJ, Osgood ND, Padula WV, Higashi MK, et al. Applying dynamic simulation modeling methods in health care delivery research—the SIMULATE checklist: report of the ISPOR simulation modeling emerging good practices task force. Val Health. 2015;18(1):5–16.

Basco-Carrera L, Warren A, van Beek E, Jonoski A, Giardino A. Collaborative modelling or participatory modelling? A framework for water resources management. Environ Model Softw. 2017;91:95–110.

Gilbert N, Ahrweiler P, Barbrook-Johnson P, Narasimhan KP, Wilkinson H. Computational modelling of public policy: reflections on practice. J Artif Soc Soc Simul. 2018;21(1):14.

Barbrook-Johnson P, Badham J, Gilbert N. Uses of agent-based modeling for health communication: the TELL ME case study. Health Commun. 2017;32(8):939–44.

Johnson P. Agent-based models as “interested amateurs.” Land. 2015;4(2):281–99.

Jordan R, Gray S, Zellner M, Glynn PD, Voinov A, Hedelin B, et al. Twelve questions for the participatory modeling community. Earth’s Future. 2018;6(8):1046–57.

Elsawah S, Filatova T, Jakeman AJ, Kettner AJ, Zellner ML, Athanasiadis IN, et al. Eight grand challenges in socio-environmental systems modeling. Soc Environ Syst Model. 2020;2:16226.

Adams S, Rhodes T, Lancaster K. New directions for participatory modelling in health: redistributing expertise in relation to localised matters of concern. Glob Public Health. 2021. https://doi.org/10.1080/17441692.2021.1998575 .

Landström C, Whatmore SJ, Lane SN, Odoni NA, Ward N, Bradley S. Coproducing flood risk knowledge: redistributing expertise in critical ‘participatory modelling.’ Environ Plan A. 2011;43(7):1617–33.

Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, et al. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006;6(1):1–13.

Boell SK, Cecez-Kecmanovic D. A hermeneutic approach for conducting literature reviews and literature searches. Commun Assoc Inf Syst. 2014;34(1):12.

Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews? Eur J Clin Invest. 2018. https://doi.org/10.1111/eci.12931 .

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005;61(2):417–30.

O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51.

Atkinson J-A, Skinner A, Lawson K, Rosenberg S, Hickie IB. Bringing new tools, a regional focus, resource-sensitivity, local engagement and necessary discipline to mental health policy and planning. BMC Public Health. 2020;20:1–9.

Ibrahim Shire M, Jun GT, Robinson S. Healthcare workers’ perspectives on participatory system dynamics modelling and simulation: designing safe and efficient hospital pharmacy dispensing systems together. Ergonomics. 2020;63(8):1044–56.

Gilbert N, Terna P. How to build and use agent-based models in social science. Mind Soc. 2000;1(1):57–72.

O’Donnell E, Atkinson J-A, Freebairn L, Rychetnik L. Participatory simulation modelling to inform public health policy and practice: rethinking the evidence hierarchies. J Public Health Policy. 2017;38(2):203–15.

Atkinson J-A, Skinner A, Hackney S, Mason L, Heffernan M, Currier D, et al. Systems modelling and simulation to inform strategic decision making for suicide prevention in rural New South Wales (Australia). Aust N Z J Psychiatry. 2020;54(9):892–901.

Langellier BA, Kuhlberg JA, Ballard EA, Slesinski SC, Stankov I, Gouveia N, et al. Using community-based system dynamics modeling to understand the complex systems that influence health in cities: the SALURBAL study. Health Place. 2019;60:102215.

Occhipinti J-A, Skinner A, Carter S, Heath J, Lawson K, McGill K, et al. Federal and state cooperation necessary but not sufficient for effective regional mental health systems: insights from systems modelling and simulation. Sci Rep. 2021;11(1):1–12.

Freebairn L. “Turning mirrors into windows”: a study of participatory dynamic simulation modelling to inform health policy decisions. Sydney: The University of Notre Dame Australia; 2019.

Matthews RB, Gilbert NG, Roach A, Polhill JG, Gotts NM. Agent-based land-use models: a review of applications. Landscape Ecol. 2007;22(10):1447–59.

Osgood N. Frontiers in health modelling. In: El-Sayed AM, Galea S, editors. Systems science and population health. Oxford: Oxford University Press; 2017.

Freebairn L, Atkinson J-A, Osgood ND, Kelly PM, McDonnell G, Rychetnik L. Turning conceptual systems maps into dynamic simulation models: an Australian case study for diabetes in pregnancy. PLoS ONE. 2019;14(6):e0218875.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Brugnach M, Tagg A, Keil F, de Lange WJ. Uncertainty matters: computer models at the science—policy interface. Water Resour Manage. 2007;21(7):1075–90.

Freebairn L, Rychetnik L, Atkinson J-A, Kelly P, McDonnell G, Roberts N, et al. Knowledge mobilisation for policy development: implementing systems approaches through participatory dynamic simulation modelling. Health Res Policy Syst. 2017;15(1):1–12.

Freebairn L, Atkinson J-A, Kelly PM, McDonnell G, Rychetnik L. Decision makers’ experience of participatory dynamic simulation modelling: methods for public health policy. BMC Med Inform Decis Mak. 2018;18(1):1–14.

de Oliveira Morais LM, Kuhlberg J, Ballard E, Indvik K, Rocha SC, Sales DM, et al. Promoting knowledge to policy translation for urban health using community-based system dynamics in Brazil. Health Res Policy Syst. 2021;19(1):1–15.

Siokou C, Morgan R, Shiell A. Group model building: a participatory approach to understanding and acting on systems. Public Health Res Pract. 2014;25(1):e2511404.

Hosseinichimeh N, Kim H, Ebrahimvandi A, Iams J, Andersen D. Using a stakeholder analysis to improve systems modelling of health issues: the impact of progesterone therapy on infant mortality in Ohio. Syst Res Behav Sci. 2019;36(4):476–93.

Muttalib F, Ballard E, Langton J, Malone S, Fonseca Y, Hansmann A, et al. Application of systems dynamics and group model building to identify barriers and facilitators to acute care delivery in a resource limited setting. BMC Health Serv Res. 2021;21(1):1–10.

Deutsch AR, Lustfield R, Jalali MS. Community-based system dynamics modelling of stigmatized public health issues: increasing diverse representation of individuals with personal experiences. Syst Res Behav Sci. 2021. https://doi.org/10.2139/ssrn.3573207 .

Popay J, Williams G, Thomas C, Gatrell T. Theorising inequalities in health: the place of lay knowledge. Sociol Health Illn. 1998;20(5):619–44.

Voinov A, Kolagani N, McCall MK, Glynn PD, Kragt ME, Ostermann FO, et al. Modelling with stakeholders—next generation. Environ Model Softw. 2016;77:196–220.

Sterling EJ, Zellner M, Jenni KE, Leong K, Glynn PD, BenDor TK, et al. Try, try again: lessons learned from success and failure in participatory modeling. Elementa. 2019;7:9.

Williams DS. Enhancing autonomy for climate change adaptation using participatory modeling. Weather Clim Soc. 2020;12(4):667–78.

Van Bruggen A, Nikolic I, Kwakkel J. Modeling with stakeholders for transformative change. Sustainability. 2019;11(3):825.

Bou Nassar JA, Malard JJ, Adamowski JF, Ramírez Ramírez M, Medema W, Tuy H. Multi-level storylines for participatory modeling–involving marginalized communities in Tz’olöj Ya’, Mayan Guatemala. Hydrol Earth Syst Sci. 2021;25(3):1283–306.

Smetschka B, Gaube V. Co-creating formalized models: participatory modelling as method and process in transdisciplinary research and its impact potentials. Environ Sci Policy. 2020;103:41–9.

Atkinson J-A, Knowles D, Wiggers J, Livingston M, Room R, Prodan A, et al. Harnessing advances in computer simulation to inform policy and planning to reduce alcohol-related harms. Int J Public Health. 2018;63(4):537–46.

Voinov A, Gaddis EB. Values in participatory modeling: theory and practice. In: Gray S, Paolisso M, Jordan R, Gray S, editors. Environmental modeling with stakeholders. Berlin: Springer; 2017. p. 47–63.

Falconi SM, Palmer RN. An interdisciplinary framework for participatory modeling design and evaluation—what makes models effective participatory decision tools? Water Resour Res. 2017;53(2):1625–45.

Rouwette EA, Korzilius H, Vennix JA, Jacobs E. Modeling as persuasion: the impact of group model building on attitudes and behavior. Syst Dyn Rev. 2011;27(1):1–21.

Frerichs L, Lich KH, Funchess M, Burrell M, Cerulli C, Bedell P, et al. Applying critical race theory to group model building methods to address community violence. Prog Commun Health Partnersh. 2016;10(3):443.

Rouwette E, Bleijenbergh I, Vennix J. Group model-building to support public policy: addressing a conflicted situation in a problem neighbourhood. Syst Res Behav Sci. 2016;33(1):64–78.

Gray SA, Gray S, Cox LJ, Henly-Shepard S. Mental modeler: a fuzzy-logic cognitive mapping modeling tool for adaptive environmental management. In: 2013 46th Hawaii International Conference on System Sciences; IEEE. 2013.

Huang J, Hmelo-Silver CE, Jordan R, Gray S, Frensley T, Newman G, et al. Scientific discourse of citizen scientists: Models as a boundary object for collaborative problem solving. Comput Hum Behav. 2018;87:480–92.

Singer A, Gray S, Sadler A, Olabisi LS, Metta K, Wallace R, et al. Translating community narratives into semi-quantitative models to understand the dynamics of socio-environmental crises. Environ Model Softw. 2017;97:46–55.

Star SL, Griesemer JR. Institutional ecology, translations’ and boundary objects: amateurs and professionals in Berkeley’s Museum of Vertebrate Zoology, 1907–39. Soc Stud Sci. 1989;19(3):387–420.

Black LJ, Andersen DF. Using visual representations as boundary objects to resolve conflict in collaborative model-building approaches. Syst Res Behav Sci. 2012;29(2):194–208.

Latour B. Science in action: how to follow scientists and engineers through society. Cambridge: Harvard University Press; 1987.

Schubert C. Situating technological and societal futures: pragmatist engagements with computer simulations and social dynamics. Technol Soc. 2015;40:4–13.

Aykut SC, Demortain D, Benbouzid B. The politics of anticipatory expertise: plurality and contestation of futures knowledge in governance â introduction to the special issue. Sci Technol Stud. 2019;32(4):2–12.

Rhodes T, Lancaster K. Excitable models: Projections, targets, and the making of futures without disease. Sociol Health Illn. 2021. https://doi.org/10.1111/1467-9566.13263 .

Lahsen M. Seductive simulations? Uncertainty distribution around climate models. Soc Stud Sci. 2005;35(6):895–922.

Callon M, Law J. On qualculation, agency, and otherness. Environ Plan D. 2005;23(5):717–33.

Rhodes T, Lancaster K, Lees S, Parker M. Modelling the pandemic: attuning models to their contexts. BMJ Glob Health. 2020;5(6):e002914.

Rhodes T, Lancaster K. Evidence-making interventions in health: a conceptual framing. Soc Sci Med. 2019;238:112488.

Lee GY, Hickie IB, Occhipinti J-A, Song YJC, Camacho S, Skinner A, et al. Participatory systems modelling for youth mental health: an evaluation study applying a comprehensive multi-scale framework. Int J Environ Res Public Health. 2022;19(7):4015.

Lee GY, Hickie IB, Occhipinti J-A, Song YJC, Skinner A, Camacho S, et al. Presenting a comprehensive multi-scale evaluation framework for participatory modelling programs: a scoping review. PLoS ONE. 2022;17(4):e0266125.

Atkinson J-A, Wells R, Page A, Dominello A, Haines M, Wilson A. Applications of system dynamics modelling to support health policy. Public Health Res Pract. 2015;25(3):e2531531.

Rhodes T, Lancaster K, Rosengarten M. A model society: maths, models and expertise in viral outbreaks. Crit Public Health. 2020;30:253–6.

Rhodes T, Lancaster K. Mathematical models as public troubles in COVID-19 infection control: following the numbers. Health Sociol Rev. 2020;29(2):177–94.

Saltelli A, Bammer G, Bruno I, Charters E, Di Fiore M, Didier E, et al. Five ways to ensure that models serve society: a manifesto. Nature. 2020. https://doi.org/10.1038/d41586-020-01812-9 .

Download references

Acknowledgements

This research has been supported by The Australian Prevention Partnership Centre, funded by the NHMRC, the Australian Government Department of Health, ACT Health, Cancer Council Australia, NSW Ministry of Health, Wellbeing SA, Tasmanian Department of Health, and VicHealth. It is administered by the Sax Institute.

Author information

Authors and affiliations.

The Australian Prevention Partnership Centre, Sydney, Australia

Victoria Loblay & Louise Freebairn

Youth Mental Health and Technology Team, Brain and Mind Centre, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia

Victoria Loblay, Louise Freebairn & Jo-An Occhipinti

Menzies Centre for Health Policy and Economics, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia

Louise Freebairn

Computer Simulation & Advanced Research Technologies (CSART), Sydney, NSW, Australia

Jo-An Occhipinti

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, VL, LF and JO; methodology, VL; formal analysis, VL, LF and JO; writing—original draft preparation, VL; writing—review and editing, LF and JO; All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Victoria Loblay .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

: Standards for reporting qualitative research.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Cite this article.

Loblay, V., Freebairn, L. & Occhipinti, JA. Conceptualising the value of simulation modelling for public engagement with policy: a critical literature review. Health Res Policy Sys 21 , 123 (2023). https://doi.org/10.1186/s12961-023-01069-4

Download citation

Received : 10 August 2022

Accepted : 04 November 2023

Published : 27 November 2023

DOI : https://doi.org/10.1186/s12961-023-01069-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Participatory modelling
• Complex systems
• Public policy
• Knowledge generation
• Community engagement
• Communication
• Transparency
• Transformational change
• Multiplicity
• Policymaking

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]