literature review wikipedia

News alert: UC Berkeley has announced its next university librarian

Secondary menu

• Log in to your Library account
• Hours and Maps
• Connect from Off Campus
• UC Berkeley Home

Search form

Conducting a literature review: why do a literature review, why do a literature review.

• How To Find "The Literature"
• Found it -- Now What?

Besides the obvious reason for students -- because it is assigned! -- a literature review helps you explore the research that has come before you, to see how your research question has (or has not) already been addressed.

You identify:

• core research in the field
• experts in the subject area
• methodology you may want to use (or avoid)
• gaps in knowledge -- or where your research would fit in

It Also Helps You:

• Publish and share your findings
• Justify requests for grants and other funding
• Identify best practices to inform practice
• Set wider context for a program evaluation
• Compile information to support community organizing

Great brief overview, from NCSU

Want To Know More?

• Next: How To Find "The Literature" >>
• Last Updated: Apr 25, 2024 1:10 PM
• URL: https://guides.lib.berkeley.edu/litreview

• Literature review
• 2 Primary and secondary research
• 5 References

Overview [ edit | edit source ]

A literature review is a synthesis of past academic discussion and research findings.

Primary and secondary research [ edit | edit source ]

Literature reviews can be considered secondary research, which may then lead on to original research (primary research). Sometimes the boundary between primary and secondary research is not clear (e.g., a critical review of previously published ideas and data can lead to the discovery of new ideas).

Types [ edit | edit source ]

Literature reviews can be [1] :

• Scoping (Munn et al., 2018)
• Traditional or narrative
• Systematic (Munn et al., 2018)
• Meta-analytic
• Meta-synthesis

See also [ edit | edit source ]

• Formulating literature reviews (UC-Pharmacy-Research Wikiversity)
• Literature review (Wikipedia)
• Literature review (UC-Pharmacy-Research Wikiversity)

References [ edit | edit source ]

Navigation menu

• University of Texas Libraries

Literature Reviews

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

• Provide a comprehensive and updated review of the literature;
• Explain why this review has taken place;
• Articulate a position or hypothesis;
• Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

• Demonstrate how a study fills a gap in research
• Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

• Organizes or describes a topic
• Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

• It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
• It may be too focused. Some niche studies may miss the bigger picture.
• It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
• It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

• October 26, 2022 recording
• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 18, 2024 11:38 AM
• URL: https://libguides.usc.edu/writingguide

Conducting a Literature Review: Home

What is a literature review.

A literature review is a body of text that aims to review the critical points of current knowledge on a particular topic. Most often associated with science-oriented literature, such as a thesis, the literature review usually proceeds a research proposal, methodology and results section. Its ultimate goals is to bring the reader up to date with current literature on a topic and forms that basis for another goal, such as the justification for future research in the area. (retrieved from http://en.wikipedia.org/wiki/Literature_review )

Print Resources

Many resources on research methodology include a chapter on literature review. Try these titles:

• SAGE Research Methods "SAGE Research Methods (SRM) is a research tool supported by a newly devised taxonomy that links content and methods terms. It provides the most comprehensive picture available today of research methods (quantitative, qualitative and mixed methods)across the social and behavioural sciences."

There are many resources available on the internet and in print to help you conduct a literature review. For graduate students working on a thesis, the most important resource is your graduate committee chair. Also, if you are an ILR student, don't forget to speak with one of Catherwood's reference librarians. They have subject matter expertise and can help you find research materials, as well as show you relevant databases and resources, including Zotero , an online bibliographic management system. Cornell students can also take advantage of the  John S. Knight Institute for Writing in the Disciplines , which offers a walk-in tutoring Service.

Additional Resources for Writing Literature Reviews

• Library Research at Cornell by Michael Engle Last Updated Mar 22, 2024 6824 views this year
• The Literature Review: A Few Tips on Conducting Writing Advice from the University of Toronto Health Sciences Writing Centre.
• Write a Literature Review Research guide from the University Library, UC Santa Cruz
• Learn How to Write a Review of Literature From the Writing Center at University of Wisconsin - Madison.

Searching the Library Catalog

Conducting a guided keyword search from the  Cornell University Library Catalog , using the terms "research methodology" or "qualitative research" will provide additional results, and adding the term "social sciences" to the search will help narrow the results. You can also use this list of the classification numbers for theses by department . We also offer a finding guide to dissertations and theses that you may find useful!

We also recommend you look through other theses - often your graduate chair will have copies.

• Last Updated: Feb 4, 2021 9:22 AM
• URL: https://guides.library.cornell.edu/ilrlitreview

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Protocol for a Systematic Literature Review of Research on the Wikipedia

2009, Proceedings of the International Conference …

Related Papers

Digital Society, 2009. ICDS'09. Third International …

Chitu Okoli

SSRN Electronic Journal

nicolas jullien

Arto Lanamäki , Chitu Okoli , Finn Å Nielsen , Mohamad Mehdi

Journal of the Association for Information Science and Technology

Arto Lanamäki

Dimitrios Xanthidis , Eisa Aleisa

2009 IEEE Symposium on Computers and Communications

Felipe Ortega

to the Special Libraries …

Lisa Kruesi

Eduard Aibar

This issue looks in depth at the multiplicity of the social and cultural impacts of Wikipedia. The articles analyse issues including its development and the consequences for the commercial sector and the public image of large corporations (in the article by Marcia W. DiStaso and Marcus Messner) or its role in the diffusion of culture and architectural heritage (in the article by Emilio José Rodriguez et al.). The article by Antoni Oliver and Salvador Climent details the use of Wikipedia as a structured knowledge corpus, in the framework of the state of the art in natural language processing research. In turn, the article by David Gómez proposes the concept of wikimediasphere and shows how Wikipedia actually forms part of a very dense ecosystem of projects that, though they share common elements, act with a high level of autonomy as nodes on a wider network. Lastly, the article by Nathaniel Tkacz analyses the practical and epistemological implications of one of the basic pillars of Wikipedia's core content policy - the Neutral Point of View - and its relation to a specific concept of truth.

Reinoso Peinado

Dimitrios Xanthidis

This research study aims to investigate the role of Wikipedia in contemporary knowledge production as it is viewed by the various users of such tools in Greece. Academics, scientists, scholars and professionals provide a variety of argumentation on the pros and cons of the role of Wikipedia in contemporary knowledge production some of them rather conflicting to each other. A pre-tested questionnaire was used for data collection. The data were collected at various establishments and a total of about 171 individuals from all walks of life, different backgrounds and age and cultures responded. The study results are clear. Wikipedia is a successful story, regardless of its credibility, despite its authors&#39; anonymity and often lack a significant and/or particular academic or professional merit, but because it is a good start and a quick and easy source of finding free information that either will be confirmed by other more official sources or that can be used by &quot;casual&quot; in...

RELATED PAPERS

Vania Lamilla

Minuten der Weisheit

Radio Science

CHO JUI (RAY) WENG

Sociologias

Daniel Castaño Zapata

Angeles Luengo

Summer School Unila, Marwadewa, Vern, & Zagreb Univ.

Fuad Abdulgani

Ranju Singh

Journal of magnetic resonance

Jukka Jokisaari

Fórum Linguístico

Pedro Navarro

Mark Feigenson

Environmental Research

Gerd Sällsten

IMF Working Papers

Suchanan Tambunlertchai

Learning Community : Jurnal Pendidikan Luar Sekolah

WIWIN HERWINA

Journal of Investigative Dermatology

African Journal of Science, Technology, Innovation and Development

alem mezgebo hailu

Sam Thiagalingam

Cor et Vasa

Jan Janousek

Physica Medica

Gerard Den Heeten

Alioune Aidara Diouf

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• Introduction
• Welcome to Advanced Writing!
• Author Biographies
• Acknowledgments
• UNIT 1. WRITING TOOLS
• 1. What's Advanced About Advanced Writing?
• 2. Writing Tools
• 3. Writing Processes
• 4. Grammar & Mechanics
• UNIT 2. ACADEMIC AUDIENCES
• 7. Writing for Academic Audiences
• 8. Finding & Evaluating Sources
• 9. Discussing & Citing Sources
• 10. Defining Literature Reviews
• 11. Planning Literature Reviews
• 12. Writing Literature Reviews
• 13. Crafting Proposals
• UNIT 3. GENERAL AUDIENCES
• 14. Writing for General Audiences
• 15. Applying for Jobs & Graduate Schools
• 16. Creating Public Texts
• 17. Presenting
• Translations

Defining Literature Reviews

Choose a sign-in option.

Tools and Settings

Questions and Tasks

Citation and Embed Code

Learning Outcomes

In this chapter, you'll

• learn the definition of a literature review
• understand the difference between a literature review and a traditional argumentative research paper
• learn how literature reviews are used in the scholarly research process
• understand the purpose of a literature review

10.1 What's a Literature Review?

To answer this question, I have to tell you a story, so stick with me.

The straightforward answer is that a literature review  is a review or synthesis of all the research published on a certain topic. But I'd rather explain it from a skateboarder's perspective:

One of my favorite movie series is the 1980s classic Back to the Future trilogy where the bodacious skater Marty McFly time-travels to the future and sees the coolest thing I could imagine: a hoverboard. As a kid, I was a dabbling skateboarder at the time I saw this movie and thought if I could just have one of those rad, flying hoverboards, all my troubles would disappear. It was an optimistic time.

Trouble is, hoverboards are really hard to make. We've already passed the year 2015 when the "Future" of Back to the Future Part 2 takes place, and guess what? No hoverboards. I know what you're thinking: you've seen a skateboard-like, two-wheeled device marketed with the name "Hoverboard" that teenagers ride around parks, sidewalks, and their parents' basements. But you're wrong: that's just an electric, no-handled scooter that occasionally catches on fire and burns people's houses down. I'm not satisfied.

I want a real hoverboard. That you ride in the air . So how can we know when real hoverboards will be available? How can we know where the technology is now? Will we know a real hoverboard when we see one? Tony Hawk, the best skateboarder of all time (whose face was incidentally taped to my wall in the '80s) recently filmed a 2-minute video of how far real hoverboard technology has come—filmed on the very day Marty McFly supposedly went to the future: October 21, 2015:

Tony Hawk and the cutting edge of hoverboard research

Although this "hoverboard" was really a huge black rectangle the size of Delaware floating only an inch off the ground, and although Tony Hawk fell off a lot, he was technically in the air, so I'm taking that as a good sign. Then recently, a professional jet ski rider broke the world record for longest time "hovering" in the air with a highly dangerous jet-engine-propelled contraption called Flyboard Air. It's also definitely a step in the right direction, but there's a big problem (beside extreme danger): it's projected to cost around $250,000. Sadly, not in my price range.

The good news is, now we've found the point where hoverboard research actually is. The bad news: we have to face the sad truth that it might still be a while before we get real Marty-McFly-approved flying hoverboards that non-billionaires can ride. But at least now we know. Because knowing is half the battle.

The State of a Field on a Topic

That leads me to literature reviews. Whenever you want to know the state of a field of research like how far hoverboard technology has come, the best way to find out is probably not YouTube videos. It turns out you can do something much more reliable: conduct a literature review . In this case, "literature" doesn't mean the Victorian novels you read in English class, it means all the research published on a certain topic. So a literature review is simply a review or a synthesis of the research published on a topic.

Researchers and companies today don't just start projects out of the blue—they do their homework first by finding out what others have already done. So if you want to make a hoverboard, you don't just go to Home Depot and buy random parts—you research what already exists and talk to the experts so you don't have to reinvent the wheel (get it?).

"If I have seen further, it is by standing on the shoulders of giants." —Sir Isaac Newton (and the motto of Google Scholar)

Before good researchers set up any surveys or experiments, or even write a proposal for funding, they figure out exactly which research questions have already been asked and answered. Same goes for anyone wanting to make a product that will sell. But more importantly, they look for the gaps  in the research or marketplace where answers have yet to be found. And then they focus their own work on filling in some of those gaps. That'll be your job, too.

In other words, the goal of a literature review is to find the sweet spot where the most promising research is happening now—we call that the cutting edge. ( The Cutting Edge is also a cheesy '90s movie about ice skating, so be sure not to get them confused.) As Marty McFly would say, finding the cutting edge is pretty gnarly because that's where the adventure begins. So grab the closest thing you have to a hoverboard, and let's go.

10.2 How is a Literature Review different from a typical Research Paper?

*Suggestion: Just for fun, as you read this next section, set the mood by playing one of these songs called "Old Friends." You'll see how it's relevant in a minute.

Simon & Garfunkel "Old Friends" —old-school classic

Ben Rector "Old Friends" —w/ high school band Euromart, in the garage of the house he grew up in

Willie Nelson, Roger Miller, Ray Price “Old Friends” —country Western style

You've probably been writing research papers most of your life—you know, starting from the five-paragraph essay you learned in high school to the term paper you wrote last semester that had a thesis statement and lots of quotes. So it can seem daunting to switch gears to writing a literature review, but there are some distinct advantages to making the switch (that I'll get to later). The trick is first understanding the difference between the two.

Research Papers are Thesis Driven

The difference between a typical research paper and a literature review is your purpose and strategy. Tell me if this sounds familiar: when you're assigned to write a research paper, you start with a thesis or argument that you'd like to make. Your thesis has to do with a new way to look at something or a change you'd like to see in the future. Then you search for sources that support your point. You might adjust your thesis if you come across sources that challenge your claim, but generally, the sources you've gathered become evidence for your thesis and you use them to support your point. In other words, your argumentative research paper is driven by your thesis .

Literature Reviews are Source Driven

In contrast, when you write a Literature Review, the sources themselves dictate what you'll say in your paper. Remember, your goal is to tell your audience the state of the field on a topic—what's been happening in the published research—so you can find the cutting edge and where the research gaps are. Therefore, you need to find and evaluate the most relevant sources surrounding a topic and then write a review based on what you find . You can't decide on a thesis statement or know what points you'll make before you start because you have to find out what researchers are doing before you can report on that. Simply put, your literature review is driven by your sources .

You'll still have an overarching point/thesis that controls your literature review paper structure, but it will be a claim about what patterns you found in the research— not an argument about a change you want to see in the future or a new way to look at something. And you'll decide on your thesis much later in the writing process. Here's a table that compares the writing process of a traditional research paper with that of a literature review:

Literature Reviews: Catching up with Old Friends

Here's where the song comes in—another way to think about the difference between literature reviews and the traditional research papers you're used to writing (with props to Professor David Taylor from the University of Maryland ). What do you do when you meet an old friend? You ask,

"How are you? What have you been up to? Fill me in!"

Literature Reviews are like getting filled in by an old friend . Only this time, you're explaining how a field of research has gotten to the present (like how far hoverboard technology has come). But like a conversation with an old friend, you want to review only the details most relevant to the situation. You don't usually give a moment-by-moment chronology of what you've done in your life (no one has time for that); rather, you talk in terms of categories—work, family, travel, etc. This is like the synthesis that happens in a Literature Review. As you read sources about a specific topic, you'll look for themes, for similarities and differences, for points of agreement and disagreement, for gaps in the research that haven't been filled in yet. Those themes become the categories you'll talk about in your literature review so your audience will understand the big picture about your topic.

Why Literature Reviews?

There are some huge advantages to trying out this new genre of a literature review as opposed to a traditional research paper. We learned in Chapter 9 Talking About Sources that most college students do not engage with their sources—most students merely find quotes to fit their argument and plug them into their papers without actually understanding—or perhaps even reading—their sources. Many even quote directly from the abstract, not even bothering to read past there. Even more telling is that in the Howard and Jamieson study mentioned in Chapter 9, the number of students who summarized their sources in their papers was only 6%. That's probably because summary is a higher-level task that requires actually understanding the sources instead of merely extracting one quote at a time. And without summary, you can't have synthesis—the cornerstone of a good literature review. It's as if students are spending a lot of time exercising their research muscles, but have completely missed their most essential summary and synthesis muscles—their core, if you will.

Because of findings like this, many instructors are now assigning literature review papers rather than traditional research papers in order to isolate those core muscles that need exercise the most—the summarizing and synthesizing muscles. Because Literature Reviews are source-driven and require understanding and summarizing many sources, they are the perfect assignment for strengthening the missing skills in your writing repertoire. We've found that when our students write literature reviews, they learn (sometimes for the first time) the true steps of the research process; how to read, understand, and summarize sources; and even more importantly, they can finally see the big picture and synthesize those sources to understand the state of a field.

And by strengthening your ability to evaluate and synthesize sources, you'll actually strengthen your other writing skills as well, including your traditional research paper writing skills. You'll find after you focus on the literature review that finding and evaluating any type of source will be much easier—just like strengthening your core muscles will help you with all types of exercise.

This might sound crazy, but I actually get emails every semester from former students thanking me for assigning them a literature review because it's helped them with research projects in their next classes or jobs. You, too, can get rock hard research abs by learning how to write a literature review!

10.3 Literature Reviews in the Real World

In case you're not convinced enough to take on the task of writing a literature review, I thought I'd mention that literature reviews come up in the real world in many different venues that you'll encounter in your schooling and career. You'll see that they often follow similar patterns and purposes:

Grant Proposals

Any grant proposal submitted to request research funding begins with an extensive literature review to justify the need for the research funds. If you can prove there's a gap in knowledge, it makes it that much easier to convince your audience to give you funding to fill that gap. More on that in Chapter 12: Proposals .

IMRAD Articles

IMRAD (pronounced "im-rad") stands for Introduction, Methods, Results, and Discussion and is the most common genre published in the social sciences and sciences. Most of the sources you gather will likely be IMRAD-format papers or will at least begin with an introduction. The "I" in IMRAD stands for Introduction and usually consists of a review of the literature on the authors' research topic. The author(s) usually use the Introduction section to report on the published literature about their research topic and reveal the trends and gaps in current research. An added benefit to beginning an article this way is that by showing the gaps in the research, the author(s) can justify their own research and explain the significance of the topic they chose to examine. Clever!

It's kind of like a superhero moment when someone publishes that they have a problem (e.g., turns on the Bat Signal or yells "Help!"), and Batman or Superman or Wonder Woman swoops in and saves the day. The authors lay bare a problem or gap in current research, and then they reveal the research they did to solve that problem, fill that gap. Ta da! Research saves the day. 

As you might guess, the sections following the Introduction (Methods, Results, Discussion) describe the primary research the author(s) conducted to answer their research question. First they report on their quantitative and/or qualitative M ethods (M in IMRAD) including statistical analyses. Then they publish their R esults (R in IMRAD). Finally, the author(s) embark on a D iscussion (D in IMRAD) of their results in the context of the greater field of research and make suggestions for future research. This starts the research cycle over again as someone else reads their article as part of their own review of the literature and discovers a gap in the research that can be filled by new primary research. 

Published Literature Reviews

In the world of academia, literature reviews can also be published on their own. For example, if someone does an extensive  investigation into an important topic, the publishers of academic journals will often publish that literature review on its own to help other researchers understand that topic better. 

Market Analyses

Popular literature reviews.

Lest you think nerdy academics and business gurus are the only ones who rely on literature reviews, recall the last time you went on Wikipedia . If you think about it, Wikipedia is really just a giant Literature Review on millions of topics. Although the information on Wikipedia is not formally peer-reviewed like the reviews published in academic journals, they do cite all their sources and frequently revise to keep the information current. Clearly there's a market for relevant information. If you really want your mind to explode Inception -style, look up "Wikipedia" on Wikipedia and you'll find a literature review about a literature review. Crazy.

All this is a long way of saying that literature reviews are a part of life. And they're a huge part of any scholarly discipline, any successful business, and even our basic human desire for knowledge. So if you're ready to learn how to tackle your own literature review, let's talk about the first steps in Chapter 11 How to Plan a Literature Review .

Brigham Young University

Cristie Cowles Charles teaches writing and literature courses at Brigham Young University in Provo, Utah. She enjoys sparking a love for writing in her students--or at least a love for having had written (it's always worth it in the end, right?). She thinks pumpkin pie counts as a vegetable, is married to a super hot mechanical engineering and neuroscience professor (yes, they exist), and adores her five magnificent children.

This content is provided to you freely by BYU Open Learning Network.

Access it online or download it at https://open.byu.edu/advancedwriting/literature_review .

• PRO Courses Guides New Tech Help Pro Expert Videos About wikiHow Pro Upgrade Sign In
• EDIT Edit this Article
• EXPLORE Tech Help Pro About Us Random Article Quizzes Request a New Article Community Dashboard This Or That Game Popular Categories Arts and Entertainment Artwork Books Movies Computers and Electronics Computers Phone Skills Technology Hacks Health Men's Health Mental Health Women's Health Relationships Dating Love Relationship Issues Hobbies and Crafts Crafts Drawing Games Education & Communication Communication Skills Personal Development Studying Personal Care and Style Fashion Hair Care Personal Hygiene Youth Personal Care School Stuff Dating All Categories Arts and Entertainment Finance and Business Home and Garden Relationship Quizzes Cars & Other Vehicles Food and Entertaining Personal Care and Style Sports and Fitness Computers and Electronics Health Pets and Animals Travel Education & Communication Hobbies and Crafts Philosophy and Religion Work World Family Life Holidays and Traditions Relationships Youth
• Browse Articles
• Learn Something New
• Quizzes Hot
• This Or That Game
• Train Your Brain
• Explore More
• Support wikiHow
• About wikiHow
• Log in / Sign up
• Education and Communications
• College University and Postgraduate
• Academic Writing

How to Do a Literature Review

Last Updated: March 27, 2023 Fact Checked

This article was co-authored by Alexander Ruiz, M.Ed. . Alexander Ruiz is an Educational Consultant and the Educational Director of Link Educational Institute, a tutoring business based in Claremont, California that provides customizable educational plans, subject and test prep tutoring, and college application consulting. With over a decade and a half of experience in the education industry, Alexander coaches students to increase their self-awareness and emotional intelligence while achieving skills and the goal of achieving skills and higher education. He holds a BA in Psychology from Florida International University and an MA in Education from Georgia Southern University. There are 13 references cited in this article, which can be found at the bottom of the page. This article has been fact-checked, ensuring the accuracy of any cited facts and confirming the authority of its sources. This article has been viewed 320,983 times.

Some people might think of a literature review as reading a book and then giving it a thumbs up or thumbs down. Nope, not so. A literature review is a review of various pieces of literature on one topic, ranging from series of books to shorter pieces like pamphlets. Sometimes, the literary review is a part of a larger research paper. Its purpose is to prevent duplication of efforts, resolve conflicts, and point the way for further research.

Before Writing

• How many sources should you include? Does he/she want a specific number of each type? Do they have to be at least semi-current?
• In discussing your themes, are you just summarizing or critiquing? Some reviews require a thesis, some may not.
• Should you offer your opinion on your sources?
• Do you need to provide background information, such as definitions or histories, to aid in your audience's understanding?
• Is there a page or word requirement?

• Get current. If you are writing a review in the humanities, history, or social sciences, you can afford to be less concerned with timing (in fact, changing opinions throughout history may be an aspect of your paper). But if you are writing a literary review for the sciences, say, on treating diabetes, information from 5 years ago could already be obsolete. Sort through current bibliographies or literature reviews in the field to get a sense of what your discipline expects. [2] X Trustworthy Source University of North Carolina Writing Center UNC's on-campus and online instructional service that provides assistance to students, faculty, and others during the writing process Go to source

• Read between the lines. You're not necessarily looking for explicit content. Is there an aspect of the field that is missing? Are your sources all prescribing to one specific theory? Do you see trends being revealed? This will help you structure your paper immensely, zeroing in on what will give your paper purpose.

• For example, "Current trends in [topic] are A, B, and C," or "The X Theory is assumed by most sources from 1985 on." Stating something like this begs a few questions, making your review more interesting and meaningful: How will trends change in the future? What if the assumed theories are wrong?
• Again, this is not new information. You are not analyzing the material and coming up with your own, fresh perspective on it. You are simply acting like a computer--noting patterns, holes, and assumptions all your sources are taking.

• What are the author's credentials? How are their arguments supported (narratives, statistics, historical findings, etc.)?
• Is the author's perspective unbiased and objective? Are they ignoring any data to make their points seem stronger?
• How persuasive are they? Do any of their points leave a bit to be desired?
• Does their work lead to a greater understanding of the subject? [5] X Research source

Constructing Your Paper

• Help the reader along by letting them know what kind of ride they're in for. If you are employing a thesis statement, place it toward the end of your introductory paragraph. At the end, your reader should be anticipating getting into the evidence and bulk of your paper.

• Arrange it chronologically. If you are dealing with varying opinions by era or changing trends over time, chronological organization may make the most sense.
• Arrange it by publication. This organizational method fares well if each publication has a different stance. If there is a natural progression (radical to conservative, for example) between the sources, this works swimmingly.
• Arrange it by trend. If you are noticing patterns in your sources, arranging them by the trends they suggest may be the most obvious structure. Certain sources may, together, suggest one pattern that shifts over time, region, or other variable.
• Arrange it thematically. This highly depends on your thesis statement and what sources you have chosen. If you are choosing a focus that is more abstract ("Colonialism is depicted as evil," for example), the subsections may be arranged on the different methods employed to put the theme across.

• You may make your conclusion suggestive. Where might the discussion proceed if someone else picked it up where you left off? What are the consequences of the patterns and holes in today's sources?

• However, use quotes sparingly. The survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. [10] X Trustworthy Source University of North Carolina Writing Center UNC's on-campus and online instructional service that provides assistance to students, faculty, and others during the writing process Go to source Some short ones are fine, sure, but all in all, it should be written by you.

• When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. [12] X Trustworthy Source University of North Carolina Writing Center UNC's on-campus and online instructional service that provides assistance to students, faculty, and others during the writing process Go to source Then, relate it to the context of your review.
• Some professors may require that you evaluate the sources and conclude which pieces add the greatest contribution to the field. If yours is keen on this, determine your take in the introduction and string it throughout your paper.

Revising Your Work

• Does your instructor require APA formatting? What should your margins be? Headers, footers, footnotes, and page numbers? How do they want your name, headings, and subheadings? How do they want your works cited page?

• With everything said as clear as day, does it flow together? Do you transition well not only from paragraph to paragraph, but from sentence to sentence? Be sure your evidence lines up with the support and your arrangement of sources flows logically.
• Eliminate useless jargon or slang. You may have grown an entirely new vocabulary during this endeavor, but your professor has not. Write a paper that can be read by the masses. Don't make it overly esoteric.

• It's best to have someone else go over your work, too. You may have read it so many times you can no longer see you lapsed into Portuguese absent-mindedly. A different set of eyes can locate mistakes you may not have seen, ask questions you didn't realize were left unaddressed, or seek clarification on the foggier points.

Community Q&A

• Use correct citations. Your assignment will likely specify what kinds of formats you should use for citations within the text. Often, professors are looking for strict use of these formats as part of the grade. Thanks Helpful 0 Not Helpful 0
• Outline your literature review. It will help you order your thoughts into an organized presentation, making the paper ultimately easier to write. Thanks Helpful 0 Not Helpful 0

• Avoid plagiarism. Using your own words will help you avoid plagiarism, which many academic departments take very seriously. Evidence of plagiarism can get students suspended or otherwise disciplined (this includes a failing paper). Make sure to correctly attribute any direct quotes. Thanks Helpful 14 Not Helpful 3

You Might Also Like

• ↑ https://guides.lib.uoguelph.ca/c.php?g=130964&p=5000948
• ↑ 2.0 2.1 2.2 2.3 http://writingcenter.unc.edu/handouts/literature-reviews/
• ↑ https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003149
• ↑ https://library.concordia.ca/help/writing/literature-review.php
• ↑ http://guides.library.ucsc.edu/content.php?pid=364099&sid=2979684
• ↑ https://owl.purdue.edu/owl/research_and_citation/conducting_research/writing_a_literature_review.html
• ↑ https://www.ed.ac.uk/institute-academic-development/study-hub/learning-resources/literature-review
• ↑ https://www.unr.edu/writing-speaking-center/student-resources/writing-speaking-resources/editing-and-proofreading-techniques
• How to Conduct Academic Research
• How to Create a Successful Project (for School)
• How to Begin Writing a Research Paper
• How to Be More Analytical
• How to Find Primary Source Documents
• How to Complete a Project on Time

About This Article

To do a literature review, start by finding a variety of reliable sources that all relate to one topic or theme. Then, read through the sources and come up with a thesis statement for your paper. Once you have your thesis, explain how the sources you used back up your thesis in the body of your literature review. You can arrange the sources chronologically, by publication, or even thematically. For help writing an introduction and conclusion for a literature review, keep reading! Did this summary help you? Yes No

• Send fan mail to authors

Reader Success Stories

Maryam Lawal

Nov 12, 2016

Did this article help you?

Jul 7, 2016

Galal Sheher

Oct 22, 2016

Alljey Gimpes

Jun 15, 2016

Jvert Smith

Oct 4, 2018

Featured Articles

Trending Articles

Watch Articles

• Terms of Use
• Privacy Policy
• Do Not Sell or Share My Info
• Not Selling Info

wikiHow Tech Help Pro:

Develop the tech skills you need for work and life

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• References that anyone...

References that anyone can edit: review of Wikipedia citations in peer reviewed health science literature

• Related content
• Peer review
• M Dylan Bould , staff anesthesiologist 1 ,
• Emily S Hladkowicz , research assistant 2 ,
• Ashlee-Ann E Pigford , research assistant 2 ,
• Lee-Anne Ufholz , director 3 ,
• Tatyana Postonogova , research associate 4 ,
• Eunkyung Shin , research assistant 5 ,
• Sylvain Boet , staff anesthesiologist 6
• 1 Department of Anesthesiology, Children’s Hospital of Eastern Ontario, University of Ottawa, 401 Smyth Road, Ottawa, ON, Canada, K1H 8L1
• 2 Department of Anesthesiology, Ottawa Hospital Research Institute, Ottawa
• 3 Health Sciences Library, University of Ottawa, Ottawa
• 4 Allan Waters Family Simulation Centre, Li Ka Shing Knowledge Institute, St Michael’s Hospital, Toronto, ON, Canada
• 5 Department of Surgery, St Michael’s Hospital, University of Toronto, Toronto
• 6 Department of Anesthesiology, The Ottawa Hospital, University of Ottawa, Ottawa
• Correspondence to: M D Bould  dbould{at}cheo.on.ca
• Accepted 10 February 2014

Objectives To examine indexed health science journals to evaluate the prevalence of Wikipedia citations, identify the journals that publish articles with Wikipedia citations, and determine how Wikipedia is being cited.

Design Bibliometric analysis.

Study selection Publications in the English language that included citations to Wikipedia were retrieved using the online databases Scopus and Web of Science.

Data sources To identify health science journals, results were refined using Ulrich’s database, selecting for citations from journals indexed in Medline, PubMed, or Embase. Using Thomson Reuters Journal Citation Reports, 2011 impact factors were collected for all journals included in the search.

Data extraction Resulting citations were thematically coded, and descriptive statistics were calculated.

Results 1433 full text articles from 1008 journals indexed in Medline, PubMed, or Embase with 2049 Wikipedia citations were accessed. The frequency of Wikipedia citations has increased over time; most citations occurred after December 2010. More than half of the citations were coded as definitions (n=648; 31.6%) or descriptions (n=482; 23.5%). Citations were not limited to journals with a low or no impact factor; the search found Wikipedia citations in many journals with high impact factors.

Conclusions Many publications are citing information from a tertiary source that can be edited by anyone, although permanent, evidence based sources are available. We encourage journal editors and reviewers to use caution when publishing articles that cite Wikipedia.

Introduction

Launched on 15 January 2001, Wikipedia is self described as “a free, collaboratively edited, and multilingual internet encyclopedia supported by the non-profit Wikimedia Foundation.” 1 As of 2012 Wikipedia is the largest online reference site, 2 3 and it is reported to be the most used online healthcare resource globally. 4 However, the assessment of Wikipedia as a credible source for information has been debated since its origin. 5 The impermanent nature of Wikipedia entries and concerns about quality have been raised as important matters. 6

The literature on Wikipedia has concentrated on evaluating content and ensuring that users have access to appropriate information. 5 Only 13% of Wikipedia articles had identifiable errors when assessed by academics. 7 Giles and colleagues found that the number of factual errors, omissions, or misleading statements in Wikipedia articles was comparable to the Encyclopaedia Britannica. 8 In general, Wikipedia articles reference academic literature, 9 10 and medical articles are overseen by WikiProject Medicine, which is driven by editorial oversight. 2 4 Several studies have confirmed the accuracy of health specific Wikipedia articles and discuss the potential value in education of patients. 2 3 8 11 12 13 14 15 Wikipedia has offered an innovative way to provide free access to information for people around the world. 2 4 5 To further improve the quality of these articles, medical professionals are being encouraged to contribute to Wikipedia. 2 16 17 18 Studies have also examined groups that use Wikipedia, noting that, in addition to patients and nursing students, 2 3 10 19 medical students and residents commonly access Wikipedia to acquire health information. 20 21 22 23 Although physicians have been discouraged from relying on Wikipedia, 16 one study showed that in practice use of Wikipedia is as high as 70% among junior physicians. 22 In general, consultation of Wikipedia is a growing trend among academics. For example, among more than 1000 authors in Nature , 17% reported that they consulted Wikipedia on a weekly basis. 8 As the public and healthcare professionals increasingly accept Wikipedia as a source of information, new questions emerge such as the appropriateness of citing Wikipedia in academic publications.

The controversial nature of citing Wikipedia as a reference source for academic information is threefold. Firstly, a theoretical concern exists that anyone with access to the internet can alter Wikipedia. This raises questions about the spread of unintentional misinformation, which is fuelled by acts of intentional vandalism that have gained media attention and have challenged the public’s perception of Wikipedia. 24 25 26 However, the use of a wiki model addresses the fear that that the information on Wikipedia is not guaranteed to be correct. The concept of a wiki is based on the assumption that the majority is correct: if someone writes an erroneous statement on Wikipedia, the probability is considered to be high that someone else from the majority that is presumed to be correct will identify the error. This model has many advantages and allows Wikipedia to be a free source of a vast amount of collaborative information. 2 4 5 However, the scientific community still has concerns about the academic integrity of a model that in theory could be edited by anyone.

Secondly, the changing nature of Wikipedia makes permanent versions difficult to access. Although Wikipedia maintains a detailed history of previous Wikipedia pages, current academic citation systems rarely require a citation to include time stamped access information detailed to the second. As Wikipedia pages are in constant change, unlike paper encyclopaedias in which readers can confidently find the permanent cited source, readers might find confirming that the Wikipedia reference is the exact version cited challenging, particularly if detailed time stamps are not included. 27

Thirdly, citing tertiary sources such as Wikipedia, which are resources that compile or provide digests of secondary sources, has literary problems. 28 Secondary sources are books, articles, or unpublished literature that provide an interpretation of primary sources, which include original data, manuscripts, records, or documents. International guidelines state that authors should provide direct references to original research sources, 29 so citing Wikipedia or any other tertiary source in the academic literature opposes literary practice. To date, only one study has examined the frequency of citation of Wikipedia in the academic literature. 30 To our knowledge, no one has examined the frequency of Wikipedia citations in the health science literature and the ways in which Wikipedia citations are used. We aimed to evaluate the prevalence of Wikipedia citations in indexed health science journals, identify the health science journals that publish articles with Wikipedia citations, and determine how Wikipedia is being cited.

Data collection

In February 2012 one investigator (LU) searched the ISI Web of Science and Scopus online databases to identify articles in the peer reviewed health science literature that had directly cited Wikipedia since its inception in 2001. She searched references of all articles in both databases on the same day for the word “Wikipedia” or any possible derivation to account for spelling errors. The word could appear anywhere in the reference, so not all articles necessarily referenced Wikipedia but could be articles about Wikipedia. To focus on indexed health science literature, we refined the results by using Ulrich’s Database to identify articles from journals indexed in at least one of Medline, PubMed, or Embase. We retrieved full text for the articles (including reviews, original research, editorials, letters to the editor, and case reports) identified in the search through the University of Ottawa library and associated Canadian inter-library loan services. We excluded citations when articles were not written in English or when full text was unavailable. Although citations originated in the health science literature, the topic of the citation could extend beyond health, so we included all Wikipedia pages that were cited in the academic literature. In the event that an article cited Wikipedia more than once, we considered all citations and created duplicate entries for that article.

We used Thomson Reuters Journal Citation Reports (JCR) to collected journals’ 2011 impact factors (a measure that reflects how frequently the average article in a journal has been cited in a particular period). 31 32 JCR impact factors provide quantitative evidence about the position of one journal in relation to the competition and offer an approximation of the prestige of a journal. When interpreting the impact factor, one should consider only journals within the same subject category as the scores are relative. 32 JCR coverage includes the world’s leading journals and offers a systematic, quantifiable means to critically evaluate them by measuring the influence and impact of research at the journal and category levels, as well as showing the relation between citing and cited journals. We did not recover other journal metrics because of the unique nature of JCR and its pre-eminence as a proxy to measure a journal’s importance within a field.

Data extraction/coding system

Using an iterative process, we developed a descriptive coding strategy. Three investigators (MDB, SB, ESH) reached consensus on the distinction between categories. After independently reviewing 15% of citations, the three investigators met as a team to discuss their coding strategies. They developed an initial coding strategy and recoded citations with the new guide. The team continued this iterative process of coding, meeting, and refining the guide before the three authors reached agreement and the coding strategy was finalised. Emergent sub-codes were integrated into the coding system as they arose. Using the coding system, one investigator (ESH) systematically coded all Wikipedia citations. To ensure coding reliability, another investigator (AEP) coded 100 random citations with the same coding system.

We used SPSS 17.0 for statistical analysis. We calculated descriptive statistics for the frequency of Wikipedia citations by thematic code, the impact factor of journals citing Wikipedia (median, range), and the frequency of these statistics by year. We used the intra-class correlation coefficient to assess the inter-rater reliability.

We recovered 2359 publications (2307 from Scopus and an additional 52 from Web of Science) in our search. After excluding duplicates, non-English journals, and articles that did not directly cite Wikipedia, we retrieved full text for 1433 articles from 1008 indexed journals, with 2049 Wikipedia citations. In total, 2011 impact factors were available for 1420 citations in 980 articles from 650 indexed journals and were found in the JCR database.

Table 1 ⇓ shows the frequency of Wikipedia citations since 2001, the nature of citations (thematic code), and related impact factor information (median and range). The intra-class correlation coefficient for the two coders was 0.91 (P<0.001), indicating a high degree of inter-rater reliability. More than half of the citations were coded as definitions (n=648; 31.6%) or descriptions (n=482; 23.5%). For the purpose of this study, of the 13 categories, we considered citations (n=82; 4.0%) from only two categories (Citations about Wikipedia, and Wikipedia used in methods) to be the most appropriate uses of Wikipedia, as in these cases Wikipedia was the original source of information. Furthermore, we recovered 97 (4.8%) citations in which Wikipedia was cited in place of an original research study.

 Frequency of Wikipedia citations since 2001, categorised by code and impact factor

• View inline

The median impact factor of journals citing Wikipedia was 2.0 and has remained fairly consistent over time. However, the total number of Wikipedia citations has increased each year since 2004 except for between 2009 and 2010. The figure ⇓ shows the number of articles that cite Wikipedia at least once by year up to 21 November 2013. Consistently since the inception of Wikipedia, journals with high impact factors have continued to publish references to Wikipedia. Table 2 ⇓ illustrates the journals with the top 25 highest impact factors that emerged in our study. These journals accounted for 2.2% (n=3) of cases in which Wikipedia was the most appropriate citations (as previously defined). Table 3 ⇓ specifies the titles and types of articles published in these journals, including the number of Wikipedia citations in each article. Journals with available impact factors that cited Wikipedia more than 10 times included Explore: The Journal of Science and Healing (n=53), Child’s Nervous System (n=26), BMC Bioinformatics (n=21), Theoretical Biology and Medical Modelling (n=20), Journal of Medical Internet Research (n=17), Health Information and Libraries Journal (n=13), BMJ (n=13), JALA—Journal of the Association for Laboratory Automation (n=12), and American Journal of Forensic Medicine and Pathology (n=11).

Number of articles that cite Wikipedia at least once by year since 2001. *Data from February 2012 to November 2013 were not included in detailed analysis

• Download figure
• Open in new tab
• Download powerpoint

 25 journals with highest impact factors that have cited Wikipedia since 2001

 25 journals with highest impact factors: article titles, types, and number of Wikipedia citations

Our findings illustrate a relatively small but increasing frequency of citations of Wikipedia in indexed health sciences literature. We retrieved 2049 Wikipedia citations, in 1433 articles from 1008 journals indexed in Medline, PubMed, or Embase. Most of these citations occurred after December 2010 and were consistently found in journals with low impact factors and journals without impact factors, as well as in journals with high impact factors, including Nature , Science , and the BMJ . Using the descriptive coding strategy, we found a wide variety of uses for Wikipedia citations in the literature, the most common being definitions and descriptive statements.

Strengths and limitations

As the first study to describe the citation practices of Wikipedia (how, where, when, and so on) in indexed health science literature, our study adds insight into the role that Wikipedia may play in academic literature. We also provide information about the quality of the journals that cite Wikipedia, measured through impact factors from the Journal Citation Reports. The study searched citations from two well recognised databases (Scopus and Web of Science) and included all English language journals available in full text at Canadian university libraries, so some citations may have been excluded from the search because of the search criteria. However, the study used two comprehensive databases that probably captured most citations from journals with reaching influence as well as high impact factors. Although we report an increasing number of Wikipedia citations, the health science literature is expansive; therefore, papers that have cited Wikipedia remain a small minority of all published papers, albeit with increasing frequency.

Comparison with other studies

Active research on Wikipedia has examined the role of academic citations in Wikipedia, content validity, and interactions between Wikipedia and user groups. 5 The only other study to examine Wikipedia in the scholarly references was completed by Park in 2011. 30 In this bibliometric analysis, Park also used the ISI Web of Science and Scopus databases to identify the total number of studies (n=1746), as well as leading authors, their institutional affiliations, most frequent publication sources, main academic fields, and other statistics on the frequency with which scholarly articles cite Wikipedia. Our study took Park’s analysis further. Firstly, we used Ulrich’s database to refine search findings to be specific to health sciences literature. Secondly, we examined how Wikipedia is being cited. Both studies show a general increase in frequency of citations over time, suggesting that if Park’s data were to extend beyond 2010 they would continue to show increasing Wikipedia citations. While Park identified the publications that cited Wikipedia most often, our study also provides impact factors, adding information about journals’ impact.

Contextualisation and policy implications

Just because more researchers are citing Wikipedia does not necessarily justify it as a valid source of information for citation. Wikipedia itself has cautioned against using Wikipedia as a source, 33 and some universities have gone so far as to ban students from citing Wikipedia. 34 Recognising that learning modalities are changing and evolving increasingly towards online and e-resources, 21 23 35 36 we believe that ensuring that peer reviewed academic literature aligns with the International Committee of Medical Journal Editors (ICMJE) guidelines is still important. Relevant to this study, the ICMJE guidelines state that “Readers should therefore be provided with direct references to original research sources whenever possible.” 29 Although people have been argued that Wikipedia is comparable to an encyclopedia in terms of accuracy, 8 37 this argument overlooks the literary problems associated with citing a tertiary source intended to direct researchers to an appropriate reference. 34 We echo the findings of other researchers who believe that Wikipedia should not be cited when a more authoritative—that is, primary, permanent, peer reviewed, evidence based—source exists 6 ; however, Wikipedia may be the most appropriate source to cite for a definition of Wikipedia and in situations in which Wikipedia is used as part of the scientific methods (for example, a search strategy). Outside of those rare instances, arguing that citation of Wikipedia in the academic literature is appropriate is difficult. Our “original research” category further illustrates how Wikipedia was often cited when the authors could have cited an original research study. For example, one article stated that “Some researchers propose that vitamin D supplementation may be beneficial in the treatment and prevention of some types of cancer,” 38 and rather than citing the original sources provided on the Wikipedia page, the author instead cited Wikipedia.

Evidence based medicine requires that clinical decisions are made using professional expertise in conjunction with replicable evidence from systematic research. 39 As Wikipedia entries are constantly changing, they may be difficult to access in the form that was originally cited by the time the work is published, reducing the reader’s ability to reproduce the author’s process. 27 In a study by Peoples, 6 most citations to Wikipedia did not reference the date and time at which the entry was visited. Although Wikipedia does have a history function that allows users to access previous versions of the page, this function may not be obvious to most users, and determining which version was accessed for citations that do not denote the date and time of the visit can be difficult. If Wikipedia is to be cited in the future, not only the date accessed but also the time should be included so that future researchers can access the entry as it was originally viewed. Thus, the process of peer review should include close attention to the references that appear in articles before recommending them for publication and potentially approving information that contradicts evidence based research.

Future directions

Although the proportion of retrieved Wikipedia citations in the indexed health science literature was relatively small, the increasing trend of citation is still important to note. The study of how Wikipedia is used in the academic literature is relatively young, and beginning an open discourse around the appropriateness of using non-permanent Wikipedia citations in the academic literature seems timely. The relationship between academic publication and Wikipedia remains largely understudied, and international guidelines such as those of the ICJME, World Association of Medical Editors, and Council of Science Editors lack editorial guidance on the subject. Although our study does not provide evidence of harm by these citations, even those that substitute for a permanent primary research source, we emphasise the need for a consistent voice on how Wikipedia should be used in the academic literature. This study begs the broader question of “what is an appropriate reference source?” Is it something that is generated by the scientific method, is replicable, and undergoes rigorous peer review or is it something that is collaboratively generated, readily open to editing, and broadly accepted? We call attention to the need to work with the Wikipedia community to establish guidelines not only for reviewers and editors but also for academics drafting publications.

An increasing number of peer reviewed academic papers in the health sciences are citing Wikipedia. The apparent increase in the frequency of citations of Wikipedia may suggest a lack of understanding by authors, reviewers, or editors of the mechanisms by which Wikipedia evolves. Although only a very small proportion of citations are of Wikipedia pages, the possibility for the spread of misinformation from an unverified source is at odds with the principles of robust scientific methodology and could potentially affect care of patients. We caution against this trend and suggest that editors and reviewers insist on citing primary sources of information where possible.

What is already known on this topic

The use of Wikipedia as a source of academic information has been debated since its origin, but it is increasingly cited in peer reviewed health science literature

Although studies have examined the content and vailidity of Wikipedia, no evidence has been published about how this resource is being used in health science literature

What this paper adds

Although a few instances exist that may warrant using Wikipedia as a reference, Wikipedia is often cited when permanent, evidence based sources are available

Authors, reviewers, and editors should use caution when publishing articles that include Wikipedia citations

Cite this as: BMJ 2014;348:g1585

Contributors: MDB and SB were involved in the conception and design of the study, the analysis of the data, and critical review of the manuscript. ESH was involved in the conception and design of the study, the analysis and interpretation of data, and critical review of the manuscript. AEP was involved in the analysis and interpretation of data and drafted the manuscript. LU was involved in the conception and design of the study, interpretation of the data, and critical review of the manuscript. TP and ES were involved in collection and analysis of data and critically revised the manuscript. All authors had full access to all data and reviewed the final version to be published. MDB and SB are the guarantors.

Funding: The study was funded by the Children’s Hospital of Eastern Ontario Department of Anesthesiology, University of Ottawa, and the Ottawa Hospital Department of Anesthesiology, University of Ottawa. The funding sources had no role in the study design; the collection, analysis, and interpretation of data; or the writing of the article and the decision to submit it for publication.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Ethical approval: Not needed. All the data presented are in the public domain. Citation information and full text articles were acquired through the ISI Web of Science and Scopus databases. As the accepted practice is not to seek consent to report on information in the public domain, we did not do so for this study, and nor did we seek approval from our research ethics board.

Transparency: The lead author (MDB) affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Data sharing: No additional data available.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/ .

• ↵ Wikipedia. Wikipedia. 2012 [updated 19:11, 28 October 2012; cited 30 October 2012]. http://en.wikipedia.org/wiki/Wikipedia .
• ↵ Heilman JM, Kemmann E, Bonert M, Chatterjee A, Ragar B, Beards GM, et al. Wikipedia: a key tool for global public health promotion. J Med Internet Res 2011 ; 13 : e14 . OpenUrl CrossRef PubMed
• ↵ Thomas GR, Eng L, de Wolff JF, Grover SC. An evaluation of wikipedia as a resource for patient education in nephrology. Semin Dial 2013 ; 26 : 159 -63. OpenUrl CrossRef PubMed
• ↵ Heilman J. Online encyclopedia provides free health info for all. Interview by Fiona Fleck. Bull World Health Organ 2013 ; 91 : 8 -9. OpenUrl CrossRef PubMed Web of Science
• ↵ Okoli C, Mehdi M, Mesgari M, Nielsen FÅ, Lanamäki A. The people’s encyclopedia under the gaze of the sages: a systematic review of scholarly research on Wikipedia. 2012. http://dx.doi.org/10.2139/ssrn.2021326 .
• ↵ Peoples LF. The citation of Wikipedia in judicial opinions. Yale Journal of Law and Technology 2009 ;12.
• ↵ Chesney T. An empirical examination of wikipedia’s credibility. First Monday 2006 ;11(11).
• ↵ Giles J. Internet encycolpedias go head to head. Nature 2005 ; 438 : 900 -1. OpenUrl CrossRef PubMed Web of Science
• ↵ Nielsen FÅ. Scientific citations in Wikipedia. First Monday 2007 ;12(8).
• ↵ Haigh CA. Wikipedia as an evidence source for nursing and healthcare students. Nurse Educ Today 2011 ; 31 : 135 -9. OpenUrl CrossRef PubMed Web of Science
• ↵ Czarnecka-Kujawa K, Abdalian R, Grover SC. The quality of open access and open source internet material in gastroenterology: is Wikipedia appropriate for knowledge transfer to patients? [abstract M1042]. Gastroenterology 2008 ; 134 : A325 -6. OpenUrl CrossRef
• ↵ Giles J. Wikipedia rival calls in the experts. Nature 2006 ; 443 : 493 . OpenUrl CrossRef PubMed Web of Science
• ↵ Rajagopalan MS, Khanna VK, Leiter Y, Stott M, Showalter TN, Dicker AP, et al. Patient-oriented cancer information on the internet: a comparison of wikipedia and a professionally maintained database. J Oncol Pract 2011 ; 7 : 319 -23. OpenUrl Abstract / FREE Full Text
• ↵ Van der Marel S, Duijvestein M, Hardwick JC, van den Brink GR, Veenendaal R, Hommes DW, et al. Quality of web-based information on inflammatory bowel diseases. Inflamm Bowel Dis 2009 ; 15 : 1891 -6. OpenUrl CrossRef PubMed Web of Science
• ↵ Richardson CG, Hamadani LG, Gotay C. Quantifying Canadians’ use of the internet as a source of information on behavioural risk factor modifications related to cancer prevention. Chronic Dis Inj Can 2013 ; 33 : 123 -8. OpenUrl PubMed
• ↵ Metcalfe D, Powell J. Should doctors spurn Wikipedia? J R Soc Med 2011 ; 104 : 488 -9. OpenUrl FREE Full Text
• ↵ Callis KL, Christ LR, Resasco J, Armitage DW, Ash JD, Caughlin TT, et al. Improving Wikipedia: educational opportunity and professional responsibility. Trends Ecol Evol 2009 ; 24 : 177 -9. OpenUrl CrossRef PubMed Web of Science
• ↵ Wikipedia. Wikipedia:Plain and simple guide for medical editors. 2013 [updated 13:43, 17 May 2013‎; cited 4 December 2103]. http://en.wikipedia.org/wiki/Wikipedia:Plain_and_simple_guide_for_medical_editors .
• ↵ Leithner A, Maurer-Ertl W, Glehr M, Friesenbichler J, Leithner K, Windhager R. Wikipedia and osteosarcoma: a trustworthy patients’ information? J Am Med Inform Assoc 2010 ; 17 : 373 -4. OpenUrl Abstract / FREE Full Text
• ↵ Allahwala UK, Nadkarni A, Sebaratnam DF. Wikipedia use amongst medical students—new insights into the digital revolution. Med Teach 2013 ; 35 : 337 . OpenUrl
• ↵ Duran-Nelson A, Gladding S, Beattie J, Nixon LJ. Should we Google it? Resource use by internal medicine residents for point-of-care clinical decision making. Acad Med 2013 ; 88 : 788 -94. OpenUrl CrossRef PubMed Web of Science
• ↵ Hughes B, Joshi I, Lemonde H, Wareham J. Junior physician’s use of Web 2.0 for information seeking and medical education: a qualitative study. Int J Med Inf 2009 ; 78 : 645 -55. OpenUrl CrossRef PubMed Web of Science
• ↵ Judd T, Kennedy G. Expediency-based practice? Medical students’ reliance on Google and Wikipedia for biomedical inquiries. Br J Educ Technol 2011 ; 42 : 351 -60. OpenUrl CrossRef
• ↵ Vandals prompt Wikipedia to ponder editing changes. ABC News 2009 Jan 28.
• ↵ Martin L. Wikipedia fights off cyber vandals. The Guardian 2006 Jun 18.
• ↵ Kleeman J. Wikipedia fights vandalism. New Zealand Herald 2007 Apr 2.
• ↵ Knapp MM. eBay, Wikipedia, and the future of the footnote. Theatre Hist Stud 2008 ; 28 : 36 -41. OpenUrl CrossRef
• ↵ Prytherch RJ. Harrod’s librarians’ glossary and reference book: a directory of over 10,200 terms, organizations, projects and acronyms in the areas of information management, library science, publishing and archive management: Ashgate Publishing, 2005.
• ↵ International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. 2013 [cited 15 Feb 2014]. www.icmje.org/icmje-recommendations.pdf .
• ↵ Park TK. The visibility of Wikipedia in scholarly publications. First Monday 2011 ;16(1).
• ↵ Garfield E. The Thomson Reuters impact factor. Curr Contents 1994 ; 25 : 3 -7. OpenUrl
• ↵ Reuters T. Journal Citation Reports. 2013. http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/ .
• ↵ Wikipedia. Wikipedia:Citing Wikipedia. 2013 [updated 19:49, 5 June 2013; cited 6 June 2013]. http://en.wikipedia.org/wiki/Wikipedia:Citing_Wikipedia .
• ↵ Waters NL. Why you can’t cite Wikipedia in my class. Commun ACM 2007 ; 50 : 15 -7. OpenUrl
• ↵ Matava CT, Rosen D, Siu E, Bould DM. eLearning among Canadian anesthesia residents: a survey of podcast use and content needs. BMC Med Educ 2013 ; 13 : 59 . OpenUrl CrossRef PubMed
• ↵ Mostyn A, Jenkinson CM, McCormick D, Meade O, Lymn JS. An exploration of student experiences of using biology podcasts in nursing training. BMC Med Educ 2013 ; 13 : 12 . OpenUrl CrossRef PubMed
• ↵ Reavley NJ, Mackinnon AJ, Morgan AJ, Alvarez-Jimenez M, Hetrick SE, Killackey E, et al. Quality of information sources about mental disorders: a comparison of Wikipedia with centrally controlled web and printed sources. Psychol Med 2012 ; 42 : 1753 -62. OpenUrl CrossRef PubMed
• ↵ Chakraborti C, K. Vitamin D as a promising anticancer agent. Indian J Pharmacol 2010 ; 43 : 113 -20. OpenUrl
• ↵ Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ 1996 ; 312 : 71 -2. OpenUrl FREE Full Text

Advertisement

Supported by

Alice Munro, Nobel Laureate and Master of the Short Story, Dies at 92

Her stories were widely considered to be without equal, a mixture of ordinary people and extraordinary themes.

• Share full article

By Anthony DePalma

Alice Munro, the revered Canadian author who started writing short stories because she did not think she had the time or the talent to master novels, then stubbornly dedicated her long career to churning out psychologically dense stories that dazzled the literary world and earned her the Nobel Prize in Literature, died on Monday night in Port Hope, Ontario, east of Toronto. She was 92.

A spokesman for her publisher, Penguin Random House Canada, confirmed the death, at a nursing home. Ms. Munro’s health had declined since at least 2009, when she said she’d had heart bypass surgery and had been treated for cancer, though she continued to write.

Ms. Munro was a member of the rare breed of writer, like Katherine Anne Porter and Raymond Carver, who made their reputations in the notoriously difficult literary arena of the short story, and did so with great success. Her tales — many of them focused on women at different stages of their lives coping with complex desires — were so eagerly received and gratefully read that she attracted a whole new generation of readers.

Ms. Munro’s stories were widely considered to be without equal, a mixture of ordinary people and extraordinary themes. She portrayed small-town folks, often in rural southwestern Ontario, facing situations that made the fantastic seem an everyday occurrence. Some of her characters were fleshed out so completely through generations and across continents that readers reached a level of intimacy with them that usually comes only with a full-length novel.

She achieved such compactness through exquisite craftsmanship and a degree of precision that did not waste words. Other writers declared some of her stories to be near-perfect — a heavy burden for a writer of modest personal character who had struggled to overcome a lack of self-confidence at the beginning of her career, when she left the protective embrace of her quiet hometown and ventured into the competitive literary scene.

Her insecurity, however powerfully she felt it, was never noticed by her fellow writers, who celebrated her craftsmanship and freely lent her their highest praise.

The Irish novelist Edna O’Brien ranked Ms. Munro with William Faulkner and James Joyce as writers who had influenced her work. Joyce Carol Oates said Munro stories “have the density — moral, emotional, sometimes historical — of other writers’ novels.” And the novelist Richard Ford once made it clear that questioning Ms. Munro’s mastery over the short story would be akin to doubting the hardness of a diamond or the bouquet of a ripened peach.

“With Alice it’s like a shorthand,” Mr. Ford said. “You’ll just mention her, and everybody just kind of generally nods that she’s just sort of as good as it gets.”

In awarding her the Nobel in 2013 , when she was 82, the Swedish Academy cited her 14 collections of stories and referred to her as “a master of the contemporary short story,” praising her ability to “accommodate the entire epic complexity of the novel in just a few short pages.”

As famous for the refined exuberance of her prose as for the modesty of her personal life, Ms. Munro declined to travel to Sweden to accept her Nobel, saying she was too frail. In place of the formal lecture that winners traditionally give, she taped a long interview in Victoria, British Columbia, where she had been visiting when her award was announced. When asked if the process of writing her stories had consumed her entirely, she responded that it did, then added, “But you know, I always got lunch for my children.”

During the presentation of the taped interview at the Swedish Academy, the Swedish actress Pernilla August read an excerpt from Ms. Munro’s story “Carried Away,” a multi-decade tale of dashed expectations that typified the complicated, often disappointing, world of her stories.

“She had a picture taken. She knew how she wanted it to be,” the excerpt read. “She would have liked to wear a simple white blouse, a peasant girl’s smock with the string open at the neck. She did not own a blouse of that description and in fact had only seen them in pictures. And she would have liked to let her hair down. Or if it had to be up, she would have liked it piled very loosely and bound with strings of pearls.

“Instead she wore her blue silk shirtwaist and bound her hair as usual. She thought the picture made her look rather pale, hollow-eyed. Her expression was sterner and more foreboding than she had intended. She sent it anyway.”

‘Our Chekhov’

Ms. Munro’s early success in Canada, where her first collection of stories, “Dance of the Happy Shades” (1968), won the Governor General’s Literary Award, the equivalent of a Pulitzer Prize for fiction, spread to the United States after her stories began to be published in The New Yorker in 1977. She was an important member of a generation of Canadian writers, along with Margaret Atwood and Michael Ondaatje, whose celebrity reached far beyond the country’s borders.

Ms. Munro went on to win the Governor General’s award twice more, along with two Giller Prizes, another important national award in Canada, and many other honors. In 2009, she withdrew her collection “Too Much Happiness” from consideration for yet another Giller because she believed that a younger writer should have a chance to win it.

That same year she was awarded the Man Booker International Prize for her lifelong body of work, which the judges claimed was “practically perfect.” The awards committee commented that although she was known mostly as a short-story writer, “she brings as much depth, wisdom and precision to every story as most novelists bring to a lifetime of novels.”

“To read Alice Munro is to learn something every time that you never thought of before,” the judges said.

As her many-layered style developed, her short stories came to be neither short nor simply stories — she included 15 stories in her first book, but only eight or nine longer ones in some of her most recent collections. The greater length of each story gave her room to explore the psychological profiles of her characters more fully, and the resulting works are tightly woven tapestries of great tension, lasting resonance and stunning breadth that combine the emotional thrust of a novel with the pinpoint power of a masterful poem.

Over the years, her stories seemed to grow darker and more paradoxical, even though she often described her own life as ordinary and generally upbeat. Often her characters were simple people confronting unusual circumstances. But those situations could be odd, even bizarre, such as an accident in which a soldier who returned from war is decapitated after his sleeve is caught in a factory machine, or the actions of an unattractive girl who steals so much money from her parents’ store to pay boys for sex that her parents are forced to declare bankruptcy. The women in her stories tended to be emotionally pierced — divorced women, adulteresses and noble victims of life’s vicissitudes.

Like Faulkner, Eudora Welty and the other Southern writers she admired, Ms. Munro was capable of breathing life into an entire world — for her, the importunate countryside of southwestern Ontario and the placid, occasionally threatening presence of Lake Huron.

Cynthia Ozick called her “our Chekhov,” and the description stuck.

In a 2009 review of “Too Much Happiness,” Michiko Kakutani of The New York Times described the collection’s title story as “a brilliant distillation of her Chekhovian art.”

Never a Novel

Ms. Munro was able to live a life remarkable for its normalcy. Her days, like her characters’, were filled with quotidian routines punctuated by the explosive mystery of happenstance and accident.

Outside of a decade spent on the west coast of Canada during her first marriage, she lived with a great deal of satisfaction in the Ontario bramble she celebrated in her stories, quietly composing them in the house where her second husband was raised, not far from the place where she was born.

Perhaps the question that most dogged her throughout her long career was why, with her abundant talents and perceptive eye, she restricted herself to what is generally seen as the limited world of the short story rather than launch into the glittery universe of the novel.

“I don’t really understand a novel,” Ms. Munro confessed to Mervyn Rothstein of The Times in a 1986 interview. “I don’t understand where the excitement is supposed to come in a novel, and I do in a story. There’s a kind of tension that if I’m getting a story right I can feel right away.”

While one of her early collections, “Lives of Girls and Women,” is sometimes called a novel, Ms. Munro and her longtime editor at Alfred A. Knopf, Ann Close, considered it a collection of linked stories.

“Once I started to write that, I was off,” she told The Paris Review. “Then I made a big mistake. I tried to make it a regular novel, an ordinary sort of childhood adolescence novel. About March I saw it wasn’t working. It didn’t feel right to me, and I thought I would have to abandon it. I was very depressed. Then it came to me that what I had to do was pull it apart and put it in the story form. Then I could handle it.”

At times she swore she would never write a novel — almost dismissing the challenge as too great for her to even attempt. But at other times she seemed to wistfully wonder, as one of her characters might, how different her life might have been had she written a blockbuster novel.

“I’m thinking of something now, how it might be a novel, but I bet you it won’t be,” she said in a 1998 interview, just after publication of her widely acclaimed collection “The Love of a Good Woman.” She confessed that on occasion she had experimented with stretching her stories into novels but said she found that the stories “start to sag” when she did so, as though being taken beyond their natural limits. Still, the lure never completely evaporated. “My ambition is to write a novel before I die,” she said, also in 1998.

She never did.

Shortly before receiving her Nobel in 2013, Ms. Munro told several interviewers that she had decided to stop writing. As far back as 2009, she had disclosed her cancer diagnosis and that she’d undergone heart bypass surgery. Her declining health had robbed her of strength, but she also remarked that she’d been writing since she was 20 and had grown weary of what Del, a character in “Lives of Girls and Women” who is generally taken to be Ms. Munro’s proxy, says is a writer’s only duty, which is “to produce a masterpiece.”

“That’s a long time to be working,” Ms. Munro said, “and I thought maybe it’s time to take it easy.”

Rural Beginnings

Alice Ann Laidlaw was born on July 10, 1931, in the village of Wingham, Ontario, hard by the banks of Lake Huron. She was the first of three children of Robert Eric Laidlaw and Anne Clarke (Chamney) Laidlaw. Her father had tried his luck at the rather exotic undertaking of raising silver foxes and mink, but when that failed he went through a number of professions, including stints as foundry watchman and turkey farmer.

When Anne Laidlaw developed Parkinson’s disease, it fell to Alice, not yet a teenager but the oldest of the three children, to care for her mother, an experience that she wove through her writing. She was able to attend college after winning a two-year scholarship to the University of Western Ontario in London, Ontario, about 65 miles south of Wingham.

She majored in English but initially kept her ambition to write fiction to herself. She dropped out before completing her studies and married a fellow student, James Munro. She sold her first short work of fiction, a story, to the radio service of the Canadian Broadcasting Corporation.

The Munros settled in Vancouver and had two children; a third died at birth. Ms. Munro said the domestic demands of those years — balancing parenthood with her dream of writing, “getting apple juice, answering the phone and letting the cat in” — left her no time or energy for ambitious projects like writing novels. Instead, she dedicated herself to mastering the short story, a form that she felt she could manage in between raising her children and taking care of her house.

In 1963, Ms. Munro and her husband moved to Victoria, where she helped him found a bookstore, Munro’s, and gave birth to another daughter. The marriage ended in 1973, and she moved back to Ontario.

By then, her literary reputation in Canada was established. In 1968, her first book, “Dance of the Happy Shades,” a collection of short stories compiled over a dozen years, introduced readers to what would later be widely recognized as “Alice Munro Country” — the rigidly introspective landscape of solitary country roads and stolid houses of yellow brick within which shy lives and solemn secrets unfolded.

“Everybody knows what a house does, how it encloses space and makes connections between one enclosed space and another and presents what is outside in a new way,” she wrote in a 1982 essay. “That is the nearest I can come to explaining what a story is for me.”

Her stories are blanketed with countless small but sharp observations that animate Munro Country. For instance, in “Spaceships Have Landed,” a story in the collection “Open Secrets” (1994), the main character drunkenly flirts with her boyfriend’s friend, only to be grossly insulted by him. The next day, she calls him to the porch of her house and confronts him while using a piece of steel wool to clean freshly laid eggs.

Such details evoke a sense of the semirural Canadian backcountry, a quiet land where people never deliberately call attention to themselves and the ordinariness of life can be suddenly disrupted by accidents, arrivals and unanticipated departures.

Although Ms. Munro was most often described as a Canadian writer, her stories evoked not Canada itself but the bittersweet triumphs, mishaps and humiliations of small town life. And in the end, every landscape served as backdrop for her central themes, which were the unpredictability of life and the betrayals that women suffer or commit — scenes redolent with autobiography.

In “The Albanian Virgin,” a celebrated story featuring a rare exotic setting as well as the familiar Canadian landscape, the female protagonist runs a bookstore in Victoria and dreamily contemplates the errant directions taken by her life: “But I was not despondent. I had made a desperate change in my life, and in spite of the regrets that I suffered every day, I was proud of that. I felt as if I had finally come out into the world in a new, true, skin.”

A Publicity-Shy ‘Plodder’

Ms. Munro shunned much of the publicity usually associated with literary success and limited her book tour appearances and readings. She often referred to herself in a self-deprecating way; she said she had not “come out of the closet” as a professional writer until she was 40, and she called herself a “plodder” because of the slow and deliberate way she worked, often writing in her nightclothes for several hours in the morning and then extensively revising her stories before sending them off.

But to critics, there was nothing plodding about her stories, which were put together so seamlessly that the many flashbacks, flash-forwards and shifts in time and place that she employed happened without notice. She often started her stories at a point where other authors might end theirs, and continued them well past the climax or denouement that would have satisfied others less driven by the twists of fate. Inevitably, this left readers to work out who exactly the narrator was and how one character was related to another.

Eventually, though, every piece would fit together. “It’s like a child’s puzzle,” the novelist Anne Tyler once said of Ms. Munro’s work. “In the most successful of the stories, the end result is a satisfying click as everything settles precisely into place.”

After the turbulence and dislocation she went through before Ms. Munro turned 40, her life and career clicked satisfyingly into place when she returned to southern Ontario. She started seeing Gerald Fremlin, a geographer, and after a brief romance married him and moved into the house in Clinton, Ontario, where he was raised.

She is survived by her daughters, Sheila, Jenny and Andrea. Sheila Munro is the author of the 2001 memoir “Lives of Mothers and Daughters: Growing Up With Alice Munro.”

She embarked on an ambitious schedule of publishing a collection of short stories every three or four years, winning praise and admiration across Canada, where she comes close to being a household literary saint. After receiving her first Governor General’s award, she won it twice more, for “Who Do You Think You Are?” in 1978 and for “The Progress of Love” in 1986.

In 1998, she received the Giller Prize for “The Love of a Good Woman,” and in 2004 she picked up another for “Runaway.” After the National Book Critics Circle agreed for the first time to consider authors from outside the United States for its award, Ms. Munro won in 1998 for “The Love of a Good Woman.”

As if she were a character in one of her stories, plagued by bad timing and unlucky happenstance, Ms. Munro was not at home when the Swedish Academy called to tell her that she had won; it had to leave a telephone message. She was in Victoria visiting her daughter, who heard the news and woke her mother at 4 a.m. Still groggy when interviewed by the CBC, Ms. Munro admitted that she’d forgotten that the prize was to be awarded that day, calling it “a splendid thing to happen,” adding, “more than I can say.”

Struggling to control her emotions, she reflected on her success and what it might mean for literature. “My stories have gotten around quite remarkably for short stories,” she told the interviewer. “I would really hope that this would make people see the short story as an important art, not something you play around with until you got a novel written.”

Lisa D. Awano and Sofia Poznansky contributed reporting.

An earlier version of this obituary misspelled the given name of an author who praised Ms. Munro’s writing. She is Anne Tyler, not Ann.

How we handle corrections