empirical model literature review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 14 May 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 

How to write a good literature review 

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 

Find academic papers related to your research topic faster. Try Research on Paperpal  

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!    

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as you go with Paperpal Research. Start now for free.   

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface with the option to save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 
• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, how to write a high-quality conference paper, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai, what are scholarly sources and where can you..., how to write a hypothesis types and examples , measuring academic success: definition & strategies for excellence, what is academic writing: tips for students.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Module 2 Chapter 3: What is Empirical Literature & Where can it be Found?

In Module 1, you read about the problem of pseudoscience. Here, we revisit the issue in addressing how to locate and assess scientific or empirical literature . In this chapter you will read about:

• distinguishing between what IS and IS NOT empirical literature
• how and where to locate empirical literature for understanding diverse populations, social work problems, and social phenomena.

Probably the most important take-home lesson from this chapter is that one source is not sufficient to being well-informed on a topic. It is important to locate multiple sources of information and to critically appraise the points of convergence and divergence in the information acquired from different sources. This is especially true in emerging and poorly understood topics, as well as in answering complex questions.

What Is Empirical Literature

Social workers often need to locate valid, reliable information concerning the dimensions of a population group or subgroup, a social work problem, or social phenomenon. They might also seek information about the way specific problems or resources are distributed among the populations encountered in professional practice. Or, social workers might be interested in finding out about the way that certain people experience an event or phenomenon. Empirical literature resources may provide answers to many of these types of social work questions. In addition, resources containing data regarding social indicators may also prove helpful. Social indicators are the “facts and figures” statistics that describe the social, economic, and psychological factors that have an impact on the well-being of a community or other population group.The United Nations (UN) and the World Health Organization (WHO) are examples of organizations that monitor social indicators at a global level: dimensions of population trends (size, composition, growth/loss), health status (physical, mental, behavioral, life expectancy, maternal and infant mortality, fertility/child-bearing, and diseases like HIV/AIDS), housing and quality of sanitation (water supply, waste disposal), education and literacy, and work/income/unemployment/economics, for example.

Three characteristics stand out in empirical literature compared to other types of information available on a topic of interest: systematic observation and methodology, objectivity, and transparency/replicability/reproducibility. Let’s look a little more closely at these three features.

Systematic Observation and Methodology. The hallmark of empiricism is “repeated or reinforced observation of the facts or phenomena” (Holosko, 2006, p. 6). In empirical literature, established research methodologies and procedures are systematically applied to answer the questions of interest.

Objectivity. Gathering “facts,” whatever they may be, drives the search for empirical evidence (Holosko, 2006). Authors of empirical literature are expected to report the facts as observed, whether or not these facts support the investigators’ original hypotheses. Research integrity demands that the information be provided in an objective manner, reducing sources of investigator bias to the greatest possible extent.

Transparency and Replicability/Reproducibility.   Empirical literature is reported in such a manner that other investigators understand precisely what was done and what was found in a particular research study—to the extent that they could replicate the study to determine whether the findings are reproduced when repeated. The outcomes of an original and replication study may differ, but a reader could easily interpret the methods and procedures leading to each study’s findings.

What is NOT Empirical Literature

By now, it is probably obvious to you that literature based on “evidence” that is not developed in a systematic, objective, transparent manner is not empirical literature. On one hand, non-empirical types of professional literature may have great significance to social workers. For example, social work scholars may produce articles that are clearly identified as describing a new intervention or program without evaluative evidence, critiquing a policy or practice, or offering a tentative, untested theory about a phenomenon. These resources are useful in educating ourselves about possible issues or concerns. But, even if they are informed by evidence, they are not empirical literature. Here is a list of several sources of information that do not meet the standard of being called empirical literature:

• your course instructor’s lectures
• political statements
• advertisements
• newspapers & magazines (journalism)
• television news reports & analyses (journalism)
• many websites, Facebook postings, Twitter tweets, and blog postings
• the introductory literature review in an empirical article

You may be surprised to see the last two included in this list. Like the other sources of information listed, these sources also might lead you to look for evidence. But, they are not themselves sources of evidence. They may summarize existing evidence, but in the process of summarizing (like your instructor’s lectures), information is transformed, modified, reduced, condensed, and otherwise manipulated in such a manner that you may not see the entire, objective story. These are called secondary sources, as opposed to the original, primary source of evidence. In relying solely on secondary sources, you sacrifice your own critical appraisal and thinking about the original work—you are “buying” someone else’s interpretation and opinion about the original work, rather than developing your own interpretation and opinion. What if they got it wrong? How would you know if you did not examine the primary source for yourself? Consider the following as an example of “getting it wrong” being perpetuated.

Example: Bullying and School Shootings . One result of the heavily publicized April 1999 school shooting incident at Columbine High School (Colorado), was a heavy emphasis placed on bullying as a causal factor in these incidents (Mears, Moon, & Thielo, 2017), “creating a powerful master narrative about school shootings” (Raitanen, Sandberg, & Oksanen, 2017, p. 3). Naturally, with an identified cause, a great deal of effort was devoted to anti-bullying campaigns and interventions for enhancing resilience among youth who experience bullying.  However important these strategies might be for promoting positive mental health, preventing poor mental health, and possibly preventing suicide among school-aged children and youth, it is a mistaken belief that this can prevent school shootings (Mears, Moon, & Thielo, 2017). Many times the accounts of the perpetrators having been bullied come from potentially inaccurate third-party accounts, rather than the perpetrators themselves; bullying was not involved in all instances of school shooting; a perpetrator’s perception of being bullied/persecuted are not necessarily accurate; many who experience severe bullying do not perpetrate these incidents; bullies are the least targeted shooting victims; perpetrators of the shooting incidents were often bullying others; and, bullying is only one of many important factors associated with perpetrating such an incident (Ioannou, Hammond, & Simpson, 2015; Mears, Moon, & Thielo, 2017; Newman &Fox, 2009; Raitanen, Sandberg, & Oksanen, 2017). While mass media reports deliver bullying as a means of explaining the inexplicable, the reality is not so simple: “The connection between bullying and school shootings is elusive” (Langman, 2014), and “the relationship between bullying and school shooting is, at best, tenuous” (Mears, Moon, & Thielo, 2017, p. 940). The point is, when a narrative becomes this publicly accepted, it is difficult to sort out truth and reality without going back to original sources of information and evidence.

What May or May Not Be Empirical Literature: Literature Reviews

Investigators typically engage in a review of existing literature as they develop their own research studies. The review informs them about where knowledge gaps exist, methods previously employed by other scholars, limitations of prior work, and previous scholars’ recommendations for directing future research. These reviews may appear as a published article, without new study data being reported (see Fields, Anderson, & Dabelko-Schoeny, 2014 for example). Or, the literature review may appear in the introduction to their own empirical study report. These literature reviews are not considered to be empirical evidence sources themselves, although they may be based on empirical evidence sources. One reason is that the authors of a literature review may or may not have engaged in a systematic search process, identifying a full, rich, multi-sided pool of evidence reports.

There is, however, a type of review that applies systematic methods and is, therefore, considered to be more strongly rooted in evidence: the systematic review .

Systematic review of literature. A systematic reviewis a type of literature report where established methods have been systematically applied, objectively, in locating and synthesizing a body of literature. The systematic review report is characterized by a great deal of transparency about the methods used and the decisions made in the review process, and are replicable. Thus, it meets the criteria for empirical literature: systematic observation and methodology, objectivity, and transparency/reproducibility. We will work a great deal more with systematic reviews in the second course, SWK 3402, since they are important tools for understanding interventions. They are somewhat less common, but not unheard of, in helping us understand diverse populations, social work problems, and social phenomena.

Locating Empirical Evidence

Social workers have available a wide array of tools and resources for locating empirical evidence in the literature. These can be organized into four general categories.

Journal Articles. A number of professional journals publish articles where investigators report on the results of their empirical studies. However, it is important to know how to distinguish between empirical and non-empirical manuscripts in these journals. A key indicator, though not the only one, involves a peer review process . Many professional journals require that manuscripts undergo a process of peer review before they are accepted for publication. This means that the authors’ work is shared with scholars who provide feedback to the journal editor as to the quality of the submitted manuscript. The editor then makes a decision based on the reviewers’ feedback:

• Accept as is
• Accept with minor revisions
• Request that a revision be resubmitted (no assurance of acceptance)

When a “revise and resubmit” decision is made, the piece will go back through the review process to determine if it is now acceptable for publication and that all of the reviewers’ concerns have been adequately addressed. Editors may also reject a manuscript because it is a poor fit for the journal, based on its mission and audience, rather than sending it for review consideration.

Indicators of journal relevance. Various journals are not equally relevant to every type of question being asked of the literature. Journals may overlap to a great extent in terms of the topics they might cover; in other words, a topic might appear in multiple different journals, depending on how the topic was being addressed. For example, articles that might help answer a question about the relationship between community poverty and violence exposure might appear in several different journals, some with a focus on poverty, others with a focus on violence, and still others on community development or public health. Journal titles are sometimes a good starting point but may not give a broad enough picture of what they cover in their contents.

In focusing a literature search, it also helps to review a journal’s mission and target audience. For example, at least four different journals focus specifically on poverty:

• Journal of Children & Poverty
• Journal of Poverty
• Journal of Poverty and Social Justice
• Poverty & Public Policy

Let’s look at an example using the Journal of Poverty and Social Justice . Information about this journal is located on the journal’s webpage: http://policy.bristoluniversitypress.co.uk/journals/journal-of-poverty-and-social-justice . In the section headed “About the Journal” you can see that it is an internationally focused research journal, and that it addresses social justice issues in addition to poverty alone. The research articles are peer-reviewed (there appear to be non-empirical discussions published, as well). These descriptions about a journal are almost always available, sometimes listed as “scope” or “mission.” These descriptions also indicate the sponsorship of the journal—sponsorship may be institutional (a particular university or agency, such as Smith College Studies in Social Work ), a professional organization, such as the Council on Social Work Education (CSWE) or the National Association of Social Work (NASW), or a publishing company (e.g., Taylor & Frances, Wiley, or Sage).

Indicators of journal caliber.  Despite engaging in a peer review process, not all journals are equally rigorous. Some journals have very high rejection rates, meaning that many submitted manuscripts are rejected; others have fairly high acceptance rates, meaning that relatively few manuscripts are rejected. This is not necessarily the best indicator of quality, however, since newer journals may not be sufficiently familiar to authors with high quality manuscripts and some journals are very specific in terms of what they publish. Another index that is sometimes used is the journal’s impact factor . Impact factor is a quantitative number indicative of how often articles published in the journal are cited in the reference list of other journal articles—the statistic is calculated as the number of times on average each article published in a particular year were cited divided by the number of articles published (the number that could be cited). For example, the impact factor for the Journal of Poverty and Social Justice in our list above was 0.70 in 2017, and for the Journal of Poverty was 0.30. These are relatively low figures compared to a journal like the New England Journal of Medicine with an impact factor of 59.56! This means that articles published in that journal were, on average, cited more than 59 times in the next year or two.

Impact factors are not necessarily the best indicator of caliber, however, since many strong journals are geared toward practitioners rather than scholars, so they are less likely to be cited by other scholars but may have a large impact on a large readership. This may be the case for a journal like the one titled Social Work, the official journal of the National Association of Social Workers. It is distributed free to all members: over 120,000 practitioners, educators, and students of social work world-wide. The journal has a recent impact factor of.790. The journals with social work relevant content have impact factors in the range of 1.0 to 3.0 according to Scimago Journal & Country Rank (SJR), particularly when they are interdisciplinary journals (for example, Child Development , Journal of Marriage and Family , Child Abuse and Neglect , Child Maltreatmen t, Social Service Review , and British Journal of Social Work ). Once upon a time, a reader could locate different indexes comparing the “quality” of social work-related journals. However, the concept of “quality” is difficult to systematically define. These indexes have mostly been replaced by impact ratings, which are not necessarily the best, most robust indicators on which to rely in assessing journal quality. For example, new journals addressing cutting edge topics have not been around long enough to have been evaluated using this particular tool, and it takes a few years for articles to begin to be cited in other, later publications.

Beware of pseudo-, illegitimate, misleading, deceptive, and suspicious journals . Another side effect of living in the Age of Information is that almost anyone can circulate almost anything and call it whatever they wish. This goes for “journal” publications, as well. With the advent of open-access publishing in recent years (electronic resources available without subscription), we have seen an explosion of what are called predatory or junk journals . These are publications calling themselves journals, often with titles very similar to legitimate publications and often with fake editorial boards. These “publications” lack the integrity of legitimate journals. This caution is reminiscent of the discussions earlier in the course about pseudoscience and “snake oil” sales. The predatory nature of many apparent information dissemination outlets has to do with how scientists and scholars may be fooled into submitting their work, often paying to have their work peer-reviewed and published. There exists a “thriving black-market economy of publishing scams,” and at least two “journal blacklists” exist to help identify and avoid these scam journals (Anderson, 2017).

This issue is important to information consumers, because it creates a challenge in terms of identifying legitimate sources and publications. The challenge is particularly important to address when information from on-line, open-access journals is being considered. Open-access is not necessarily a poor choice—legitimate scientists may pay sizeable fees to legitimate publishers to make their work freely available and accessible as open-access resources. On-line access is also not necessarily a poor choice—legitimate publishers often make articles available on-line to provide timely access to the content, especially when publishing the article in hard copy will be delayed by months or even a year or more. On the other hand, stating that a journal engages in a peer-review process is no guarantee of quality—this claim may or may not be truthful. Pseudo- and junk journals may engage in some quality control practices, but may lack attention to important quality control processes, such as managing conflict of interest, reviewing content for objectivity or quality of the research conducted, or otherwise failing to adhere to industry standards (Laine & Winker, 2017).

One resource designed to assist with the process of deciphering legitimacy is the Directory of Open Access Journals (DOAJ). The DOAJ is not a comprehensive listing of all possible legitimate open-access journals, and does not guarantee quality, but it does help identify legitimate sources of information that are openly accessible and meet basic legitimacy criteria. It also is about open-access journals, not the many journals published in hard copy.

An additional caution: Search for article corrections. Despite all of the careful manuscript review and editing, sometimes an error appears in a published article. Most journals have a practice of publishing corrections in future issues. When you locate an article, it is helpful to also search for updates. Here is an example where data presented in an article’s original tables were erroneous, and a correction appeared in a later issue.

• Marchant, A., Hawton, K., Stewart A., Montgomery, P., Singaravelu, V., Lloyd, K., Purdy, N., Daine, K., & John, A. (2017). A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLoS One, 12(8): e0181722. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5558917/
• Marchant, A., Hawton, K., Stewart A., Montgomery, P., Singaravelu, V., Lloyd, K., Purdy, N., Daine, K., & John, A. (2018).Correction—A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLoS One, 13(3): e0193937.  http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0193937

Search Tools. In this age of information, it is all too easy to find items—the problem lies in sifting, sorting, and managing the vast numbers of items that can be found. For example, a simple Google® search for the topic “community poverty and violence” resulted in about 15,600,000 results! As a means of simplifying the process of searching for journal articles on a specific topic, a variety of helpful tools have emerged. One type of search tool has previously applied a filtering process for you: abstracting and indexing databases . These resources provide the user with the results of a search to which records have already passed through one or more filters. For example, PsycINFO is managed by the American Psychological Association and is devoted to peer-reviewed literature in behavioral science. It contains almost 4.5 million records and is growing every month. However, it may not be available to users who are not affiliated with a university library. Conducting a basic search for our topic of “community poverty and violence” in PsychINFO returned 1,119 articles. Still a large number, but far more manageable. Additional filters can be applied, such as limiting the range in publication dates, selecting only peer reviewed items, limiting the language of the published piece (English only, for example), and specified types of documents (either chapters, dissertations, or journal articles only, for example). Adding the filters for English, peer-reviewed journal articles published between 2010 and 2017 resulted in 346 documents being identified.

Just as was the case with journals, not all abstracting and indexing databases are equivalent. There may be overlap between them, but none is guaranteed to identify all relevant pieces of literature. Here are some examples to consider, depending on the nature of the questions asked of the literature:

• Academic Search Complete—multidisciplinary index of 9,300 peer-reviewed journals
• AgeLine—multidisciplinary index of aging-related content for over 600 journals
• Campbell Collaboration—systematic reviews in education, crime and justice, social welfare, international development
• Google Scholar—broad search tool for scholarly literature across many disciplines
• MEDLINE/ PubMed—National Library of medicine, access to over 15 million citations
• Oxford Bibliographies—annotated bibliographies, each is discipline specific (e.g., psychology, childhood studies, criminology, social work, sociology)
• PsycINFO/PsycLIT—international literature on material relevant to psychology and related disciplines
• SocINDEX—publications in sociology
• Social Sciences Abstracts—multiple disciplines
• Social Work Abstracts—many areas of social work are covered
• Web of Science—a “meta” search tool that searches other search tools, multiple disciplines

Placing our search for information about “community violence and poverty” into the Social Work Abstracts tool with no additional filters resulted in a manageable 54-item list. Finally, abstracting and indexing databases are another way to determine journal legitimacy: if a journal is indexed in a one of these systems, it is likely a legitimate journal. However, the converse is not necessarily true: if a journal is not indexed does not mean it is an illegitimate or pseudo-journal.

Government Sources. A great deal of information is gathered, analyzed, and disseminated by various governmental branches at the international, national, state, regional, county, and city level. Searching websites that end in.gov is one way to identify this type of information, often presented in articles, news briefs, and statistical reports. These government sources gather information in two ways: they fund external investigations through grants and contracts and they conduct research internally, through their own investigators. Here are some examples to consider, depending on the nature of the topic for which information is sought:

• Agency for Healthcare Research and Quality (AHRQ) at https://www.ahrq.gov/
• Bureau of Justice Statistics (BJS) at https://www.bjs.gov/
• Census Bureau at https://www.census.gov
• Morbidity and Mortality Weekly Report of the CDC (MMWR-CDC) at https://www.cdc.gov/mmwr/index.html
• Child Welfare Information Gateway at https://www.childwelfare.gov
• Children’s Bureau/Administration for Children & Families at https://www.acf.hhs.gov
• Forum on Child and Family Statistics at https://www.childstats.gov
• National Institutes of Health (NIH) at https://www.nih.gov , including (not limited to):
• National Institute on Aging (NIA at https://www.nia.nih.gov
• National Institute on Alcohol Abuse and Alcoholism (NIAAA) at https://www.niaaa.nih.gov
• National Institute of Child Health and Human Development (NICHD) at https://www.nichd.nih.gov
• National Institute on Drug Abuse (NIDA) at https://www.nida.nih.gov
• National Institute of Environmental Health Sciences at https://www.niehs.nih.gov
• National Institute of Mental Health (NIMH) at https://www.nimh.nih.gov
• National Institute on Minority Health and Health Disparities at https://www.nimhd.nih.gov
• National Institute of Justice (NIJ) at https://www.nij.gov
• Substance Abuse and Mental Health Services Administration (SAMHSA) at https://www.samhsa.gov/
• United States Agency for International Development at https://usaid.gov

Each state and many counties or cities have similar data sources and analysis reports available, such as Ohio Department of Health at https://www.odh.ohio.gov/healthstats/dataandstats.aspx and Franklin County at https://statisticalatlas.com/county/Ohio/Franklin-County/Overview . Data are available from international/global resources (e.g., United Nations and World Health Organization), as well.

Other Sources. The Health and Medicine Division (HMD) of the National Academies—previously the Institute of Medicine (IOM)—is a nonprofit institution that aims to provide government and private sector policy and other decision makers with objective analysis and advice for making informed health decisions. For example, in 2018 they produced reports on topics in substance use and mental health concerning the intersection of opioid use disorder and infectious disease,  the legal implications of emerging neurotechnologies, and a global agenda concerning the identification and prevention of violence (see http://www.nationalacademies.org/hmd/Global/Topics/Substance-Abuse-Mental-Health.aspx ). The exciting aspect of this resource is that it addresses many topics that are current concerns because they are hoping to help inform emerging policy. The caution to consider with this resource is the evidence is often still emerging, as well.

Numerous “think tank” organizations exist, each with a specific mission. For example, the Rand Corporation is a nonprofit organization offering research and analysis to address global issues since 1948. The institution’s mission is to help improve policy and decision making “to help individuals, families, and communities throughout the world be safer and more secure, healthier and more prosperous,” addressing issues of energy, education, health care, justice, the environment, international affairs, and national security (https://www.rand.org/about/history.html). And, for example, the Robert Woods Johnson Foundation is a philanthropic organization supporting research and research dissemination concerning health issues facing the United States. The foundation works to build a culture of health across systems of care (not only medical care) and communities (https://www.rwjf.org).

While many of these have a great deal of helpful evidence to share, they also may have a strong political bias. Objectivity is often lacking in what information these organizations provide: they provide evidence to support certain points of view. That is their purpose—to provide ideas on specific problems, many of which have a political component. Think tanks “are constantly researching solutions to a variety of the world’s problems, and arguing, advocating, and lobbying for policy changes at local, state, and federal levels” (quoted from https://thebestschools.org/features/most-influential-think-tanks/ ). Helpful information about what this one source identified as the 50 most influential U.S. think tanks includes identifying each think tank’s political orientation. For example, The Heritage Foundation is identified as conservative, whereas Human Rights Watch is identified as liberal.

While not the same as think tanks, many mission-driven organizations also sponsor or report on research, as well. For example, the National Association for Children of Alcoholics (NACOA) in the United States is a registered nonprofit organization. Its mission, along with other partnering organizations, private-sector groups, and federal agencies, is to promote policy and program development in research, prevention and treatment to provide information to, for, and about children of alcoholics (of all ages). Based on this mission, the organization supports knowledge development and information gathering on the topic and disseminates information that serves the needs of this population. While this is a worthwhile mission, there is no guarantee that the information meets the criteria for evidence with which we have been working. Evidence reported by think tank and mission-driven sources must be utilized with a great deal of caution and critical analysis!

In many instances an empirical report has not appeared in the published literature, but in the form of a technical or final report to the agency or program providing the funding for the research that was conducted. One such example is presented by a team of investigators funded by the National Institute of Justice to evaluate a program for training professionals to collect strong forensic evidence in instances of sexual assault (Patterson, Resko, Pierce-Weeks, & Campbell, 2014): https://www.ncjrs.gov/pdffiles1/nij/grants/247081.pdf . Investigators may serve in the capacity of consultant to agencies, programs, or institutions, and provide empirical evidence to inform activities and planning. One such example is presented by Maguire-Jack (2014) as a report to a state’s child maltreatment prevention board: https://preventionboard.wi.gov/Documents/InvestmentInPreventionPrograming_Final.pdf .

When Direct Answers to Questions Cannot Be Found. Sometimes social workers are interested in finding answers to complex questions or questions related to an emerging, not-yet-understood topic. This does not mean giving up on empirical literature. Instead, it requires a bit of creativity in approaching the literature. A Venn diagram might help explain this process. Consider a scenario where a social worker wishes to locate literature to answer a question concerning issues of intersectionality. Intersectionality is a social justice term applied to situations where multiple categorizations or classifications come together to create overlapping, interconnected, or multiplied disadvantage. For example, women with a substance use disorder and who have been incarcerated face a triple threat in terms of successful treatment for a substance use disorder: intersectionality exists between being a woman, having a substance use disorder, and having been in jail or prison. After searching the literature, little or no empirical evidence might have been located on this specific triple-threat topic. Instead, the social worker will need to seek literature on each of the threats individually, and possibly will find literature on pairs of topics (see Figure 3-1). There exists some literature about women’s outcomes for treatment of a substance use disorder (a), some literature about women during and following incarceration (b), and some literature about substance use disorders and incarceration (c). Despite not having a direct line on the center of the intersecting spheres of literature (d), the social worker can develop at least a partial picture based on the overlapping literatures.

Figure 3-1. Venn diagram of intersecting literature sets.

Take a moment to complete the following activity. For each statement about empirical literature, decide if it is true or false.

Social Work 3401 Coursebook Copyright © by Dr. Audrey Begun is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License , except where otherwise noted.

Share This Book

Towards a Consensus Model: Literature Review of How Science Teachers’ Pedagogical Content Knowledge Is Investigated in Empirical Studies

• First Online: 29 January 2019

Cite this chapter

• Kennedy Kam Ho Chan 4 &
• Anne Hume 5  

3658 Accesses

54 Citations

This chapter presents a systematic review of the science education literature to identify how researchers investigate science teachers’ pedagogical content knowledge (PCK). Specifically, we focus on empirical studies of individual science teachers’ PCK published in peer-reviewed science education and teacher education journals since 2008. For each of the reviewed studies, we identify (1) the research context of the investigation; (2) the major purpose of the study; (3) the conceptualisation of PCK in the study; (4) the data sources used to investigate teachers’ PCK; and (5) the approaches used to determine teachers’ PCK. Using this collated information, we provide an overview of how the PCK concept is used, interpreted and investigated within the science education community. The review reveals that researchers conceptualise and operationalise PCK differently. Consequently, they investigate PCK in highly diverse ways and use a wide range of data sources and approaches to capture and determine teachers’ PCK, which in turn generates different kinds of qualitative and quantitative data. Collectively, our findings reveal gaps in the PCK literature and highlight several points of divergence in thinking around the PCK concept within the PCK research community in the field of science education. The findings also provide evidence from the literature supporting the need to build upon and further refine the Consensus Model  (CM) that emerged from the first (1st) PCK Summit in 2012 to further science education research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Articles that were accepted and published online, even if not assigned to an issue, before 31 December 2017 were included in this review.

In line with Shulman ( 1987 ), we use the term category to refer to a distinct domain of knowledge. PCK components refer to the knowledge components of PCK.

CoRe stands for content representation—a portrayal of PCK structured by big ideas related to a topic with responses to key prompts (Loughran, Mulhall, & Berry, 2004 ).

Despite the cyclical nature of the teaching process, it can be difficult to determine whether a particular activity belongs to the pre-active or the post-active phase of the teaching cycle. As such, analysis of student work, predicting student responses in assessment tasks, classifying assessment questions and reflecting on one’s own teaching actions were assigned to the post-active phase of teaching.

Because 9 studies involved both pre-service and in-service teachers, the total number of studies is 108, rather than 99.

Because 10 studies involved both primary and secondary school teachers, the total number of studies is 109, rather than 99.

The total number does not add up to 99 as some studies used both approaches.

The total number does not add up to 64 as two studies contained separate tasks that allowed the examination of the teaching artefacts/actions only in one task and the teachers’ decisions and reasoning around the other teaching task.

The total number does not add up to 60 as four studies used both simulated teaching tasks and investigated real life in situ.

Abbitt, J. T. (2011). Measuring technological pedagogical content knowledge in preservice teacher education. Journal of Research on Technology in Education, 43 (4), 281–300.

Article   Google Scholar  

Abd-El-Khalick, F. (2006). Pre-service and experienced biology teachers’ global and specific subject matter structures: Implications for conceptions of pedagogical content knowledge. EURASIA Journal of Mathematics, Science & Technology Education, 2 (1), 1–29.

Abell, S. K. (2007). Research on science teacher knowledge. In S. K. Abell & N. Lederman (Eds.), Handbook of research on science education (pp. 1105–1149). Mahwah, NJ: Lawrence Erlbaum Associates.

Google Scholar  

Abell, S. K. (2008). Twenty years later: Does pedagogical content knowledge remain a useful idea? International Journal of Science Education, 30 (10), 1405–1416.

Abell, S. K., Rogers, M. A. P., Hanuscin, D. L., Lee, M. H., & Gagnon, M. J. (2008). Preparing the next generation of science teacher educators: A model for developing PCK for teaching science teachers. Journal of Science Teacher Education, 20 (1), 77–93.

Adadan, E., & Oner, D. (2014). Exploring the progression in preservice chemistry teachers’ pedagogical content knowledge representations: The case of “Behavior of gases”. Research in Science Education, 44 (6), 829–858.

Akerson, V. L., Pongsanon, K., Park Rogers, M. A., Carter, I., & Galindo, E. (2017). Exploring the use of lesson study to develop elementary preservice teachers’ pedagogical content knowledge for teaching nature of science. International Journal of Science and Mathematics Education, 15 (2), 293–312.

Akin, F. N., & Uzuntiryaki-Kondakçı, E. (2018). The nature of the interplay among components of pedagogical content knowledge in reaction rate and chemical equilibrium topics of novice and experienced chemistry teachers. Chemistry Education Research and Practice .

Alonzo, A. C., & Kim, J. (2016). Declarative and dynamic pedagogical content knowledge as elicited through two video-based interview methods. Journal of Research in Science Teaching, 53 (8), 1259–1286.

Alonzo, A. C., Kobarg, M., & Seidel, T. (2012). Pedagogical content knowledge as reflected in teacher–student interactions: Analysis of two video cases. Journal of Research in Science Teaching, 49 (10), 1211–1239.

Alvarado, C., Cañada, F., Garritz, A., & Mellado, V. (2015). Canonical pedagogical content knowledge by CoRes for teaching acid–base chemistry at high school. Chemistry Education Research and Practice, 16 (3), 603–618.

An, S., Kulm, G., & Wu, Z. (2004). The pedagogical content knowledge of middle school, mathematics teachers in China and the U.S. Journal of Mathematics Teacher Education, 7 (2), 145–172.

Avraamidou, L., & Zembal-Saul, C. (2005). Giving priority to evidence in science teaching: A first-year elementary teacher’s specialized practices and knowledge. Journal of Research in Science Teaching, 42 (9), 965–986.

Aydin, S., & Boz, Y. (2013). The nature of integration among PCK components: A case study of two experienced chemistry teachers. Chemistry Education Research and Practice, 14 (4), 615–624.

Aydin, S., Demirdöğen, B., Akin, F. N., Uzuntiryaki-Kondakçı, E., & Tarkin, A. (2015). The nature and development of interaction among components of pedagogical content knowledge in practicum. Teaching and Teacher Education, 46, 37–50.

Aydin, S., Demirdöğen, B., Tarkin, A., Kutucu, E. S., Ekiz, B., Akin, F. N., et al. (2013). Providing a set of research-based practices to support preservice teachers’ long-term professional development as learners of science teaching. Science Education, 97 (6), 903–935.

Aydin, S., Friedrichsen, P. J., Boz, Y., & Hanuscin, D. L. (2014). Examination of the topic-specific nature of pedagogical content knowledge in teaching electrochemical cells and nuclear reactions. Chemistry Education Research and Practice, 15 (4), 658–674.

Barendsen, E., & Henze, I. (2017). Relating teacher PCK and teacher practice using classroom observation. Research in Science Education .

Barnett, E., & Friedrichsen, P. J. (2015). Educative mentoring: How a mentor supported a preservice biology teacher’s pedagogical content knowledge development. Journal of Science Teacher Education, 26 (7), 647–668.

Baxter, J. A., & Lederman, N. G. (1999). Assessment and measurement of pedagogical content knowledge. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge: The construct and its implications for science education (pp. 147–161). Dordrecht: Kluwer Academic.

Bektas, O., Ekiz, B., Tuysuz, M., Kutucu, E. S., Tarkin, A., & Uzuntiryaki-Kondakçı, E. (2013). Pre-service chemistry teachers’ pedagogical content knowledge of the nature of science in the particle nature of matter. Chemistry Education Research and Practice, 14 (2), 201–213.

Bennett, J., Lubben, F., & Hogarth, S. (2007). Bringing science to life: A synthesis of the research evidence on the effects of context-based and STS approaches to science teaching. Science Education, 91 (3), 347–370.

Bergqvist, A., & Chang Rundgren, S.-N. (2017). The influence of textbooks on teachers’ knowledge of chemical bonding representations relative to students’ difficulties understanding. Research in Science & Technological Education, 35 (2), 215–237.

Bergqvist, A., Drechsler, M., & Chang Rundgren, S.-N. (2016). Upper secondary teachers’ knowledge for teaching chemical bonding models. International Journal of Science Education, 38 (2), 298–318.

Berry, A., Depaepe, F., & van Driel, J. H. (2016). Pedagogical content knowledge in teacher education. In J. Loughran & M. L. Hamilton (Eds.), International handbook of teacher education (Vol. 1, pp. 347–386). Singapore: Springer Singapore.

Chapter   Google Scholar  

Berry, A., Friedrichsen, P. J., & Loughran, J. (2015). Re-examining pedagogical content knowledge in science education . New York: Routledge.

Book   Google Scholar  

Bindernagel, J. A., & Eilks, I. (2009). Evaluating roadmaps to portray and develop chemistry teachers’ PCK about curricular structures concerning sub-microscopic models. Chemistry Education Research and Practice, 10 (2), 77–85.

Blömeke, S., Suhl, U., & Kaiser, G. (2011). Teacher education effectiveness: Quality and equity of future primary teachers’ mathematics and mathematics pedagogical content knowledge. Journal of Teacher Education, 62 (2), 154–171.

Boesdorfer, S., & Lorsbach, A. (2014). PCK in action: Examining one chemistry teacher’s practice through the lens of her orientation toward science teaching. International Journal of Science Education, 36 (13), 2111–2132.

Bravo, P., & Cofré, H. (2016). Developing biology teachers’ pedagogical content knowledge through learning study: The case of teaching human evolution. International Journal of Science Education, 38 (16), 2500–2527.

Brown, P., Friedrichsen, P. J., & Abell, S. K. (2013). The development of prospective secondary biology teachers PCK. Journal of Science Teacher Education, 24 (1), 133–155.

Burton, E. P. (2013). Student work products as a teaching tool for nature of science pedagogical knowledge: A professional development project with in-service secondary science teachers. Teaching and Teacher Education, 29, 156–166.

Cetin-Dindar, A., Boz, Y., Yildiran Sonmez, D., & Demirci Celep, N. (2018). Development of pre-service chemistry teachers’ technological pedagogical content knowledge. Chemistry Education Research and Practice .

Chan, K. K. H., & Yung, B. H. W. (2015). On-site pedagogical content knowledge development. International Journal of Science Education, 37 (8), 1246–1278.

Chan, K. K. H., & Yung, B. H. W. (2017). Developing pedagogical content knowledge for teaching a new topic: More than teaching experience and subject matter knowledge. Research in Science Education , 1–33.

Charalambous, C. Y. (2016). Investigating the knowledge needed for teaching mathematics. Journal of Teacher Education, 67 (3), 220–237.

Chen, B., & Wei, B. (2015). Examining chemistry teachers’ use of curriculum materials: in view of teachers’ pedagogical content knowledge. Chemistry Education Research and Practice, 16 (2), 260–272.

Cochran, K. F., DeRuiter, J. A., & King, R. A. (1993). Pedagogical content knowing: An integrative model for teacher preparation. Journal of Teacher Education, 44 (4), 263–272.

Cochran, K. F., & Jones, L. L. (1998). The subject matter knowledge of preservice science teachers. In B. J. Fraser & K. G. Tobin (Eds.), International handbook of science education (Vol. 2, pp. 707–718). Dordrecht: Kluwer.

Cohen, R., & Yarden, A. (2009). Experienced junior-high-school teachers’ PCK in light of a curriculum change: “The cell is to be studied longitudinally”. Research in Science Education, 39 (1), 131–155.

Cook, S. D. N., & Brown, J. S. (1999). Bridging epistemologies: The generative dance between organizational knowledge and organizational knowing. Organization Science, 10 (4), 381–400.

Daehler, K. R., & Shinohara, M. (2001). A complete circuit is a complete circle: Exploring the potential of case materials and methods to develop teachers’ content knowledge and pedagogical content knowledge of science. Research in Science Education, 31 (2), 267–288.

Davidowitz, B., & Potgieter, M. (2016). Use of the Rasch measurement model to explore the relationship between content knowledge and topic-specific pedagogical content knowledge for organic chemistry. International Journal of Science Education, 38 (9), 1483–1503.

Davis, E., & Krajcik, J. (2005). Designing educative curriculum materials to promote teacher learning. Educational Researcher, 34 (3), 3–14.

Demirdöğen, B. (2016). Interaction between science teaching orientation and pedagogical content knowledge components. Journal of Science Teacher Education, 27 (5), 495–532.

Demirdöğen, B., Hanuscin, D. L., Uzuntiryaki-Kondakçı, E., & Köseoğlu, F. (2016). Development and nature of preservice chemistry teachers’ pedagogical content knowledge for nature of science. Research in Science Education, 46 (6), 831–855.

Demirdöğen, B., & Uzuntiryaki-Kondakçı, E. (2016). Closing the gap between beliefs and practice: Change of pre-service chemistry teachers’ orientations during a PCK-based NOS course. Chemistry Education Research and Practice, 17 (4), 818–841.

Denzin, N. K. (1989). The research act: A theoretical introduction to sociological methods (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.

Depaepe, F., Verschaffel, L., & Kelchtermans, G. (2013). Pedagogical content knowledge: A systematic review of the way in which the concept has pervaded mathematics educational research. Teaching and Teacher Education, 34, 12–25.

Diezmann, C. M., & Watters, J. J. (2015). The knowledge base of subject matter experts in teaching: A case study of a professional scientist as a beginning teacher. International Journal of Science and Mathematics Education, 13 (6), 1517–1537.

Donnelly, D. F., & Hume, A. (2015). Using collaborative technology to enhance pre-service teachers’ pedagogical content knowledge in Science. Research in Science & Technological Education, 33 (1), 61–87.

Driver, R., Asoko, H., Leach, J., Scott, P., & Mortimer, E. (1994). Constructing scientific knowledge in the classroom. Educational Researcher, 23 (7), 5–12.

Faikhamta, C., & Clarke, A. (2012). A self-study of a Thai teacher educator developing a better understanding of PCK for teaching about teaching science. Research in Science Education, 43 (3), 955–979.

Falk, A. (2012). Teachers learning from professional development in elementary science: Reciprocal relations between formative assessment and pedagogical content knowledge. Science Education, 96 (2), 265–290.

Fernández-Balboa, J.-M., & Stiehl, J. (1995). The generic nature of pedagogical content knowledge among college professors. Teaching and Teacher Education, 11 (3), 293–306.

Findlay, M., & Bryce, T. G. K. (2012). From teaching physics to teaching children: Beginning teachers learning from pupils. International Journal of Science Education, 34 (17), 2727–2750.

Fischer, H. E., Borowski, A., & Tepner, O. (2012). Professional knowledge of science teachers. In B. J. Fraser, K. Tobin, & C. J. McRobbie (Eds.), Second international handbook of science education (pp. 435–448). Dordrecht: Springer Netherlands.

Förtsch, C., Werner, S., von Kotzebue, L., & Neuhaus, B. J. (2016). Effects of biology teachers’ professional knowledge and cognitive activation on students’ achievement. International Journal of Science Education, 38 (17), 2642–2666.

Fraser, S. P. (2016). Pedagogical content knowledge (PCK): Exploring its usefulness for science lecturers in higher education. Research in Science Education, 46 (1), 141–161.

Friedrichsen, P. J., Abell, S. K., Pareja, E. M., Brown, P. L., Lankford, D. M., & Volkmann, M. J. (2009). Does teaching experience matter? Examining biology teachers’ prior knowledge for teaching in an alternative certification program. Journal of Research in Science Teaching, 46 (4), 357–383.

Friedrichsen, P. J., van Driel, J. H., & Abell, S. K. (2011). Taking a closer look at science teaching orientations. Science Education, 95 (2), 358–376.

Geddis, A. N., Onslow, B., Beynon, C., & Oesch, J. (1993). Transforming content knowledge: Learning to teach about isotopes. Science Education, 77 (6), 575–591.

Gess-Newsome, J. (1999). Pedagogical content knowledge: An introduction and orientation. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge: The construct and its implications for science education (pp. 3–20). Dordrecht: Kluwer Academic.

Gess-Newsome, J. (2015). A model of teacher professional knowledge and skill including PCK: Results of the thinking from the PCK Summit. In A. Berry, P. J. Friedrichsen, & J. Loughran (Eds.), Re-examining pedagogical content knowledge in science education (pp. 28–42). New York: Routledge.

Gess-Newsome, J., Taylor, J. A., Carlson, J., Gardner, A. L., Wilson, C. D., & Stuhlsatz, M. A. M. (2017). Teacher pedagogical content knowledge, practice, and student achievement. International Journal of Science Education , 1–20.

Grossman, P. L. (1990). The making of a teacher: Teacher knowledge and teacher education . New York: Teachers College Press.

Großschedl, J., Harms, U., Kleickmann, T., & Glowinski, I. (2015). Preservice biology teachers’ professional knowledge: Structure and learning opportunities. Journal of Science Teacher Education, 26 (3), 291–318.

Großschedl, J., Mahler, D., Kleickmann, T., & Harms, U. (2014). Content-related knowledge of biology teachers from secondary schools: Structure and learning opportunities. International Journal of Science Education, 36 (14), 2335–2366.

Guerriero, S. (2017). Pedagogical knowledge and the changing nature of the teaching profession . Paris: OECD Publishing.

Hallman-Thrasher, A., Connor, J., & Sturgill, D. (2017). Strong discipline knowledge cuts both ways for novice mathematics and science teachers. International Journal of Science and Mathematics Education .

Hanuscin, D. L. (2013). Critical incidents in the development of pedagogical content knowledge for teaching the nature of science: A prospective elementary teacher’s journey. Journal of Science Teacher Education, 24 (6), 933–956.

Hanuscin, D. L., Lee, M. H., & Akerson, V. L. (2011). Elementary teachers’ pedagogical content knowledge for teaching the nature of science. Science Education, 95 (1), 145–167.

Hashweh, M. Z. (2005). Teacher pedagogical constructions: A reconfiguration of pedagogical content knowledge. Teachers and Teaching: Theory and Practice, 11 (3), 273–292.

Heller, J. I., Daehler, K. R., Wong, N., Shinohara, M., & Miratrix, L. W. (2012). Differential effects of three professional development models on teacher knowledge and student achievement in elementary science. Journal of Research in Science Teaching, 49 (3), 333–362.

Henze, I., van Driel, J. H., & Verloop, N. (2008). Development of experienced science teachers’ pedagogical content knowledge of models of the solar system and the universe. International Journal of Science Education, 30 (10), 1321–1342.

Hill, H. C., Ball, D. L., & Schilling, S. G. (2008). Unpacking pedagogical content knowledge: Conceptualizing and measuring teachers’ topic-specific knowledge of students. Journal for Research in Mathematics Education, 39 (4), 372–400.

Jackson, P. W. (1986). Life in classrooms . New York: Holt, Rinehart & Winston.

Jin, H., Shin, H., Johnson, M. E., Kim, J., & Anderson, C. W. (2015). Developing learning progression-based teacher knowledge measures. Journal of Research in Science Teaching, 52 (9), 1269–1295.

Jüttner, M., & Neuhaus, B. J. (2012). Development of items for a pedagogical content knowledge test based on empirical analysis of pupils’ errors. International Journal of Science Education, 34 (7), 1125–1143.

Kaiser, G., Blömeke, S., König, J., Busse, A., Döhrmann, M., & Hoth, J. (2017). Professional competencies of (prospective) mathematics teachers—Cognitive versus situated approaches. Educational Studies in Mathematics, 94 (2), 161–182.

Kanter, D. E., & Konstantopoulos, S. (2010). The impact of a project-based science curriculum on minority student achievement, attitudes, and careers: The effects of teacher content and pedagogical content knowledge and inquiry-based practices. Science Education, 94 (5), 855–887.

Käpylä, M., Heikkinen, J. P., & Asunta, T. (2009). Influence of content knowledge on pedagogical content knowledge: The case of teaching photosynthesis and plant growth. International Journal of Science Education, 31 (10), 1395–1415.

Kaya, O. N. (2009). The nature of relationships among the components of pedagogical content knowledge of preservice science teachers: ‘Ozone layer depletion’ as an example. International Journal of Science Education, 31 (7), 961–988.

Keller, M. M., Neumann, K., & Fischer, H. E. (2017). The impact of physics teachers’ pedagogical content knowledge and motivation on students’ achievement and interest. Journal of Research in Science Teaching, 54 (5), 586–614.

Kellner, E., Gullberg, A., Attorps, I., Thorén, I., & Tärneberg, R. (2011). Prospective teachers’ initial conceptions about pupils’ difficulties in science and mathematics: A potential resource in teacher education. International Journal of Science and Mathematics Education, 9 (4), 843–866.

Kersting, N. B. (2008). Using video clips of mathematics classroom instruction as item prompts to measure teachers’ knowledge of teaching mathematics. Educational and Psychological Measurement, 68 (5), 845–861.

Kersting, N. B., Sutton, T., Kalinec-Craig, C., Stoehr, K. J., Heshmati, S., Lozano, G., et al. (2016). Further exploration of the classroom video analysis (CVA) instrument as a measure of usable knowledge for teaching mathematics: Taking a knowledge system perspective. ZDM Mathematics Education, 48 (1), 97–109.

Khourey-Bowers, C., & Fenk, C. (2009). Influence of constructivist professional development on chemistry content knowledge and scientific model development. Journal of Science Teacher Education, 20 (5), 437–457.

Kind, V. (2009). Pedagogical content knowledge in science education: Perspectives and potential for progress. Studies in Science Education, 45 (2), 169–204.

Kind, V. (2017). Development of evidence-based, student-learning-oriented rubrics for pre-service science teachers’ pedagogical content knowledge. International Journal of Science Education , 1–33.

Kirschner, S., Borowski, A., Fischer, H. E., Gess-Newsome, J., & von Aufschnaiter, C. (2016). Developing and evaluating a paper-and-pencil test to assess components of physics teachers’ pedagogical content knowledge. International Journal of Science Education, 38 (8), 1343–1372.

Knievel, I., Lindmeier, A. M., & Heinze, A. (2015). Beyond knowledge: Measuring primary teachers’ subject-specific competences in and for teaching Mathematics with items based on video vignettes. International Journal of Science and Mathematics Education, 13 (2), 309–329.

König, J., Blömeke, S., Klein, P., Suhl, U., Busse, A., & Kaiser, G. (2014). Is teachers’ general pedagogical knowledge a premise for noticing and interpreting classroom situations? A video-based assessment approach. Teaching and Teacher Education, 38, 76–88.

Krepf, M., Plöger, W., Scholl, D., & Seifert, A. (2017). Pedagogical content knowledge of experts and novices—What knowledge do they activate when analyzing science lessons? Journal of Research in Science Teaching , 1–23.

Lin, J.-W. (2016). Do skilled elementary teachers hold scientific conceptions and can they accurately predict the type and source of students’ preconceptions of electric circuits? International Journal of Science and Mathematics Education, 14 (2), 287–307.

Lin, J.-W. (2017). A comparison of experienced and preservice elementary school teachers’ content knowledge and pedagogical content knowledge about electric circuits. EURASIA Journal of Mathematics, Science and Technology Education, 13 (3).

Lin, J.-W., & Chiu, M. H. (2010). The mismatch between students’ mental models of acids/bases and their sources and their teacher’s anticipations thereof. International Journal of Science Education, 32 (12), 1617–1646.

Loughran, J., Milroy, P., Berry, A., Gunstone, R., & Mulhall, P. (2001). Documenting science teachers’ pedagogical content knowledge through PaP-eRs. Research in Science Education, 31 (2), 289–307.

Loughran, J., Mulhall, P., & Berry, A. (2004). In search of pedagogical content knowledge in science: Developing ways of articulating and documenting professional practice. Journal of Research in Science Teaching, 41 (4), 370–391.

Lucero, M. M., Petrosino, A. J., & Delgado, C. (2017). Exploring the relationship between secondary science teachers’ subject matter knowledge and knowledge of student conceptions while teaching evolution by natural selection. Journal of Research in Science Teaching, 54 (2), 219–246.

Luft, J. A. (2009). Beginning secondary science teachers in different induction programmes: The first year of teaching. International Journal of Science Education, 31 (17), 2355–2384.

Luft, J. A., Firestone, J. B., Wong, S. S., Ortega, I., Adams, K., & Bang, E. (2011). Beginning secondary science teacher induction: A two-year mixed methods study. Journal of Research in Science Teaching, 48 (10), 1199–1224.

Magnusson, S., Krajcik, J., & Borko, H. (1999). Nature, sources, and development of pedagogical content knowledge for science teaching. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge: The construct and its implications for science education (pp. 95–132). Dordrecht: Kluwer Academic.

Mahler, D., Großschedl, J., & Harms, U. (2017). Using doubly latent multilevel analysis to elucidate relationships between science teachers’ professional knowledge and students’ performance. International Journal of Science Education, 39 (2), 213–237.

Marks, R. (1990). Pedagogical content knowledge: From a mathematical case to a modified conception. Journal of Teacher Education, 41 (3), 3–11.

Marshall, J. C., Smart, J., & Alston, D. M. (2016). Development and validation of Teacher Intentionality of Practice Scale (TIPS): A measure to evaluate and scaffold teacher effectiveness. Teaching and Teacher Education, 59, 159–168.

Matthew, J. K., Tae Seob, S., & Punya, M. (2012). How do we measure TPACK? Let me count the ways. In N. R. Robert, R. R. Christopher, & L. N. Margaret (Eds.), Educational technology, teacher knowledge, and classroom impact: A research handbook on frameworks and approaches (pp. 16–31). Hershey, PA, USA: IGI Global.

Mavhunga, E. (2016). Transfer of the pedagogical transformation competence across chemistry topics. Chemistry Education Research and Practice, 17 (4), 1081–1097.

Mavhunga, E., & Rollnick, M. (2013). Improving PCK of chemical equilibrium in pre-service teachers. African Journal of Research in Mathematics, Science and Technology Education, 17 (1–2), 113–125.

Mavhunga, E., & Rollnick, M. (2016). Teacher- or learner-centred? Science teacher beliefs related to topic specific pedagogical content knowledge: A South African case study. Research in Science Education, 46 (6), 831–855.

McNeill, K. L., González-Howard, M., Katsh-Singer, R., & Loper, S. (2016). Pedagogical content knowledge of argumentation: Using classroom contexts to assess high-quality PCK rather than pseudoargumentation. Journal of Research in Science Teaching, 53 (2), 261–290.

Mellado, V., Bermejo, M. L., Blanco, L. J., & Ruiz, C. (2007). The classroom practice of a prospective secondary biology teacher and his conceptions of the nature of science and of teaching and learning science. International Journal of Science and Mathematics Education, 6 (1), 37–62.

Melo-Niño, L. V., Cañada, F., & Mellado, V. (2017a). Exploring the emotions in pedagogical content knowledge about the electric field. International Journal of Science Education, 39 (8), 1025–1044.

Melo-Niño, L. V., Cañada, F., & Mellado, V. (2017b). Initial characterization of colombian high school physics teachers’ pedagogical content knowledge on electric fields. Research in Science Education, 47 (1), 25–48.

Meschede, N., Fiebranz, A., Möller, K., & Steffensky, M. (2017). Teachers’ professional vision, pedagogical content knowledge and beliefs: On its relation and differences between pre-service and in-service teachers. Teaching and Teacher Education, 66, 158–170.

Miller, M. (2007). Pedagogical content knowledge. In G. Bodner & M. Orgill (Eds.), Theoretical frameworks for research in chemistry/science education (pp. 86–106). Upper Saddle River, NJ: Pearson Prentice Hall.

Monet, J. A., & Etkina, E. (2008). Fostering self-reflection and meaningful learning: Earth science professional development for middle school science teachers. Journal of Science Teacher Education, 19 (5), 455–475.

Moodley, K., & Gaigher, E. (2017). Teaching electric circuits: Teachers’ perceptions and learners’ misconceptions. Research in Science Education .

Mthethwa-Kunene, E., Onwu, G. O., & de Villiers, R. (2015). Exploring biology teachers’ pedagogical content knowledge in the teaching of genetics in Swaziland science classrooms. International Journal of Science Education, 37 (7), 1140–1165.

National Council for Accreditation of Teacher Education. (2008). Professional standards for the accreditation of teacher preparation institutions . Washington, DC: National Council for Accreditation of Teacher Education.

National Research Council. (1996). The national science education standards . Washington, DC: National Academy Press.

Nilsson, P. (2014). When teaching makes a difference: Developing science teachers’ pedagogical content knowledge through learning study. International Journal of Science Education, 36 (11), 1794–1814.

Nilsson, P., & Elm, A. (2017). Capturing and developing early childhood teachers’ science pedagogical content knowledge through CoRes. Journal of Science Teacher Education , 1–19.

Nilsson, P., & Loughran, J. (2012). Exploring the development of pre-service science elementary teachers’ pedagogical content knowledge. Journal of Science Teacher Education, 23 (7), 699–721.

Nilsson, P., & van Driel, J. H. (2010). Teaching together and learning together—Primary science student teachers’ and their mentors’ joint teaching and learning in the primary classroom. Teaching and Teacher Education, 26 (6), 1309–1318.

Nilsson, P., & Vikström, A. (2015). Making PCK explicit—Capturing science teachers’ pedagogical content knowledge (PCK) in the science classroom. International Journal of Science Education, 37 (17), 2836–2857.

Oh, P. S., & Kim, K. S. (2013). Pedagogical transformations of science content knowledge in Korean elementary classrooms. International Journal of Science Education, 35 (9), 1590–1624.

Park, S., & Chen, Y.-C. (2012). Mapping out the integration of the components of pedagogical content knowledge (PCK): Examples from high school biology classrooms. Journal of Research in Science Teaching, 49 (7), 922–941.

Park, S., Jang, J.-Y., Chen, Y.-C., & Jung, J. (2011). Is pedagogical content knowledge (PCK) necessary for reformed science teaching?: Evidence from an empirical study. Research in Science Education, 41 (2), 245–260.

Park, S., & Oliver, J. S. (2008a). National Board Certification (NBC) as a catalyst for teachers’ learning about teaching: The effects of the NBC process on candidate teachers’ PCK development. Journal of Research in Science Teaching, 45 (7), 812–834.

Park, S., & Oliver, J. S. (2008b). Revisiting the conceptualisation of pedagogical content knowledge (PCK): PCK as a conceptual tool to understand teachers as professionals. Research in Science Education, 38 (3), 261–284.

Park, S., Suh, J., & Seo, K. (2017). Development and validation of measures of secondary science teachers’ PCK for teaching photosynthesis. Research in Science Education , 1–25.

Paulick, I., Großschedl, J., Harms, U., & Möller, J. (2016). Preservice teachers’ professional knowledge and its relation to academic self-concept. Journal of Teacher Education, 67 (3), 173–182.

Piliouras, P., Plakitsi, K., Seroglou, F., & Papantoniou, G. (2017). Teaching explicitly and reflecting on elements of nature of science: A discourse-focused professional development program with four fifth-grade teachers. Research in Science Education , 1–26.

Qhobela, M., & Kolitsoe Moru, E. (2014). Examining secondary school physics teachers’ beliefs about teaching and classroom practices in Lesotho as a foundation for professional development. International Journal of Science and Mathematics Education, 12 (6), 1367–1392.

Rollnick, M. (2017). Learning about semi conductors for teaching—The role played by content knowledge in Pedagogical Content Knowledge (PCK) development. Research in Science Education , 1–36.

Rollnick, M., Bennett, J., Rhemtula, M., Dharsey, N., & Ndlovu, T. (2008). The place of subject matter knowledge in pedagogical content knowledge: A case study of South African teachers teaching the amount of substance and chemical equilibrium. International Journal of Science Education, 30 (10), 1365–1387.

Rosenkränzer, F., Hörsch, C., Schuler, S., & Riess, W. (2017). Student teachers’ pedagogical content knowledge for teaching systems thinking: Effects of different interventions. International Journal of Science Education , 1–20.

Roth, K. J., Garnier, H. E., Chen, C., Lemmens, M., Schwille, K., & Wickler, N. I. Z. (2011). Videobased lesson analysis: Effective science PD for teacher and student learning. Journal of Research in Science Teaching, 48 (2), 117–148.

Sadler, P. M., Sonnert, G., Coyle, H. P., Cook-Smith, N., & Miller, J. L. (2013). The Influence of teachers’ knowledge on student learning in middle school physical science classrooms. American Educational Research Journal, 50 (5), 1020–1049.

Salloum, S. L., & BouJaoude, S. (2008). Careful! It is H 2 O? Teachers’ conceptions of chemicals. International Journal of Science Education, 30 (1), 33–64.

Scharfenberg, F.-J., & Bogner, F. X. (2016). A new role change approach in pre-service teacher education for developing pedagogical content knowledge in the context of a student outreach lab. Research in Science Education, 46 (5), 743–766.

Schmelzing, S., van Driel, J. H., Jüttner, M., Brandenbusch, S., Sandmann, A., & Neuhaus, B. J. (2013). Development, evaluation and validation of a paper-and-pencil test for measuring two components of biology teachers’ pedagogical content knowledge concerning the “cardiovascular system” International Journal of Science and Mathematics Education, 11 (6), 1369–1390.

Schneider, R. M., & Plasman, K. (2011). Science teacher learning progressions. Review of Educational Research, 81 (4), 530–565.

Settlage, J. (2013). On acknowledging PCK’s shortcomings. Journal of Science Teacher Education, 24 (1), 1–12.

Seung, E. (2013). The process of physics teaching assistants’ pedagogical content knowledge development. International Journal of Science and Mathematics Education, 11 (6), 1303–1326.

Shavelson, R. J., Ruiz-Primo, M. A., & Wiley, E. W. (2005). Windows into the mind. Higher Education, 49 (4), 413–430.

Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15 (2), 4–14.

Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57 (1), 1–22.

Shulman, L. S. (2015). PCK: Its genesis and exodus. In A. Berry, P. J. Friedrichsen, & J. Loughran (Eds.), Re-examining pedagogical content knowledge in science education (pp. 3–13). New York: Routledge.

Sickel, A. J., & Friedrichsen, P. J. (2017). Using multiple lenses to examine the development of beginning biology teachers’ pedagogical content knowledge for teaching natural selection simulations. Research in Science Education , 1–42.

Smit, R., Rietz, F., & Kreis, A. (2017). What are the effects of science lesson planning in peers?—Analysis of attitudes and knowledge based on an actor–partner interdependence model. Research in Science Education , 1–18.

Smit, R., Weitzel, H., Blank, R., Rietz, F., Tardent, J., & Robin, N. (2017). Interplay of secondary pre-service teacher content knowledge (CK), pedagogical content knowledge (PCK) and attitudes regarding scientific inquiry teaching within teacher training. Research in Science & Technological Education , 1–23.

Smith, S. P., & Banilower, E. R. (2015). Assessing PCK: A new application of the uncertainty principle. In A. Berry, P. J. Friedrichsen, & J. Loughran (Eds.), Re-examining pedagogical content knowledge in science education (pp. 88–103). New York: Routledge.

Sorge, S., Kröger, J., Petersen, S., & Neumann, K. (2017). Structure and development of pre-service physics teachers’ professional knowledge. International Journal of Science Education , 1–28.

Stasinakis, P. K., & Athanasiou, K. (2016). Investigating Greek biology teachers’ attitudes towards evolution teaching with respect to their pedagogical content knowledge: Suggestions for their professional development. EURASIA Journal of Mathematics, Science & Technology Education, 13, 1605–1617.

Stender, A., Brückmann, M., & Neumann, K. (2017). Transformation of topic-specific professional knowledge into personal pedagogical content knowledge through lesson planning. International Journal of Science Education , 1–25.

Suh, J., & Park, S. (2017). Exploring the relationship between pedagogical content knowledge (PCK) and sustainability of an innovative science teaching approach. Teaching and Teacher Education, 64, 246–259.

Supprakob, S., Faikhamta, C., & Suwanruji, P. (2016). Using the lens of pedagogical content knowledge for teaching the nature of science to portray novice chemistry teachers’ transforming NOS in early years of teaching profession. Chemistry Education Research and Practice, 17 (4), 1067–1080.

Tamir, P. (1988). Subject matter and related pedagogical knowledge in teacher education. Teaching & Teacher Education, 4, 99–110.

Tay, S. L., & Yeo, J. (2017). Analysis of a physics teacher’s pedagogical ‘micro-actions’ that support 17-year-olds’ learning of free body diagrams via a modelling approach. International Journal of Science Education , 1–30.

Tepner, O., Borowski, A., Dollny, S., Fischer, H. E., Jüttner, M., Kirschner, S., et al. (2012). Modell zur Entwicklung von Testitems zur Erfassung des Professionswissens von Lehrkräften in den Naturwissenschaften [Item development model for assessing professional knowledge of science teachers]. Zeitschrift für Didaktik der Naturwissenschaften, 18, 7–28.

Uzuntiryaki-Kondakçı, E., Demirdöğen, B., Akin, F. N., Tarkin, A., & Aydin, S. (2017). Exploring the complexity of teaching: The interaction between teacher self-regulation and pedagogical content knowledge. Chemistry Education Research and Practice, 18 (1), 250–270.

van Dijk, E. M. (2009). Teachers’ views on understanding evolutionary theory: A PCK-study in the framework of the ERTE-model. Teaching and Teacher Education, 25 (2), 259–267.

van Dijk, E. M. (2014). Understanding the heterogeneous nature of science: A comprehensive notion of PCK for scientific literacy. Science Education, 98 (3), 397–411.

van Dijk, E. M., & Kattmann, U. (2007). A research model for the study of science teachers’ PCK and improving teacher education. Teaching and Teacher Education, 23 (6), 885–897.

van Driel, J. H., & Abell, S. K. (2010). Science teacher education. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (3rd ed., pp. 712–718). Oxford: Elsevier.

van Driel, J. H., & Berry, A. (2010). The teacher education knowledge base: Pedagogical content knowledge. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (3rd ed., pp. 656–661). Oxford: Elsevier.

van Driel, J. H., & Berry, A. (2012). Teacher professional development focusing on pedagogical content knowledge. Educational Researcher, 41 (1), 26–28.

van Driel, J. H., & Berry, A. (2017). Developing pre-service teachers’ pedagogical content knowledge. In D. J. Clandinin & J. Husu (Eds.), The SAGE handbook of research on teacher education (Vol. 2, pp. 561–576). Thousand Oaks: California SAGE Publications Ltd.

van Driel, J. H., Berry, A., & Meirink, J. A. (2014). Research on science teacher knowledge. In N. G. Lederman & S. K. Abell (Eds.), Handbook of research on science education (Vol. 2, pp. 848–870). New York, NY: Routledge.

van Driel, J. H., Verloop, N., & de Vos, W. (1998). Developing science teachers’ pedagogical content knowledge. Journal of Research in Science Teaching, 35 (6), 673–695.

Veal, W. R., & MaKinster, J. (1999). Pedagogical content knowledge taxonomies. Electronic Journal of Science Education, 3 (4).

Walan, S., Nilsson, P., & Ewen, B. M. (2017). Why inquiry? Primary teachers’ objectives in choosing inquiry- and context-based instructional strategies to stimulate students’ science learning. Research in Science Education , 1–20.

Wang, Y.-L., Tsai, C.-C., & Wei, S.-H. (2015). The sources of science teaching self-efficacy among elementary school teachers: A mediational model approach. International Journal of Science Education, 37 (14), 2264–2283.

Willermark, S. (2017). Technological pedagogical and content knowledge: A review of empirical studies published from 2011 to 2016. Journal of Educational Computing Research , 1–28.

Wilson, S. M., Shulman, L. S., & Richert, E. R. (1987). ‘150 different ways’ of knowing: Representations of knowledge in Teaching. In J. Calderhead (Ed.), Exploring teachers’ thinking . London: Cassell.

Wongsopawiro, D. S., Zwart, R. C., & van Driel, J. H. (2017). Identifying pathways of teachers’ PCK development. Teachers and Teaching, 23 (2), 191–210.

Zembylas, M. (2007). Emotional ecology: The intersection of emotional knowledge and pedagogical content knowledge in teaching. Teaching and Teacher Education, 23 (4), 355–367.

Zhou, S., Wang, Y., & Zhang, C. (2016). Pre-service science teachers’ PCK: Inconsistency of pre-service teachers’ predictions and student learning difficulties in Newton’s Third Law. EURASIA Journal of Mathematics, Science and Technology Education, 12 (3), 373–385.

Download references

Acknowledgements

This research was supported by the Early Career Scheme of the Research Grants Council of Hong Kong [Project Number 27608717].

Author information

Authors and affiliations.

The University of Hong Kong, Hong Kong, Hong Kong

Kennedy Kam Ho Chan

University of Waikato, Hamilton, New Zealand

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Kennedy Kam Ho Chan .

Editor information

Editors and affiliations.

Monash University, Melbourne, VIC, Australia

Rebecca Cooper

University of Potsdam, Potsdam, Germany

Andreas Borowski

An overview of the studies reviewed. Note all reviewed articles can be found in the reference list for this chapter.

1. The number of years of teaching experience (in general) of in-service teachers is given in parentheses. ‘ M ’ denotes the mean number of teaching years of the teachers in the study.

2. Grade level refers to the grades that the teachers taught or for which they were certified. Teachers from grade 1 to 6 were labelled as primary teachers, whereas those from grade 7 to 12 were categorised as secondary teachers. Studies that involved middle school teachers were categorised as investigating both primary and secondary teachers. If the studies specified that the target participants were middle school teachers, middle school was also included in the above table.

3. If teachers from more than two subject domains were included, the subject was listed as ‘science’ or ‘science and non-science’.

4. The subject domain of primary teachers is not shown as primary teachers are often generalists rather than subject specialists.

5. If more than two science topics were investigated in the study, the topics were labelled as ‘multiple’.

6. If the topics under investigation were not clearly listed, the topic was categorised as ‘not specified’.

7. Only teachers, not scientists, were included in the determination of the sample size in Kirschner et al. ( 2016 ) and Schmelzing et al. ( 2013 ).

8. As Salloum and BouJaoude ( 2008 ) adopted purposeful sampling of chemistry teachers, the subject domain was considered as ‘chemistry’.

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this chapter

Chan, K.K.H., Hume, A. (2019). Towards a Consensus Model: Literature Review of How Science Teachers’ Pedagogical Content Knowledge Is Investigated in Empirical Studies. In: Hume, A., Cooper, R., Borowski, A. (eds) Repositioning Pedagogical Content Knowledge in Teachers’ Knowledge for Teaching Science. Springer, Singapore. https://doi.org/10.1007/978-981-13-5898-2_1

Download citation

DOI : https://doi.org/10.1007/978-981-13-5898-2_1

Published : 29 January 2019

Publisher Name : Springer, Singapore

Print ISBN : 978-981-13-5897-5

Online ISBN : 978-981-13-5898-2

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 14 May 2024

Linking organisational learning, performance, and sustainable performance in universities: an empirical study in Europe

• Roba Elbawab   ORCID: orcid.org/0000-0003-2152-229X 1 , 2  

Humanities and Social Sciences Communications volume  11 , Article number:  626 ( 2024 ) Cite this article

14 Accesses

1 Altmetric

Metrics details

• Business and management

Universities are facing changes that could be adapted by learning. Organisational learning helps universities in attaining better organisational and sustainable performance. The study aims to combine and explore how organisational learning culture enables organisational learning to contribute to better organisational performance and better sustainable performance, following the natural resource-based view and organisational learning theory. The study examines the relationship between organisational learning culture, organisational learning, organisational performance, and sustainable performance in the university context from university teachers. The author collected 221 surveys from public university teachers in Europe to test the model. The results indicate a positive relationship between organisational learning culture and organisational learning. In addition to that, the positive relationship between organisational learning and organisational performance is indicated. Moreover, the results indicate a positive relationship between organisational learning and sustainable performance. The results also show that the organisational learning process mediates organisational learning culture and university performance. The study addresses a gap in the scarce studies in the university context for organisational learning and sustainable performance. Finally, this study reproduces an organisational model that has been adapted for universities.

Similar content being viewed by others

Do staff capacity and performance-based budgeting improve organisational performance? Empirical evidence from Chinese public universities

Do subcultures play a role in facilitating academic quality?—A case study of a Saudi higher education institution

The relationship between strategic human resource management, green innovation and environmental performance: a moderated-mediation model

Introduction.

Universities are facing different types of change, including digitalisation, sustainability, entrepreneurship, and innovation (Leal Filho et al. 2018 ; Pocol et al. 2022 ); it is the universities’ obligation to cope with this change (Medne et al. 2022 ; James et al. 1993 ). One of the proven ways that help universities adapt to change and increase their performance is learning, more specifically, organisational learning (Kezar and Holcombe 2019 ). In fact, organisational learning practices and processes can facilitate change and enhance organisations (Argyris and Schön 1996 ; Fiol and Lyles 1985 ; Garvin 1993 ; Huber 1991 ).

The topic of organisational learning has been discussed since the early nineties when the foundations of organisational learning were further developed during this era (Castaneda et al. 2018 ). Researchers called for more research to understand organisational learning. Organisations learn when there is information processing that leads to a change in the behaviour and the acquisition of knowledge, skills, and abilities (Kezar and Holcombe 2019 ; Flores et al. 2012 ; Jyothibabu et al. 2010 ; Jiménez Jiménez and SanzValle 2006 ; Slater and Narver 1995 ; Huber, 1991 ). Organisational learning researchers have extended their research to identify the organisational learning predictors to include organisational learning culture (Flores et al. 2012 ). Organisational learning has been discussed in several industries, but it is considered scarce in the university context (Abu‐Tineh 2011 ; Voolaid and Ehrlich 2017 ). Previous research focused on organisational learning capabilities and behaviours. However, the author will focus on organisational learning processes in this study. The organisational learning processes affect organisational performance (Bontis et al. 2002 ; Crossan and Bapuji 2003 ; Kontoghiorghes et al. 2005 ; Sun et al. 2008 ; Jyothibabu et al. 2010 ), as well as sustainability and sustainable performance (Iqbal and Ahmad 2021 ; Kordab et al. 2020 ; Bilan et al. 2020 ). However, this relationship in the university sector is understudied, with a gap in organisational learning literature. Especially since universities are considered a complex type of organisation (Elbawab 2022a ; Bleiklie and Kogan 2007 ). Sustainable development was defined as ‘development which meets the needs of the present, without compromising the ability of future generations to meet their own needs’ by the Brundtland Commission in 1987 (Basiago 1995 ). Sustainable development and learning share many important elements, including “a challenge to mental models, fostering fundamental change, engaging in extensive collaborative activity and, in some cases, revisiting core assumptions about business and its purpose” (Molnar and Mulvihill 2003 , p. 168), therefore several scholars showed the need to understand the relationship between learning in organisations and sustainability (Feeney et al. 2023 ). Sustainability is used in different domains, including economics and education (Pocol et al. 2022 ; Basiago 1995 ). In 2015, The United Nations General Assembly approved the ‘2030 Agenda for Sustainable Development’, which contains a set of measures aiming to balance economic progress and the protection of the environment (Leal Filho et al. 2018 ). The agenda consists of 17 Sustainable Development Goals (SDGs), which, among many other tasks, plan to eradicate poverty and create better health conditions in both developed and developing countries (Leal Filho et al. 2018 ). Sustainability in higher education institutions can be implemented in teaching, research, governance, and outreach (Leal Filho et al. 2023 ; Serafini et al. 2022 ). In fact, higher education’s growth contributes to society’s better sustainable development (Geng et al. 2023 ; Geng et al. 2020a ). Therefore, this is one of the reasons for the need to focus on studying sustainability in higher education. In the research area, sustainability research can be implemented by researchers from various areas who can work independently or collectively on the same project by combining their expertise (Leal Filho et al. 2023 ; Collin 2009 ). Additionally, sustainability can be implemented in research by framing higher education institutions’ research in the direction of the SDGs (Serafini et al. 2022 ). In the teaching area, sustainability can be implemented within the strategies in the curriculum development of promoting sustainability and in planning new courses (Leal Filho et al. 2023 ; Serafini et al. 2022 ). Sustainability in the teaching area can also be developed by modifying the existing curriculum with the SDGs (Leal Filho et al. 2023 ). As for governance, sustainability can be implemented by establishing indicators in rankings that evaluate the performance of higher education institutions concerning compliance with the SDGs. Sustainability within governance can also be implemented by evaluating the level of knowledge, awareness, and attitudes towards the SDGs among academic community members and educators (Serafini et al. 2022 ). Finally, for the outreach, sustainability can be implemented by disseminating SDGs by training the managers and decision-makers in civil society organisations (Serafini et al. 2022 ). In this study, the author will focus on the governance of sustainability in education. The relationship between organisational learning and sustainability has been discussed in several studies. A study developed in 2020 assessed the relationship between organisational learning and sustainable organisational performance (Kordab et al. 2020 ). Another study assessed the relationship between organisational learning and sustainability (Bilan et al. 2020 ). Subsequently, the lack of studies that evaluate organisational learning and sustainability in universities has emerged.

According to the natural resource-based view (NRBV) theory (Hart 1995 ), environmentally friendly resources and capabilities play a key competitive advantage in an organisation (Iqbal and Ahmad 2021 ; Hart 1995 ). The NRBV theory takes the capabilities and competitive advantage thinking one step further, where the theory posits that the organisation’s competitive advantage can only be sustained when the capabilities creating the advantage are supported by resources not easily duplicated by competitors (Hart 1995 ). In this study, the resource is the organisational learning. In organisational learning theory, organisational learning is defined as the change that occurs in the organisation, resulting from knowledge memorised in organisations gathered from experience and changes in behaviour resulting from such knowledge (Argote and Miron-Spektor 2011 ). These experiences and changes in behaviours that happen in the organisation are not easily duplicated by the competitors. To explain organisational learning more thoroughly and to show how organisational learning is a resource that is not easily duplicated, the author will further explain the types of learning that are crucial for the organisational learning theory (Crossan et al. 1999 ). Organisational learning has two types of learning. The first is characterised by improving the existing routines, and the second is characterised by reframing a situation or solving unclear problems (Edmondson 2002 ). Since the existing routines and situations will differ from one organisation to another, thus organisational learning can be considered a resource that is not easy to duplicate. Consequently, environmentally friendly resources should have a relationship with organisational learning. Subsequently, this study explores the relationship between organisational learning and sustainable performance (Environmental performance and social performance). The significance of the study is found at both empirical and theoretical levels. Theoretically, it is found to explore the influence of organisational learning as a process on university performance. Also, the influence of learning on sustainability and the role of learning culture among these relationships. This study adds to the theory of organisational learning and, specifically, how to treat organisational learning as a process. Moreover, another significant aspect of this study is addressing the mediation of the organisational learning process in the relationship between organisational learning culture and university performance, as organisational learning culture is proposed to directly affect higher education institutional performance (Kumar 2005 ). Furthermore, the significance of the study is found in exploring the relationship between organisational learning and sustainable performance in the university context. Finally, it empirically assesses organisational learning and all the relationships in the university’s context.

This study assesses the relationship between organisational learning culture and organisational learning processes. The study will also assess the organisational learning processes with the outcomes, where the relationship between the organisational learning process and university performance is assessed. The study will also assess the relationship between organisational learning processes and sustainable performance. Further, this research assesses the mediation between the organisational learning culture and university performance.

Literature review and hypothesis development

Organisational learning process.

Most scholars agree that organisational learning is known as the change in organisational knowledge, which is acquired through practical experiences (Argote and Miron-Spektor 2011 ), where this knowledge is then translated into the organisation’s knowledge system (Do et al. 2022 ). Organisational learning is defined as “the process by which organisations learn” (Chiva et al. 2007 , p. 224; Domínguez-Escrig et al. 2022 ). Organisational learning focuses not only on intentional learning but also on unintentional learning in the organisation (Huber 1991 ), as organisational learning helps reduce uncertainty (Schönherr et al. 2023 ). Learning can happen intentionally and unintentionally (Huber 1991 ). Intentional learning is the main process for scientists and educators. Researchers often think of it as an intentional process directed at improving effectiveness. In contrast, unintentional learning is proposed as unsystematic learning (Huber 1991 ). Even though previous research has focused on organisational learning as a culture or as an outcome, fewer have discovered organisational learning processes (Pham Thi Bich, Tran Quang 2016 ). Consequently, the author focuses on the organisational learning process due to the scarcity of research in this area. Subsequently, universities’ organisational learning will be better understood (Abu‐Tineh 2011 ) and could be enhanced.

Huber ( 1991 ) suggested that organisational learning includes four processes. The processes are information acquisition, knowledge dissemination, shared interpretation, and organisational memory (Huber 1991 ; Santos-Vijande et al. 2012 ). The relevant organisational learning processes in the university sector proposed by the study (Elbawab 2022b ) are information acquisition and knowledge dissemination. In this research, the author has empirically assessed the organisational learning processes and proposed that the relevant processes are information acquisition and knowledge dissemination. Hence, these are the processes used in this paper. The process of information acquisition is about acquiring information from various sources, either internally or externally (Huber 1991 ; Flores et al. 2012 ). The internal information is gathered from inside the organisation and from the company’s creator or previously acquired experience. As for the external information, it is gathered from the competitors and the marketplace through acknowledging and acquiring the implicit analysis of the actions of the competitors. On other occasions, organisations look for the best practices, and they solve the problems by identifying key tendencies, collecting external information, and comparing their performance with that of their relevant competitors (Santos-Vijande et al. 2012 ).

Knowledge dissemination is a process that takes place through formal settings (e.g., departmental meetings, discussion of future needs, and cross-training) and informal interactions among individuals within the organisation (Kofman and Senge 1993 ). The creation of formal networks and databases encourages communication by guaranteeing both the accuracy and the rapid dissemination of information. These initiatives need more informal exchange mechanisms to complement them so that any tacit knowledge individuals gather is transformed into explicit knowledge. Researchers perceive organisational learning as either an organisational process or an organisational capability. Organisational capability is the organisational and managerial characteristics that facilitate the organisational learning process or allow an organisation to learn (Aragón et al. 2014 ; Chiva et al. 2007 ; Tohidi et al. 2012 ). In the present study, organisational learning is viewed as a process that occurs inside the organisation on an organisational level. Organisational learning as a process focuses on the set of actions that occur in the organisation to help in the learning process. In the university context, researchers have called for more studies to understand the organisational learning process (Abu‐Tineh 2011 ). Voolaid and Ehrlich ( 2017 ) assessed organisational learning in two Estonian universities, but also from a cultural perspective and with the perception of the two universities. The researchers insisted on the scarcity of organisational learning research in higher education institutions and the need for a study that empirically assesses organisational learning in more universities from different countries and regions. Further, more research is needed to understand the multidimensionality from an aggregated perspective.

Organisational learning culture

Organisational learning culture is a general predictor of the organisational learning process. The organisational learning culture is essential for organisational learning (Flores et al. 2012 ). Higher education institutions need to adapt to the competition with new discoveries and ideas proactively. The development of a learning culture could be the key, to help through gathering, organising, sharing, and analysing the knowledge across the institution (Kumar 2005). Previously, several predictors have been mentioned for organisational learning, including knowledge-sharing behaviour (Park and Kim 2018 ), goal orientation (Chadwick and Raver 2012 ), participative decision-making, openness, learning orientation and transformational leadership (Flores et al. 2012 ). Flores et al. ( 2012 ) mentioned that these predictors are part of the culture, whereas organisational learning culture is considered a predictor that should be assessed in relation to organisational learning. Consequently, this study assesses organisational learning culture as a predictor of organisational learning. Pham Thi Bich and Tran Quang ( 2016 ) recommend that more predictors positively influencing organisational learning should be explored.

Organisational culture is a factor that facilitates organisational learning (e.g., Ahmed et al. 1999 ; Campbell and Cairns 1994 ; Conner and Clawson 2004 ; Maccoby 2003 ; Marquardt 1996 ; Marsick and Watkins 2003 ; Pedler et al. 1997 ; Rebelo and Duarte Gomes 2011 ). An organisational learning culture is described as the values, beliefs and assumptions that emphasise creating collective learning in an organisation (Sorakraikitikul and Siengthai 2014 ). Researchers have shown the importance of an organisational learning culture as a culture that creates a supportive environment. This culture enables and influences learning and knowledge sharing at the individual, team, and organisational levels (Kontoghiorghes et al. 2005 ; Marsick and Watkins 2003 ). Despite the importance of organisational learning culture in the literature (e.g., Marquardt 1996 ; Pedler et al. 1997 ), there is still a lack of research explicitly concerning learning culture (Rebelo and Duarte Gomes 2011 ) and its relationship with organisational learning. Also, a study developed by Wahda ( 2017 ) has agreed with the scarcity of studies that assess organisational learning culture in higher education institutions. In Wahda’s ( 2017 ) study, the results show that organisational learning culture is found in a university in Indonesia, and it also shows the importance of applying organisational learning culture in higher education institutions as it facilitates the learning processes. In conclusion, there is a lack of research addressing this relationship in the university context.

Previous research showed a positive relationship between participative decision making, openness and leadership and organisational learning (Flores et al. 2012 ). Since these predictors are considered part of the organisational culture, the author proposes a positive relationship between organisational learning culture and organisational learning. This study will explore the relationship between the organisational learning culture (as a predictor) and the organisational learning process in the university context. Accordingly, the author hypothesises:

H1 : There is a positive relationship between the organisational learning culture and the organisational learning process.

H1a : There is a positive relationship between system connection and dialogue and inquiry and information acquisition.

H1b : There is a positive relationship between system connection and dialogue and inquiry and knowledge dissemination.

Performance

Organisational performance.

The organisation’s performance depends on the achievement and the progress of the strategy identified by the organisation (Davies and Walters 2004 ; Mohammad 2019 ). Performance needs to meet the organisational strategies and the organisational goals because it shows the organisation’s success. Several studies have mentioned Organisational performance as an outcome of organisational learning (Aragón et al. 2014 ; Mohammad 2019 ). This research focuses on university performance. Few studies have focused on assessing the relationship between organisational learning and organisational performance for example (Bontis et al. 2002 ; Crossan and Bapuji 2003 ; Jyothibabu et al. 2010 ; Kontoghiorghes et al. 2005 ; Sun et al. 2008 ). Some previous empirical studies proposed the positive influence of organisational learning on organisational performance (Aragón et al. 2014 ; Mohammad 2019 ). According to previous research, organisational learning helps to improve the performance of an organisation.

Most previous research has focused on the relationship between organisational learning as a capability and performance (e.g., Camps and Luna-Arocas 2012 ; Hurley and Hult 1998 ; Keskin 2006 ; Rhodes et al. 2008 ). Nevertheless, the present study focuses on organisational learning as a process and its impact on university performance. In the context of universities, few empirical studies have shown a positive relationship between organisational learning and university performance (Guţă 2014 ; Pham Thi Bich and Tran Quang 2016 ). Guţă ( 2014 ) did not assess the relationship empirically, while Pham Thi Bich and Tran Quang ( 2016 ) study assessed university performance in only one university. More research is needed to assess the relationship between the organisational learning process and organisational performance (Pham Thi Bich and Tran Quang 2016 ). This paper assesses university performance from teachers’ opinions from several universities. From the previous research, the author hypothesised:

H2 : There is a positive relationship between organisational learning processes and university performance.

The relationship between organisational learning culture and organisational performance has also been discussed in the previous literature, where a positive relationship has been identified between organisational learning culture and organisational performance (Ellinger et al. 2002 ; Sorakraikitikul and Siengthai 2014 ). Organisational learning culture supports promoting and facilitating workers’ learning, hence contributing to organisational development and performance (Rebelo and Duarte Gomes 2011 ). Although there is little empirical evidence concerning the relationship between organisational learning culture and the performance of public organisations, some studies still allow us to infer that organisational learning culture is related to performance (Choi 2020 ). This paper assesses educators’ opinions in public universities, as public organisations are understudied. Hence, we hypothesised the following:

H3 : There is a positive relationship between the organisational learning culture and university performance.

The study will also assess the mediation of the organisational learning process in the relationship between organisational learning culture and university performance. Building on the previous hypotheses H1, H2 and H3, we propose that organisational learning culture solely is not sufficient to improve the university’s performance and that there is a need to involve organisational learning to enhance the university’s performance. Subsequently, we hypothesise the following:

H4 : The organisational learning process mediates the relationship between organisational learning culture and university performance.

Sustainable performance

Nowadays, sustainability has been called for in different business models (Zhang et al. 2019 ). The concept of sustainability helps organisations to improve different processes, which results in higher organisational performance (Zhang et al. 2019 ). Other studies have mentioned sustainability as an output of organisational learning (Kordab et al. 2020 ; Bilan et al. 2020 ; Iqbal and Ahmad 2021 ). The more learning that happens on an organisational level, the more sustainable the organisation is. In a study developed by Bilan et al. ( 2020 ), the authors advised that organisational learning significantly improves the firm’s sustainability. Other studies have focused on sustainable performance (Kordab et al. 2020 ; Iqbal and Ahmad 2021 ). Kordab and his colleagues mentioned the positive relationship between organisational learning and sustainable organisational performance (Kordab et al. 2020 ). The study developed by Iqbal and Ahmad ( 2021 ) states that organisational learning significantly influences sustainable performance. Another study developed in 5 companies in Norway and Italy has explored the internalisation of a sustainable environment through the learning process (Bianchi et al. 2022 ; Massimo and Nora 2022 ). The literature discussing the relationship between organisational learning and sustainability is scarce (Kordab et al. 2020 ). The earlier mentioned studies have assessed the relationship between Malaysian manufacturing organisations (Bilan et al. 2020 ), audit and consulting companies in the Middle East (Kordab et al. 2020 ), and small and medium-sized enterprises in Pakistan (Iqbal and Ahmad 2021 ). On the other hand, this study assesses the relationship between organisational learning and sustainable performance in the university context.

Sustainability in higher education institutions helps in the development of regenerative societies. This help is provided by educators as they influence ideologies and perspectives regarding sustainability in society (Leal Filho et al. 2023 ). In the review study developed by (Serafini et al. 2022 ) for the articles related to higher education institutions and SDGs, only four per cent of the studies considered professors as the target audience. Hence, in this study, the author assesses sustainable performance from educators’ perceptions as it is scarce. Therefore, the hypothesis is developed as below:

H5 : There is a positive association between organisational learning and sustainable performance.

H5a : There is a positive relationship between information acquisition, knowledge dissemination and sustainable environmental performance.

H5b : There is a positive relationship between information acquisition, knowledge dissemination and sustainable social performance.

Finally, Fig. 1 shows the proposed model of this study, which includes the proposed hypotheses.

Proposed research model.

Methodology

Data collection and sample.

This study’s sample mainly focused on university teachers from several European universities. Self-selection sampling is used in this paper; this method helps the researcher better explore the research area and understand the relationships (Saunders et al. 2007 ). The self-selection sampling method relies on the willingness of the participant to participate in the questionnaire. An email invitation with the link to the online questionnaire has been sent to a range of professors. Moreover, university teachers who accepted to participate are the ones who were considered in this study. The researcher has gathered the emails of the university teachers from the university websites.

Data were gathered through an online questionnaire Footnote 1 . The questionnaire is developed from the previous literature. The questionnaire was developed on Qualtrics, and an anonymous link was sent to the respondents. The researcher sent the questionnaire to 10366 university teachers. The university teachers are from different schools and departments, including business, psychology, science, and engineering schools. The researcher received 525 replies, which corresponds to a response rate of 5%. Of the 525 responses received, 304 were incomplete, so we excluded the incomplete questionnaires and kept 221 questionnaires. The questionnaire was sent to 53 public universities in Europe. The countries were Portugal, Spain, Italy, France, Germany and Greece.

The sample is composed of (36.7%) associate professors, (15.4%) assistant professors, (22%) full professors, (6.8%) lecturers and assistants, (3.2%) invited assistant/associate/full professors and (15.8%) respondents who did not declare their job level. Also, the majority of the sample (65.2%) has worked more than seven years in the same university and more than five years in the same team (66.5%). As for the age of the participants, (49.9%) were of age 50 years and above, and the majority of the sample (47.5%) was composed of Males, followed by (37.6%) Females and (14.9%) ‘don’t prefer to say’. The survey was conducted from October 2022 to January 2023. Two reminder emails were sent to the university teachers. Moreover, the design of this study is a correlational design, where all the proposed relationships are studied between organisational learning as a process and its antecedent and outcomes.

The constructs used to assess the indicators in this study are obtained from previous scientific studies, where their reliability and validity were previously tested and verified. Organisational learning culture was assessed using the measure of the Dimensions of Learning Organisations Questionnaire (Watkins and Marsick 1993 , 1997 ), which was adapted and validated to the university’s context in Elbawab ( 2022b ). The adapted measure included two sub-dimensions. The scale consisted of 8 items that measured the two sub-dimensions: dialogue and inquiry and system connection. The participants indicated to what extent they agreed with each of the eight items on a 7-point rating scale (1 = totally disagree, 7= totally agree). An example of the items is: “In my university, whenever academic staff state their view, they also ask what others think.” we checked the scales’ internal consistency to measure these indicators by calculating Cronbach’s alpha. The results indicate strong scale reliability for both system connection (0.88) and dialogue and inquiry (0.87).

The organisational learning process scale was assessed based on the Santos-Vijande et al. ( 2012 ) scale and then adapted to the university’s context in Elbawab ( 2022b ). The scale consists of 8 items that measure two subdimensions: information acquisition and knowledge dissemination. Individuals indicated to what extent they agreed with each of the eight items on a 7-point rating scale (1 = totally disagree, 7 = totally agree). An example of the items is: “We have a meeting schedule among departments and with the dean to integrate the existing information.” We checked the scales’ internal consistency to measure these indicators by calculating Cronbach’s alpha. The results indicate strong scale reliability for information acquisition (0.89) and knowledge dissemination (0.79).

As for the performance, one variable was used to evaluate university performance, and another was used to evaluate sustainable performance. The university performance questionnaire is based on Jyothibabu et al. ( 2010 ), but the scale is adapted to the university context. The measured scale includes seven items. All items were scored on a 7-point Likert scale (1 = totally disagree, 7= totally agree). An example of the items is: “There is continuous improvement in my university”. In this study, the Jyothibabu et al. 2010 scale has been adjusted from a 6-point to a 7-point Likert scale as the 7-point Likert scale reaches the upper limit of reliability (Allen and Seaman, 2007 ; Leung, 2011 ). Also, removing a neutral point introduces “a forced choice in the scale” (Allen and Seaman 2007 ), whereas our focus in this study is to avoid the forced choice. The reliability score of this scale in this study is 0.93; however, Jyothibabu et al. ( 2010 ) Cronbach alpha scored 0.90. In conclusion, the Cronbach alpha score in this study has improved. All factor loadings are significant ( p  < 0.05) and indicate strong factor loadings. As for sustainable performance, it is assessed based on Iqbal and Ahmad ( 2021 ), but only the environmental and social performance were adopted in this study. Since the economic performance measure mainly focuses on sales growth, income stability, profitability and return on investment. At the same time, the activities of public universities are driven by the pursuit of excellence and prestige maximisation, which does not necessarily imply economic efficiency traditionally assumed for profit-maximising business establishments (Kipesha and Msigwa 2013 ). Therefore, sustainable economic performance will not be assessed in this study for these reasons.

These measures are relevant to the university context, which follows the QS world rankings, where the environmental impact and the social impact of each university are addressed. The sustainable performance scale is adapted to the university context. All items were scored on a 7-point Likert scale (1 = totally disagree, 7 = totally agree). An example of the items is: “Your university is concerned about waste management”. The results indicate strong scale reliability for sustainable environmental performance (0.88) and sustainable social performance (0.82). All factor loadings are significant ( p  < 0.05) and indicate strong factor loadings.

Data analysis

The author assessed the descriptive statistics in this study by identifying the means and standard deviations. Also, the author measured the factor loading for all the questionnaire items. Moreover, the author tested the hypotheses with the statistical software IBM SPSS Statistics Suite, version 27. The author assessed the correlations among the relationships and assessed the models using regression analysis using SPSS. Moreover, the author used the PROCESS macro by Andrew Hayes ( 2013 ) in SPSS to test the mediation hypothesis.

The means, standard deviations, and correlations for all the variables in the study are shown in Table 1 . The results shown are a consideration of a sample where more than 65% have worked in the same university for more than seven years, and more than 74% of the sample are assistant, associate, and full professors. The highest means are for performance and sustainability environmental performance. At the same time, the lowest means are for system connection and dialogue and inquiry. Factor loadings were then calculated for all items, which were all higher than 0.64% (shown in Table 2 ).

The findings show the acceptance of H1, there is a strong significant positive relationship between dialogue and inquiry and information acquisition ( r  = 0.70, p  < 0.001), and there is likewise a strong positive significant relationship between dialogue and inquiry and knowledge dissemination ( r  = 0.64, p  < 0.001). There is a significant positive relationship between system connection and information acquisition ( r  = 0.59, p  < 0.001), and a significant positive relationship emerged between system connection and knowledge dissemination ( r  = 0.52, p  < 0.001). Multiple regression was run to predict information acquisition from dialogue, inquiry, and system connection. There is a positive effect between dialogue and inquiry and information acquisition (β = 0.520, p  < 0.01); also, there is a positive effect of system connection and information acquisition (β = 0.197, p  < 0.01). These variables statistically significantly predicted information acquisition, R2 = 0.522. All two variables added statistically significantly to the prediction, p  < 0.05. Therefore, H1a is supported. Multiple regression was run to predict knowledge dissemination from dialogue, inquiry, and system connection. There is a positive effect of dialogue and inquiry on knowledge dissemination (β = 0.544, p  < 0.01), and there is a positive effect between system connection and knowledge dissemination (β = 0.162, p  < 0.01); these variables statistically significantly predicted information acquisition, R2 = 0.429. All two variables added statistically significantly to the prediction, p  < 0.05. Therefore, H1b is supported. These findings show the positive relationship between organisational learning culture and organisational learning.

The findings also support H2. A significant positive relationship between information acquisition and university performance is found ( r  = 0.58, p  < 0.001), and there is a strong positive significant relationship between knowledge dissemination and university performance ( r  = 0.64, p  < 0.001). A multiple regression was run to predict university performance from information acquisition and knowledge dissemination. There is a positive effect between information acquisition and university performance (β = 0.250, p  < 0.01); also there is a positive effect between knowledge dissemination and university performance (β = 0.381, p  < 0.01). These variables statistically significantly predicted information acquisition, R2 = 0.451. All two variables added statistically significantly to the prediction, p  < 0.05. Therefore, H2 is supported. These findings show the positive relationship between the organisational learning process and university performance.

The findings support H3, where there is a strong significant positive relationship between system connection and university performance ( r  = 0.57, p  < 0.001), also there is a strong positive significant relationship between dialogue and inquiry and university performance ( r  = 0.66, p  < 0.001). Multiple regression was run to predict university performance from system connection, dialogue and inquiry. There is a positive effect between system connection and university performance (β = 0.184, p  < 0.01), and there is a positive effect between dialogue and inquiry and university performance (β = 0.440, p  < 0.01). These variables statistically significantly predicted information acquisition, R2 = 0.470. All two variables added statistically significantly to the prediction, p  < 0.05. Therefore, H3 is supported. This concludes the positivity of the relationship between organisational learning culture and university performance.

This study also developed a mediation analysis, H4, using Hayes ( 2013 ). Macros were developed to assess the mediation analysis of the models on SPSS. Macros help estimate the indirect effect with a bootstrap approach (Cole et al. 2008 ).

Organisational learning culture (dialogue and inquiry and system connection) has an indirect effect on university performance mediated by organisational learning processes (information acquisition and knowledge dissemination), which supports H4. Dialogue and inquiry have an indirect impact on university performance mediated by information acquisition (IE = 0.1427). The indirect effect is statistically significant; a bootstrapped 95% confidence interval around the indirect effect did not contain zero CI[0.482, 0.2423]. Also, system connection indirectly affects university performance mediated by information acquisition (IE = 0.1800); the indirect effect is a statistically significant bootstrapped 95% confidence interval around the indirect effect that did not contain zero, CI[0.1104,0.2548]. Dialogue and inquiry have an indirect impact on university performance mediated by knowledge dissemination (IE = 0.2038). The indirect effect is a statistically significant bootstrapped 95% confidence interval around the indirect effect that did not contain zero, CI [0.1306, 0.2865]. Moreover, system connection has an indirect effect on university performance mediated by knowledge dissemination (IE = 0.1970). The indirect effect is a statistically significant bootstrapped 95% confidence interval around the indirect effect that did not contain zero, CI [0.1351, 0.2638].

The findings support H5, where there is a strong significant positive relationship between Information acquisition and sustainable environmental performance ( r  = 0.57, p  < 0.001), Whereas a positive significant relationship has been found between information acquisition and sustainable social performance ( r  = 0.559, p  < 0.001). Moreover, a positive significant relationship is detected between knowledge dissemination and sustainable environmental performance ( r  = 0.526, p  < 0.001), and finally, a positive significant relationship is detected between knowledge dissemination and sustainable social performance ( r  = 0.550, p  < 0.001). A multiple regression was run to predict sustainable environmental performance from information acquisition and knowledge dissemination. There is a positive effect between information acquisition and sustainable environmental performance (β = 0.365, p  < 0.01); also, there is a positive effect between knowledge dissemination and sustainable environmental performance (β = 0.208, p  < 0.01). These variables statistically significantly predicted information acquisition, R2 = 0.362. All two variables added statistically one significantly to the prediction, p  < 0.05. Therefore, H5a is supported. A multiple regression was run to predict sustainable social performance from information acquisition and knowledge dissemination. There is a positive effect between information acquisition and sustainable social performance (β = 0.299, p  < 0.01); also, there is a positive effect between knowledge dissemination and sustainable social performance (β = 0.252, p  < 0.01). These variables statistically significantly predicted information acquisition, R2 = 0.365. All two variables added statistically significantly to the prediction, p  < 0.05. Therefore, H5 b is supported. It is deduced that a positive relationship exists between the organisational learning process and sustainable performance. The results are summarised in Table 3 , and the new proposed model is found in Fig. 2 .

Deduced research model.

Discussion and conclusion

This paper examines the impact of the organisational learning culture on the organisational learning process in the university context as well as the impact of the organisational learning process on a university’s performance and sustainable performance. The literature showed a gap where organisational learning processes are rarely assessed in universities (Abu‐Tineh 2011 ; Elbawab 2022b ). Also, most of the previous studies assess the impact of culture on organisational learning, but few studies have assessed the impact of organisational learning culture on the organisational learning process. Another gap has emerged, where the impact of organisational learning on sustainability is understudied (Alerasoul, 2022 ). Whereas this study also focuses on empirically assessing the relationship between organisational learning and sustainable performance. Learning normally empowers the occurrence of sustainability in organisations and enhances sustainability practices.

The findings of this study contribute to the literature in many ways. The findings support the positive relationship between an organisational learning culture and an organisational learning process, which supports H1. Organisational learning culture is represented by dialogues and inquiry and the system connection. Further, organisational learning processes are represented in this study as the process of information acquisition and knowledge dissemination. Dialogue and inquiry and system connection have a positive impact on information acquisition and knowledge dissemination. So, the findings indicate that the more the organisational learning culture increases in the university, the more the organisational learning processes occur. This relationship between the organisational learning culture and the organisational learning process is assessed empirically in this research, contrary to other studies where previous research always focused on organisational culture rather than organisational learning culture (Cho et al. 2013 ; Liao et al. 2012 ). This research also contributes to organisational learning research as the results indicate that the organisational learning culture is one of the antecedents to the organisational learning process. Accordingly, top management in universities needs to focus on improving the organisational learning culture to have better organisational processes. Also, human resources practitioners need to highlight the importance of maintaining organisational learning culture in organisations as it facilitates the organisational learning process.

The findings also support H2 which posits the positive relationship between the organisational learning process and a university’s performance. Our findings agree with previous research that supported the positive relationship between organisational learning and performance in organisations (Aragón et al. 2014 ; Bontis et al. 2002 ; Jyothibabu et al. 2010 ). We mainly focus on organisational learning processes that enhance a university’s performance. Our findings show that the better the information acquisition process and knowledge dissemination processes are, the better the university’s performance is going to occur in universities. So practically, the higher the efficiency of the acquisition of knowledge process, like acquiring the information from two sources is leading to a better the university’s performance. The two sources of information acquisition are internally from within the university and externally from other universities in the market. In this research, information acquisition is the process of identifying tendencies and problems, which leads to a better performance by the university.

This research has focused on the relationship between organisational learning culture and university performance. The findings in this study agree with previous studies (Ellinger et al. 2002 ; Sorakraikitikul and Siengthai 2014 ) that there is a positive relationship between the organisational learning culture and performance; the present study assessed this relationship empirically in the context of public universities while other studies have focused on various industries. (Choi 2020 ) focused on the assessment of the relationship between organisational learning culture and performance in public organisations generally. The present study focused on public universities as part of the public organisations in any country. The results show support for H3. Moreover, the findings suggest that universities that have a supportive learning culture will lead to better performance. If universities encourage a strong learning culture among their teachers, staff, and students, this will eventually lead to better performance.

Moreover, the organisational learning process mediates the relationship between organisational learning culture and university performance, as illustrated by the findings. These results support H4. Also, these results indicate that the more the top managers and human resources practitioners in universities encourage a learning culture and develop strong organisational learning processes, the higher the university’s performance will be. These findings demonstrate the need for alignment between the university learning culture and the university learning processes as they will promote better university performance. The findings indicate that organisational learning culture indirectly impacts university performance when it is mediated by organisational learning. From a theoretical lens, the organisational learning in this study is stated as a process. Hence, the organisational learning culture is considered an antecedent to the organisational learning process. In this study, the adapted model of culture (system connection and dialogue and inquiry) tests the culture with the process (information acquisition and knowledge dissemination); the results have shown both a direct and an indirect impact on university performance. The direct impact of culture on university performance has been found in the (Wahda, 2017 ) study, where this study agrees with (Wahda 2017 ) study. However, since this study has looked at organisational learning as a process and culture as an antecedent, it provided additional theoretical evidence of the indirect impact of organisational learning culture on performance, mediated by organisational learning as a process. This study also agrees with (Rebelo and Gomes 2017 ) study, where organisational culture emerges as a key concept and an essential condition that would promote and support learning in organisations. Moreover, in the following study, organisational learning culture is considered an antecedent to the organisational learning process. Whereas university performance is considered an outcome of the organisational learning process. Therefore, this study supports the (Rebelo and Duarte Gomes 2011 ) study that considers organisational learning culture as an antecedent to the organisational learning process. At the same time, the theoretical contribution that lies in this study is to address the model and consider the relationship between organisational learning culture and university performance directly and indirectly through the organisational learning process.

Another part of the framework assessed in this study is the impact of organisational learning processes on sustainable performance. Most universities nowadays focus on the importance of sustainability and apply sustainability practices. Also, the universities capitalise on the SDGs that are developed by the European Union. Whereas universities call for relating future research with the SDGs, as it helps in the sustainable development of society (Serafini et al. 2022 ). Moreover, universities focus on enhancing their sustainable environment. Still, this study addressed a significant gap, where few research studies have empirically addressed the impact of organisational learning on sustainable performance. And this research highlights the importance of applying this relationship and helps imply sustainable performance as an indicator for universities to be used in the future.

Sustainable performance in this study is considered another outcome of organisational learning. The findings of this study show support for H5. This study confirms the previous study findings (Iqbal and Ahmad 2021 ) that there is a positive impact between organisational learning and sustainable performance. However (Iqbal and Ahmad 2021 ) study did not assess the organisational learning process in universities, so this study develops it. The findings of this paper indicate the positive impact of information acquisition and knowledge dissemination on sustainable environmental performance and sustainable social performance. Therefore, when the organisational learning in universities is deepened and increased, the sustainable performance of the universities is better.

Finally, this study has created a suitable model to assess organisational learning processes, antecedents, and outcomes in universities. It has contributed to the theory of organisational learning; it shows a newly adapted model of organisational learning culture as well as organisational learning as a process and its influence on a university’s performance and the sustainable performance of the organisation. This adapted model may be considered novel because it indicates the antecedents, processes, and outcomes of organisational learning. Also, this model relates between NRBV and organisational learning theories, which is considered a theoretical contribution. Moreover, this model is tested in European universities and is considered an empirical contribution.

Implications and future research

Theoretical implications.

For the theoretical implications of describing organisational learning in universities, most previous research focused on assessing learning organisations; for example, the review developed by (Örtenblad and Koris 2014 ) showed the studies focused on assessing learning organisations. Furthermore, few studies focused on the learning processes and identified which organisational learning processes are more relevant to public universities and the education sector.

We have validated the organisational learning predictor as an organisational learning culture. Also, we have validated the organisational learning outcomes: university performance and sustainable performance.

The relevant learning processes are knowledge dissemination and information acquisition, where these processes are mainly representing the organisational learning processes in universities. This study recommends using the organisational learning developed model as it is relevant to universities.

Since researchers proposed that organisational learning culture is an important facilitator of the organisational learning process (Marsick and Watkins 2003 ). This study focused on assessing organisational learning culture’s impact on the organisational learning process. Previous research mainly focused on assessing the organisational culture’s impact on organisational learning (e.g., Oh and Han 2020 ; Rebelo and Duarte Gomes 2011 ). Few researchers focused on the impact of organisational learning culture on organisational learning. Previous research showed organisational culture as decision-making processes, openness, learning orientation, and leadership (Flores et al. 2012 ). In this study, organisational learning culture was described as the collective learning culture that enhances the organisational learning activities (Sorakraikitikul and Siengthai 2014 ). Subsequently, it is evident that this gap was addressed and studied in this study. The findings show a strong relationship between organisational learning culture and organisational learning processes in education.

Moreover, the mediation of organisational learning processes between organisational learning culture and university performance is another theoretical contribution to organisational studies. As the previous research mainly focused on the impact of organisational learning culture on organisational performance (Sorakraikitikul and Siengthai 2014 ), few studies focused on mediation analysis. The findings of this study show that organisational learning culture indirectly impacts university performance when it is mediated by both organisational learning processes (information acquisition and knowledge dissemination). These findings show the importance of having an effective learning culture and efficient organisational learning processes on the organisational level as they help improve university performance.

Previous studies focused on assessing the relationship between organisational learning and organisational performance (Bontis et al. 2002 ; Jyothibabu et al. 2010 ). Scarce studies focused on university performance, while in the meantime, university performance reveals the success of the university and the achievement of its goals. Finally, this study has contributed to organisational studies by exploring the relationship between organisational learning and sustainable performance and curating a model specifically for higher education institutions.

Practical implications

This research serves universities; the implications of this research could be adapted to various faculties. The result of this investigation recommends that when universities work on organisational learning culture, it is followed by enhancing the organisational learning process. Subsequently, the organisational performance and sustainable performance will be improved. This section will provide implications and countermeasures for different stakeholders related to the universities.

To develop the organisational learning culture, universities should work on their dialogue and inquiry process and their system connection process. A learning culture that promotes more dialogue and inquiry among the university members is the target. Hence, the organisational learning culture should be encouraged at all university levels, including among deans, heads of departments, directors, and teachers. The author of the study suggests that deans hold organisational meetings for the faculty members (for example, semester meetings and monthly meetings) mainly to discuss the faculty’s point of view, to share feedback and to empower the experimentation and questioning of the process, to reach for shared view and perspective reasonably. For the teacher, the author recommends attending the meetings with the capacity to inquire about the process, share their feedback and opinions and listen to other views. At the same time, the author recommends enhancing the organisational learning culture by focusing on the system connection.

The author recommends having an ongoing connection with the market to know their needs. Subsequently, the department heads and the director of the programs design new curricula and update the existing ones to align the curriculum with the market needs. We recommend that the teachers support the system connection by advising and recommending the new market demands and technologies in their classes and for the program directors and department heads. Subsequently, both these actions will help increase the organisational learning culture in universities.

After providing the necessary organisational learning culture, universities should develop their organisational learning process in order to increase their performance. To develop the organisational learning process, universities are recommended to capitalise on information acquisition and processes to disseminate knowledge.

For the information acquisition process, acquiring information from two sources improves the university’s performance. The two sources of information acquisition are internally from within the university and externally from other universities in the market. The author recommends that the deans, as well as the heads of departments, focus on acquiring the information internally, whether the students’ satisfaction, the feedback about the curriculum, and even the successful teaching methodologies. The deans need to focus on the university’s previous experiences and align the learning processes with the business environment. As for the external information, deans and rectors who have the bigger image need to acquire the information from competitors (e.g. international universities) and the marketplace to develop their analyses regarding each academic year and the long-term plans. The rectors will help empower the organisational learning processes to flow throughout the university. Practically, the top management in the university needs to look for the best practices internally and externally and solve problems by identifying the key tendencies, whereas in this process, the rectors, deans, program directors, heads of departments and teachers need to share and collaborate. Meanwhile, the information acquisition process is enhanced by collecting not only the information but also the best practices and pedagogies from the competitors. Also, the employers’, policymakers’, and governors’ perspectives should be acquired.

Another practice for enhancing the information acquisition process is comparing the university’s performance to other universities. Acquiring information sources could be formally through the formal reports published on a yearly basis (ex: public university performance reports, universal rankings and amount of collaboration and funds provided to this university), whereas informal sources of acquiring information like assessing the performance of the university on social media. Hence, different stakeholders are involved in the information acquisition process.

As for the second process, knowledge dissemination, the author recommends that universities focus on both the formal and informal interaction between employees to enhance the dissemination of knowledge. For the formal interactions, the author advises rectors and deans to focus on meetings and training that will enhance the continuous learning process and efficiently use the university database. Moreover, deans should enhance the formal networks among their universities and the rest of the universities, for example, through exchanging professors and publishing research papers. These practices will help disseminate the information rapidly and accurately. As for the informal interaction and communication between employees, the author advises the teachers to share knowledge, best practices, new teaching methods, and new tendencies in research and the market with each other. The author also advises university heads of departments to encourage informal interaction between teachers to support the university’s organisational learning. Since the government governors are involved in the process, the author recommends that university heads contact government governors. Government governors can help by disseminating the goals and future plans of the government, as well as the best practices from different universities so that the universities can adapt their learning processes.

Another example of an organisational learning process may be found during the departmental and pedagogical meetings, during which the future of the courses, the programmes, and the schools are discussed, this leads to better dissemination of knowledge and eventually a better performance by the university. Therefore, top managers in universities need to focus on the implementation of the organisational learning processes in their universities as it helps in having better performance and eventually adapt to change and university success (Meshari et al. 2021 ).

Another important point is that universities need to acquire sustainable performance as it is also one of the indicators of the university rankings that has been recently added to the university’s ranking (like the QS world ranking). Where the SDGs in the QS world ranking, mainly focus on the environmental impact and the social impact of each university by indicating the SDGs rating of each university regarding social and environmental impact.

Finally, top managers in universities need to adapt the universities to the change that is occurring internationally and promote having a better sustainable performance to also reflect in their ranking. These will eventually reflect in the university’s success.

Since the findings show that there is a positive impact of organisational learning on sustainable performance, then it is recommended to deliver training for top managers and decision makers in civil society organisations and government governors to enhance sustainable development outreach. Moreover, in order to focus on the importance of sustainable performance in the meetings and the conferences that are developed in the organisations.

In conclusion, all these recommendations will likely help universities achieve better performance. Top managers like deans, rectors and school heads in universities need to focus on the learning flow within the university. They need to focus on the organisational learning process at the organisational level. The author suggests for the future of this research stream to assess this model on a broader scope, as more universities from outside Europe could be evaluated. As this model showed its validity in various public universities. Also, it is recommended to assess this model in private universities, as the private universities have different regulations, organisational learning culture and performance goals.

Data availability

The dataset generated during and/or analysed during the current study is available from the corresponding author upon reasonable request.

https://iscteiul.co1.qualtrics.com/jfe/form/SV_37Cf3UZX5NPJs0d

Abu‐Tineh AM (2011) Exploring the relationship between organizational learning and career resilience among faculty members at Qatar University. Int J Edu Manag 25(6):635–650. https://doi.org/10.1108/09513541111159095

Article   Google Scholar  

Ahmed PK, Loh AYE, Zairi M (1999) Cultures for continuous improvement and learning. Tot Quality Manag 10(4-5):426–434. https://doi.org/10.1080/0954412997361

Alerasoul SA, Afeltra G, Hakala H, Minelli E, Strozzi F (2022) Organisational learning, learning organisation, and learning orientation: An integrative review and framework. Human Res Manag Rev 32(3):100854

Google Scholar  

Allen IE, Seaman CA (2007) Likert scales and data analysis. Quality Prog 40(7):64–65

Aragón M, Jimenez‐Jimenez D, Sanz Valle R (2014) Training and performance: The mediating role of organizational learning. Business Res Quart 3(10):17. https://doi.org/10.1016/j.cede.2013.05.003

Argote L, Miron-Spektor E (2011) Organizational Learning: From Experience to Knowledge. Organization Sci 22(5):1123–1137. https://doi.org/10.1287/orsc.1100.0621

Argyris C, Schön D (1978) Organizational Learning: A Theory of Action Perspective. Reis, (77/78), 345–348. https://doi.org/10.2307/40183951

Argyris C, Schön D (1996) Organizational learning: a theory of action perspective. Reading: Addison-Wesley

Basiago AD (1995) Methods of defining ‘sustainability. Sustainable Dev 3(3):109–119

Bianchi G, Testa F, Boiral O, Iraldo F (2022) Organizational Learning for Environmental Sustainability: Internalizing Lifecycle Management. Org Environ 35(1):103–129. https://doi.org/10.1177/1086026621998744

Bilan Y, Hussain HI, Haseeb M, Kot S (2020) Sustainability and economic performance: Role of organizational learning and innovation. Enginee Econ 31(1):93–103. https://doi.org/10.5755/j01.ee.31.1.24045

Bleiklie I, Kogan M (2007) Organization and governance of universities. Higher Edu Policy 20:477–493

Bontis N, Crossan MM, Hulland J (2002) Managing An Organizational Learning System By Aligning Stocks and Flows. J Manag Stud 39(4):437–469. https://doi.org/10.1111/1467-6486.t01-1-00299

Campbell T, Cairns H (1994) Developing and Measuring the Learning Organization. Industrial Commercial Training 26(7):10–15. https://doi.org/10.1108/00197859410064583

Camps J, Luna-Arocas R (2012) A Matter of Learning: How Human Resources Affect Organizational Performance. British Journal of Management, no–no. https://doi.org/10.1111/j.1467-8551.2010.00714.x

Castaneda DI, Manrique LF, Cuellar S (2018) Is organizational learning being absorbed by knowledge management? A systematic review. J Knowledge Manag 22(2):299–325. https://doi.org/10.1108/jkm-01-2017-0041

Chadwick IC, Raver JL (2012) Motivating Organizations to Learn. J Manag 41(3):957–986. https://doi.org/10.1177/0149206312443558

Chiva R, Alegre J, Lapiedra R (2007) Measuring organisational learning capability among the workforce. Int J Manpower 28(3/4):224–242. https://doi.org/10.1108/01437720710755227

Cho I, Kim JK, Park H, Cho N-H (2013) The relationship between organisational culture and service quality through organisational learning framework. Total Quality Manag Business Excellence 24(7-8):753–768. https://doi.org/10.1080/14783363.2013.791100

Choi I (2020) Moving beyond Mandates: Organizational Learning Culture, Empowerment, and Performance. Int J Public Admin 43(8):724–735. https://doi.org/10.1080/01900692.2019.1645690

Cole MS, Walter F, Bruch H (2008) Affective mechanisms linking dysfunctional behavior to performance in work teams: A moderated mediation study. J Appl Psychol 93(5):945–958. https://doi.org/10.1037/0021-9010.93.5.945

Article   PubMed   Google Scholar  

Collin A (2009) Multidisciplinary, interdisciplinary, and transdisciplinary collaboration: implications for vocational psychology. Int J Educ Vocat Guid 9(2):101–110

Conner ML, Clawson JG (Eds.). (2004) Creating a Learning Culture. https://doi.org/10.1017/cbo9781139165303

Crossan MM, Bapuji HB (2003) Examining the link between knowledge management, organizational learning and performance. Proceedings of the 5th International Conference on Organizational Learning and Knowledge, Lancaster University, Lancaster, 30 May-2 June

Crossan MM, Lane HW, White RE (1999) An Organizational Learning Framework: From Intuition to Institution. The Academy of Management Review, vol. 24, no. 3, pp. 522–537. JSTOR, www.jstor.org/stable/259140 . Accessed 28 Sept. 2020

Davies H, Walters P (2004) Emergent patterns of strategy, environment and performance in a transition economy. Strategic Manag J 25(4):347–364. https://doi.org/10.1002/smj.380

Do H, Budhwar P, Shipton H, Nguyen HD, Nguyen B (2022) Building organizational resilience, innovation through resource-based management initiatives, organizational learning and environmental dynamism. J Business Res 141:808–821

Domínguez-Escrig E, Broch FFM, Chiva R, Alcamí RL (2022) Authentic leadership: boosting organisational learning capability and innovation success. Learning Org 30(1):23–36

Edmondson AC (2002) The Local and Variegated Nature of Learning in Organizations: A Group-Level Perspective. Org Sci 13(2):128–146. https://doi.org/10.1287/orsc.13.2.128.530

Elbawab R (2022b) University Rankings and Goals: A Cluster Analysis. Economies 10(9):209

Elbawab RRKI (2022a) Predictors and outcomes of team learning in higher education institutions. Dissertation, ISCTE-University Institute of Lisbon

Ellinger AD, Ellinger AE, Yang B, Howton SW (2002) The relationship between the learning organization concept and firms’ financial performance: An empirical assessment. Hum Resource Dev Quart 13(1):5–22. https://doi.org/10.1002/hrdq.1010

Feeney M, Grohnert T, Gijselaers W, Martens P (2023) Organizations, learning, and sustainability: A cross-disciplinary review and research agenda. J Business Ethics 184(1):217–235

Fiol CM, Lyles MA (1985) Organizational Learning. Acad Manag Rev 10(4):803. https://doi.org/10.2307/258048

Flores LG, Zheng W, Rau D, Thomas CH (2012) Organizational learning: Subprocess identification, construct validation, and an empirical test of cultural antecedents. J Manag 38(2):640–667. https://doi.org/10.1177/0149206310384631

Garvin DA (1993) Building a learning organization. Harvard Business Rev 71:78–91

CAS   Google Scholar  

Geng Y, Chen L, Li J, Iqbal K (2023) Higher education and digital economy: Analysis of their coupling coordination with the Yangzte River economic Belt in China as the example. Ecol Indicators 154:110510

Geng YQ, Zuhu HW, Zhao N, Zhai QH (2020a) A New Framework to Evaluate Sustainable Higher Education: An Analysis of China. Discrete Dyn Nat Soc 2020:110514

Guţă AL (2014) Measuring organizational learning. Model testing in two Romanian universities. Manag Marketing 9(3), 253–282. http://www.managementmarketing.ro/pdf/articole/454.pdf

Hart S (1995) A natural-resource-based view of the firm. Acad Manag Rev 20(4):986–1014

Hayes AF (2013) Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. (2nd ed.). Guilford Publications

Huber GP (1991) Organizational Learning: The Contributing Processes and the Literatures. Org Sci 2(1):88–115. https://doi.org/10.1287/orsc.2.1.88

Hurley RF, Hult GTM (1998) Innovation, Market Orientation, and Organizational Learning: An Integration and Empirical Examination. J Marketing 62(3):42–54. https://doi.org/10.1177/002224299806200303

Iqbal Q, Ahmad NH (2021) Sustainable development: The colors of sustainable leadership in learning organization. Sustain Dev 29(1):108–119

James LR, Demaree RG, Wolf G (1993) r-sub(wg): An assessment of within-group interrater agreement. J Appl Psychol 78(2):306–309. https://doi.org/10.1037/0021-9010.78.2.306

Jiménez Jiménez D, SanzValle R (2006) Innovación,aprendizaje organizativo and resultados empresariales. Un estudio empírico. Cuadernos de Economía and Dirección de la Empresa 29:31–56

Jyothibabu C, Farooq A, Bhusan Pradhan B (2010) An integrated scale for measuring an organizational learning system. Learning Org 17(4):303–327. https://doi.org/10.1108/09696471011043081

Keskin H (2006) Market orientation, learning orientation, and innovation capabilities in SMEs. Eur J Innovation Manag 9(4):396–417. https://doi.org/10.1108/14601060610707849

Kezar AJ, Holcombe EM (2019) Barriers to organizational learning in a multi-institutional initiative. Higher Educ 79(6):1119–1138. https://doi.org/10.1007/s10734-019-00459-4

Kipesha EF, Msigwa R (2013) Efficiency of higher learning institutions: Evidences from public universities in Tanzania. J Educ Pract 4(7):63–73

Kofman F, Senge PM (1993) Communities of commitment: The heart of learning organizations. Org Dyn 22(2):5–23. https://doi.org/10.1016/0090-2616(93)90050-b

Kontoghiorghes C, Awbrey SM, Feurig PL (2005) Examining the Relationship Between Learning Organization Characteristics and Change Adaptation, Innovation, and Organizational Performance. Hum Res Dev Quarterly 16(2):185–211. https://doi.org/10.1002/hrdq.1133

Kordab M, Raudeliūnienė J, Meidutė-Kavaliauskienė I (2020) Mediating role of knowledge management in the relationship between organizational learning and sustainable organizational performance. Sustainability 12(23):10061

Kumar N (2005) Assessing the learning culture and performance of educational institutions. Perform Improv 44(9):27

Leal Filho W, Tripathi SK, Andrade Guerra JBSOD, Giné Garriga R, Orlovic Lovren V, Willats J (2018) Using the sustainable development goals towards a better understanding of sustainability challenges. Int J Sustain Develop World Ecol 26(2):179–190

Leal Filho, W, Trevisan LV, Rampasso IS, Anholon R, Dinis MAP, Brandli L L, ... Mazutti J (2023) When the alarm bells ring: Why the UN sustainable development goals may not be achieved by 2030. J Clean Prod 407:137108

Leung SO (2011) A comparison of psychometric properties and normality in 4-, 5-, 6-, and 11-point Likert scales. J Soc Serv Res 37(4):412–421

Article   ADS   Google Scholar  

Liao SH, Chang WJ, Hu D-C, Yueh YL (2012) Relationships among organizational culture, knowledge acquisition, organizational learning, and organizational innovation in Taiwan’s banking and insurance industries. Int J Hum Res Manag 23(1):52–70. https://doi.org/10.1080/09585192.2011.599947

Maccoby M (2003) The Human Side: The Seventh Rule: Create a Learning Culture. Res-Technol Manag 46(3):59–60. https://doi.org/10.1080/08956308.2003.11671567

Marquardt M (1996) Building the Learning Organization. McGraw-Hill, New York, NY

Marsick VJ, Watkins KE (2003) Demonstrating the Value of an Organization’s Learning Culture: The Dimensions of the Learning Organization Questionnaire. Adv Dev Hum Res 5(2):132–151. https://doi.org/10.1177/1523422303005002002

Massimo B, Nora A (2022) Barriers to organizational learning and sustainability: The case of a consumer cooperative. J Co-operative Org Manag 10(2):100182

Medne A, Lapina I, Zeps A (2022) Challenges of uncertainty in sustainable strategy development: Reconsidering the key performance indicators. Sustain 14(2):761

Meshari AZ, Othayman MB, Boy F, Doneddu D (2021) The impact of learning organizations dimensions on the organisational performance: An exploring study of Saudi Universities. Int Business Res 14(2):54

Mohammad HI (2019) Mediating effect of organizational learning and moderating role of environmental dynamism on the relationship between strategic change and firm performance. J Strat Manag 12(2):275–297. https://doi.org/10.1108/jsma-07-2018-0064

Molnar E, Mulvihill PR (2003) Sustainability-focused organi- zational learning: Recent experiences and new challenges. J Environ Planning Manag 46(2):167–176

Oh SY, Han HS (2020) Facilitating organisational learning activities: Types of organisational culture and their influence on organisational learning and performance. Knowledge Manag Res Pract 18(1):1–15

Örtenblad A, Koris R (2014) Is the learning organization idea relevant to higher educational institutions? A literature review and a “multi-stakeholder contingency approach”. Int J Educ Management 28(2):173–214

Park S, Kim E-J (2018) Fostering organizational learning through leadership and knowledge sharing. J Knowledge Manag 22(6):1408–1423. https://doi.org/10.1108/jkm-10-2017-0467

Pedler M, Burgoyne J, Boydell T (1997) The Learning Company: A Strategy for Sustainable Development, 2nd ed., McGraw-Hill, London. https://pesquisa.bvsalud.org/portal/resource/pt/pah-27346?lang=en

Pham Thi Bich N, Tran Quang H (2016) Organizational Learning in Higher Education Institutions: A Case Study of A Public University in Vietnam. J Econ Dev88–104. https://doi.org/10.33301/2016.18.02.06

Pocol CB, Stanca L, Dabija DC, Pop ID, Mișcoiu S (2022) Knowledge co-creation and sustainable education in the labor market-driven university–business environment. Front Environ Sci 10:781075

Rebelo T, Gomes AD (2017) Is organizational learning culture a good bet? An analysis of its impact on organizational profitability and customer satisfaction. Academia Revista Latinoamericana de Administración 30(3):328–343

Rebelo TM, Duarte Gomes A (2011) Conditioning factors of an organizational learning culture. J Workplace Learn 23(3):173–194. https://doi.org/10.1108/13665621111117215

Rhodes J, Lok P, Yu‐Yuan Hung R, Fang S (2008) An integrative model of organizational learning and social capital on effective knowledge transfer and perceived organizational performance. J Workplace Learn 20(4):245–258. https://doi.org/10.1108/13665620810871105

Santos-Vijande ML, López-Sánchez JÁ, Trespalacios JA (2012) How organizational learning affects a firm’s flexibility, competitive strategy, and performance. J Business Res 65(8):1079–1089. https://doi.org/10.1016/j.jbusres.2011.09.002

Santos-Vijande ML, López-Sánchez JÁ, González-Mieres C (2012) Organizational learning, innovation and performance in KIBS.Journal of Management & Organization, 1390–1447. https://doi.org/10.5172/jmo.2012.1390

Saunders M, Lewis P, Thornhill A (2007) Research methods for business students (4th. Ed.). Pearson education

Schönherr S, Eller R, Kallmuenzer A, Peters M (2023) Organisational learning and sustainable tourism: the enabling role of digital transformation. J Knowledge Manag

Serafini PG, de Moura JM, de Almeida M. R., de Rezende JFD (2022) Sustainable development goals in higher education institutions: a systematic literature review. J Clean Prod 370:133473

Slater SF, Narver JC (1995) Market Orientation and the Learning Organization. J Marketing 59(3):63–74. https://doi.org/10.1177/002224299505900306

Sorakraikitikul M, Siengthai S (2014) Organizational learning culture and workplace spirituality. Learning Org 21(3):175–192. https://doi.org/10.1108/tlo-08-2011-0046

Sun H, Ho K, Ni W (2008) The empirical relationship among Organisational Learning, Continuous Improvement and Performance Improvement. Int J Learn Change 3(1):110. https://doi.org/10.1504/ijlc.2008.018871

Tohidi H, Mohsen Seyedaliakbar S, Mandegari M (2012) Organizational learning measurement and the effect on firm innovation. J Enterprise Info Manag 25(3):219–245. https://doi.org/10.1108/17410391211224390

Voolaid K, Ehrlich Ü (2017) Organizational learning of higher education institutions: the case of Estonia. Learning Org 24(5):340–354. https://doi.org/10.1108/tlo-02-2017-0013

Wahda W (2017) Mediating effect of knowledge management on organizational learning culture toward organization performance. J Manag Dev 36(7):846–858

Watkins K, Marsick VJ (1993) Sculpting the Learning Organization. Jossey-Bass, San Francisco, CA

Watkins K, Marsick V (1997) Dimensions of the learning organization questionnaire. Partners for the Learning Organization, Warwick, RI

Zhang Y, Khan U, Lee S, Salik M (2019) The influence of management innovation and technological innovation on organization performance. A mediating role of sustainability. Sustainability 11(2):495

Download references

Author information

Authors and affiliations.

BRU-IUL, Instituto Universitário de Lisboa (ISCTE-IUL), Lisbon, Portugal

Roba Elbawab

Escola de Ciências Económicas e das Organizações, Universidade Lusófona de Humanidades e Tecnologias, Lisboa, Portugal

You can also search for this author in PubMed   Google Scholar

Contributions

RE contributed to all the sections of the article.

Corresponding author

Correspondence to Roba Elbawab .

Ethics declarations

Competing interests.

The author declares no competing interests.

Ethical approval

All questionnaires collected in this study were in accordance with the ethical standards of the university and declaration of Helsinki, the questionnaire was previously used in the following dissertation (Elbawab, R. R. K. I. (2022)a. Predictors and outcomes of team learning in higher education institutions. Dissertation, ISCTE-university institute of Lisbon).

Informed consent

Informed consent was obtained from all participants.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Elbawab, R. Linking organisational learning, performance, and sustainable performance in universities: an empirical study in Europe. Humanit Soc Sci Commun 11 , 626 (2024). https://doi.org/10.1057/s41599-024-03114-1

Download citation

Received : 20 March 2023

Accepted : 25 April 2024

Published : 14 May 2024

DOI : https://doi.org/10.1057/s41599-024-03114-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

SYSTEMATIC REVIEW article

A systematic review of empirical studies incorporating english movies as pedagogic aids in english language classroom provisionally accepted.

• 1 Department of English, School of Social Sciences and Languages, Vellore Institute of Technology, Vellore, India., India
• 2 Research Scholar, Department of English, School of Social Sciences and Languages, Vellore Institute of Technology, Vellore, Tamil Nadu, India., India
• 3 Department of English, School of Social Sciences and Languages, VIT University, India

The final, formatted version of the article will be published soon.

The use of movie as an audio-visual multimodal tool has been extensively researched, and the studies prove that they play a vital role in enhancing communicative competence. Incorporating authentic materials like movies, television series, podcasts, social media, etc. into language learning serves as a valuable resource for the learners, for it exposes them to both official and vernacular language. The current study aims to systematically analyse the preceding studies that conjoined English movies into the curriculum to teach English. It also examines and evaluates the empirical research that various researchers conducted from 2000 to 2023. The articles were primarily sourced from prominent academic databases such as ProQuest, ScienceDirect, Scopus, Web of Science, and Google Scholar. Inclusion and exclusion criteria were applied in screening the 921 sources, of which 23 empirical studies were eligible for the review as a result of a three-stage data extraction process as shown in the "Preferred Reporting Items for Systematic Reviews and Meta Analyses" (PRISMA) chart. The extraction of data from the review encompasses an overview of the empirical studies, methodologies, participants, and interventions. The extracts were systematically analysed using the software's End Note and Covidence. The analysis of the existing literature and experimental data substantiates that teaching and learning English as a second or foreign language using movies as teaching aids exhibit promising prospects for enhancing English language proficiency. The findings of the study reveal different genres of movies that aid the facilitator in producing effective instruction materials with clearly defined objectives and guided activities. It is also observed that the learners have a positive experience with long-term learning benefits.

Keywords: EndNote, Covidence, Audio-visual multimodal aids, Communicative competence, Systematic review, Teaching Supplements

Received: 13 Feb 2024; Accepted: 16 May 2024.

Copyright: © 2024 K and S.N.S. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Dr. Gandhimathi S.N.S, Department of English, School of Social Sciences and Languages, VIT University, Vellore, India

People also looked at

Happiness economics: Discovering future research trends through a systematic literature review

• Agrawal, Shruti
• Sharma, Nidhi
• Bruni, Maria Elena
• Iazzolino, Gianpaolo

Happiness Economics is an expanding field, with a growing number of studies due to the convolution in the disciplines of social sciences. The current trends in welfare economics have witnessed an increase in quantitative research approaches, reporting empirical associations between happiness and other variables. These approaches, however, have been limited only in the area of the economic sector, specifically focusing on the Gross Domestic Product (GDP). This paper takes a broader view of the topic, with the aim of identifying the research progress and the emerging trends in happiness economics. We provide a systematic literature review based on a bibliometric analysis, using Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA). Text-based cluster analysis was performed to identify the most prominent themes in the literature through cluster identification via VOSViewer and R Studio. The findings highlight five major emerging research themes, namely: (i) Paradoxes of happiness research in Economics; (ii) Happiness Economics: Bringing back Ordinalism?; (iii) Beyond GDP: Sustainability and subjective well-being; (iv) Policies to achieve welfare and happiness economy; and (v) Happiness management and organisational culture to improve productivity. Finally, on the basis of emerging themes, we propose future research propositions for each of the themes. Results demonstrate that happiness economics has a potential to address present needs and future engagements to build a better economic system and a happier society. The study provides novel and significant contributions to the existing literature by providing evidence of the past and current practices of happiness economics. Significant implications for the prospective stakeholders further improve the contribution of research.

• Happiness economics;
• Sustainability;
• Organizational culture;
• Systematic literature review