document review as a qualitative research method

Document Analysis

First Online: 02 January 2023

Cite this chapter

document review as a qualitative research method

Benjamin Kutsyuruba 4

Part of the book series: Springer Texts in Education ((SPTE))

5082 Accesses

2 Citations

This chapter describes the document analysis approach. As a qualitative method, document analysis entails a systematic procedure for reviewing and evaluating documents through finding, selecting, appraising (making sense of), and synthesizing data contained within them. This chapter outlines the brief history, method and use of document analysis, provides an outline of its process, strengths and limitations, and application, and offers further readings, resources, and suggestions for student engagement activities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Available as EPUB and PDF
Read on any device
Instant download
Own it forever
Compact, lightweight edition
Dispatched in 3 to 5 business days
Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Qualitative Text Analysis: A Systematic Approach

Systematic Reviews and Meta-Analysis: A Guide for Beginners

Qualitative Content Analysis: Theoretical Background and Procedures

Altheide, D. L. (1987). Ethnographic content analysis. Qualitative Sociology, 10 (1), 65–77.

Article Google Scholar

Altheide, D. L. (1996). Qualitative media analysis . SAGE.

Google Scholar

Altheide, D. L. (2000). Tracking discourse and qualitative document analysis. Poetics, 27 , 287–299.

Atkinson, P. A., & Coffey, A. (1997). Analysing documentary realities. In D. Silverman (Ed.), Qualitative research: Theory, method and practice (pp. 45–62). SAGE.

Berg, B. L. (2001). Qualitative research methods for social sciences . Allyn and Bacon.

Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9 (2), 27–40. https://doi.org/10.3316/qrj0902027

Bryman, A. (2003). Research methods and organization studies . Routledge.

Book Google Scholar

Cardno, C. (2018). Policy document analysis: A practical educational leadership tool and a qualitative research method. Educational Administration: Theory and Practice , 24 (4), 623–640. https://doi.org/10.14527/kuey.2018.016

Caulley, D. N. (1983). Document analysis in program evaluation. Evaluation and Program Planning, 6 , 19–29.

Corbin, J., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). SAGE.

Derrida, J. (1978). Writing and difference . Routledge & Kegan Paul.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research . Aldine De Gruyter.

Glesne, C., & Peshkin, A. (1992). Becoming qualitative researchers (2nd ed.). Longman.

Goode, W. J., & Hatt, P. K. (1952). Methods in social research . McGraw-Hill.

Hodder, I. (2000). The interpretation of documents and material culture. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 703–715). SAGE.

Krippendorff, K. (1980). Content analysis: An introduction to its methodology. SAGE.

Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human Communication Research, 28 , 587–604.

Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2010). Practical resources for assessing and reporting intercoder reliability in content analysis research projects . Retrieved March 20, 2011, from http://matthewlombard.com/reliability/index_print.html

Mayring, P. (2000). Qualitative content analysis. Forum: Qualitative social research (Vol. 1(2)). Retrieved March 22, 2011, from http://www.qualitative-research.net/index.php/fqs/article/viewArticle/1089/2385

McMillan, J. H., & Schumacher, S. (2010). Research in education: Evidence-based inquiry (7th ed.). Pearson.

Merriam, S. B. (1988a). Case study research in education: A qualitative approach . Jossey-Bass.

Merriam, S. B. (1998b). Case study research in education . Jossey-Bass.

Miller, F. A., & Alvarado, K. (2005). Incorporating documents into qualitative nursing research. Journal of Nursing Scholarship, 37 (4), 348–353.

Neuendorf, K. A. (2002). The content analysis guidebook . SAGE.

O’Leary, Z. (2014). The essential guide to doing your research project (2nd ed.). SAGE.

Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). SAGE.

Prior, L. (2003). Using documents in social research . SAGE.

Prior, L. (2008a). Document analysis. In L. Given (Ed.), The SAGE encyclopaedia of qualitative research methods (pp. 231–232). SAGE. https://doi.org/10.4135/9781412963909

Prior, L. (2008b). Repositioning documents in social research. Sociology, 42 (5), 821–836. https://doi.org/10.1177/0038038508094564

Prior, L. (2012). The role of documents in social research. In S. Delamont (Ed.), Handbook of qualitative research in education (pp. 426–438). Edward Elgar.

Salminen, A., Kauppinen, K., & Lehtovaara, M. (1997). Towards a methodology for document analysis. Journal of the American Society for Information Science, 48 (7), 644–655.

Stake, R. E. (1995). The art of case study research . SAGE.

Wharton, C. (2006). Document analysis. In V. Jupp (Ed.), The SAGE dictionary of social research methods (pp. 80–81). SAGE. https://doi.org/10.4135/9780857020116

Yin, R. K. (2009). Case study research, design and methods (4th ed.). SAGE.

Additional Reading

Kutsyuruba, B. (2017). Examining education reforms through document analysis methodology. In I. Silova, A. Korzh, S. Kovalchuk, & N. Sobe (Eds.), Reimagining Utopias: Theory and method for educational research in post-socialist contexts (pp. 199–214). Sense.

Kutsyuruba, B., Christou, T., Heggie, L., Murray, J., & Deluca, C. (2015). Teacher collaborative inquiry in Ontario: An analysis of provincial and school board policies and support documents. Canadian Journal of Educational Administration and Policy, 172 , 1–38.

Kutsyuruba, B., Godden, L., & Tregunna, L. (2014). Curbing the early-career attrition: A pan-Canadian document analysis of teacher induction and mentorship programs. Canadian Journal of Educational Administration and Policy, 161 , 1–42.

Segeren, A., & Kutsyuruba, B. (2012). Twenty years and counting: An examination of the development of equity and inclusive education policy in Ontario (1990–2010). Canadian Journal of Educational Administration and Policy, 136 , 1–38.

Online Resources

Document Analysis: A How To Guide (12:27 min) https://www.youtube.com/watch?v=vOsE9saR_ck

Document Analysis with Philip Adu (1:16:40 min) https://youtu.be/bLKBffW5JPU

Download references

Author information

Authors and affiliations.

Queen’s University, Kingston, Canada

Benjamin Kutsyuruba

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Kutsyuruba .

Editor information

Editors and affiliations.

Department of Educational Administration, College of Education, University of Saskatchewan, Saskatoon, SK, Canada

Janet Mola Okoko

Scott Tunison

Department of Educational Administration, University of Saskatchewan, Saskatoon, SK, Canada

Keith D. Walker

Rights and permissions

Reprints and permissions

Copyright information

About this chapter

Kutsyuruba, B. (2023). Document Analysis. In: Okoko, J.M., Tunison, S., Walker, K.D. (eds) Varieties of Qualitative Research Methods. Springer Texts in Education. Springer, Cham. https://doi.org/10.1007/978-3-031-04394-9_23

Download citation

DOI : https://doi.org/10.1007/978-3-031-04394-9_23

Published : 02 January 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-04396-3

Online ISBN : 978-3-031-04394-9

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Publish with us

Policies and ethics

Find a journal
Track your research

To read this content please select one of the options below:

Please note you do not have access to teaching notes, document analysis as a qualitative research method.

Qualitative Research Journal

ISSN : 1443-9883

Article publication date: 3 August 2009

This article examines the function of documents as a data source in qualitative research and discusses document analysis procedure in the context of actual research experiences. Targeted to research novices, the article takes a nuts‐and‐bolts approach to document analysis. It describes the nature and forms of documents, outlines the advantages and limitations of document analysis, and offers specific examples of the use of documents in the research process. The application of document analysis to a grounded theory study is illustrated.

Content analysis
Grounded theory
Thematic analysis
Triangulation

Bowen, G.A. (2009), "Document Analysis as a Qualitative Research Method", Qualitative Research Journal , Vol. 9 No. 2, pp. 27-40. https://doi.org/10.3316/QRJ0902027

Emerald Group Publishing Limited

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

Search Menu
Sign in through your institution
Advance Articles
Editor's Choice
Supplements
Open Access Articles
Research Collections
Review Collections
Author Guidelines
Submission Site
Open Access Options
Self-Archiving Policy
About Health Policy and Planning
About the London School of Hygiene and Tropical Medicine
HPP at a glance
Editorial Board
Advertising and Corporate Services
Journals Career Network
Journals on Oxford Academic
Books on Oxford Academic

Article Contents

Introduction, what is document analysis, the read approach, supplementary data, acknowledgements.

< Previous

Document analysis in health policy research: the READ approach

Article contents
Figures & tables

Sarah L Dalglish, Hina Khalid, Shannon A McMahon, Document analysis in health policy research: the READ approach, Health Policy and Planning , Volume 35, Issue 10, December 2020, Pages 1424–1431, https://doi.org/10.1093/heapol/czaa064

Permissions Icon Permissions

Document analysis is one of the most commonly used and powerful methods in health policy research. While existing qualitative research manuals offer direction for conducting document analysis, there has been little specific discussion about how to use this method to understand and analyse health policy. Drawing on guidance from other disciplines and our own research experience, we present a systematic approach for document analysis in health policy research called the READ approach: (1) ready your materials, (2) extract data, (3) analyse data and (4) distil your findings. We provide practical advice on each step, with consideration of epistemological and theoretical issues such as the socially constructed nature of documents and their role in modern bureaucracies. We provide examples of document analysis from two case studies from our work in Pakistan and Niger in which documents provided critical insight and advanced empirical and theoretical understanding of a health policy issue. Coding tools for each case study are included as Supplementary Files to inspire and guide future research. These case studies illustrate the value of rigorous document analysis to understand policy content and processes and discourse around policy, in ways that are either not possible using other methods, or greatly enrich other methods such as in-depth interviews and observation. Given the central nature of documents to health policy research and importance of reading them critically, the READ approach provides practical guidance on gaining the most out of documents and ensuring rigour in document analysis.

Rigour in qualitative research is judged partly by the use of deliberate, systematic procedures; however, little specific guidance is available for analysing documents, a nonetheless common method in health policy research.

Document analysis is useful for understanding policy content across time and geographies, documenting processes, triangulating with interviews and other sources of data, understanding how information and ideas are presented formally, and understanding issue framing, among other purposes.

The READ (Ready materials, Extract data, Analyse data, Distil) approach provides a step-by-step guide to conducting document analysis for qualitative policy research.

The READ approach can be adapted to different purposes and types of research, two examples of which are presented in this article, with sample tools in the Supplementary Materials .

Document analysis (also called document review) is one of the most commonly used methods in health policy research; it is nearly impossible to conduct policy research without it. Writing in early 20th century, Weber (2015) identified the importance of formal, written documents as a key characteristic of the bureaucracies by which modern societies function, including in public health. Accordingly, critical social research has a long tradition of documentary review: Marx analysed official reports, laws, statues, census reports and newspapers and periodicals over a nearly 50-year period to come to his world-altering conclusions ( Harvey, 1990 ). Yet in much of social science research, ‘documents are placed at the margins of consideration,’ with privilege given to the spoken word via methods such as interviews, possibly due to the fact that many qualitative methods were developed in the anthropological tradition to study mainly pre-literate societies ( Prior, 2003 ). To date, little specific guidance is available to help health policy researchers make the most of these wells of information.

The term ‘documents’ is defined here broadly, following Prior, as physical or virtual artefacts designed by creators, for users, to function within a particular setting ( Prior, 2003 ). Documents exist not as standalone objects of study but must be understood in the social web of meaning within which they are produced and consumed. For example, some analysts distinguish between public documents (produced in the context of public sector activities), private documents (from business and civil society) and personal documents (created by or for individuals, and generally not meant for public consumption) ( Mogalakwe, 2009 ). Documents can be used in a number of ways throughout the research process ( Bowen, 2009 ). In the planning or study design phase, they can be used to gather background information and help refine the research question. Documents can also be used to spark ideas for disseminating research once it is complete, by observing the ways those who will use the research speak to and communicate ideas with one another.

Documents can also be used during data collection and analysis to help answer research questions. Recent health policy research shows that this can be done in at least four ways. Frequently, policy documents are reviewed to describe the content or categorize the approaches to specific health problems in existing policies, as in reviews of the composition of drowning prevention resources in the United States or policy responses to foetal alcohol spectrum disorder in South Africa ( Katchmarchi et al. , 2018 ; Adebiyi et al. , 2019 ). In other cases, non-policy documents are used to examine the implementation of health policies in real-world settings, as in a review of web sources and newspapers analysing the functioning of community health councils in New Zealand ( Gurung et al. , 2020 ). Perhaps less frequently, document analysis is used to analyse policy processes, as in an assessment of multi-sectoral planning process for nutrition in Burkina Faso ( Ouedraogo et al. , 2020 ). Finally, and most broadly, document analysis can be used to inform new policies, as in one study that assessed cigarette sticks as communication and branding ‘documents,’ to suggest avenues for further regulation and tobacco control activities ( Smith et al. , 2017 ).

This practice paper provides an overarching method for conducting document analysis, which can be adapted to a multitude of research questions and topics. Document analysis is used in most or all policy studies; the aim of this article is to provide a systematized method that will enhance procedural rigour. We provide an overview of document analysis, drawing on guidance from disciplines adjacent to public health, introduce the ‘READ’ approach to document analysis and provide two short case studies demonstrating how document analysis can be applied.

Document analysis is a systematic procedure for reviewing or evaluating documents, which can be used to provide context, generate questions, supplement other types of research data, track change over time and corroborate other sources ( Bowen, 2009 ). In one commonly cited approach in social research, Bowen recommends first skimming the documents to get an overview, then reading to identify relevant categories of analysis for the overall set of documents and finally interpreting the body of documents ( Bowen, 2009 ). Document analysis can include both quantitative and qualitative components: the approach presented here can be used with either set of methods, but we emphasize qualitative ones, which are more adapted to the socially constructed meaning-making inherent to collaborative exercises such as policymaking.

The study of documents as a research method is common to a number of social science disciplines—yet in many of these fields, including sociology ( Mogalakwe, 2009 ), anthropology ( Prior, 2003 ) and political science ( Wesley, 2010 ), document-based research is described as ill-considered and underutilized. Unsurprisingly, textual analysis is perhaps most developed in fields such as media studies, cultural studies and literary theory, all disciplines that recognize documents as ‘social facts’ that are created, consumed, shared and utilized in socially organized ways ( Atkinson and Coffey, 1997 ). Documents exist within social ‘fields of action,’ a term used to designate the environments within which individuals and groups interact. Documents are therefore not mere records of social life, but integral parts of it—and indeed can become agents in their own right ( Prior, 2003 ). Powerful entities also manipulate the nature and content of knowledge; therefore, gaps in available information must be understood as reflecting and potentially reinforcing societal power relations ( Bryman and Burgess, 1994 ).

Document analysis, like any research method, can be subject to concerns regarding validity, reliability, authenticity, motivated authorship, lack of representativity and so on. However, these can be mitigated or avoided using standard techniques to enhance qualitative rigour, such as triangulation (within documents and across methods and theoretical perspectives), ensuring adequate sample size or ‘engagement’ with the documents, member checking, peer debriefing and so on ( Maxwell, 2005 ).

Document analysis can be used as a standalone method, e.g. to analyse the contents of specific types of policy as they evolve over time and differ across geographies, but document analysis can also be powerfully combined with other types of methods to cross-validate (i.e. triangulate) and deepen the value of concurrent methods. As one guide to public policy research puts it, ‘almost all likely sources of information, data, and ideas fall into two general types: documents and people’ ( Bardach and Patashnik, 2015 ). Thus, researchers can ask interviewees to address questions that arise from policy documents and point the way to useful new documents. Bardach and Patashnik suggest alternating between documents and interviews as sources as information, as one tends to lead to the other, such as by scanning interviewees’ bookshelves and papers for titles and author names ( Bardach and Patashnik, 2015 ). Depending on your research questions, document analysis can be used in combination with different types of interviews ( Berner-Rodoreda et al. , 2018 ), observation ( Harvey, 2018 ), and quantitative analyses, among other common methods in policy research.

The READ approach to document analysis is a systematic procedure for collecting documents and gaining information from them in the context of health policy studies at any level (global, national, local, etc.). The steps consist of: (1) ready your materials, (2) extract data, (3) analyse data and (4) distil your findings. We describe each of these steps in turn.

Step 1. Ready your materials

At the outset, researchers must set parameters in terms of the nature and number (approximately) of documents they plan to analyse, based on the research question. How much time will you allocate to the document analysis, and what is the scope of your research question? Depending on the answers to these questions, criteria should be established around (1) the topic (a particular policy, programme, or health issue, narrowly defined according to the research question); (2) dates of inclusion (whether taking the long view of several decades, or zooming in on a specific event or period in time); and (3) an indicative list of places to search for documents (possibilities include databases such as Ministry archives; LexisNexis or other databases; online searches; and particularly interview subjects). For difficult-to-obtain working documents or otherwise non-public items, bringing a flash drive to interviews is one of the best ways to gain access to valuable documents.

For research focusing on a single policy or programme, you may review only a handful of documents. However, if you are looking at multiple policies, health issues, or contexts, or reviewing shorter documents (such as newspaper articles), you may look at hundreds, or even thousands of documents. When considering the number of documents you will analyse, you should make notes on the type of information you plan to extract from documents—i.e. what it is you hope to learn, and how this will help answer your research question(s). The initial criteria—and the data you seek to extract from documents—will likely evolve over the course of the research, as it becomes clear whether they will yield too few documents and information (a rare outcome), far too many documents and too much information (a much more common outcome) or documents that fail to address the research question; however, it is important to have a starting point to guide the search. If you find that the documents you need are unavailable, you may need to reassess your research questions or consider other methods of inquiry. If you have too many documents, you can either analyse a subset of these ( Panel 1 ) or adopt more stringent inclusion criteria.

Exploring the framing of diseases in Pakistani media

In Table 1 , we present a non-exhaustive list of the types of documents that can be included in document analyses of health policy issues. In most cases, this will mean written sources (policies, reports, articles). The types of documents to be analysed will vary by study and according to the research question, although in many cases, it will be useful to consult a mix of formal documents (such as official policies, laws or strategies), ‘gray literature’ (organizational materials such as reports, evaluations and white papers produced outside formal publication channels) and, whenever possible, informal or working documents (such as meeting notes, PowerPoint presentations and memoranda). These latter in particular can provide rich veins of insight into how policy actors are thinking through the issues under study, particularly for the lucky researcher who obtains working documents with ‘Track Changes.’ How you prioritize documents will depend on your research question: you may prioritize official policy documents if you are studying policy content, or you may prioritize informal documents if you are studying policy process.

Types of documents that can be consulted in studies of health policy

During this initial preparatory phase, we also recommend devising a file-naming system for your documents (e.g. Author.Date.Topic.Institution.PDF), so that documents can be easily retrieved throughout the research process. After extracting data and processing your documents the first time around, you will likely have additional ‘questions’ to ask your documents and need to consult them again. For this reason, it is important to clearly name source files and link filenames to the data that you are extracting (see sample naming conventions in the Supplementary Materials ).

Step 2. Extract data

Data can be extracted in a number of ways, and the method you select for doing so will depend on your research question and the nature of your documents. One simple way is to use an Excel spreadsheet where each row is a document and each column is a category of information you are seeking to extract, from more basic data such as the document title, author and date, to theoretical or conceptual categories deriving from your research question, operating theory or analytical framework (Panel 2). Documents can also be imported into thematic coding software such as Atlas.ti or NVivo, and data extracted that way. Alternatively, if the research question focuses on process, documents can be used to compile a timeline of events, to trace processes across time. Ask yourself, how can I organize these data in the most coherent manner? What are my priority categories? We have included two different examples of data extraction tools in the Supplementary Materials to this article to spark ideas.

Case study Documents tell part of the story in Niger

Document analyses are first and foremost exercises in close reading: documents should be read thoroughly, from start to finish, including annexes, which may seem tedious but which sometimes produce golden nuggets of information. Read for overall meaning as you extract specific data related to your research question. As you go along, you will begin to have ideas or build working theories about what you are learning and observing in the data. We suggest capturing these emerging theories in extended notes or ‘memos,’ as used in Grounded Theory methodology ( Charmaz, 2006 ); these can be useful analytical units in themselves and can also provide a basis for later report and article writing.

As you read more documents, you may find that your data extraction tool needs to be modified to capture all the relevant information (or to avoid wasting time capturing irrelevant information). This may require you to go back and seek information in documents you have already read and processed, which will be greatly facilitated by a coherent file-naming system. It is also useful to keep notes on other documents that are mentioned that should be tracked down (sometimes you can write the author for help). As a general rule, we suggest being parsimonious when selecting initial categories to extract from data. Simply reading the documents takes significant time in and of itself—make sure you think about how, exactly, the specific data you are extracting will be used and how it goes towards answering your research questions.

Step 3. Analyse data

As in all types of qualitative research, data collection and analysis are iterative and characterized by emergent design, meaning that developing findings continually inform whether and how to obtain and interpret data ( Creswell, 2013 ). In practice, this means that during the data extraction phase, the researcher is already analysing data and forming initial theories—as well as potentially modifying document selection criteria. However, only when data extraction is complete can one see the full picture. For example, are there any documents that you would have expected to find, but did not? Why do you think they might be missing? Are there temporal trends (i.e. similarities, differences or evolutions that stand out when documents are ordered chronologically)? What else do you notice? We provide a list of overarching questions you should think about when viewing your body of document as a whole ( Table 2 ).

Questions to ask your overall body of documents

HIV and viral hepatitis articles by main frames (%). Note: The percentage of articles is calculated by dividing the number of articles appearing in each frame for viral hepatitis and HIV by the respectivenumber of sampled articles for each disease (N = 137 for HIV; N = 117 for hepatitis). Time frame: 1 January 2006 to 30 September 2016

Representations of progress toward Millennium Development Goal 4 in Nigerien policy documents. Sources: clockwise from upper left: ( WHO 2006 ); ( Institut National de la Statistique 2010 ); ( Ministè re de la Santé Publique 2010 ); ( Unicef 2010 )

In addition to the meaning-making processes you are already engaged in during the data extraction process, in most cases, it will be useful to apply specific analysis methodologies to the overall corpus of your documents, such as policy analysis ( Buse et al. , 2005 ). An array of analysis methodologies can be used, both quantitative and qualitative, including case study methodology, thematic content analysis, discourse analysis, framework analysis and process tracing, which may require differing levels of familiarity and skills to apply (we highlight a few of these in the case studies below). Analysis can also be structured according to theoretical approaches. When it comes to analysing policies, process tracing can be particularly useful to combine multiple sources of information, establish a chronicle of events and reveal political and social processes, so as to create a narrative of the policy cycle ( Yin, 1994 ; Shiffman et al. , 2004 ). Practically, you will also want to take a holistic view of the documents’ ‘answers’ to the questions or analysis categories you applied during the data extraction phase. Overall, what did the documents ‘say’ about these thematic categories? What variation did you find within and between documents, and along which axes? Answers to these questions are best recorded by developing notes or memos, which again will come in handy as you write up your results.

As with all qualitative research, you will want to consider your own positionality towards the documents (and their sources and authors); it may be helpful to keep a ‘reflexivity’ memo documenting how your personal characteristics or pre-standing views might influence your analysis ( Watt, 2007 ).

Step 4. Distil your findings

You will know when you have completed your document review when one of the three things happens: (1) completeness (you feel satisfied you have obtained every document fitting your criteria—this is rare), (2) out of time (this means you should have used more specific criteria), and (3) saturation (you fully or sufficiently understand the phenomenon you are studying). In all cases, you should strive to make the third situation the reason for ending your document review, though this will not always mean you will have read and analysed every document fitting your criteria—just enough documents to feel confident you have found good answers to your research questions.

Now it is time to refine your findings. During the extraction phase, you did the equivalent of walking along the beach, noticing the beautiful shells, driftwood and sea glass, and picking them up along the way. During the analysis phase, you started sorting these items into different buckets (your analysis categories) and building increasingly detailed collections. Now you have returned home from the beach, and it is time to clean your objects, rinse them of sand and preserve only the best specimens for presentation. To do this, you can return to your memos, refine them, illustrate them with graphics and quotes and fill in any incomplete areas. It can also be illuminating to look across different strands of work: e.g. how did the content, style, authorship, or tone of arguments evolve over time? Can you illustrate which words, concepts or phrases were used by authors or author groups?

Results will often first be grouped by theoretical or analytic category, or presented as a policy narrative, interweaving strands from other methods you may have used (interviews, observation, etc.). It can also be helpful to create conceptual charts and graphs, especially as this corresponds to your analytical framework (Panels 1 and 2). If you have been keeping a timeline of events, you can seek out any missing information from other sources. Finally, ask yourself how the validity of your findings checks against what you have learned using other methods. The final products of the distillation process will vary by research study, but they will invariably allow you to state your findings relative to your research questions and to draw policy-relevant conclusions.

Document analysis is an essential component of health policy research—it is also relatively convenient and can be low cost. Using an organized system of analysis enhances the document analysis’s procedural rigour, allows for a fuller understanding of policy process and content and enhances the effectiveness of other methods such as interviews and non-participant observation. We propose the READ approach as a systematic method for interrogating documents and extracting study-relevant data that is flexible enough to accommodate many types of research questions. We hope that this article encourages discussion about how to make best use of data from documents when researching health policy questions.

Supplementary data are available at Health Policy and Planning online.

The data extraction tool in the Supplementary Materials for the iCCM case study (Panel 2) was conceived of by the research team for the multi-country study ‘Policy Analysis of Community Case Management for Childhood and Newborn Illnesses’. The authors thank Sara Bennett and Daniela Rodriguez for granting permission to publish this tool. S.M. was supported by The Olympia-Morata-Programme of Heidelberg University. The funders had no role in the decision to publish, or preparation of the manuscript. The content is the responsibility of the authors and does not necessarily represent the views of any funder.

Conflict of interest statement . None declared.

Ethical approval. No ethical approval was required for this study.

Abdelmutti N , Hoffman-Goetz L. 2009 . Risk messages about HPV, cervical cancer, and the HPV vaccine Gardasil: a content analysis of Canadian and U.S. national newspaper articles . Women & Health 49 : 422 – 40 .

Google Scholar

Adebiyi BO , Mukumbang FC , Beytell A-M. 2019 . To what extent is fetal alcohol spectrum disorder considered in policy-related documents in South Africa? A document review . Health Research Policy and Systems 17 :

Atkinson PA , Coffey A. 1997 . Analysing documentary realities. In: Silverman D (ed). Qualitative Research: Theory, Method and Practice . London : SAGE .

Google Preview

Bardach E , Patashnik EM. 2015 . Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving . Los Angeles : SAGE .

Bennett S , Dalglish SL , Juma PA , Rodríguez DC. 2015 . Altogether now… understanding the role of international organizations in iCCM policy transfer . Health Policy and Planning 30 : ii26 – 35 .

Berner-Rodoreda A , Bärnighausen T , Kennedy C et al. 2018 . From doxastic to epistemic: a typology and critique of qualitative interview styles . Qualitative Inquiry 26 : 291 – 305 . 1077800418810724.

Bowen GA. 2009 . Document analysis as a qualitative research method . Qualitative Research Journal 9 : 27 – 40 .

Bryman A. 1994 . Analyzing Qualitative Data .

Buse K , Mays N , Walt G. 2005 . Making Health Policy . New York : Open University Press .

Charmaz K. 2006 . Constructing Grounded Theory: A Practical Guide through Qualitative Analysis . London : SAGE .

Claassen L , Smid T , Woudenberg F , Timmermans DRM. 2012 . Media coverage on electromagnetic fields and health: content analysis of Dutch newspaper articles and websites . Health, Risk & Society 14 : 681 – 96 .

Creswell JW. 2013 . Qualitative Inquiry and Research Design . Thousand Oaks, CA : SAGE .

Dalglish SL , Rodríguez DC , Harouna A , Surkan PJ. 2017 . Knowledge and power in policy-making for child survival in Niger . Social Science & Medicine 177 : 150 – 7 .

Dalglish SL , Surkan PJ , Diarra A , Harouna A , Bennett S. 2015 . Power and pro-poor policies: the case of iCCM in Niger . Health Policy and Planning 30 : ii84 – 94 .

Entman RM. 1993 . Framing: toward clarification of a fractured paradigm . Journal of Communication 43 : 51 – 8 .

Fournier G , Djermakoye IA. 1975 . Village health teams in Niger (Maradi Department). In: Newell KW (ed). Health by the People . Geneva : WHO .

Gurung G , Derrett S , Gauld R. 2020 . The role and functions of community health councils in New Zealand’s health system: a document analysis . The New Zealand Medical Journal 133 : 70 – 82 .

Harvey L. 1990 . Critical Social Research . London : Unwin Hyman .

Harvey SA. 2018 . Observe before you leap: why observation provides critical insights for formative research and intervention design that you’ll never get from focus groups, interviews, or KAP surveys . Global Health: Science and Practice 6 : 299 – 316 .

Institut National de la Statistique. 2010. Rapport National sur les Progrès vers l'atteinte des Objectifs du Millénaire pour le Développement. Niamey, Niger: INS.

Kamarulzaman A. 2013 . Fighting the HIV epidemic in the Islamic world . Lancet 381 : 2058 – 60 .

Katchmarchi AB , Taliaferro AR , Kipfer HJ. 2018 . A document analysis of drowning prevention education resources in the United States . International Journal of Injury Control and Safety Promotion 25 : 78 – 84 .

Krippendorff K. 2004 . Content Analysis: An Introduction to Its Methodology . SAGE .

Marten R. 2019 . How states exerted power to create the Millennium Development Goals and how this shaped the global health agenda: lessons for the sustainable development goals and the future of global health . Global Public Health 14 : 584 – 99 .

Maxwell JA. 2005 . Qualitative Research Design: An Interactive Approach , 2 nd edn. Thousand Oaks, CA : Sage Publications .

Mayring P. 2004 . Qualitative Content Analysis . In: Flick U, von Kardorff E, Steinke I (eds). A Companion to Qualitative Research . SAGE .

Ministère de la Santé Publique. 2010. Enquête nationale sur la survie des enfants de 0 à 59 mois et la mortalité au Niger 2010. Niamey, Niger: MSP.

Mogalakwe M. 2009 . The documentary research method—using documentary sources in social research . Eastern Africa Social Science Research Review 25 : 43 – 58 .

Nelkin D. 1991 . AIDS and the news media . The Milbank Quarterly 69 : 293 – 307 .

Ouedraogo O , Doudou MH , Drabo KM et al. 2020 . Policy overview of the multisectoral nutrition planning process: the progress, challenges, and lessons learned from Burkina Faso . The International Journal of Health Planning and Management 35 : 120 – 39 .

Prior L. 2003 . Using Documents in Social Research . London: SAGE .

Shiffman J , Stanton C , Salazar AP. 2004 . The emergence of political priority for safe motherhood in Honduras . Health Policy and Planning 19 : 380 – 90 .

Smith KC , Washington C , Welding K et al. 2017 . Cigarette stick as valuable communicative real estate: a content analysis of cigarettes from 14 low-income and middle-income countries . Tobacco Control 26 : 604 – 7 .

Strömbäck J , Dimitrova DV. 2011 . Mediatization and media interventionism: a comparative analysis of Sweden and the United States . The International Journal of Press/Politics 16 : 30 – 49 .

UNICEF. 2010. Maternal, Newborn & Child Surival Profile. Niamey, Niger: UNICEF

Watt D. 2007 . On becoming a qualitative researcher: the value of reflexivity . Qualitative Report 12 : 82 – 101 .

Weber M. 2015 . Bureaucracy. In: Waters T , Waters D (eds). Rationalism and Modern Society: New Translations on Politics, Bureaucracy, and Social Stratification . London : Palgrave MacMillan .

Wesley JJ. 2010 . Qualitative Document Analysis in Political Science.

World Health Organization. 2006. Country Health System Fact Sheet 2006: Niger. Niamey, Niger: WHO.

Yin R. 1994 . Case Study Research: Design and Methods . Thousand Oaks, CA : Sage .

Supplementary data

Email alerts, citing articles via.

Recommend to Your Librarian

Affiliations

Online ISSN 1460-2237
Copyright © 2024 The London School of Hygiene and Tropical Medicine and Oxford University Press
About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Copyright © 2024 Oxford University Press
Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

No products in the cart.

The Basics of Document Analysis

Document analysis is the process of reviewing or evaluating documents both printed and electronic in a methodical manner. The document analysis method, like many other qualitative research methods, involves examining and interpreting data to uncover meaning, gain understanding, and come to a conclusion.

20k grant for early career researchers banner

What is Meant by Document Analysis?

Document analysis pertains to the process of interpreting documents for an assessment topic by the researcher as a means of giving voice and meaning. In Document Analysis as a Qualitative Research Method by Glenn A. Bowen , document analysis is described as, “... a systematic procedure for reviewing or evaluating documents—both printed and electronic (computer-based and Internet-transmitted) material. Like other analytical methods in qualitative research, document analysis requires that data be examined and interpreted in order to elicit meaning, gain understanding, and develop empirical knowledge.”

During the analysis of documents, the content is categorized into distinct themes, similar to the way transcripts from interviews or focus groups are analyzed. The documents may also be graded or scored using a rubric.

Document analysis is a social research method of great value, and it plays a crucial role in most triangulation methods, combining various methods to study a particular phenomenon.

>> View Webinar: How-To’s for Data Analysis

Documents fall into three main categories:

Personal Documents: A personal account of an individual's beliefs, actions, and experiences. The following are examples: e-mails, calendars, scrapbooks, Facebook posts, incident reports, blogs, duty logs, newspapers, and reflections or journals.
Public Records: Records of an organization's activities that are maintained continuously over time. These include mission statements, student transcripts, annual reports, student handbooks, policy manuals, syllabus, and strategic plans.
Physical Evidence: Artifacts or items found within a study setting, also referred to as artifacts. Among these are posters, flyers, agendas, training materials, and handbooks.

The qualitative researcher generally makes use of two or more resources, each using a different data source and methodology, to achieve convergence and corroboration. An important purpose of triangulating evidence is to establish credibility through a convergence of evidence. Corroboration of findings across data sets reduces the possibility of bias, by examining data gathered in different ways.

It is important to note that document analysis differs from content analysis as content analysis refers to more than documents. As part of their definition for content analysis, Columbia Mailman School of Public Health states that, “Sources of data could be from interviews, open-ended questions, field research notes, conversations, or literally any occurrence of communicative language (such as books, essays, discussions, newspaper headlines, speeches, media, historical documents).

How Do You Do Document Analysis?

In order for a researcher to obtain reliable results from document analysis, a detailed planning process must be undertaken. The following is an outline of an eight-step planning process that should be employed in all textual analysis including document analysis techniques.

Identify the texts you want to analyze such as samples, population, participants, and respondents.
You should consider how texts will be accessed, paying attention to any cultural or linguistic barriers.
Acknowledge and resolve biases.
Acquire appropriate research skills.
Strategize for ensuring credibility.
Identify the data that is being sought.
Take into account ethical issues.
Keep a backup plan handy.

Researchers can use a wide variety of texts as part of their research, but the most common source is likely to be written material. Researchers often ask how many documents they should collect. There is an opinion that a wide selection of documents is preferable, but the issue should probably revolve more around the quality of the document than its quantity.

Why is Document Analysis Useful?

Different types of documents serve different purposes. They provide background information, indicate potential interview questions, serve as a mechanism for monitoring progress and tracking changes within a project, and allow for verification of any claims or progress made.

You can triangulate your claims about the phenomenon being studied using document analysis by using multiple sources and other research gathering methods.

Below are the advantages and disadvantages of document analysis

Document analysis may assist researchers in determining what questions to ask your interviewees, as well as provide insight into what to watch out for during your participant observation.
It is particularly useful to researchers who wish to focus on specific case studies
It is inexpensive and quick in cases where data is easily obtainable.
Documents provide specific and reliable data, unaffected by researchers' presence unlike with other research methods like participant observation.

Disadvantages

It is likely that the documents researchers obtain are not complete or written objectively, requiring researchers to adopt a critical approach and not assume their contents are reliable or unbiased.
There may be a risk of information overload due to the number of documents involved. Researchers often have difficulties determining what parts of each document are relevant to the topic being studied.
It may be necessary to anonymize documents and compare them with other documents.

How NVivo Can Help with Document Analysis

Analyzing copious amounts of data and information can be a daunting and time-consuming prospect. Luckily, qualitative data analysis tools like NVivo can help!

NVivo’s AI-powered autocoding text analysis tool can help you efficiently analyze data and perform thematic analysis . By automatically detecting, grouping, and tagging noun phrases, you can quickly identify key themes throughout your documents – aiding in your evaluation.

Additionally, once you start coding part of your data, NVivo’s smart coding can take care of the rest for you by using machine learning to match your coding style. After your initial coding, you can run queries and create visualizations to expand on initial findings and gain deeper insights.

These features allow you to conduct data analysis on large amounts of documents – improving the efficiency of this qualitative research method. Learn more about these features in the webinar, NVivo 14: Thematic Analysis Using NVivo.

>> Watch Webinar NVivo 14: Thematic Analysis Using NVivo

A QDA recipe? A ten-step approach for qualitative document analysis using MAXQDA

Guest post by Professional MAXQDA Trainer Dr. Daniel Rasch .

Introduction

Qualitative text or document analysis has evolved into one of the most used qualitative methods across several disciplines ( Kuckartz, 2014 & Mayring, 2010). Its straightforward structure and procedure enable the researcher to adapt the method to his or her special case – nearly to every need.

A ten-steps-approach for qualitative document analysis using MAXQDA

This article proposes a recipe of ten simple steps for conducting qualitative document analyses (QDA) using MAXQDA (see table 1 for an overview).

Table 1: Overview of the “QDA recipe”

The ten steps for conducting qualitative document analyses using MAXQDA

Step 1: the research question(s).

As always, research begins with the question(s). Three aspects should be covered when dealing with the research question(s):

What do you want to find out exactly,
what relevance does your research on this exact question have, and
what contribution is your research going to make to your discipline?

Highlight these questions in your introduction and make your research stand out.

Step 2: Data collection and data sampling

After you have decided on the questions, you should think about how to answer them. What kind of qualitative data will best answer your question? Interviews – how many and with whom? Documents – which ones and where to collect them from?

At this point, you can already start thinking about validity: are you going to use a representative or a biased sample? Check the different options for sampling and its effects on validity ( Krippendorff, 2019 ).

Step 3: Select and prepare the data

For this step, MAXQDA 2020 is an excellent tool to help you prepare the selected data for any further steps . Whatever type of qualitative data you choose, you can import it into MAXQDA and then you can have MAXQDA assist in transcribing it. In the end, qualitative document analysis is all about written forms of communication (Kuckartz, 2014).

Figure 1: Import the data you have chosen or selected

Step 4: Codebook development

It takes time to develop a solid codebook. Working deductively, the process is a little easier with codes deriving from the theoretical considerations in the context of your research. Inductively, there are various steps you can use, ranging from creative coding to in-vivo-codes.

Content-wise, you can apply all sorts of codes, such as themes or evaluations, two of the most commonly used styles of content analysis (see thematic and evaluative content analysis in Kuckartz, 2014).

Figure 2: coding options in MAXQDA

a brief definition,
a long definition,
criteria for when to use the code,
criteria for when not to use the code, and
an example.

Using MAXQDA’s code memos simplify the process of creating and maintaining a good codebook . First, you can always go back to the codes and view and review your codebook within your project, and second, you can simply export the codebook as an attachment or appendix for publication purposes (use: Reports > Codebook ).

Figure 3: Creating a new code with code memo

Step 5: Unitizing and coding instructions

Before the process of coding starts, it is necessary to decide on the units of, as well as the rules for, coding. It is especially important to decide on your unit of coding (sentences, paragraphs, quasi-sentences, etc.). Coding rules help to keep this choice consistent and support you to stick to your research question(s) because every passage you code and every memo you write should be done in order to answer your research question(s). Decision rules should be added: what are you going to do if a passage does not fit in your subcodes but should be coded because it is important for your research question?

Step 6: Trial, training, reliability

Trial runs are of major importance. Not only do they show you, which codes work and which do not, but they also help you to rethink your choices in terms of the unit of coding, the content of the codebook, and reliability. Since there are different options for the latter, stick to what works best for you: either a qualitative comparison of what you have coded or quantitative indicators like Krippendorff’s alpha if need be .

You can test yourself or a team you work with and there might even be some situations, where a reliability test is not helpful or needed. When testing the codebook, be sure to test the variability of your collected documents and be sure that the entire codebook is tested.

MAXQDA helps you compare different forms of agreement for more an unlimited number of texts, divided into two different document groups (one document group coded by coder 1, a second document group coded by coder 2 – be aware, that you can also test yourself and be coder 2 yourself).

Figure 4: Intercoder agreement

Step 7: Revision and modification

After checking, which codes work and which do not, you can revise the codebook and modify it. As Schreier puts it: “No coding frame (codebook – DR) is perfect” (Schreier, 2012: 147).

Step 8: Coding

There are many different coding strategies, but one thing is for sure: qualitative work needs time and reading, as well as working with the material over and over again.

One coding strategy might be to first make yourself comfortable with the documents and start coding after second or third reading only. Another strategy is to concentrate on some of your codes first and do a second round of coding with the other codes later.

Step 9: Analyze and compare

Analyze and compare – these two words are the essence of the qualitative analysis at this step. At the core of each qualitative document analysis is the description of the content and the comparison of these contents between the documents you analyze.

After everything has been coded, you can make use of different analysis strategies: paraphrase, write summaries, look for intersections of codes, patterns of likeliness between the documents using simple or complex queries.

Figure 5: different analysis strategies in MAXQDA

Step 10: Interpretation and presentation

Reporting and summarizing qualitative findings is difficult. Most often, we find simple descriptions of the content with the use of quotations, paraphrases or other references to the text. However, MAXQDA makes it fast and easier with many options to choose from . The easiest way is to generate a table to sum up your findings – if your data or the findings allow for this.

MAXQDA offers several options: either map relations of codes, documents or memos with the MAXMaps , create matrices between codes and documents ( Code Matrix Browser ) or codes and codes ( Code Relations Browser ) to display the distribution of codes inside your data or even using different colors to map the distribution of codes or single documents.

Figure 6: Visual Tools for presentation

The Code Matrix Browser also enables you to quantify the qualitative data using two clicks. You can export these numbers for further analysis with statistical packages, to run causal relation and effect calculations, such as regressions or correlations ( Rasch, 2018 ).

Summary and adoption

Qualitative document analysis is one of the most popular techniques and adaptable to nearly every field. MAXQDA is a software tool that offers many options to make your analysis and therefore your research easier .

The recipe works best for theory-driven, deductive coding. However, it can be also used for inductive, explorative work by switching some of these steps around: for example, your codebook development might be one step to do during or after the trial and testing, since codes are developed inductively during the coding process. Still, it is important to define these codes properly.

The above-mentioned recipe has been used as a basis for several publications by the author. Starting with simple comparison of qualitative and quantitative text analysis ( Boräng et al., 2014 ), to the usage of the qualitative data as a basis for regression models ( Eising et al., 2015 ; Eising et al., 2017 ) to a book using mixed methods and therefore both qualitative and quantitative data analysis ( Rasch, 2018 ).

About the author

Daniel Rasch is a post-doctoral researcher in political science at the German University of Administrative Sciences, Speyer. He received his Ph.D. with a mixed methods analysis of lobbyists‘ success in the European Union. He focuses on the quantification of qualitative data. He is an experienced MAXQDA lecturer and has been a Professional MAXQDA Trainer since 2012.

MAXQDA Newsletter

Our research and analysis tips, straight to your inbox.

Document analysis in health policy research: the READ approach

Sarah l dalglish.

1 Department of International Health, Johns Hopkins School of Public Health, 615 N. Wolfe St, Baltimore, MD 21205, USA

2 Institute for Global Health, University College London, Institute for Global Health 3rd floor, 30 Guilford Street, London WC1N 1EH, UK

Hina Khalid

3 School of Humanities and Social Sciences, Information Technology University, Arfa Software Technology Park, Ferozepur Road, Lahore 54000, Pakistan

Shannon A McMahon

4 Heidelberg Institute of Global Health, Medical Faculty and University Hospital, University of Heidelberg, Im Neuenheimer Feld 130/3, 69120 Heidelberg, Germany

Associated Data

Key Messages

Rigour in qualitative research is judged partly by the use of deliberate, systematic procedures; however, little specific guidance is available for analysing documents, a nonetheless common method in health policy research.
Document analysis is useful for understanding policy content across time and geographies, documenting processes, triangulating with interviews and other sources of data, understanding how information and ideas are presented formally, and understanding issue framing, among other purposes.
The READ (Ready materials, Extract data, Analyse data, Distil) approach provides a step-by-step guide to conducting document analysis for qualitative policy research.
The READ approach can be adapted to different purposes and types of research, two examples of which are presented in this article, with sample tools in the Supplementary Materials .

Introduction

What is document analysis?

The READ approach

Step 1. Ready your materials

Exploring the framing of diseases in Pakistani media

Types of documents that can be consulted in studies of health policy

Step 2. Extract data

Case study Documents tell part of the story in Niger

Step 3. Analyse data

Questions to ask your overall body of documents

An external file that holds a picture, illustration, etc.
Object name is czaa064f1.jpg

An external file that holds a picture, illustration, etc.
Object name is czaa064f2.jpg

Step 4. Distil your findings

Supplementary Data

Supplementary data are available at Health Policy and Planning online.

Supplementary Material

Czaa064_supplementary_data, acknowledgements.

Conflict of interest statement . None declared.

Ethical approval. No ethical approval was required for this study.

Abdelmutti N, Hoffman-Goetz L. 2009. Risk messages about HPV, cervical cancer, and the HPV vaccine Gardasil: a content analysis of Canadian and U.S. national newspaper articles . Women & Health 49 : 422–40. [ PubMed ] [ Google Scholar ]
Adebiyi BO, Mukumbang FC, Beytell A-M. 2019. To what extent is fetal alcohol spectrum disorder considered in policy-related documents in South Africa? A document review . Health Research Policy and Systems 17 : [ PMC free article ] [ PubMed ] [ Google Scholar ]
Atkinson PA, Coffey A. 1997. Analysing documentary realities In: Silverman D (ed). Qualitative Research: Theory, Method and Practice . London: SAGE. [ Google Scholar ]
Bardach E, Patashnik EM. 2015. Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving . Los Angeles: SAGE. [ Google Scholar ]
Bennett S, Dalglish SL, Juma PA, Rodríguez DC. 2015. Altogether now… understanding the role of international organizations in iCCM policy transfer . Health Policy and Planning 30 : ii26–35. [ PubMed ] [ Google Scholar ]
Berner-Rodoreda A, Bärnighausen T, Kennedy C et al. 2018. From doxastic to epistemic: a typology and critique of qualitative interview styles . Qualitative Inquiry 26 : 291–305. 1077800418810724. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Bowen GA. 2009. Document analysis as a qualitative research method . Qualitative Research Journal 9 : 27–40. [ Google Scholar ]
Bryman A. 1994. Analyzing Qualitative Data .
Buse K, Mays N, Walt G. 2005. Making Health Policy . New York: Open University Press. [ Google Scholar ]
Charmaz K. 2006. Constructing Grounded Theory: A Practical Guide through Qualitative Analysis . London: SAGE. [ PubMed ] [ Google Scholar ]
Claassen L, Smid T, Woudenberg F, Timmermans DRM. 2012. Media coverage on electromagnetic fields and health: content analysis of Dutch newspaper articles and websites . Health, Risk & Society 14 : 681–96. [ Google Scholar ]
Creswell JW. 2013. Qualitative Inquiry and Research Design . Thousand Oaks, CA: SAGE. [ Google Scholar ]
Dalglish SL, Rodríguez DC, Harouna A, Surkan PJ. 2017. Knowledge and power in policy-making for child survival in Niger . Social Science & Medicine 177 : 150–7. [ PubMed ] [ Google Scholar ]
Dalglish SL, Surkan PJ, Diarra A, Harouna A, Bennett S. 2015. Power and pro-poor policies: the case of iCCM in Niger . Health Policy and Planning 30 : ii84–94. [ PubMed ] [ Google Scholar ]
Entman RM. 1993. Framing: toward clarification of a fractured paradigm . Journal of Communication 43 : 51–8. [ Google Scholar ]
Fournier G, Djermakoye IA. 1975. Village health teams in Niger (Maradi Department) In: Newell KW (ed). Health by the People . Geneva: WHO. [ Google Scholar ]
Gurung G, Derrett S, Gauld R. 2020. The role and functions of community health councils in New Zealand’s health system: a document analysis . The New Zealand Medical Journal 133 : 70–82. [ PubMed ] [ Google Scholar ]
Harvey L. 1990. Critical Social Research . London: Unwin Hyman. [ Google Scholar ]
Harvey SA. 2018. Observe before you leap: why observation provides critical insights for formative research and intervention design that you’ll never get from focus groups, interviews, or KAP surveys . Global Health: Science and Practice 6 : 299–316. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Institut National de la Statistique. 2010. Rapport National sur les Progrès vers l'atteinte des Objectifs du Millénaire pour le Développement. Niamey, Niger: INS. [ Google Scholar ]
Kamarulzaman A. 2013. Fighting the HIV epidemic in the Islamic world . Lancet 381 : 2058–60. [ PubMed ] [ Google Scholar ]
Katchmarchi AB, Taliaferro AR, Kipfer HJ. 2018. A document analysis of drowning prevention education resources in the United States . International Journal of Injury Control and Safety Promotion 25 : 78–84. [ PubMed ] [ Google Scholar ]
Krippendorff K. 2004. Content Analysis: An Introduction to Its Methodology . SAGE. [ Google Scholar ]
Marten R. 2019. How states exerted power to create the Millennium Development Goals and how this shaped the global health agenda: lessons for the sustainable development goals and the future of global health . Global Public Health 14 : 584–99. [ PubMed ] [ Google Scholar ]
Maxwell JA. 2005. Qualitative Research Design: An Interactive Approach , 2 nd edn. Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
Mayring P. 2004. Qualitative Content Analysis . In: Flick U, von Kardorff E, Steinke I (eds). A Companion to Qualitative Research SAGE. [ Google Scholar ]
Ministère de la Santé Publique. 2010. Enquête nationale sur la survie des enfants de 0 à 59 mois et la mortalité au Niger 2010. Niamey, Niger: MSP. [ Google Scholar ]
Mogalakwe M. 2009. The documentary research method—using documentary sources in social research . Eastern Africa Social Science Research Review 25 : 43–58. [ Google Scholar ]
Nelkin D. 1991. AIDS and the news media . The Milbank Quarterly 69 : 293–307. [ PubMed ] [ Google Scholar ]
Ouedraogo O, Doudou MH, Drabo KM et al. 2020. Policy overview of the multisectoral nutrition planning process: the progress, challenges, and lessons learned from Burkina Faso . The International Journal of Health Planning and Management 35 : 120–39. [ PubMed ] [ Google Scholar ]
Prior L. 2003. Using Documents in Social Research . London: SAGE. [ Google Scholar ]
Shiffman J, Stanton C, Salazar AP. 2004. The emergence of political priority for safe motherhood in Honduras . Health Policy and Planning 19 : 380–90. [ PubMed ] [ Google Scholar ]
Smith KC, Washington C, Welding K et al. 2017. Cigarette stick as valuable communicative real estate: a content analysis of cigarettes from 14 low-income and middle-income countries . Tobacco Control 26 : 604–7. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Strömbäck J, Dimitrova DV. 2011. Mediatization and media interventionism: a comparative analysis of Sweden and the United States . The International Journal of Press/Politics 16 : 30–49. [ Google Scholar ]
UNICEF. 2010. Maternal, Newborn & Child Surival Profile. Niamey, Niger: UNICEF . [ Google Scholar ]
Watt D. 2007. On becoming a qualitative researcher: the value of reflexivity . Qualitative Report 12 : 82–101. [ Google Scholar ]
Weber M. 2015. Bureaucracy In: Waters T, Waters D (eds). Rationalism and Modern Society: New Translations on Politics, Bureaucracy, and Social Stratification . London: Palgrave MacMillan. [ Google Scholar ]
Wesley JJ. 2010. Qualitative Document Analysis in Political Science.
World Health Organization. 2006. Country Health System Fact Sheet 2006: Niger. Niamey, Niger: WHO.
Yin R. 1994. Case Study Research: Design and Methods . Thousand Oaks, CA: Sage. [ Google Scholar ]

This website may not work correctly because your browser is out of date. Please update your browser .

Data collection methods for evaluation: Document review

Resource link.

Data collection methods for evaluation - document review (PDF, 162KB)

This resource from the Centers for Disease Control and Prevention (CDC) provides a brief guide to using document review as a data collection method for evaluation.

This guide provides an overview of when to use document review, how to plan and conduct it, and its advantages and disadvantages. It is noted that document review is helpful for gathering background information, determining if program implementation reflects program plans, and developing other data collection tools for evaluation.

Document review has several advantages, including being relatively inexpensive, providing a behind-the-scenes look at a program that may not be directly observable, and bringing up issues not noted by other means. However, there are also potential disadvantages, such as information being incomplete or inaccurate, biased due to selective survival of information, and time-consuming to collect, review, and analyze many documents.

Centers for Disease Control and Prevention (CDC), (2018). Data collection methods for evaluation: Document review (No. 18). U.S. Dept. of Health and Human Services. https://www.cdc.gov/healthyyouth/evaluation/pdf/brief18.pdf

'Data collection methods for evaluation: Document review' is referenced in:

Existing documents
Best of AEA365: Approaching document review in a systematic way

Open access
Published: 16 May 2024

Integrating qualitative research within a clinical trials unit: developing strategies and understanding their implementation in contexts

Jeremy Segrott ORCID: orcid.org/0000-0001-6215-0870 1 ,
Sue Channon 2 ,
Amy Lloyd 4 ,
Eleni Glarou 2 , 3 ,
Josie Henley 5 ,
Jacqueline Hughes 2 ,
Nina Jacob 2 ,
Sarah Milosevic 2 ,
Yvonne Moriarty 2 ,
Bethan Pell 6 ,
Mike Robling 2 ,
Heather Strange 2 ,
Julia Townson 2 ,
Qualitative Research Group &
Lucy Brookes-Howell 2

Trials volume 25 , Article number: 323 ( 2024 ) Cite this article

360 Accesses

6 Altmetric

Metrics details

Background/aims

The value of using qualitative methods within clinical trials is widely recognised. How qualitative research is integrated within trials units to achieve this is less clear. This paper describes the process through which qualitative research has been integrated within Cardiff University’s Centre for Trials Research (CTR) in Wales, UK. We highlight facilitators of, and challenges to, integration.

We held group discussions on the work of the Qualitative Research Group (QRG) within CTR. The content of these discussions, materials for a presentation in CTR, and documents relating to the development of the QRG were interpreted at a workshop attended by group members. Normalisation Process Theory (NPT) was used to structure analysis. A writing group prepared a document for input from members of CTR, forming the basis of this paper.

Actions to integrate qualitative research comprised: its inclusion in Centre strategies; formation of a QRG with dedicated funding/roles; embedding of qualitative research within operating systems; capacity building/training; monitoring opportunities to include qualitative methods in studies; maximising the quality of qualitative research and developing methodological innovation. Facilitators of these actions included: the influence of the broader methodological landscape within trial/study design and its promotion of the value of qualitative research; and close physical proximity of CTR qualitative staff/students allowing sharing of methodological approaches. Introduction of innovative qualitative methods generated interest among other staff groups. Challenges included: pressure to under-resource qualitative components of research, preference for a statistical stance historically in some research areas and funding structures, and difficulties faced by qualitative researchers carving out individual academic profiles when working across trials/studies.

Conclusions

Given that CTUs are pivotal to the design and conduct of RCTs and related study types across multiple disciplines, integrating qualitative research into trials units is crucial if its contribution is to be fully realised. We have made explicit one trials unit’s experience of embedding qualitative research and present this to open dialogue on ways to operationalise and optimise qualitative research in trials. NPT provides a valuable framework with which to theorise these processes, including the importance of sense-making and legitimisation when introducing new practices within organisations.

Peer Review reports

The value of using qualitative methods within randomised control trials (RCTs) is widely recognised [ 1 , 2 , 3 ]. Qualitative research generates important evidence on factors affecting trial recruitment/retention [ 4 ] and implementation, aiding interpretation of quantitative data [ 5 ]. Though RCTs have traditionally been viewed as sitting within a positivist paradigm, recent methodological innovations have developed new trial designs that draw explicitly on both quantitative and qualitative methods. For instance, in the field of complex public health interventions, realist RCTs seek to understand the mechanisms through which interventions generate hypothesised impacts, and how interactions across different implementation contexts form part of these mechanisms. Proponents of realist RCTs—which integrate experimental and realist paradigms—highlight the importance of using quantitative and qualitative methods to fully realise these aims and to generate an understanding of intervention mechanisms and how context shapes them [ 6 ].

A need for guidance on how to conduct good quality qualitative research is being addressed, particularly in relation to feasibility studies for RCTs [ 7 ] and process evaluations embedded within trials of complex interventions [ 5 ]. There is also guidance on the conduct of qualitative research within trials at different points in the research cycle, including development, conduct and reporting [ 8 , 9 ].

A high proportion of trials are based within or involve clinical trials units (CTUs). In the UK the UKCRC Registered CTU Network describes them as:

… specialist units which have been set up with a specific remit to design, conduct, analyse and publish clinical trials and other well-designed studies. They have the capability to provide specialist expert statistical, epidemiological, and other methodological advice and coordination to undertake successful clinical trials. In addition, most CTUs will have expertise in the coordination of trials involving investigational medicinal products which must be conducted in compliance with the UK Regulations governing the conduct of clinical trials resulting from the EU Directive for Clinical Trials.

Thus, CTUs provide the specialist methodological expertise needed for the conduct of trials, and in the case of trials of investigational medicinal products, their involvement may be mandated to ensure compliance with relevant regulations. As the definition above suggests, CTUs also conduct and support other types of study apart from RCTs, providing a range of methodological and subject-based expertise.

However, despite their central role in the conduct and design of trials, (and other evaluation designs) little has been written about how CTUs have integrated qualitative work within their organisation at a time when such methods are, as stated above, now recognised as an important aspect of RCTs and evaluation studies more generally. This is a significant gap, since integration at the organisational level arguably shapes how qualitative research is integrated within individual studies, and thus it is valuable to understand how CTUs have approached the task. There are different ways of involving qualitative work in trials units, such as partnering with other departments (e.g. social science) or employing qualitative researchers directly. Qualitative research can be imagined and configured in different ways—as a method that generates data to inform future trial and intervention design, as an embedded component within an RCT or other evaluation type, or as a parallel strand of research focusing on lived experiences of illness, for instance. Understanding how trials units have integrated qualitative research is valuable, as it can shed light on which strategies show promise, and in which contexts, and how qualitative research is positioned within the field of trials research, foregrounding the value of qualitative research. However, although much has been written about its use within trials, few accounts exist of how trials units have integrated qualitative research within their systems and structures.

This paper discusses the process of embedding qualitative research within the work of one CTU—Cardiff University’s Centre for Trials Research (CTR). It highlights facilitators of this process and identifies challenges to integration. We use the Normalisation Process Theory (NPT) as a framework to structure our experience and approach. The key gap addressed by this paper is the implementation of strategies to integrate qualitative research (a relatively newly adopted set of practices and processes) within CTU systems and structures. We acknowledge from the outset that there are multiple ways of approaching this task. What follows therefore is not a set of recommendations for a preferred or best way to integrate qualitative research, as this will comprise diverse actions according to specific contexts. Rather, we examine the processes through which integration occurred in our own setting and highlight the potential value of these insights for others engaged in the work of promoting qualitative research within trials units.

Background to the integration of qualitative research within CTR

The CTR was formed in 2015 [ 10 ]. It brought together three existing trials units at Cardiff University: the South East Wales Trials Unit, the Wales Cancer Trials Unit, and the Haematology Clinical Trials Unit. From its inception, the CTR had a stated aim of developing a programme of qualitative research and integrating it within trials and other studies. In the sections below, we map these approaches onto the framework offered by Normalisation Process Theory to understand the processes through which they helped achieve embedding and integration of qualitative research.

CTR’s aims (including those relating to the development of qualitative research) were included within its strategy documents and communicated to others through infrastructure funding applications, annual reports and its website. A Qualitative Research Group (QRG), which had previously existed within the South East Wales Trials Unit, with dedicated funding for methodological specialists and group lead academics, was a key mechanism through which the development of a qualitative portfolio was put into action. Integration of qualitative research within Centre systems and processes occurred through the inclusion of qualitative research in study adoption processes and representation on committees. The CTR’s study portfolio provided a basis to track qualitative methods in new and existing studies, identify opportunities to embed qualitative methods within recently adopted studies (at the funding application stage) and to manage staff resources. Capacity building and training were an important focus of the QRG’s work, including training courses, mentoring, creation of an academic network open to university staff and practitioners working in the field of healthcare, presentations at CTR staff meetings and securing of PhD studentships. Standard operating procedures and methodological guidance on the design and conduct of qualitative research (e.g. templates for developing analysis plans) aimed to create a shared understanding of how to undertake high-quality research, and a means to monitor the implementation of rigorous approaches. As the QRG expanded its expertise it sought to develop innovative approaches, including the use of visual [ 11 ] and ethnographic methods [ 12 ].

Understanding implementation—Normalisation Process Theory (NPT)

Normalisation Process Theory (NPT) provides a model with which to understand the implementation of new sets of practices and their normalisation within organisational settings. The term ‘normalisation’ refers to how new practices become routinised (part of the everyday work of an organisation) through embedding and integration [ 13 , 14 ]. NPT defines implementation as ‘the social organisation of work’ and is concerned with the social processes that take place as new practices are introduced. Embedding involves ‘making practices routine elements of everyday life’ within an organisation. Integration takes the form of ‘sustaining embedded practices in social contexts’, and how these processes lead to the practices becoming (or not becoming) ‘normal and routine’ [ 14 ]. NPT is concerned with the factors which promote or ‘inhibit’ attempts to embed and integrate the operationalisation of new practices [ 13 , 14 , 15 ].

Embedding new practices is therefore achieved through implementation—which takes the form of interactions in specific contexts. Implementation is operationalised through four ‘generative mechanisms’— coherence , cognitive participation , collective action and reflexive monitoring [ 14 ]. Each mechanism is characterised by components comprising immediate and organisational work, with actions of individuals and organisations (or groups of individuals) interdependent. The mechanisms operate partly through forms of investment (i.e. meaning, commitment, effort, and comprehension) [ 14 ].

Coherence refers to how individuals/groups make sense of, and give meaning to, new practices. Sense-making concerns the coherence of a practice—whether it ‘holds together’, and its differentiation from existing activities [ 15 ]. Communal and individual specification involve understanding new practices and their potential benefits for oneself or an organisation. Individuals consider what new practices mean for them in terms of tasks and responsibilities ( internalisation ) [ 14 ].

NPT frames the second mechanism, cognitive participation , as the building of a ‘community of practice’. For a new practice to be initiated, individuals and groups within an organisation must commit to it [ 14 , 15 ]. Cognitive participation occurs through enrolment —how people relate to the new practice; legitimation —the belief that it is right for them to be involved; and activation —defining which actions are necessary to sustain the practice and their involvement [ 14 ]. Making the new practices work may require changes to roles (new responsibilities, altered procedures) and reconfiguring how colleagues work together (changed relationships).

Third, Collective Action refers to ‘the operational work that people do to enact a set of practices’ [ 14 ]. Individuals engage with the new practices ( interactional workability ) reshaping how members of an organisation interact with each other, through creation of new roles and expectations ( relational interaction ) [ 15 ]. Skill set workability concerns how the work of implementing a new set of practices is distributed and the necessary roles and skillsets defined [ 14 ]. Contextual integration draws attention to the incorporation of a practice within social contexts, and the potential for aspects of these contexts, such as systems and procedures, to be modified as a result [ 15 ].

Reflexive monitoring is the final implementation mechanism. Collective and individual appraisal evaluate the value of a set of practices, which depends on the collection of information—formally and informally ( systematisation ). Appraisal may lead to reconfiguration in which procedures of the practice are redefined or reshaped [ 14 , 15 ].

We sought to map the following: (1) the strategies used to embed qualitative research within the Centre, (2) key facilitators, and (3) barriers to their implementation. Through focused group discussions during the monthly meetings of the CTR QRG and in discussion with the CTR senior management team throughout 2019–2020 we identified nine types of documents (22 individual documents in total) produced within the CTR which had relevant information about the integration of qualitative research within its work (Table 1 ). The QRG had an ‘open door’ policy to membership and welcomed all staff/students with an interest in qualitative research. It included researchers who were employed specifically to undertake qualitative research and other staff with a range of study roles, including trial managers, statisticians, and data managers. There was also diversity in terms of career stage, including PhD students, mid-career researchers and members of the Centre’s Executive team. Membership was therefore largely self-selected, and comprised of individuals with a role related to, or an interest in, embedding qualitative research within trials. However, the group brought together diverse methodological perspectives and was not solely comprised of methodological ‘champions’ whose job it was to promote the development of qualitative research within the centre. Thus whilst the group (and by extension, the authors of this paper) had a shared appreciation of the value of qualitative research within a trials centre, they also brought varied methodological perspectives and ways of engaging with it.

All members of the QRG ( n = 26) were invited to take part in a face-to-face, day-long workshop in February 2019 on ‘How to optimise and operationalise qualitative research in trials: reflections on CTR structure’. The workshop was attended by 12 members of staff and PhD students, including members of the QRG and the CTR’s senior management team. Recruitment to the workshop was therefore inclusive, and to some extent opportunistic, but all members of the QRG were able to contribute to discussions during regular monthly group meetings and the drafting of the current paper.

The aim of the workshop was to bring together information from the documents in Table 1 to generate discussion around the key strategies (and their component activities) that had been adopted to integrate qualitative research into CTR, as well as barriers to, and facilitators of, their implementation. The agenda for the workshop involved four key areas: development and history of the CTR model; mapping the current model within CTR; discussing the structure of other CTUs; and exploring the advantages and disadvantages of the CTR model.

During the workshop, we discussed the use of NPT to conceptualise how qualitative research had been embedded within CTR’s systems and practices. The group produced spider diagrams to map strategies and actions on to the four key domains (or ‘generative mechanisms’ of NPT) summarised above, to aid the understanding of how they had functioned, and the utility of NPT as a framework. This is summarised in Table 2 .

Detailed notes were made during the workshop. A core writing group then used these notes and the documents in Table 1 to develop a draft of the current paper. This was circulated to all members of the CTR QRG ( n = 26) and stored within a central repository accessible to them to allow involvement and incorporate the views of those who were not able to attend the workshop. This draft was again presented for comments in the monthly CTR QRG meeting in February 2021 attended by n = 10. The Standards for QUality Improvement Reporting Excellence 2.0 (SQUIRE) guidelines were used to inform the structure and content of the paper (see supplementary material) [ 16 ].

In the following sections, we describe the strategies CTR adopted to integrate qualitative research. These are mapped against NPT’s four generative mechanisms to explore the processes through which the strategies promoted integration, and facilitators of and barriers to their implementation. A summary of the strategies and their functioning in terms of the generative mechanisms is provided in Table 2 .

Coherence—making sense of qualitative research

In CTR, many of the actions taken to build a portfolio of qualitative research were aimed at enabling colleagues, and external actors, to make sense of this set of methodologies. Centre-level strategies and grant applications for infrastructure funding highlighted the value of qualitative research, the added benefits it would bring, and positioned it as a legitimate set of practices alongside existing methods. For example, a 2014 application for renewal of trials unit infrastructure funding stated:

We are currently in the process of undertaking […] restructuring for our qualitative research team and are planning similar for trial management next year. The aim of this restructuring is to establish greater hierarchical management and opportunities for staff development and also provide a structure that can accommodate continuing growth.

Within the CTR, various forms of communication on the development of qualitative research were designed to enable staff and students to make sense of it, and to think through its potential value for them, and ways in which they might engage with it. These included presentations at staff meetings, informal meetings between project teams and the qualitative group lead, and the visibility of qualitative research on the public-facing Centre website and Centre committees and systems. For instance, qualitative methods were included (and framed as a distinct set of practices) within study adoption forms and committee agendas. Information for colleagues described how qualitative methods could be incorporated within funding applications for RCTs and other evaluation studies to generate new insights into questions research teams were already keen to answer, such as influences on intervention implementation fidelity. Where externally based chief investigators approached the Centre to be involved in new grant applications, the existence of the qualitative team and group lead enabled the inclusion of qualitative research to be actively promoted at an early stage, and such opportunities were highlighted in the Centre’s brochure for new collaborators. Monthly qualitative research network meetings—advertised across CTR and to external research collaborators, were also designed to create a shared understanding of qualitative research methods and their utility within trials and other study types (e.g. intervention development, feasibility studies, and observational studies). Training events (discussed in more detail below) also aided sense-making.

Several factors facilitated the promotion of qualitative research as a distinctive and valuable entity. Among these was the influence of the broader methodological landscape within trial design which was promoting the value of qualitative research, such as guidance on the evaluation of complex interventions by the Medical Research Council [ 17 ], and the growing emphasis placed on process evaluations within trials (with qualitative methods important in understanding participant experience and influences on implementation) [ 5 ]. The attention given to lived experience (both through process evaluations and the move to embed public involvement in trials) helped to frame qualitative research within the Centre as something that was appropriate, legitimate, and of value. Recognition by research funders of the value of qualitative research within studies was also helpful in normalising and legitimising its adoption within grant applications.

The inclusion of qualitative methods within influential methodological guidance helped CTR researchers to develop a ‘shared language’ around these methods, and a way that a common understanding of the role of qualitative research could be generated. One barrier to such sense-making work was the varying extent to which staff and teams had existing knowledge or experience of qualitative research. This varied across methodological and subject groups within the Centre and reflected the history of the individual trials units which had merged to form the Centre.

Cognitive participation—legitimising qualitative research

Senior CTR leaders promoted the value and legitimacy of qualitative research. Its inclusion in centre strategies, infrastructure funding applications, and in public-facing materials (e.g. website, investigator brochures), signalled that it was appropriate for individuals to conduct qualitative research within their roles, or to support others in doing so. Legitimisation also took place through informal channels, such as senior leadership support for qualitative research methods in staff meetings and participation in QRG seminars. Continued development of the QRG (with dedicated infrastructure funding) provided a visible identity and equivalence with other methodological groups (e.g. trial managers, statisticians).

Staff were asked to engage with qualitative research in two main ways. First, there was an expansion in the number of staff for whom qualitative research formed part of their formal role and responsibilities. One of the three trials units that merged to form CTR brought with it a qualitative team comprising methodological specialists and a group lead. CTR continued the expansion of this group with the creation of new roles and an enlarged nucleus of researchers for whom qualitative research was the sole focus of their work. In part, this was linked to the successful award of projects that included a large qualitative component, and that were coordinated by CTR (see Table 3 which describes the PUMA study).

Members of the QRG were encouraged to develop their own research ideas and to gain experience as principal investigators, and group seminars were used to explore new ideas and provide peer support. This was communicated through line management, appraisal, and informal peer interaction. Boundaries were not strictly demarcated (i.e. staff located outside the qualitative team were already using qualitative methods), but the new team became a central focus for developing a growing programme of work.

Second, individuals and studies were called upon to engage in new ways with qualitative research, and with the qualitative team. A key goal for the Centre was that groups developing new research ideas should give more consideration in general to the potential value and inclusion of qualitative research within their funding applications. Specifically, they were asked to do this by thinking about qualitative research at an early point in their application’s development (rather than ‘bolting it on’ after other elements had been designed) and to draw upon the expertise and input of the qualitative team. An example was the inclusion of questions on qualitative methods within the Centre’s study adoption form and representation from the qualitative team at the committee which reviewed new adoption requests. Where adoption requests indicated the inclusion of qualitative methods, colleagues were encouraged to liaise with the qualitative team, facilitating the integration of its expertise from an early stage. Qualitative seminars offered an informal and supportive space in which researchers could share initial ideas and refine their methodological approach. The benefits of this included the provision of sufficient time for methodological specialists to be involved in the design of the proposed qualitative component and ensuring adequate costings had been drawn up. At study adoption group meetings, scrutiny of new proposals included consideration of whether new research proposals might be strengthened through the use of qualitative methods where these had not initially been included. Meetings of the QRG—which reviewed the Centre’s portfolio of new studies and gathered intelligence on new ideas—also helped to identify, early on, opportunities to integrate qualitative methods. Communication across teams was useful in identifying new research ideas and embedding qualitative researchers within emerging study development groups.

Actions to promote greater use of qualitative methods in funding applications fed through into a growing number of studies with a qualitative component. This helped to increase the visibility and legitimacy of qualitative methods within the Centre. For example, the PUMA study [ 12 ], which brought together a large multidisciplinary team to develop and evaluate a Paediatric early warning system, drew heavily on qualitative methods, with the qualitative research located within the QRG. The project introduced an extensive network of collaborators and clinical colleagues to qualitative methods and how they could be used during intervention development and the generation of case studies. Further information about the PUMA study is provided in Table 3 .

Increasing the legitimacy of qualitative work across an extensive network of staff, students and collaborators was a complex process. Set within the continuing dominance of quantitative methods with clinical trials, there were variations in the extent to which clinicians and other collaborators embraced the value of qualitative methods. Research funding schemes, which often continued to emphasise the quantitative element of randomised controlled trials, inevitably fed through into the focus of new research proposals. Staff and external collaborators were sometimes uncertain about the added value that qualitative methods would bring to their trials. Across the CTR there were variations in the speed at which qualitative research methods gained legitimacy, partly based on disciplinary traditions and their influences. For instance, population health trials, often located within non-health settings such as schools or community settings, frequently involved collaboration with social scientists who brought with them experience in qualitative methods. Methodological guidance in this field, such as MRC guidance on process evaluations, highlighted the value of qualitative methods and alternatives to the positivist paradigm, such as the value of realist RCTs. In other, more clinical areas, positivist paradigms had greater dominance. Established practices and methodological traditions across different funders also influenced the ease of obtaining funding to include qualitative research within studies. For drugs trials (CTIMPs), the influence of regulatory frameworks on study design, data collection and the allocation of staff resources may have played a role. Over time, teams gained repeated experience of embedding qualitative research (and researchers) within their work and took this learning with them to subsequent studies. For example, the senior clinician quoted within the PUMA case study (Table 3 below) described how they had gained an appreciation of the rigour of qualitative research and an understanding of its language. Through these repeated interactions, embedding of qualitative research within studies started to become the norm rather than the exception.

Collective action—operationalising qualitative research

Collective action concerns the operationalisation of new practices within organisations—the allocation and management of the work, how individuals interact with each other, and the work itself. In CTR the formation of a Qualitative Research Group helped to allocate and organise the work of building a portfolio of studies. Researchers across the Centre were called upon to interact with qualitative research in new ways. Presentations at staff meetings and the inclusion of qualitative research methods in portfolio study adoption forms were examples of this ( interactive workability ). It was operationalised by encouraging study teams to liaise with the qualitative research lead. Development of standard operating procedures, templates for costing qualitative research and methodological guidance (e.g. on analysis plans) also helped encourage researchers to interact with these methods in new ways. For some qualitative researchers who had been trained in the social sciences, working within a trials unit meant that they needed to interact in new and sometimes unfamiliar ways with standard operating procedures, risk assessments, and other trial-based systems. Thus, training needs and capacity-building efforts were multidirectional.

Whereas there had been a tendency for qualitative research to be ‘bolted on’ to proposals for RCTs, the systems described above were designed to embed thinking about the value and design of the qualitative component from the outset. They were also intended to integrate members of the qualitative team with trial teams from an early stage to promote effective integration of qualitative methods within larger trials and build relationships over time.

Standard Operating Procedures (SOPs), formal and informal training, and interaction between the qualitative team and other researchers increased the relational workability of qualitative methods within the Centre—the confidence individuals felt in including these methods within their studies, and their accountability for doing so. For instance, study adoption forms prompted researchers to interact routinely with the qualitative team at an early stage, whilst guidance on costing grants provided clear expectations about the resources needed to deliver a proposed set of qualitative data collection.

Formation of the Qualitative Research Group—comprised of methodological specialists, created new roles and skillsets ( skill set workability ). Research teams were encouraged to draw on these when writing funding applications for projects that included a qualitative component. Capacity-building initiatives were used to increase the number of researchers with the skills needed to undertake qualitative research, and for these individuals to develop their expertise over time. This was achieved through formal training courses, academic seminars, mentoring from experienced colleagues, and informal knowledge exchange. Links with external collaborators and centres engaged in building qualitative research supported these efforts. Within the Centre, the co-location of qualitative researchers with other methodological and trial teams facilitated knowledge exchange and building of collaborative relationships, whilst grouping of the qualitative team within a dedicated office space supported a collective identity and opportunities for informal peer support.

Some aspects of the context in which qualitative research was being developed created challenges to operationalisation. Dependence on project grants to fund qualitative methodologists meant that there was a continuing need to write further grant applications whilst limiting the amount of time available to do so. Similarly, researchers within the team whose role was funded largely by specific research projects could sometimes find it hard to create sufficient time to develop their personal methodological interests. However, the cultivation of a methodologically varied portfolio of work enabled members of the team to build significant expertise in different approaches (e.g. ethnography, discourse analysis) that connected individual studies.

Reflexive monitoring—evaluating the impact of qualitative research

Inclusion of questions/fields relating to qualitative research within the Centre’s study portfolio database was a key way in which information was collected ( systematisation ). It captured numbers of funding applications and funded studies, research design, and income generation. Alongside this database, a qualitative resource planner spreadsheet was used to link individual members of the qualitative team with projects and facilitate resource planning, further reinforcing the core responsibilities and roles of qualitative researchers within CTR. As with all staff in the Centre, members of the qualitative team were placed on ongoing rather than fixed-term contracts, reflecting their core role within CTR. Planning and strategy meetings used the database and resource planner to assess the integration of qualitative research within Centre research, identify opportunities for increasing involvement, and manage staff recruitment and sustainability of researcher posts. Academic meetings and day-to-day interaction fulfilled informal appraisal of the development of the group, and its position within the Centre. Individual appraisal was also important, with members of the qualitative team given opportunities to shape their role, reflect on progress, identify training needs, and further develop their skillset, particularly through line management systems.

These forms of systematisation and appraisal were used to reconfigure the development of qualitative research and its integration within the Centre. For example, group strategies considered how to achieve long-term integration of qualitative research from its initial embedding through further promoting the belief that it formed a core part of the Centre’s business. The visibility and legitimacy of qualitative research were promoted through initiatives such as greater prominence on the Centre’s website. Ongoing review of the qualitative portfolio and discussion at academic meetings enabled the identification of areas where increased capacity would be helpful, both for qualitative staff, and more broadly within the Centre. This prompted the qualitative group to develop an introductory course to qualitative methods open to all Centre staff and PhD students, aimed at increasing understanding and awareness. As the qualitative team built its expertise and experience it also sought to develop new and innovative approaches to conducting qualitative research. This included the use of visual and diary-based methods [ 11 ] and the adoption of ethnography to evaluate system-level clinical interventions [ 12 ]. Restrictions on conventional face-to-face qualitative data collection due to the COVID-19 pandemic prompted rapid adoption of virtual/online methods for interviews, observation, and use of new internet platforms such as Padlet—a form of digital note board.

In this paper, we have described the work undertaken by one CTU to integrate qualitative research within its studies and organisational culture. The parallel efforts of many trials units to achieve these goals arguably come at an opportune time. The traditional designs of RCTs have been challenged and re-imagined by the increasing influence of realist evaluation [ 6 , 18 ] and the widespread acceptance that trials need to understand implementation and intervention theory as well as assess outcomes [ 17 ]. Hence the widespread adoption of embedded mixed methods process evaluations within RCTs. These broad shifts in methodological orthodoxies, the production of high-profile methodological guidance, and the expectations of research funders all create fertile ground for the continued expansion of qualitative methods within trials units. However, whilst much has been written about the importance of developing qualitative research and the possible approaches to integrating qualitative and quantitative methods within studies, much less has been published on how to operationalise this within trials units. Filling this lacuna is important. Our paper highlights how the integration of a new set of practices within an organisation can become embedded as part of its ‘normal’ everyday work whilst also shaping the practices being integrated. In the case of CTR, it could be argued that the integration of qualitative research helped shape how this work was done (e.g. systems to assess progress and innovation).

In our trials unit, the presence of a dedicated research group of methodological specialists was a key action that helped realise the development of a portfolio of qualitative research and was perhaps the most visible evidence of a commitment to do so. However, our experience demonstrates that to fully realise the goal of developing qualitative research, much work focuses on the interaction between this ‘new’ set of methods and the organisation into which it is introduced. Whilst the team of methodological specialists was tasked with, and ‘able’ to do the work, the ‘work’ itself needed to be integrated and embedded within the existing system. Thus, alongside the creation of a team and methodological capacity, promoting the legitimacy of qualitative research was important to communicate to others that it was both a distinctive and different entity, yet similar and equivalent to more established groups and practices (e.g. trial management, statistics, data management). The framing of qualitative research within strategies, the messages given out by senior leaders (formally and informally) and the general visibility of qualitative research within the system all helped to achieve this.

Normalisation Process Theory draws our attention to the concepts of embedding (making a new practice routine, normal within an organisation) and integration —the long-term sustaining of these processes. An important process through which embedding took place in our centre concerned the creation of messages and systems that called upon individuals and research teams to interact with qualitative research. Research teams were encouraged to think about qualitative research and consider its potential value for their studies. Critically, they were asked to do so at specific points, and in particular ways. Early consideration of qualitative methods to maximise and optimise their inclusion within studies was emphasised, with timely input from the qualitative team. Study adoption systems, centre-level processes for managing financial and human resources, creation of a qualitative resource planner, and awareness raising among staff, helped to reinforce this. These processes of embedding and integration were complex and they varied in intensity and speed across different areas of the Centre’s work. In part this depended on existing research traditions, the extent of prior experience of working with qualitative researchers and methods, and the priorities of subject areas and funders. Centre-wide systems, sometimes linked to CTR’s operation as a CTU, also helped to legitimise and embed qualitative research, lending it equivalence with other research activity. For example, like all CTUs, CTR was required to conform with the principles of Good Clinical Practice, necessitating the creation of a quality management system, operationalised through standard operating procedures for all areas of its work. Qualitative research was included, and became embedded, within these systems, with SOPs produced to guide activities such as qualitative analysis.

NPT provides a helpful way of understanding how trials units might integrate qualitative research within their work. It highlights how new practices interact with existing organisational systems and the work needed to promote effective interaction. That is, alongside the creation of a team or programme of qualitative research, much of the work concerns how members of an organisation understand it, engage with it, and create systems to sustain it. Embedding a new set of practices may be just as important as the quality or characteristics of the practices themselves. High-quality qualitative research is of little value if it is not recognised and drawn upon within new studies for instance. NPT also offers a helpful lens with which to understand how integration and embedding occur, and the mechanisms through which they operate. For example, promoting the legitimacy of a new set of practices, or creating systems that embed it, can help sustain these practices by creating an organisational ambition and encouraging (or requiring) individuals to interact with them in certain ways, redefining their roles accordingly. NPT highlights the ways in which integration of new practices involves bi-directional exchanges with the organisation’s existing practices, with each having the potential to re-shape the other as interaction takes place. For instance, in CTR, qualitative researchers needed to integrate and apply their methods within the quality management and other systems of a CTU, such as the formalisation of key processes within standard operating procedures, something less likely to occur outside trials units. Equally, project teams (including those led by externally based chief investigators) increased the integration of qualitative methods within their overall study design, providing opportunities for new insights on intervention theory, implementation and the experiences of practitioners and participants.

We note two aspects of the normalisation processes within CTR that are slightly less well conceptualised by NPT. The first concerns the emphasis within coherence on identifying the distinctiveness of new practices, and how they differ from existing activities. Whilst differentiation was an important aspect of the integration of qualitative research in CTR, such integration could be seen as operating partly through processes of de-differentiation, or at least equivalence. That is, part of the integration of qualitative research was to see it as similar in terms of rigour, coherence, and importance to other forms of research within the Centre. To be viewed as similar, or at least comparable to existing practices, was to be legitimised.

Second, whilst NPT focuses mainly on the interaction between a new set of practices and the organisational context into which it is introduced, our own experience of introducing qualitative research into a trials unit was shaped by broader organisational and methodological contexts. For example, the increasing emphasis placed upon understanding implementation processes and the experiences of research participants in the field of clinical trials (e.g. by funders), created an environment conducive to the development of qualitative research methods within our Centre. Attempts to integrate qualitative research within studies were also cross-organisational, given that many of the studies managed within the CTR drew together multi-institutional teams. This provided important opportunities to integrate qualitative research within a portfolio of studies that extended beyond CTR and build a network of collaborators who increasingly included qualitative methods within their funding proposals. The work of growing and integrating qualitative research within a trials unit is an ongoing one in which ever-shifting macro-level influences can help or hinder, and where the organisations within which we work are never static in terms of barriers and facilitators.

The importance of utilising qualitative methods within RCTs is now widely recognised. Increased emphasis on the evaluation of complex interventions, the influence of realist methods directing greater attention to complexity and the widespread adoption of mixed methods process evaluations are key drivers of this shift. The inclusion of qualitative methods within individual trials is important and previous research has explored approaches to their incorporation and some of the challenges encountered. Our paper highlights that the integration of qualitative methods at the organisational level of the CTU can shape how they are taken up by individual trials. Within CTR, it can be argued that qualitative research achieved high levels of integration, as conceptualised by Normalisation Process Theory. Thus, qualitative research became recognised as a coherent and valuable set of practices, secured legitimisation as an appropriate focus of individual and organisational activity and benefitted from forms of collective action which operationalised these organisational processes. Crucially, the routinisation of qualitative research appeared to be sustained, something which NPT suggests helps define integration (as opposed to initial embedding). However, our analysis suggested that the degree of integration varied by trial area. This variation reflected a complex mix of factors including disciplinary traditions, methodological guidance, existing (un)familiarity with qualitative research, and the influence of regulatory frameworks for certain clinical trials.

NPT provides a valuable framework with which to understand how these processes of embedding and integration occur. Our use of NPT draws attention to the importance of sense-making and legitimisation as important steps in introducing a new set of practices within the work of an organisation. Integration also depends, across each mechanism of NPT, on the building of effective relationships, which allow individuals and teams to work together in new ways. By reflecting on our experiences and the decisions taken within CTR we have made explicit one such process for embedding qualitative research within a trials unit, whilst acknowledging that approaches may differ across trials units. Mindful of this fact, and the focus of the current paper on one trials unit’s experience, we do not propose a set of recommendations for others who are working to achieve similar goals. Rather, we offer three overarching reflections (framed by NPT) which may act as a useful starting point for trials units (and other infrastructures) seeking to promote the adoption of qualitative research.

First, whilst research organisations such as trials units are highly heterogenous, processes of embedding and integration, which we have foregrounded in this paper, are likely to be important across different contexts in sustaining the use of qualitative research. Second, developing a plan for the integration of qualitative research will benefit from mapping out the characteristics of the extant system. For example, it is valuable to know how familiar staff are with qualitative research and any variations across teams within an organisation. Thirdly, NPT frames integration as a process of implementation which operates through key generative mechanisms— coherence , cognitive participation , collective action and reflexive monitoring . These mechanisms can help guide understanding of which actions help achieve embedding and integration. Importantly, they span multiple aspects of how organisations, and the individuals within them, work. The ways in which people make sense of a new set of practices ( coherence ), their commitment towards it ( cognitive participation ), how it is operationalised ( collective action ) and the evaluation of its introduction ( reflexive monitoring ) are all important. Thus, for example, qualitative research, even when well organised and operationalised within an organisation, is unlikely to be sustained if appreciation of its value is limited, or people are not committed to it.

We present our experience of engaging with the processes described above to open dialogue with other trials units on ways to operationalise and optimise qualitative research in trials. Understanding how best to integrate qualitative research within these settings may help to fully realise the significant contribution which it makes the design and conduct of trials.

Availability of data and materials

Some documents cited in this paper are either freely available from the Centre for Trials Research website or can be requested from the author for correspondence.

O’Cathain A, Thomas KJ, Drabble SJ, Rudolph A, Hewison J. What can qualitative research do for randomised controlled trials? A systematic mapping review. BMJ Open. 2013;3(6):e002889.

Article PubMed PubMed Central Google Scholar

O’Cathain A, Thomas KJ, Drabble SJ, Rudolph A, Goode J, Hewison J. Maximising the value of combining qualitative research and randomised controlled trials in health research: the QUAlitative Research in Trials (QUART) study – a mixed methods study. Health Technol Assess. 2014;18(38):1–197.

Clement C, Edwards SL, Rapport F, Russell IT, Hutchings HA. Exploring qualitative methods reported in registered trials and their yields (EQUITY): systematic review. Trials. 2018;19(1):589.

Hennessy M, Hunter A, Healy P, Galvin S, Houghton C. Improving trial recruitment processes: how qualitative methodologies can be used to address the top 10 research priorities identified within the PRioRiTy study. Trials. 2018;19:584.

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350(mar19 6):h1258.

Bonell C, Fletcher A, Morton M, Lorenc T, Moore L. Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Soc Sci Med. 2012;75(12):2299–306.

Article PubMed Google Scholar

O’Cathain A, Hoddinott P, Lewin S, Thomas KJ, Young B, Adamson J, et al. Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers. Pilot Feasibility Stud. 2015;1:32.

Cooper C, O’Cathain A, Hind D, Adamson J, Lawton J, Baird W. Conducting qualitative research within Clinical Trials Units: avoiding potential pitfalls. Contemp Clin Trials. 2014;38(2):338–43.

Rapport F, Storey M, Porter A, Snooks H, Jones K, Peconi J, et al. Qualitative research within trials: developing a standard operating procedure for a clinical trials unit. Trials. 2013;14:54.

Cardiff University. Centre for Trials Research. Available from: https://www.cardiff.ac.uk/centre-for-trials-research . Accessed 10 May 2024.

Pell B, Williams D, Phillips R, Sanders J, Edwards A, Choy E, et al. Using visual timelines in telephone interviews: reflections and lessons learned from the star family study. Int J Qual Methods. 2020;19:160940692091367.

Thomas-Jones E, Lloyd A, Roland D, Sefton G, Tume L, Hood K, et al. A prospective, mixed-methods, before and after study to identify the evidence base for the core components of an effective Paediatric Early Warning System and the development of an implementation package containing those core recommendations for use in th. BMC Pediatr. 2018;18:244.

May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M, et al. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res. 2007;7:148.

May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43(3):535–54.

Article Google Scholar

May CR, Mair F, Finch T, Macfarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4:29.

Ogrinc G, Davies L, Goodman D, Batalden PB, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): Revised publication guidelines from a detailed consensus process. BMJ Quality and Safety. 2016;25:986-92.

Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.

Jamal F, Fletcher A, Shackleton N, Elbourne D, Viner R, Bonell C. The three stages of building and testing mid-level theories in a realist RCT: a theoretical and methodological case-example. Trials. 2015;16(1):466.

Download references

Acknowledgements

Members of the Centre for Trials Research (CTR) Qualitative Research Group were collaborating authors: C Drew (Senior Research Fellow—Senior Trial Manager, Brain Health and Mental Wellbeing Division), D Gillespie (Director, Infection, Inflammation and Immunity Trials, Principal Research Fellow), R Hale (now Research Associate, School of Social Sciences, Cardiff University), J Latchem-Hastings (now Lecturer and Postdoctoral Fellow, School of Healthcare Sciences, Cardiff University), R Milton (Research Associate—Trial Manager), B Pell (now PhD student, DECIPHer Centre, Cardiff University), H Prout (Research Associate—Qualitative), V Shepherd (Senior Research Fellow), K Smallman (Research Associate), H Stanton (Research Associate—Senior Data Manager). Thanks are due to Kerry Hood and Aimee Grant for their involvement in developing processes and systems for qualitative research within CTR.

No specific grant was received to support the writing of this paper.

Author information

Authors and affiliations.

Centre for Trials Research, DECIPHer Centre, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK

Jeremy Segrott

Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK

Sue Channon, Eleni Glarou, Jacqueline Hughes, Nina Jacob, Sarah Milosevic, Yvonne Moriarty, Mike Robling, Heather Strange, Julia Townson & Lucy Brookes-Howell

Division of Population Medicine, School of Medicine, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK

Eleni Glarou

Wales Centre for Public Policy, Cardiff University, Sbarc I Spark, Maindy Road, Cardiff, CF24 4HQ, UK

School of Social Sciences, Cardiff University, King Edward VII Avenue, Cardiff, CF10 3WA, UK

Josie Henley

DECIPHer Centre, School of Social Sciences, Cardiff University, Sbarc I Spark, Maindy Road, Cardiff, CF24 4HQ, UK

Bethan Pell

You can also search for this author in PubMed Google Scholar

Qualitative Research Group

, D. Gillespie
, J. Latchem-Hastings
, R. Milton
, V. Shepherd
, K. Smallman
& H. Stanton

Contributions

JS contributed to the design of the work and interpretation of data and was responsible for leading the drafting and revision of the paper. SC contributed to the design of the work, the acquisition of data and the drafting and revision of the paper. AL contributed to the design of the work, the acquisition of data and the drafting and revision of the paper. EG contributed to a critical review of the manuscript and provided additional relevant references. JH provided feedback on initial drafts of the paper and contributed to subsequent revisions. JHu provided feedback on initial drafts of the paper and contributed to subsequent revisions. NG provided feedback on initial drafts of the paper and contributed to subsequent revisions. SM was involved in the acquisition and analysis of data and provided a critical review of the manuscript. YM was involved in the acquisition and analysis of data and provided a critical review of the manuscript. MR was involved in the interpretation of data and critical review and revision of the paper. HS contributed to the conception and design of the work, the acquisition and analysis of data, and the revision of the manuscript. JT provided feedback on initial drafts of the paper and contributed to subsequent revisions. LB-H made a substantial contribution to the design and conception of the work, led the acquisition and analysis of data, and contributed to the drafting and revision of the paper.

Corresponding author

Correspondence to Jeremy Segrott .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval was not sought as no personal or identifiable data was collected.

Consent for publication

Competing interests.

All authors are or were members of staff or students in the Centre for Trials Research. JS is an associate editor of Trials .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Segrott, J., Channon, S., Lloyd, A. et al. Integrating qualitative research within a clinical trials unit: developing strategies and understanding their implementation in contexts. Trials 25 , 323 (2024). https://doi.org/10.1186/s13063-024-08124-7

Download citation

Received : 20 October 2023

Accepted : 17 April 2024

Published : 16 May 2024

DOI : https://doi.org/10.1186/s13063-024-08124-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Qualitative research
Qualitative methods
Trials units
Normalisation Process Theory
Randomised controlled trials

ISSN: 1745-6215

Submission enquiries: Access here and click Contact Us
General enquiries: [email protected]

CRO Platform

Test your insights. Run experiments. Win. Or learn. And then win.

eCommerce Customer Analytics Platform

Acquisition matters. But retention matters more. Understand, monitor & nurture the best customers.

Case Studies
Ebooks, Tools, Templates
Digital Marketing Glossary
eCommerce Growth Stories
eCommerce Growth Show
Help & Technical Documentation

CRO Guide > Chapter 3.1

Qualitative Research: Definition, Methodology, Limitation & Examples

Qualitative research is a method focused on understanding human behavior and experiences through non-numerical data. Examples of qualitative research include:

One-on-one interviews,
Focus groups, Ethnographic research,
Case studies,
Record keeping,
Qualitative observations

In this article, we’ll provide tips and tricks on how to use qualitative research to better understand your audience through real world examples and improve your ROI. We’ll also learn the difference between qualitative and quantitative data.

Marketers often seek to understand their customers deeply. Qualitative research methods such as face-to-face interviews, focus groups, and qualitative observations can provide valuable insights into your products, your market, and your customers’ opinions and motivations. Understanding these nuances can significantly enhance marketing strategies and overall customer satisfaction.

What is Qualitative Research

Qualitative research is a market research method that focuses on obtaining data through open-ended and conversational communication. This method focuses on the “why” rather than the “what” people think about you. Thus, qualitative research seeks to uncover the underlying motivations, attitudes, and beliefs that drive people’s actions.

Let’s say you have an online shop catering to a general audience. You do a demographic analysis and you find out that most of your customers are male. Naturally, you will want to find out why women are not buying from you. And that’s what qualitative research will help you find out.

In the case of your online shop, qualitative research would involve reaching out to female non-customers through methods such as in-depth interviews or focus groups. These interactions provide a platform for women to express their thoughts, feelings, and concerns regarding your products or brand. Through qualitative analysis, you can uncover valuable insights into factors such as product preferences, user experience, brand perception, and barriers to purchase.

Types of Qualitative Research Methods

Qualitative research methods are designed in a manner that helps reveal the behavior and perception of a target audience regarding a particular topic.

The most frequently used qualitative analysis methods are one-on-one interviews, focus groups, ethnographic research, case study research, record keeping, and qualitative observation.

1. One-on-one interviews

Conducting one-on-one interviews is one of the most common qualitative research methods. One of the advantages of this method is that it provides a great opportunity to gather precise data about what people think and their motivations.

Spending time talking to customers not only helps marketers understand who their clients are, but also helps with customer care: clients love hearing from brands. This strengthens the relationship between a brand and its clients and paves the way for customer testimonials.

A company might conduct interviews to understand why a product failed to meet sales expectations.
A researcher might use interviews to gather personal stories about experiences with healthcare.

These interviews can be performed face-to-face or on the phone and usually last between half an hour to over two hours.

When a one-on-one interview is conducted face-to-face, it also gives the marketer the opportunity to read the body language of the respondent and match the responses.

2. Focus groups

Focus groups gather a small number of people to discuss and provide feedback on a particular subject. The ideal size of a focus group is usually between five and eight participants. The size of focus groups should reflect the participants’ familiarity with the topic. For less important topics or when participants have little experience, a group of 10 can be effective. For more critical topics or when participants are more knowledgeable, a smaller group of five to six is preferable for deeper discussions.

The main goal of a focus group is to find answers to the “why”, “what”, and “how” questions. This method is highly effective in exploring people’s feelings and ideas in a social setting, where group dynamics can bring out insights that might not emerge in one-on-one situations.

A focus group could be used to test reactions to a new product concept.
Marketers might use focus groups to see how different demographic groups react to an advertising campaign.

One advantage that focus groups have is that the marketer doesn’t necessarily have to interact with the group in person. Nowadays focus groups can be sent as online qualitative surveys on various devices.

Focus groups are an expensive option compared to the other qualitative research methods, which is why they are typically used to explain complex processes.

3. Ethnographic research

Ethnographic research is the most in-depth observational method that studies individuals in their naturally occurring environment.

This method aims at understanding the cultures, challenges, motivations, and settings that occur.

A study of workplace culture within a tech startup.
Observational research in a remote village to understand local traditions.

Ethnographic research requires the marketer to adapt to the target audiences’ environments (a different organization, a different city, or even a remote location), which is why geographical constraints can be an issue while collecting data.

This type of research can last from a few days to a few years. It’s challenging and time-consuming and solely depends on the expertise of the marketer to be able to analyze, observe, and infer the data.

4. Case study research

The case study method has grown into a valuable qualitative research method. This type of research method is usually used in education or social sciences. It involves a comprehensive examination of a single instance or event, providing detailed insights into complex issues in real-life contexts.

Analyzing a single school’s innovative teaching method.
A detailed study of a patient’s medical treatment over several years.

Case study research may seem difficult to operate, but it’s actually one of the simplest ways of conducting research as it involves a deep dive and thorough understanding of the data collection methods and inferring the data.

5. Record keeping

Record keeping is similar to going to the library: you go over books or any other reference material to collect relevant data. This method uses already existing reliable documents and similar sources of information as a data source.

Historical research using old newspapers and letters.
A study on policy changes over the years by examining government records.

This method is useful for constructing a historical context around a research topic or verifying other findings with documented evidence.

6. Qualitative observation

Qualitative observation is a method that uses subjective methodologies to gather systematic information or data. This method deals with the five major sensory organs and their functioning, sight, smell, touch, taste, and hearing.

Sight : Observing the way customers visually interact with product displays in a store to understand their browsing behaviors and preferences.
Smell : Noting reactions of consumers to different scents in a fragrance shop to study the impact of olfactory elements on product preference.
Touch : Watching how individuals interact with different materials in a clothing store to assess the importance of texture in fabric selection.
Taste : Evaluating reactions of participants in a taste test to identify flavor profiles that appeal to different demographic groups.
Hearing : Documenting responses to changes in background music within a retail environment to determine its effect on shopping behavior and mood.

Below we are also providing real-life examples of qualitative research that demonstrate practical applications across various contexts:

Qualitative Research Real World Examples

Let’s explore some examples of how qualitative research can be applied in different contexts.

1. Online grocery shop with a predominantly male audience

Method used: one-on-one interviews.

Let’s go back to one of the previous examples. You have an online grocery shop. By nature, it addresses a general audience, but after you do a demographic analysis you find out that most of your customers are male.

One good method to determine why women are not buying from you is to hold one-on-one interviews with potential customers in the category.

Interviewing a sample of potential female customers should reveal why they don’t find your store appealing. The reasons could range from not stocking enough products for women to perhaps the store’s emphasis on heavy-duty tools and automotive products, for example. These insights can guide adjustments in inventory and marketing strategies.

2. Software company launching a new product

Method used: focus groups.

Focus groups are great for establishing product-market fit.

Let’s assume you are a software company that wants to launch a new product and you hold a focus group with 12 people. Although getting their feedback regarding users’ experience with the product is a good thing, this sample is too small to define how the entire market will react to your product.

So what you can do instead is holding multiple focus groups in 20 different geographic regions. Each region should be hosting a group of 12 for each market segment; you can even segment your audience based on age. This would be a better way to establish credibility in the feedback you receive.

3. Alan Pushkin’s “God’s Choice: The Total World of a Fundamentalist Christian School”

Method used: ethnographic research.

Moving from a fictional example to a real-life one, let’s analyze Alan Peshkin’s 1986 book “God’s Choice: The Total World of a Fundamentalist Christian School”.

Peshkin studied the culture of Bethany Baptist Academy by interviewing the students, parents, teachers, and members of the community alike, and spending eighteen months observing them to provide a comprehensive and in-depth analysis of Christian schooling as an alternative to public education.

The study highlights the school’s unified purpose, rigorous academic environment, and strong community support while also pointing out its lack of cultural diversity and openness to differing viewpoints. These insights are crucial for understanding how such educational settings operate and what they offer to students.

Even after discovering all this, Peshkin still presented the school in a positive light and stated that public schools have much to learn from such schools.

Peshkin’s in-depth research represents a qualitative study that uses observations and unstructured interviews, without any assumptions or hypotheses. He utilizes descriptive or non-quantifiable data on Bethany Baptist Academy specifically, without attempting to generalize the findings to other Christian schools.

4. Understanding buyers’ trends

Method used: record keeping.

Another way marketers can use quality research is to understand buyers’ trends. To do this, marketers need to look at historical data for both their company and their industry and identify where buyers are purchasing items in higher volumes.

For example, electronics distributors know that the holiday season is a peak market for sales while life insurance agents find that spring and summer wedding months are good seasons for targeting new clients.

5. Determining products/services missing from the market

Conducting your own research isn’t always necessary. If there are significant breakthroughs in your industry, you can use industry data and adapt it to your marketing needs.

The influx of hacking and hijacking of cloud-based information has made Internet security a topic of many industry reports lately. A software company could use these reports to better understand the problems its clients are facing.

As a result, the company can provide solutions prospects already know they need.

Real-time Customer Lifetime Value (CLV) Benchmark Report

See where your business stands compared to 1,000+ e-stores in different industries.

35 reports by industry and business size.

Qualitative Research Approaches

Once the marketer has decided that their research questions will provide data that is qualitative in nature, the next step is to choose the appropriate qualitative approach.

The approach chosen will take into account the purpose of the research, the role of the researcher, the data collected, the method of data analysis , and how the results will be presented. The most common approaches include:

Narrative : This method focuses on individual life stories to understand personal experiences and journeys. It examines how people structure their stories and the themes within them to explore human existence. For example, a narrative study might look at cancer survivors to understand their resilience and coping strategies.
Phenomenology : attempts to understand or explain life experiences or phenomena; It aims to reveal the depth of human consciousness and perception, such as by studying the daily lives of those with chronic illnesses.
Grounded theory : investigates the process, action, or interaction with the goal of developing a theory “grounded” in observations and empirical data.
Ethnography : describes and interprets an ethnic, cultural, or social group;
Case study : examines episodic events in a definable framework, develops in-depth analyses of single or multiple cases, and generally explains “how”. An example might be studying a community health program to evaluate its success and impact.

How to Analyze Qualitative Data

Analyzing qualitative data involves interpreting non-numerical data to uncover patterns, themes, and deeper insights. This process is typically more subjective and requires a systematic approach to ensure reliability and validity.

1. Data Collection

Ensure that your data collection methods (e.g., interviews, focus groups, observations) are well-documented and comprehensive. This step is crucial because the quality and depth of the data collected will significantly influence the analysis.

2. Data Preparation

Once collected, the data needs to be organized. Transcribe audio and video recordings, and gather all notes and documents. Ensure that all data is anonymized to protect participant confidentiality where necessary.

3. Familiarization

Immerse yourself in the data by reading through the materials multiple times. This helps you get a general sense of the information and begin identifying patterns or recurring themes.

Develop a coding system to tag data with labels that summarize and account for each piece of information. Codes can be words, phrases, or acronyms that represent how these segments relate to your research questions.

Descriptive Coding : Summarize the primary topic of the data.
In Vivo Coding : Use language and terms used by the participants themselves.
Process Coding : Use gerunds (“-ing” words) to label the processes at play.
Emotion Coding : Identify and record the emotions conveyed or experienced.

5. Thematic Development

Group codes into themes that represent larger patterns in the data. These themes should relate directly to the research questions and form a coherent narrative about the findings.

6. Interpreting the Data

Interpret the data by constructing a logical narrative. This involves piecing together the themes to explain larger insights about the data. Link the results back to your research objectives and existing literature to bolster your interpretations.

7. Validation

Check the reliability and validity of your findings by reviewing if the interpretations are supported by the data. This may involve revisiting the data multiple times or discussing the findings with colleagues or participants for validation.

8. Reporting

Finally, present the findings in a clear and organized manner. Use direct quotes and detailed descriptions to illustrate the themes and insights. The report should communicate the narrative you’ve built from your data, clearly linking your findings to your research questions.

Limitations of qualitative research

The disadvantages of qualitative research are quite unique. The techniques of the data collector and their own unique observations can alter the information in subtle ways. That being said, these are the qualitative research’s limitations:

1. It’s a time-consuming process

The main drawback of qualitative study is that the process is time-consuming. Another problem is that the interpretations are limited. Personal experience and knowledge influence observations and conclusions.

Thus, qualitative research might take several weeks or months. Also, since this process delves into personal interaction for data collection, discussions often tend to deviate from the main issue to be studied.

2. You can’t verify the results of qualitative research

Because qualitative research is open-ended, participants have more control over the content of the data collected. So the marketer is not able to verify the results objectively against the scenarios stated by the respondents. For example, in a focus group discussing a new product, participants might express their feelings about the design and functionality. However, these opinions are influenced by individual tastes and experiences, making it difficult to ascertain a universally applicable conclusion from these discussions.

3. It’s a labor-intensive approach

Qualitative research requires a labor-intensive analysis process such as categorization, recording, etc. Similarly, qualitative research requires well-experienced marketers to obtain the needed data from a group of respondents.

4. It’s difficult to investigate causality

Qualitative research requires thoughtful planning to ensure the obtained results are accurate. There is no way to analyze qualitative data mathematically. This type of research is based more on opinion and judgment rather than results. Because all qualitative studies are unique they are difficult to replicate.

5. Qualitative research is not statistically representative

Because qualitative research is a perspective-based method of research, the responses given are not measured.

Comparisons can be made and this can lead toward duplication, but for the most part, quantitative data is required for circumstances that need statistical representation and that is not part of the qualitative research process.

While doing a qualitative study, it’s important to cross-reference the data obtained with the quantitative data. By continuously surveying prospects and customers marketers can build a stronger database of useful information.

Quantitative vs. Qualitative Research

Qualitative and quantitative research side by side in a table

Image source

Quantitative and qualitative research are two distinct methodologies used in the field of market research, each offering unique insights and approaches to understanding consumer behavior and preferences.

As we already defined, qualitative analysis seeks to explore the deeper meanings, perceptions, and motivations behind human behavior through non-numerical data. On the other hand, quantitative research focuses on collecting and analyzing numerical data to identify patterns, trends, and statistical relationships.

Let’s explore their key differences:

Nature of Data:

Quantitative research : Involves numerical data that can be measured and analyzed statistically.
Qualitative research : Focuses on non-numerical data, such as words, images, and observations, to capture subjective experiences and meanings.

Research Questions:

Quantitative research : Typically addresses questions related to “how many,” “how much,” or “to what extent,” aiming to quantify relationships and patterns.
Qualitative research: Explores questions related to “why” and “how,” aiming to understand the underlying motivations, beliefs, and perceptions of individuals.

Data Collection Methods:

Quantitative research : Relies on structured surveys, experiments, or observations with predefined variables and measures.
Qualitative research : Utilizes open-ended interviews, focus groups, participant observations, and textual analysis to gather rich, contextually nuanced data.

Analysis Techniques:

Quantitative research: Involves statistical analysis to identify correlations, associations, or differences between variables.
Qualitative research: Employs thematic analysis, coding, and interpretation to uncover patterns, themes, and insights within qualitative data.

Do Conversion Rate Optimization the Right way.

Explore helps you make the most out of your CRO efforts through advanced A/B testing, surveys, advanced segmentation and optimised customer journeys.

An isometric image of an adobe adobe adobe adobe ad.

If you haven’t subscribed yet to our newsletter, now is your chance!

A man posing happily in front of a vivid purple background for an engaging blog post.

Like what you’re reading?

Join the informed ecommerce crowd.

We will never bug you with irrelevant info.

By clicking the Button, you confirm that you agree with our Terms and Conditions .

Continue your Conversion Rate Optimization Journey

Last modified: January 3, 2023
Conversion Rate Optimization , User Research

Valentin Radu

We’re a team of people that want to empower marketers around the world to create marketing campaigns that matter to consumers in a smart way. Meet us at the intersection of creativity, integrity, and development, and let us show you how to optimize your marketing.

Our Software

> Book a Demo
> Partner Program
> Affiliate Program
Blog Sitemap
Terms and Conditions
Privacy & Security
Cookies Policy
REVEAL Terms and Conditions

Open access
Published: 21 May 2024

The bright side of sports: a systematic review on well-being, positive emotions and performance

David Peris-Delcampo 1 ,
Antonio Núñez 2 ,
Paula Ortiz-Marholz 3 ,
Aurelio Olmedilla 4 ,
Enrique Cantón 1 ,
Javier Ponseti 2 &
Alejandro Garcia-Mas 2

BMC Psychology volume 12 , Article number: 284 ( 2024 ) Cite this article

357 Accesses

Metrics details

The objective of this study is to conduct a systematic review regarding the relationship between positive psychological factors, such as psychological well-being and pleasant emotions, and sports performance.

This study, carried out through a systematic review using PRISMA guidelines considering the Web of Science, PsycINFO, PubMed and SPORT Discus databases, seeks to highlight the relationship between other more ‘positive’ factors, such as well-being, positive emotions and sports performance.

The keywords will be decided by a Delphi Method in two rounds with sport psychology experts.

Participants

There are no participants in the present research.

The main exclusion criteria were: Non-sport thema, sample younger or older than 20–65 years old, qualitative or other methodology studies, COVID-related, journals not exclusively about Psychology.

Main outcomes measures

We obtained a first sample of 238 papers, and finally, this sample was reduced to the final sample of 11 papers.

The results obtained are intended to be a representation of the ‘bright side’ of sports practice, and as a complement or mediator of the negative variables that have an impact on athletes’ and coaches’ performance.

Conclusions

Clear recognition that acting on intrinsic motivation continues to be the best and most effective way to motivate oneself to obtain the highest levels of performance, a good perception of competence and a source of personal satisfaction.

Peer Review reports

Introduction

In recent decades, research in the psychology of sport and physical exercise has focused on the analysis of psychological variables that could have a disturbing, unfavourable or detrimental role, including emotions that are considered ‘negative’, such as anxiety/stress, sadness or anger, concentrating on their unfavourable relationship with sports performance [ 1 , 2 , 3 , 4 ], sports injuries [ 5 , 6 , 7 ] or, more generally, damage to the athlete’s health [ 8 , 9 , 10 ]. The study of ‘positive’ emotions such as happiness or, more broadly, psychological well-being, has been postponed at this time, although in recent years this has seen an increase that reveals a field of study of great interest to researchers and professionals [ 11 , 12 , 13 ] including physiological, psychological, moral and social beneficial effects of the physical activity in comic book heroes such as Tintin, a team leader, which can serve as a model for promoting healthy lifestyles, or seeking ‘eternal youth’ [ 14 ].

Emotions in relation to their effects on sports practice and performance rarely go in one direction, being either negative or positive—generally positive and negative emotions do not act alone [ 15 ]. Athletes experience different emotions simultaneously, even if they are in opposition and especially if they are of mild or moderate intensity [ 16 ]. The athlete can feel satisfied and happy and at the same time perceive a high level of stress or anxiety before a specific test or competition. Some studies [ 17 ] have shown how sports participation and the perceived value of elite sports positively affect the subjective well-being of the athlete. This also seems to be the case in non-elite sports practice. The review by Mansfield et al. [ 18 ] showed that the published literature suggests that practising sports and dance, in a group or supported by peers, can improve the subjective well-being of the participants, and also identifies negative feelings towards competence and ability, although the quantity and quality of the evidence published is low, requiring better designed studies. All these investigations are also supported by the development of the concept of eudaimonic well-being [ 19 ], which is linked to the development of intrinsic motivation, not only in its aspect of enjoyment but also in its relationship with the perception of competition and overcoming and achieving goals, even if this is accompanied by other unpleasant hedonic emotions or even physical discomfort. Shortly after a person has practised sports, he will remember those feelings of exhaustion and possibly stiffness, linked to feelings of satisfaction and even enjoyment.

Furthermore, the mediating role of parents, coaches and other psychosocial agents can be significant. In this sense, Lemelin et al. [ 20 ], with the aim of investigating the role of autonomy support from parents and coaches in the prediction of well-being and performance of athletes, found that autonomy support from parents and coaches has positive relationships with the well-being of the athlete, but that only coach autonomy support is associated with sports performance. This research suggests that parents and coaches play important but distinct roles in athlete well-being and that coach autonomy support could help athletes achieve high levels of performance.

On the other hand, an analysis of emotions in the sociocultural environment in which they arise and gain meaning is always interesting, both from an individual perspective and from a sports team perspective. Adler et al. [ 21 ] in a study with military teams showed that teams with a strong emotional culture of optimism were better positioned to recover from poor performance, suggesting that organisations that promote an optimistic culture develop more resilient teams. Pekrun et al. [ 22 ] observed with mathematics students that individual success boosts emotional well-being, while placing people in high-performance groups can undermine it, which is of great interest in investigating the effectiveness and adjustment of the individual in sports teams.

There is still little scientific literature in the field of positive emotions and their relationship with sports practice and athlete performance, although their approach has long had its clear supporters [ 23 , 24 ]. It is comforting to observe the significant increase in studies in this field, since some authors (e.g [ 25 , 26 ]). . , point out the need to overcome certain methodological and conceptual problems, paying special attention to the development of specific instruments for the evaluation of well-being in the sports field and evaluation methodologies.

As McCarthy [ 15 ] indicates, positive emotions (hedonically pleasant) can be the catalysts for excellence in sport and deserve a space in our research and in professional intervention to raise the level of athletes’ performance. From a holistic perspective, positive emotions are permanently linked to psychological well-being and research in this field is necessary: firstly because of the leading role they play in human behaviour, cognition and affection, and secondly, because after a few years of international uncertainty due to the COVID-19 pandemic and wars, it seems ‘healthy and intelligent’ to encourage positive emotions for our athletes. An additional reason is that they are known to improve motivational processes, reducing abandonment and negative emotional costs [ 11 ]. In this vein, concepts such as emotional intelligence make sense and can help to identify and properly manage emotions in the sports field and determine their relationship with performance [ 27 ] that facilitates the inclusion of emotional training programmes based on the ‘bright side’ of sports practice [ 28 ].

Based on all of the above, one might wonder how these positive emotions are related to a given event and what role each one of them plays in the athlete’s performance. Do they directly affect performance, or do they affect other psychological variables such as concentration, motivation and self-efficacy? Do they favour the availability and competent performance of the athlete in a competition? How can they be regulated, controlled for their own benefit? How can other psychosocial agents, such as parents or coaches, help to increase the well-being of their athletes?

This work aims to enhance the leading role, not the secondary, of the ‘good and pleasant side’ of sports practice, either with its own entity, or as a complement or mediator of the negative variables that have an impact on the performance of athletes and coaches. Therefore, the objective of this study is to conduct a systematic review regarding the relationship between positive psychological factors, such as psychological well-being and pleasant emotions, and sports performance. For this, the methodological criteria that constitute the systematic review procedure will be followed.

Materials and methods

This study was carried out through a systematic review using PRISMA (Preferred Reporting Items for Systematic Reviews) guidelines considering the Web of Science (WoS) and Psycinfo databases. These two databases were selected using the Delphi method [ 29 ]. It does not include a meta-analysis because there is great data dispersion due to the different methodologies used [ 30 ].

The keywords will be decided by the Delphi Method in two rounds with sport psychology experts. The results obtained are intended to be a representation of the ‘bright side’ of sports practice, and as a complement or mediator of the negative variables that have an impact on athletes’ and coaches’ performance.

It was determined that the main construct was to be psychological well-being, and that it was to be paired with optimism, healthy practice, realisation, positive mood, and performance and sport. The search period was limited to papers published between 2000 and 2023, and the final list of papers was obtained on February 13 , 2023. This research was conducted in two languages—English and Spanish—and was limited to psychological journals and specifically those articles where the sample was formed by athletes.

Each word was searched for in each database, followed by searches involving combinations of the same in pairs and then in trios. In relation to the results obtained, it was decided that the best approach was to group the words connected to positive psychology on the one hand, and on the other, those related to self-realisation/performance/health. In this way, it used parentheses to group words (psychological well-being; or optimism; or positive mood) with the Boolean ‘or’ between them (all three refer to positive psychology); and on the other hand, it grouped those related to performance/health/realisation (realisation; or healthy practice or performance), separating both sets of parentheses by the Boolean ‘and’’. To further filter the search, a keyword included in the title and in the inclusion criteria was added, which was ‘sport’ with the Boolean ‘and’’. In this way, the search achieved results that combined at least one of the three positive psychology terms and one of the other three.

Results (first phase)

The mentioned keywords were cross-matched, obtaining the combination with a sufficient number of papers. From the first research phase, the total number of papers obtained was 238. Then screening was carried out by 4 well-differentiated phases that are summarised in Fig. 1 . These phases helped to reduce the original sample to a more accurate one.

Phases of the selection process for the final sample. Four phases were carried out to select the final sample of articles. The first phase allowed the elimination of duplicates. In the second stage, those that, by title or abstract, did not fit the objectives of the article were eliminated. Previously selected exclusion criteria were applied to the remaining sample. Thus, in phase 4, the final sample of 11 selected articles was obtained

Results (second phase)

The first screening examined the title, and the abstract if needed, excluding the papers that were duplicated, contained errors or someone with formal problems, low N or case studies. This screening allowed the initial sample to be reduced to a more accurate one with 109 papers selected.

Results (third phase)

This was followed by the second screening to examine the abstract and full texts, excluding if necessary papers related to non-sports themes, samples that were too old or too young for our interests, papers using qualitative methodologies, articles related to the COVID period, or others published in non-psychological journals. Furthermore, papers related to ‘negative psychological variables’’ were also excluded.

Results (fourth phase)

At the end of this second screening the remaining number of papers was 11. In this final phase we tried to organise the main characteristics and their main conclusions/results in a comprehensible list (Table 1 ). Moreover, in order to enrich our sample of papers, we decided to include some articles from other sources, mainly those presented in the introduction to sustain the conceptual framework of the concept ‘bright side’ of sports.

The usual position of the researcher of psychological variables that affect sports performance is to look for relationships between ‘negative’ variables, first in the form of basic psychological processes, or distorting cognitive behavioural, unpleasant or evaluable as deficiencies or problems, in a psychology for the ‘risk’ society, which emphasises the rehabilitation that stems from overcoming personal and social pathologies [ 31 ], and, lately, regarding the affectation of the athlete’s mental health [ 32 ]. This fact seems to be true in many cases and situations and to openly contradict the proclaimed psychological benefits of practising sports (among others: Cantón [ 33 ], ; Froment and González [ 34 ]; Jürgens [ 35 ]).

However, it is possible to adopt another approach focused on the ‘positive’ variables, also in relation to the athlete’s performance. This has been the main objective of this systematic review of the existing literature and far from being a novel approach, although a minority one, it fits perfectly with the definition of our area of knowledge in the broad field of health, as has been pointed out for some time [ 36 , 37 ].

After carrying out the aforementioned systematic review, a relatively low number of articles were identified by experts that met the established conditions—according to the PRISMA method [ 37 , 38 , 39 , 40 ]—regarding databases, keywords, and exclusion and inclusion criteria. These precautions were taken to obtain the most accurate results possible, and thus guarantee the quality of the conclusions.

The first clear result that stands out is the great difficulty in finding articles in which sports ‘performance’ is treated as a well-defined study variable adapted to the situation and the athletes studied. In fact, among the results (11 papers), only 3 associate one or several positive psychological variables with performance (which is evaluated in very different ways, combining objective measures with other subjective ones). This result is not surprising, since in several previous studies (e.g. Nuñez et al. [ 41 ]) using a systematic review, this relationship is found to be very weak and nuanced by the role of different mediating factors, such as previous sports experience or the competitive level (e.g. Rascado, et al. [ 42 ]; Reche, Cepero & Rojas [ 43 ]), despite the belief—even among professional and academic circles—that there is a strong relationship between negative variables and poor performance, and vice versa, with respect to the positive variables.

Regarding what has been evidenced in relation to the latter, even with these restrictions in the inclusion and exclusion criteria, and the filters applied to the first findings, a true ‘galaxy’ of variables is obtained, which also belong to different categories and levels of psychological complexity.

A preliminary consideration regarding the current paradigm of sport psychology: although it is true that some recent works have already announced the swing of the pendulum on the objects of study of PD, by returning to the study of traits and dispositions, and even to the personality of athletes [ 43 , 44 , 45 , 46 ], our results fully corroborate this trend. Faced with five variables present in the studies selected at the end of the systematic review, a total of three traits/dispositions were found, which were also the most repeated—optimism being present in four articles, mental toughness present in three, and finally, perfectionism—as the representative concepts of this field of psychology, which lately, as has already been indicated, is significantly represented in the field of research in this area [ 46 , 47 , 48 , 49 , 50 , 51 , 52 ]. In short, the psychological variables that finally appear in the selected articles are: psychological well-being (PWB) [ 53 ]; self-compassion, which has recently been gaining much relevance with respect to the positive attributional resolution of personal behaviours [ 54 ], satisfaction with life (balance between sports practice, its results, and life and personal fulfilment [ 55 ], the existence of approach-achievement goals [ 56 ], and perceived social support [ 57 ]). This last concept is maintained transversally in several theoretical frameworks, such as Sports Commitment [ 58 ].

The most relevant concept, both quantitatively and qualitatively, supported by the fact that it is found in combination with different variables and situations, is not a basic psychological process, but a high-level cognitive construct: psychological well-being, in its eudaimonic aspect, first defined in the general population by Carol Ryff [ 59 , 60 ] and introduced at the beginning of this century in sport (e.g., Romero, Brustad & García-Mas [ 13 ], ; Romero, García-Mas & Brustad [ 61 ]). It is important to note that this concept understands psychological well-being as multifactorial, including autonomy, control of the environment in which the activity takes place, social relationships, etc.), meaning personal fulfilment through a determined activity and the achievement or progress towards goals and one’s own objectives, without having any direct relationship with simpler concepts, such as vitality or fun. In the selected studies, PWB appears in five of them, and is related to several of the other variables/traits.

The most relevant result regarding this variable is its link with motivational aspects, as a central axis that relates to different concepts, hence its connection to sports performance, as a goal of constant improvement that requires resistance, perseverance, management of errors and great confidence in the possibility that achievements can be attained, that is, associated with ideas of optimism, which is reflected in expectations of effectiveness.

If we detail the relationships more specifically, we can first review this relationship with the ‘way of being’, understood as personality traits or behavioural tendencies, depending on whether more or less emphasis is placed on their possibilities for change and learning. In these cases, well-being derives from satisfaction with progress towards the desired goal, for which resistance (mental toughness) and confidence (optimism) are needed. When, in addition, the search for improvement is constant and aiming for excellence, its relationship with perfectionism is clear, although it is a factor that should be explored further due to its potential negative effect, at least in the long term.

The relationship between well-being and satisfaction with life is almost tautological, in the precise sense that what produces well-being is the perception of a relationship or positive balance between effort (or the perception of control, if we use stricter terminology) and the results thereof (or the effectiveness of such control). This direct link is especially important when assessing achievement in personally relevant activities, which, in the case of the subjects evaluated in the papers, specifically concern athletes of a certain level of performance, which makes it a more valuable objective than would surely be found in the general population. And precisely because of this effect of the value of performance for athletes of a certain level, it also allows us to understand how well-being is linked to self-compassion, since as a psychological concept it is very close to that of self-esteem, but with a lower ‘demand’ or a greater ‘generosity’, when we encounter failures, mistakes or even defeats along the way, which offers us greater protection from the risk of abandonment and therefore reinforces persistence, a key element for any successful sports career [ 62 ].

It also has a very direct relationship with approach-achievement goals, since precisely one of the central aspects characterising this eudaimonic well-being and differentiating it from hedonic well-being is specifically its relationship with self-determined and persistent progress towards goals or achievements with incentive value for the person, as is sports performance evidently [ 63 ].

Finally, it is interesting to see how we can also find a facet or link relating to the aspects that are more closely-related to the need for human affiliation, with feeling part of a group or human collective, where we can recognise others and recognise ourselves in the achievements obtained and the social reinforcement of those themselves, as indicated by their relationship with perceived social support. This construct is very labile, in fact it is common to find results in which the pressure of social support is hardly differentiated, for example, from the parents of athletes and/or their coaches [ 64 ]. However, its relevance within this set of psychological variables and traits is proof of its possible conceptual validity.

Analysing the results obtained, the first conclusion is that in no case is an integrated model based solely on ‘positive’ variables or traits obtained, since some ‘negative’ ones appear (anxiety, stress, irrational thoughts), affecting the former.

The second conclusion is that among the positive elements the variable coping strategies (their use, or the perception of their effectiveness) and the traits of optimism, perfectionism and self-compassion prevail, since mental strength or psychological well-being (which also appear as important, but with a more complex nature) are seen to be participated in by the aforementioned traits.

Finally, it must be taken into account that the generation of positive elements, such as resilience, or the learning of coping strategies, are directly affected by the educational style received, or by the culture in which the athlete is immersed. Thus, the applied potential of these findings is great, but it must be calibrated according to the educational and/or cultural features of the specific setting.

Limitations

The limitations of this study are those evident and common in SR methodology using the PRISMA system, since the selection of keywords (and their logical connections used in the search), the databases, and the inclusion/exclusion criteria bias the work in its entirety and, therefore, constrain the generalisation of the results obtained.

Likewise, the conclusions must—based on the above and the results obtained—be made with the greatest concreteness and simplicity possible. Although we have tried to reduce these limitations as much as possible through the use of experts in the first steps of the method, they remain and must be considered in terms of the use of the results.

Future developments

Undoubtedly, progress is needed in research to more precisely elucidate the role of well-being, as it has been proposed here, from a bidirectional perspective: as a motivational element to push towards improvement and the achievement of goals, and as a product or effect of the self-determined and competent behaviour of the person, in relation to different factors, such as that indicated here of ‘perfectionism’ or the potential interference of material and social rewards, which are linked to sports performance—in our case—and that could act as a risk factor so that our achievements, far from being a source of well-being and satisfaction, become an insatiable demand in the search to obtain more and more frequent rewards.

From a practical point of view, an empirical investigation should be conducted to see if these relationships hold from a statistical point of view, either in the classical (correlational) or in the probabilistic (Bayesian Networks) plane.

The results obtained in this study, exclusively researched from the desk, force the authors to develop subsequent empirical and/or experimental studies in two senses: (1) what interrelationships exist between the so called ‘positive’ and ‘negative’ psychological variables and traits in sport, and in what sense are each of them produced; and, (2) from a global, motivational point of view, can currently accepted theoretical frameworks, such as SDT, easily accommodate this duality, which is becoming increasingly evident in applied work?

Finally, these studies should lead to proposals applied to the two fields that have appeared to be relevant: educational and cultural.

Application/transfer of results

A clear application of these results is aimed at guiding the training of sports and physical exercise practitioners, directing it towards strategies for assessing achievements, improvements and failure management, which keep them in line with well-being enhancement, eudaimonic, intrinsic and self-determined, which enhances the quality of their learning and their results and also favours personal health and social relationships.

Data availability

There are no further external data.

Cantón E, Checa I. Los estados emocionales y su relación con las atribuciones y las expectativas de autoeficacia en El deporte. Revista De Psicología Del Deporte. 2012;21(1):171–6.

Google Scholar

Cantón E, Checa I, Espejo B. (2015). Evidencias de validez convergente y test-criterio en la aplicación del Instrumento de Evaluación de Emociones en la Competición Deportiva. 24(2), 311–313.

Olmedilla A, Martins B, Ponseti-Verdaguer FJ, Ruiz-Barquín R, García-Mas A. It is not just stress: a bayesian Approach to the shape of the Negative Psychological Features Associated with Sport injuries. Healthcare. 2022;10(2):236. https://doi.org/10.3390/healthcare10020236 .

Article Google Scholar

Ong NCH, Chua JHE. Effects of psychological interventions on competitive anxiety in sport: a meta-analysis. Psycholy Sport Exerc. 2015;52:101836. https://doi.org/10.1016/j.psychsport.2020.101836 .

Candel MJ, Mompeán R, Olmedilla A, Giménez-Egido JM. Pensamiento catastrofista y evolución del estado de ánimo en futbolistas lesionados (Catastrophic thinking and temporary evolf mood state in injured football players). Retos. 2023;47:710–9.

Li C, Ivarsson A, Lam LT, Sun J. Basic Psychological needs satisfaction and frustration, stress, and sports Injury among University athletes: a Four-Wave prospective survey. Front Psychol. 2019;26:10. https://doi.org/10.3389/fpsyg.2019.00665 .

Wiese-Bjornstal DM. Psychological predictors and consequences of injuries in sport settings. In: Anshel MH, Petrie TA, Steinfelt JA, editors. APA handbook of sport and exercise psychology, volume 1: Sport psychology. Volume 1. Washington: American Psychological Association; 2019. pp. 699–725. https://doi.org/10.1037/0000123035 .

Chapter Google Scholar

Godoy PS, Redondo AB, Olmedilla A. (2022). Indicadores De Salud mental en jugadoras de fútbol en función de la edad. J Univers Mov Perform 21(5).

Golding L, Gillingham RG, Perera NKP. The prevalence of depressive symptoms in high-performance athletes: a systematic review. Physician Sportsmed. 2020;48(3):247–58. https://doi.org/10.1080/00913847.2020.1713708 .

Xanthopoulos MS, Benton T, Lewis J, Case JA, Master CL. Mental Health in the Young Athlete. Curr Psychiatry Rep. 2020;22(11):1–15. https://doi.org/10.1007/s11920-020-01185-w .

Cantón E, Checa I, Vellisca-González MY. Bienestar psicológico Y ansiedad competitiva: El Papel De las estrategias de afrontamiento / competitive anxiety and Psychological Well-being: the role of coping strategies. Revista Costarricense De Psicología. 2015;34(2):71–8.

Hahn E. Emotions in sports. In: Hackfort D, Spielberg CD, editors. Anxiety in Sports. Taylor & Francis; 2021. pp. 153–62. ISBN: 9781315781594.

Carrasco A, Brustad R, García-Mas A. Bienestar psicológico Y Su uso en la psicología del ejercicio, la actividad física y El Deporte. Revista Iberoamericana De psicología del ejercicio y El Deporte. 2007;2(2):31–52.

García-Mas A, Olmedilla A, Laffage-Cosnier S, Cruz J, Descamps Y, Vivier C. Forever Young! Tintin’s adventures as an Example of Physical Activity and Sport. Sustainability. 2021;13(4):2349. https://doi.org/10.3390/su13042349 .

McCarthy P. Positive emotion in sport performance: current status and future directions. Int Rev Sport Exerc Psycholy. 2011;4(1):50–69. https://doi.org/10.1080/1750984X.2011.560955 .

Cerin E. Predictors of competitive anxiety direction in male Tae Kwon do practitioners: a multilevel mixed idiographic/nomothetic interactional approach. Psychol Sport Exerc. 2004;5(4):497–516. https://doi.org/10.1016/S1469-0292(03)00041-4 .

Silva A, Monteiro D, Sobreiro P. Effects of sports participation and the perceived value of elite sport on subjective well-being. Sport Soc. 2020;23(7):1202–16. https://doi.org/10.1080/17430437.2019.1613376 .

Mansfield L, Kay T, Meads C, Grigsby-Duffy L, Lane J, John A, et al. Sport and dance interventions for healthy young people (15–24 years) to promote subjective well-being: a systematic review. BMJ Open. 2018;8(7). https://doi.org/10.1136/bmjopen-2017-020959 . e020959.

Ryff CD. Happiness is everything, or is it? Explorations on the meaning of psychological well-being. J Personal Soc Psychol. 1989;57(6):1069–81. https://doi.org/10.1037/0022-3514.57.6.1069 .

Lemelin E, Verner-Filion J, Carpentier J, Carbonneau N, Mageau G. Autonomy support in sport contexts: the role of parents and coaches in the promotion of athlete well-being and performance. Sport Exerc Perform Psychol. 2022;11(3):305–19. https://doi.org/10.1037/spy0000287 .

Adler AB, Bliese PD, Barsade SG, Sowden WJ. Hitting the mark: the influence of emotional culture on resilient performance. J Appl Psychol. 2022;107(2):319–27. https://doi.org/10.1037/apl0000897 .

Article PubMed Google Scholar

Pekrun R, Murayama K, Marsh HW, Goetz T, Frenzel AC. Happy fish in little ponds: testing a reference group model of achievement and emotion. J Personal Soc Psychol. 2019;117(1):166–85. https://doi.org/10.1037/pspp0000230 .

Seligman M. Authentic happiness. New York: Free Press/Simon and Schuster; 2002.

Seligman M, Florecer. La Nueva psicología positiva y la búsqueda del bienestar. Editorial Océano; 2016.

Giles S, Fletcher D, Arnold R, Ashfield A, Harrison J. Measuring well-being in Sport performers: where are we now and how do we Progress? Sports Med. 2020;50(7):1255–70. https://doi.org/10.1007/s40279-020-01274-z .

Article PubMed PubMed Central Google Scholar

Piñeiro-Cossio J, Fernández-Martínez A, Nuviala A, Pérez-Ordás R. Psychological wellbeing in Physical Education and School sports: a systematic review. Int J Environ Res Public Health. 2021;18(3):864. https://doi.org/10.3390/ijerph18030864 .

Gómez-García L, Olmedilla-Zafra A, Peris-Delcampo D. Inteligencia emocional y características psicológicas relevantes en mujeres futbolistas profesionales. Revista De Psicología Aplicada Al Deporte Y El Ejercicio Físico. 2023;15(72). https://doi.org/10.5093/rpadef2022a9 .

Balk YA, Englert C. Recovery self-regulation in sport: Theory, research, and practice. International Journal of Sports Science and Coaching. SAGE Publications Inc.; 2020. https://doi.org/10.1177/1747954119897528 .

King PR Jr, Beehler GP, Donnelly K, Funderburk JS, Wray LO. A practical guide to applying the Delphi Technique in Mental Health Treatment Adaptation: the example of enhanced problem-solving training (E-PST). Prof Psychol Res Pract. 2021;52(4):376–86. https://doi.org/10.1037/pro0000371 .

Glass G. Primary, secondary, and Meta-Analysis of Research. Educational Researcher. 1976;5(10):3. https://doi.org/10.3102/0013189X005010003 .

Gillham J, Seligman M. Footsteps on the road to a positive psychology. Behav Res Ther. 1999;37:163–73. https://doi.org/10.1016/s0005-7967( . 99)00055 – 8.

Castillo J. Salud mental en El Deporte individual: importancia de estrategias de afrontamiento eficaces. Fundación Universitaria Católica Lumen Gentium; 2021.

Cantón E. Deporte, salud, bienestar y calidad de vida. Cuad De Psicología Del Deporte. 2001;1(1):27–38.

Froment F, García-González A. Retos. 2017;33:3–9. https://doi.org/10.47197/retos.v0i33.50969 . Beneficios de la actividad física sobre la autoestima y la calidad de vida de personas mayores (Benefits of physical activity on self-esteem and quality of life of older people).

Jürgens I. Práctica deportiva y percepción de calidad de vida. Revista Int De Med Y Ciencias De La Actividad Física Y Del Deporte. 2006;6(22):62–74.

Carpintero H. (2004). Psicología, Comportamiento Y Salud. El Lugar De La Psicología en Los campos de conocimiento. Infocop Num Extr, 93–101.

Page M, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C, et al. Declaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas. Rev Esp Cardiol. 2001;74(9):790–9.

Royo M, Biblio-Guías. Revisiones sistemáticas: PRISMA 2020: guías oficiales para informar (redactar) una revisión sistemática. Universidad De Navarra. 2020. https://doi.org/10.1016/j.recesp.2021.06.016 .

Urrútia G, Bonfill X. PRISMA declaration: a proposal to improve the publication of systematic reviews and meta-analyses. Medicina Clínica. 2010;135(11):507–11. https://doi.org/10.1016/j.medcli.2010.01.015 .

Núñez A, Ponseti FX, Sesé A, Garcia-Mas A. Anxiety and perceived performance in athletes and musicians: revisiting Martens. Revista De Psicología. Del Deporte/Journal Sport Psychol. 2020;29(1):21–8.

Rascado S, Rial-Boubeta A, Folgar M, Fernández D. Niveles De rendimiento y factores psicológicos en deportistas en formación. Reflexiones para entender la exigencia psicológica del alto rendimiento. Revista Iberoamericana De Psicología Del Ejercicio Y El Deporte. 2014;9(2):373–92.

Reche-García C, Cepero M, Rojas F. Efecto De La Experiencia deportiva en las habilidades psicológicas de esgrimistas del ranking nacional español. Cuad De Psicología Del Deporte. 2010;10(2):33–42.

Kang C, Bennett G, Welty-Peachey J. Five dimensions of brand personality traits in sport. Sport Manage Rev. 2016;19(4):441–53. https://doi.org/10.1016/j.smr.2016.01.004 .

De Vries R. The main dimensions of Sport personality traits: a Lexical Approach. Front Psychol. 2020;23:11. https://doi.org/10.3389/fpsyg.2020.02211 .

Laborde S, Allen M, Katschak K, Mattonet K, Lachner N. Trait personality in sport and exercise psychology: a mapping review and research agenda. Int J Sport Exerc Psychol. 2020;18(6):701–16. https://doi.org/10.1080/1612197X.2019.1570536 .

Stamp E, Crust L, Swann C, Perry J, Clough P, Marchant D. Relationships between mental toughness and psychological wellbeing in undergraduate students. Pers Indiv Differ. 2015;75:170–4. https://doi.org/10.1016/j.paid.2014.11.038 .

Nicholls A, Polman R, Levy A, Backhouse S. Mental toughness, optimism, pessimism, and coping among athletes. Personality Individ Differences. 2008;44(5):1182–92. https://doi.org/10.1016/j.paid.2007.11.011 .

Weissensteiner JR, Abernethy B, Farrow D, Gross J. Distinguishing psychological characteristics of expert cricket batsmen. J Sci Med Sport. 2012;15(1):74–9. https://doi.org/10.1016/j.jsams.2011.07.003 .

García-Naveira A, Díaz-Morales J. Relationship between optimism/dispositional pessimism, performance and age in competitive soccer players. Revista Iberoamericana De Psicología Del Ejercicio Y El Deporte. 2010;5(1):45–59.

Reche C, Gómez-Díaz M, Martínez-Rodríguez A, Tutte V. Optimism as contribution to sports resilience. Revista Iberoamericana De Psicología Del Ejercicio Y El Deporte. 2018;13(1):131–6.

Lizmore MR, Dunn JGH, Causgrove Dunn J. Perfectionistic strivings, perfectionistic concerns, and reactions to poor personal performances among intercollegiate athletes. Psychol Sport Exerc. 2017;33:75–84. https://doi.org/10.1016/j.psychsport.2017.07.010 .

Mansell P. Stress mindset in athletes: investigating the relationships between beliefs, challenge and threat with psychological wellbeing. Psychol Sport Exerc. 2021;57:102020. https://doi.org/10.1016/j.psychsport.2021.102020 .

Reis N, Kowalski K, Mosewich A, Ferguson L. Exploring Self-Compassion and versions of masculinity in men athletes. J Sport Exerc Psychol. 2019;41(6):368–79. https://doi.org/10.1123/jsep.2019-0061 .

Cantón E, Checa I, Budzynska N, Canton E, Esquiva Iy, Budzynska N. (2013). Coping, optimism and satisfaction with life among Spanish and Polish football players: a preliminary study. Revista de Psicología del Deporte. 22(2), 337–43.

Mulvenna M, Adie J, Sage L, Wilson N, Howat D. Approach-achievement goals and motivational context on psycho-physiological functioning and performance among novice basketball players. Psychol Sport Exerc. 2020;51:101714. https://doi.org/10.1016/j.psychsport.2020.101714 .

Malinauskas R, Malinauskiene V. The mediation effect of Perceived Social support and perceived stress on the relationship between Emotional Intelligence and Psychological Wellbeing in male athletes. Jorunal Hum Kinetics. 2018;65(1):291–303. https://doi.org/10.2478/hukin-2018-0017 .

Scanlan T, Carpenter PJ, Simons J, Schmidt G, Keeler B. An introduction to the Sport Commitment Model. J Sport Exerc Psychol. 1993;1(1):1–15. https://doi.org/10.1123/jsep.15.1.1 .

Ryff CD. Eudaimonic well-being, inequality, and health: recent findings and future directions. Int Rev Econ. 2017;64(2):159–78. https://doi.org/10.1007/s12232-017-0277-4 .

Ryff CD, Singer B. The contours of positive human health. Psychol Inq. 1998;9(1):1–28. https://doi.org/10.1207/s15327965pli0901_1 .

Romero-Carrasco A, García-Mas A, Brustad RJ. Estado del arte, y perspectiva actual del concepto de bienestar psicológico en psicología del deporte. Revista Latinoam De Psicología. 2009;41(2):335–47.

James IA, Medea B, Harding M, Glover D, Carraça B. The use of self-compassion techniques in elite footballers: mistakes as opportunities to learn. Cogn Behav Therapist. 2022;15:e43. https://doi.org/10.1017/S1754470X22000411 .

Fernández-Río J, Cecchini JA, Méndez-Giménez A, Terrados N, García M. Understanding olympic champions and their achievement goal orientation, dominance and pursuit and motivational regulations: a case study. Psicothema. 2018;30(1):46–52. https://doi.org/10.7334/psicothema2017.302 .

Ortiz-Marholz P, Chirosa LJ, Martín I, Reigal R, García-Mas A. Compromiso Deportivo a través del clima motivacional creado por madre, padre y entrenador en jóvenes futbolistas. J Sport Psychol. 2016;25(2):245–52.

Ortiz-Marholz P, Gómez-López M, Martín I, Reigal R, García-Mas A, Chirosa LJ. Role played by the coach in the adolescent players’ commitment. Studia Physiol. 2016;58(3):184–98. https://doi.org/10.21909/sp.2016.03.716 .

Download references

This research received no external funding.

Author information

Authors and affiliations.

General Psychology Department, Valencia University, Valencia, 46010, Spain

David Peris-Delcampo & Enrique Cantón

Basic Psychology and Pedagogy Departments, Balearic Islands University, Palma de Mallorca, 07122, Spain

Antonio Núñez, Javier Ponseti & Alejandro Garcia-Mas

Education and Social Sciences Faculty, Andres Bello University, Santiago, 7550000, Chile

Paula Ortiz-Marholz

Personality, Evaluation and Psychological Treatment Deparment, Murcia University, Campus MareNostrum, Murcia, 30100, Spain

Aurelio Olmedilla

You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, AGM, EC and ANP.; planification, AO; methodology, ANP, AGM and PO.; software, ANP, DP and PO.; validation, ANP and PO.; formal analysis, DP, PO and ANP; investigation, DP, PO and ANP.; resources, DVP and JP; data curation, AO and DP.; writing—original draft preparation, ANP, DP and AGM; writing—review and editing, EC and JP.; visualization, ANP and PO.; supervision, AGM.; project administration, DP.; funding acquisition, DP and JP. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Antonio Núñez .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Informed consent statement

Consent for publication, competing interests.

The authors declare no conflict of interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Peris-Delcampo, D., Núñez, A., Ortiz-Marholz, P. et al. The bright side of sports: a systematic review on well-being, positive emotions and performance. BMC Psychol 12 , 284 (2024). https://doi.org/10.1186/s40359-024-01769-8

Download citation

Received : 04 October 2023

Accepted : 07 May 2024

Published : 21 May 2024

DOI : https://doi.org/10.1186/s40359-024-01769-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Positive emotions
Sports performance

BMC Psychology

ISSN: 2050-7283

General enquiries: [email protected]

Open access
Published: 14 May 2024

Health systems challenges, mitigation strategies and adaptations to maintain essential health services during the COVID-19 pandemic: learnings from the six geopolitical regions in Nigeria

Segun Bello 1 ,
Rachel Neill 2 ,
Ayodele S Jegede 1 ,
Eniola A. Bamgboye 1 ,
Mobolaji M. Salawu 1 ,
Rotimi Felix Afolabi 1 ,
Charles Nzelu 3 ,
Ngozi Azodo 3 ,
Anthony Adoghe 3 ,
Munirat Ogunlayi 2 ,
Saudatu Umma Yaradua 2 ,
William Wang 4 ,
Anne Liu 4 &
Olufunmilayo I. Fawole 1

BMC Health Services Research volume 24 , Article number: 625 ( 2024 ) Cite this article

260 Accesses

1 Altmetric

Metrics details

The COVID-19 pandemic control strategies disrupted the smooth delivery of essential health services (EHS) globally. Limited evidence exists on the health systems lens approach to analyzing the challenges encountered in maintaining EHS during the COVID-19 pandemic. This study aimed to identify the health system challenges encountered and document the mitigation strategies and adaptations made across geopolitical zones (GPZs) in Nigeria.

The national qualitative survey of key actors across the six GPZs in Nigeria involved ten states and the Federal Capital Territory (FCT) which were selected based on resilience, COVID-19 burden and security considerations. A pre-tested key informant guide was used to collect data on service utilization, changes in service utilization, reasons for changes in primary health centres’ (PHCs) service volumes, challenges experienced by health facilities in maintaining EHS, mitigation strategies implemented and adaptations to service delivery. Emerging sub-themes were categorized under the appropriate pillars of the health system.

A total of 22 respondents were interviewed. The challenges experienced in maintaining EHS cut across the pillars of the health systems including: Human resources shortage, shortages in the supply of personal protective equipments, fear of contracting COVID-19 among health workers misconception, ignorance, socio-cultural issues, lockdown/transportation and lack of equipment/waiting area (. The mitigation strategies included improved political will to fund health service projects, leading to improved accessibility, affordability, and supply of consumables. The health workforce was motivated by employing, redeploying, training, and incentivizing. Service delivery was reorganized by rescheduling appointments and prioritizing some EHS such as maternal and childcare. Sustainable systems adaptations included IPC and telehealth infrastructure, training and capacity building, virtual meetings and community groups set up for sensitization and engagement.

The mitigation strategies and adaptations implemented were important contributors to EHS recovery especially in the high resilience LGAs and have implications for future epidemic preparedness plans.

Peer Review reports

The COVID-19 pandemic remains the biggest global health systems shock of the 21st century leading to about 6.8 million deaths as of 26th February, 2023 [ 1 ]. The interventions implemented to control the pandemic have had far-reaching consequences, ranging from disruptions to socio-economic activities, to decline in health services provision and utilization. According to the World Health Organization (WHO), countries henceforth need to make trade-offs between the scale of direct response to health threats and the actions geared towards maintaining essential health service delivery, to mitigate the risk of system collapse [ 2 ].

Disruptions are defined as “unforeseen events that interfere with the provision of healthcare goods and services” [ 3 ]. During the COVID-19 pandemic, disruptions in health service delivery and decline in essential health services utilization was documented across all health systems including high, medium and low-income countries [ 4 ]. These disruptions were attributed to aspects of the COVID-19 pandemic response including lockdowns and reorganization of health service delivery with a shift in focus to COVID-19 control [ 2 ]. For example, in Europe, screening for cancers decreased by as much as 65 − 95% during the early phase of the pandemic [ 5 ]. In Africa, several health programmes including the malaria elimination programme, HIV/tuberculosis control, diabetes, and hypertension services were deprioritized during the pandemic [ 6 , 7 ]. Heavy declines were also reported for maternal, child health and immunization programmes [ 8 , 9 ]. effectively threatening the gains achieved in health programme outcomes over decades of investment [ 10 ]. These health programmes reported decline in service output as well as set- backs in performance indicators as similarly demonstrated during the West Africa Ebola outbreak pandemic. Analysis of the 2014–2015 Ebola outbreak suggested that the number of deaths caused by measles, malaria, HIV/AIDS and tuberculosis attributable to health systems failure during the Ebola outbreak exceeded deaths from Ebola [ 11 , 12 , 13 , 14 ].

The WHO health systems framework describes the core building blocks or pillars of the health systems which contribute to the resilience of a health system [ 15 ]. The performance of the health system in handling health crisis depends on its baseline capacity predating the crisis, as well as the magnitude of the crisis [ 2 ]. Kruk et al. defined health systems resilience as ‘the capacity of health actors, institutions, and populations to prepare for and effectively respond to crises; maintain core functions when a crisis hits; and, informed by lessons learned during the crisis, reorganize if conditions require it.’ [ 16 ] Thus, apart from maintaining the core functions of a health system, resilience includes the health system’s ability to transform, evolve and enhance its performance in improving the health of the population [ 17 ].

A well-prepared health system should have the capacity to maintain essential health services delivery to reduce morbidity and mortality from sources other than the cause of the health systems shock, throughout the duration of an emergency. Both demand and supply factors have been documented as challenges mitigating against the maintenance and utilization of essential services across health systems during the COVID-19 pandemic. The pandemic increased the workload for health systems, resulting in pressure and inadequate health workforce all over the world [ 5 ]. However, LMICs have been particularly affected from operating more vulnerable health systems with challenges that predated the COVID-19 pandemic. To compound the challenges of human resource shortages, about 50% of health facilities across Africa reported COVID-19 infection among staff, shortages in personal protective equipment (PPE), underfunding, reduced supply of medications and poor information systems [ 6 , 18 ]. Most African countries are dependent on importation of essential medicines and products. These countries were affected by the disruptions in the global supply chain because drugs were not readily available or were expensive because of the high demand relative to supply [ 19 ]. Patients expressed difficulties in accessing medicines due to the high cost [ 20 ].

Geographic variability in the level of disruptions and restorations to EHS were reported within countries [ 21 ]. The COVID-19 high burden states/areas were likely to have experienced a higher level of restrictions and enforcements of protocols which could affect the levels of disruptions and the time taken for restorations. Furthermore, recovery may be slow, temporary, or partial depending on sub-national health systems resilience. Reported innovative adaptations to halt or reverse decline in EHS delivery included home delivery, use of phones, improved triaging, shift to remote consultations, and expansion of the scope of work of community health workers and task shifting [ 21 , 22 ]. Limited evidence exists on the health systems lens approach to analyzing the challenges encountered in maintaining EHS delivery during the COVID-19 pandemic, particularly at the sub-national levels. Therefore, this study aimed to identify the health system challenges encountered during the COVID-19 pandemic and document the mitigation strategies and adaptations made across the geopolitical zones (GPZs) in Nigeria. The learnings will guide policymakers, decision makers and health administrators on how to improve health systems in Nigeria to ensure that they are resilient and prepared to respond to public health emergencies. Learnings from Nigeria especially on the mitigation and adaptation strategies may be transferrable to similar decentralized health systems.

Study setting

The study was qualitative in design involving interviews of key persons at state ministries of health (SMoH) and State Primary Health Care Development Agencies (SPHCDA) across the six geopolitical zones of Nigeria. Following the Alma Ata declaration in 1978, the primary health care (PHC) system became the fulcrum of health systems development in Nigeria. Not much progress was made in PHC however, until 1985 when the then Minister of Health adopted 52 Local Government Areas (LGAs) to build models based on the Alma Ata declaration [ 23 ]. Thereafter, the model was expanded to include all LGAs and the responsibility for overseeing the working of the PHC including immunization, antenatal care services was devolved to the LGAs [ 23 ]. PHC in Nigeria focuses on preventive services including immunization, antenatal care services, as well as the provision of basic health care services at the grass root level [ 23 ].

The Primary Health Care Under One Roof policy was introduced in 2010 and approved in 2011 [ 24 ]. It aims to strengthen the national health system by integrating all PHC services under one authority. By implications, all resources for PHC implementation are to be repositioned from all agencies, departments and ministries to the new State PHC development agencies or boards [ 24 ]. This initiative produced some improvements in health outcomes [ 25 ].

Like many other African countries, Nigeria has consistently failed to implement the 2001 Abuja declaration at which African heads of state pledged to allocate 15% of the annual national budget to health [ 26 , 27 ]. Currently, the PHC system has deteriorated with most of the 30,000 PHC facilities across the country lacking the capacity to provide essential healthcare services thereby, transferring enormous pressure to the higher levels of healthcare [ 28 ]. The challenges PHC facilities experienced before COVID-19 included poor staffing, inadequate equipment, poor distribution of health workers, poor quality of healthcare services, poor condition of infrastructure, and lack of essential drug supply.

Study design and approach

The study was part of a large national qualitative survey on resilience of the health system which aimed at identifying the key challenges to maintaining essential health services during the pandemic, from the perspective of subnational actors. This current report focused on the regional level data, but the other aspect of the study focused on comparative LGA-level data on how some LGAs overcame challenges and sustained essential health services, while comparable, neighbouring LGAs experienced ongoing disruptions [ 29 ].

Study site and participants’ selection

The study enrolled subnational actors at the state level, across the six geopolitical zones of Nigeria. These participants were engaged in the COVID-19 response and were at decision-making levels such as directors, assistant directors and heads of programmes.

Selection of study sites was guided by multiple criteria namely High Resilience (HR) LGAs; COVID 19 disease burden and regional hotspots such as LGAs with the highest cumulative cases and those with international airport or land borders; security considerations, by avoiding LGAs with considerable security challenges such as LGAs with insurgents and banditry. The procedure for identifying high resilience LGAs has been described in detail elsewhere [ 29 ]. In brief, the general outpatient (GOPD) and Ante-natal care (ANC) health services data from the National Health Management Information System (NHMIS) (January 2019 – December 2021) was analyzed using the interrupted time series. The analysis identified HR LGAs. HR LGAs were defined as LGAs which experienced a recovery in service volumes within three months of decline precipitated by the COVID-19 pandemic. LGAs were then stratified and ranked within each geopolitical zones and, in combination with COVID-19 burden and security considerations, 12 h LGAs were finally selected from 10 states and the Federal Capital Territory (FCT) across the six geopolitical zones: South-West [ 3 ], South-South [ 2 ], South-East [ 2 ], North-West [ 1 ], North-Central [ 2 ], North-East [ 1 ].

Participants for the parent study were selected purposively across state, LGA, health facilities, and community levels. However, this report is limited to the analysis of response from state-level participants across the GPZs where two participants each were selected per state.

Data collection

A key informant guide was developed following extensive review of literature on health systems resilience and essential health service maintenance (see Supplementary File). The guide was pretested among similar personnel in Nasarawa state before data collection. The interview guide was sectioned according to: profile of the study participants; services used during COVID-19; data monitoring and use; self-regulation; adaptive-short term; integrated capacities or planning; relevance to maternal neonatal and child health (MNCH); and adaptive-long term. The data presented in this report focuses on the following sections of the tool: services used during the COVID-19 pandemic and self-regulation which contained information on participants’ assessment of changes in service utilization during the COVID-19 pandemic; reasons for the changes in Primary Health Centres (PHCs) attendance, challenges experienced by facilities to maintain routine services during the pandemic, specific countermeasures that the state governments took to overcome the challenges and activities done by the state governments to encourage clients to continue to utilize the PHCs.

The Federal Ministry of Health (FMoH) led the project and played an oversight role in data collection with supervisors leading research teams to the states. The interview teams paid advocacy visits to explain the purpose of the research and obtain the support of stakeholders in the ministries of health. The interview team comprised of a supervisor, a moderator and a note taker per state. All data collectors and supervisors were trained for the purpose of this research. All participants gave informed consent before they were interviewed face-to-face and audio-recorded in their offices. A data collection pause was implemented after the first few interviews during which the interviews conducted were transcribed, reviewed and feedback were communicated to the field teams. The interview took an average of 73 min. Data was collected between June through July, 2022.

Data management

The recorded interview audios were transcribed verbatim in the original language of the interview. Transcripts were complemented with notes taken during the interviews. The transcripts, audio files and notes were labelled with unique identifiers that enabled data linkage across files. A data security protocol was implemented to safeguard against data breach. A Dropbox folder, which was only accessible to designated research team members, was created for the safe storage of the audio files, transcripts and summary notes.

Coding was done using Atlas.ti. One coder was involved in the coding the data while multiple coders coded subsets of the data for agreement. The entire research team interrogated the data and review the coding. Emerging sub-themes were categorized under the appropriate pillars of health system including (i) service delivery, (ii) health workforce, (iii) health information systems, (iv)medicines and supplies, (v) financing, (vi) leadership/governance [ 15 ].

Respondent socio-demographic characteristics

A total of 22 state-level participants were interviewed from 10 states and the Federal Capital Territory (FCT). Respondents’ age ranged from 40 to 60 years. The majority 18(82%) were male while the median total duration of employment was 23.5 years. The respondents held leadership positions in SMoH and SPHCDA, with many being Deputy/Acting Directors 6 (27%) and Directors 5 (23%) and commissioners for health 2 (9%) (Table 1 ). Most 20 (91%) had been in their current position for at least 2 years.

Health services delivery volumes at the PHC during the COVID-19 pandemic

All participants acknowledged reduction in patients’ attendance at the PHCs while some also mentioned interruption in health services delivery. The decrease in facility utilization was more pronounced during the early stage of the pandemic particularly from March 2020 through June/July, 2020. Notably, there was a drastic reduction in the antenatal clinic attendance by pregnant women and the under-5 children outpatient visits across all regions of the country.

The movement restriction during lockdowns and the fear of contracting COVID-19 were the two most prominent reasons stated for reduction in health facility patients’ attendance. Where facilities were still in operation, fear of contracting the virus among patient and health workers was common in all regions.

On the supply side, some health workers did not go to work, while some facilities were instructed to close completely. The lockdown reduced the number of health workers who were able to commute to work especially those who did not have personal means of transportation. Some state governments (such as Lagos) tried to ameliorate this by providing ambulances that took frontline workers to work. Health workers were also given stickers to identify them as essential workers so that the law enforcement agents would allow them to move through the lockdown. Like the patients, the health workers were also scared of contracting COVID-19 infection and they encouraged patients that could be managed at home to stay away from the health centers. They also referred patients very readily to the next level of health care with minimal investigations. The decline in services were attributed majorly to the COVID-19 pandemic.

On the demand side, patients were unable to travel to health facilities because of lockdown restrictions. Participants also emphasized on the economic challenges and bank closures which reduced people’s ability to purchase goods and services including healthcare. Community members exhibited fears from the belief that COVID-19 was domiciled in the health facilities. People were further afraid of being isolated in the event that they were diagnosed with COVID-19.

Differential impact of COVID-19 on LGAs

There was consensus between the participants from the different regions that urban areas had a higher burden of COVID-19 infection including disease incidence and case fatality. Consequently, there were more COVID-19 response activities in urban areas.

Participants in all regions believed that health service provision had returned to normal by June, 2022 especially for some suspended activities in the pandemic. Such activities included the integrated supportive supervision of health facilities which was believed to have returned to pre-COVID-19 levels. The isolation centers were no longer in existence and ad-hoc workers were no longer in employment. However, some COVID-19 prevention strategies such as the social mobilization, advocacy and risk communication were still on-going at the time of data collection.

Challenges faced in maintaining essential health services

Key challenges were identified by participants. Human resources shortage was the most commonly mentioned challenge from 4 GPZs, 6 states (Lagos, FCT, Imo, Kano, Abia, Ogun) of the country. Other commonly mentioned challenges included: Shortages in the supply of Personal Protective Equipments (PPEs) 4 GPZs, 5 states (Imo, Lagos, Ogun, FCT, Gombe); fear of contracting COVID-19 among health workers 4 GPZs, 4 states (Imo, Ogun, FCT, Rivers); misconception, ignorance, socio-cultural issues 2 GPZs, 2 states (Rivers, Imo); lockdown/transportation 2 GPZs, 2 states (Abia, Lagos); and lack of equipment/waiting area 2 GPZs, 2 states (FCT, Oyo). Less commonly mentioned challenges included: training gap, inadequate referral, diversion of other facility budget lines to PPEs purchase, and insecurity. The challenges considered to pre-date COVID-19 included: human resources shortage, shortages in equipment and PPEs, poor infrastructure and inadequate funding.

The challenges faced in maintaining essential health services in different health systems pillars are highlighted below with sample quotes from individual respondents (Table 2 ):

Leadership and governance

The respondent from the North Central (NC) zone explained that most of the resources allocated to various other activities in health facilities were redirected to meet the needs of COVID-19 response especially the provision of PPEs (Table 2 ):

…. the challenge of diversion… of resources [budget for other facility needs]… for PPEs.

From the Southwest (SW) zone, a respondent stated that insufficient funding had always been a challenge in carrying activities such as providing electricity in the PHCs. The challenge pre-dated COVID-19 pandemic.

“Insufficient funding has always been on ground. It is not really related to COVID-19. It has always been a case in most of the PHCs getting stipend to run the PHC like lightings, generators, pumping of water.” ( SW , )

Service delivery

The majority of the PHCs lacked infrastructure that could aid organization of services to provide physical distancing for the patients. A participant in the SW was quoted:

…majority of the health facilities do not have waiting area….

The health facilities experienced difficulty in transporting COVID-19 patients referred to isolation center for care. This was expressed by a participant in the NC zone:

“…referral, when somebody is positive having to evacuate from the hospital to the treatment center was a challenge” ( NC ) .

In the South-south (SS) and Southeast (SE) zone, the participants expressed concerns about patients’ misconceptions about COVID-19. Many patients did not believe that COVID-19 exist and as a result, were unwilling to adhere to facility COVID 19 prevention protocols. These misconceptions were reinforced by socio-cultural norms and reliance on dictates of religious leaders.

“.Misconceptions about the disease… with thoughts that there was no COVID-19 in the first place” ( SS ) .

“…the person [patient], and/or.relatives are not willing to adhere to the protocols, …what do you do?.socio-cultural issues,…where some people will say my pastor said…” ( SE ) .

Human resources

Inadequate human resource which predated the COVID-19 pandemic was expressed by both northern and southern zone respondents across six states. However, this challenge was amplified by the pandemic. There was limited number of personnel with the requisite skills to perform tasks related to the response. The task shifting strategy implemented to share task and thereby, reduce the number of health workers in facilities at any one time, also reduced the human resource capacity in the PHCs.

“The major challenge is… inadequate man power which has existed before COVID-19….” ( SW ) .

“Then during the pandemic too some health care workers absconded….health workers who had the requisite capacity were quite few” ( SE ) .

“….there was some sort of shifting done to reduce the number of health workers working at the same time…” ( NC ) .

Respondents in SW and SE also described the challenges that health workers encountered in getting to the health facility during the early period of the pandemic due to the lockdown. This was said to compound the human resource shortages.

“They [workers] find it a bit difficult to get to their work place some of them have to use their workplace as home …” ( SW ) .

“It included even the health service providers. They were locked down. They could not even access the facilities” ( SE) .

The human resources shortages in the facilities was confirmed to have been a long-standing problem that existed before the pandemic across all regions of the country which was now amplified by the pandemic.

“It [staff shortfalls] was on ground before …” ( SW ) .

“Yes, I said it that staff shortfall has been a long-term issue. The work is becoming voluminous everyday” ( SW ) .

“Of course, we have human resources gaps, before and even during the pandemic” ( NC ) .

“Well, I will say the issue of the human resource for health, it has been a long-lasting challenge even before the pandemic. So, it was now heightened by the pandemic…” ( SE ) .

Health workers’ attitude to work was stated as being a challenge to utilization of PHCs by clients. Due to the fear of contracting COVID-19, health workers were not committed to work.

“We had challenges with attitude to work you understand? Some people were more reluctant” ( SS ) .

“…health workers had a ground to be afraid because there were gaps [in] science” ( SE) .

“…even health workers were scared and they were not so committed to work because there was risk [of infection]” ( NC) .

Medicines and supplies

Respondents across most of the regions reported shortages in medical consumables such as PPEs, face masks and sanitizers especially at the beginning of the pandemic. One respondent decried challenges with the supply chain because of restricted access to PPEs even though some facilities had supplies locked up in the store.

“… it was so bad that some doctors will even use their money to buy sanitizers and face masks so as to protect themselves” (SW) .

“…dearth in supply of PPEs….but that was at the initial period. Before COVID − 19, there were no local manufacturers” (SE) .

“…when we started there was really a challenge in the facilities because even face masks were running out. Sanitizers were running out because of the increased use.” (NE) .

“…challenges about the supply chain in terms of internal access to the PPE. We put the PPE in the store and health workers in the emergency unit were not having access” (NC) .

“Rapid test for SARS-CoV-2 was not available at the beginning [of the pandemic]” as expressed by a respondent from the FCT (NC) .

The dearth in supply of consumables was confirmed to be a challenge that existed before the COVID-19 pandemic. However, the increase in the cost of some consumables such as PPE, gloves and face masks was a challenge that came with the COVID-19 pandemic.

“Dearth in supply of PPEs was actually a challenge that was in existence beforehand” ( SW) .

Mitigation strategies to health systems challenges during COVID-19 pandemic

Several interventions were implemented by state governments to address the challenges of maintaining essential health services (Table 3 ). State governments focused on the provision of consumables; recruitment, redeployment and provision of training for health workers; expansion of the infrastructural capacity; provision of vaccines, stipends, security and subsidizing health services costs. These interventions were in all regions of the country.

Political will improved during the COVID-19 pandemic, state governments were positively disposed to improving health services delivery.

“Government was ready to approve all the ongoing projects, all the ongoing services, basic medical services were being provided, they also were fighting stigma within the facilities” (NC) .

“They [government] made some services affordable, available and accessible and within the reach of the community member. They were taking services even to the community outside the facilities, services like outreach services, information dissemination and empowerment. ” (NC) .

Key interventions implemented across the regions were cascaded from state level to the LGA and facility levels down to the community. Across all regions, training and capacity building were stepped down to LGAs, facility heads and community. These activities were facilitated through LGA officers and community stakeholders.

“Health worker training was also done for health workers at the primary care centres and the secondary facilities at each of the area council. So, all the activities, all the IPC was also done.” ( NC ) .

“At the state, we have a state officer, we have the Local Government officers, we also have the health facility officers. These trainings were cascaded down from the State to the Local Government and to the health facilities to ensure that the various layers of response are well equipped in terms of capacity.” ( SE ) .

“We train and monitor. We also conduct supportive supervision from the state level down to the local government levels then to the ward and facility level; we do that routinely. We check their knowledge gap and also do on the spot training for whichever gap that we are able to identify.” ( SE ) .

“…there were trainings that we received, training upon training which usually comes from the national to the State and then we step it down to the local government and then from the local government to the wards within local governments and the facilities.” ( SW ) .

“We work with the medical officers of health in the twenty-three LGAs and the heads of facility to redistribute our staffs.” ( SS ) .

Coordination across levels of the health systems also ensured timely distribution of health facility materials:

“The moment the supply comes into the state with immediate effect they write to the MOHs (Medical office of health) in the local government stating we have some materials for you, because we do not wait until the MOHs come to collect the materials, so we send a letter to them via email communicating the delivery time. E.g. we are bringing it tomorrow morning or we are bringing it this evening be available to receive it. The moment it gets to the MOHs, the MOHs step it down to all the facilities and PHCs with immediate effect.” ( SW ) .

“The state primary health board makes funds and logistics available at the local government level” ( SW ) .

“The intervention trickles down to the facility level. The State made sure that the issue of man power, issue of adequacy of jobs you know and consumables at the health facility are addressed at the highest decision level” ( SW ) .

The COVID-19 response was supported by donor partners such as in the provision of PPEs. The government also mobilized funds from the private sector which was made available to the hospitals and PHCs.

“The state government provided PPEs, because there were also donations to them, many private sectors also donated and… they made it available for the public hospitals” ( SE) .

Regarding service delivery during the pandemic, interventions implemented included reorganizing service delivery for more facilities to render more services.

“We had to reorganize our system to ensure that more facilities in some strategic locations were rendering more services, had more people to render services, you understand, 24/7. We actually had to do that” ( SS) .

The government also built COVID-19 isolation and treatment centers to relieve the pressure on the hospitals and ensure COVID-19 patients had good care.

“They [state government] provided treatment centers for those who required admission,” ( NC) .

“Government-built isolation centers all across the 20 local government in Ogun state that is the jurisdiction.” ( SW ) .

In the Northwest (NW) zone, the government organized the Emergency Maternal and Child (EMC) services where they provided ambulances to pick up pregnant woman that required emergency surgery. Provision of ambulances was not limited to the NW region as other regions also mentioned government support by providing ambulances.

“For example, during COVID-19 pandemic people there had emergency cesarean sections especially pregnant women. There is an ambulance that picks them and there is also another one that is called EMC services, it is a special service provided by the State government for Maternal and newborn child free up to this moment” (NW) .

Intervention strategies in facilities also included prioritization of facilities in terms of services and staffing needs, rescheduling of patients’ appointment that were not emergency cases.

Clients were also redirected from facilities that were shutdown to nearby facilities that could provide treatment services. Services prioritized included patient monitoring/treatment, immunization services and provision of ambulance for transportation.

“Well, the patient monitoring evaluation and treatment were prioritized because we do not want to come down with a lot of mortality. So adequate equipment [and] consumables were provided by the State and the manpower involved were adequately remunerated and then the State paid a lot of money for them to maintain this service” ( SE ) .

“The services like maternal and child care…. those services are key. We want to make sure that mothers, pregnant mothers access care on time, the children too… Those that need to be immunized and all of that.” ( SW ) .

“The maternal, new born and child health services were prioritized and also the health workers themselves were prioritized because they are the frontliners” ( SE ) .

“…anybody that falls sick and gets to the hospital will receive care but we pay attention on pregnant women and little babies more because their own case is peculiar” ( SW ) .

“The mother too who attended antenatal clinic and even the test that will be run everything was done for free and was sponsored bby the PHC Board to the extent that they printed cards and gave it to them for free that they were not supposed to pay. The registration, everything was made free at that time. This is just to act as reliefs at that time for those who access health at the health facilities” ( SW ) .

To address the shortfall in human resources, the SW region employed health worker cadres such as doctors and nurses in batches per time, as the budget could accommodate. In some other regions such as the SE, health workers were redeployed to work at facilities which were near where they lived to improve delivery. Ad hoc staff were also engaged to work for a few months.

Workshops were organized by the state governments to train and inform the health workers on IPC and to improve their skills. This helped to alleviate their fears on contracting the virus so as to alleviate their apprehension.

Health workers including adhoc staff were motivated by increasing the hazard allowance, which led to the increment in their monthly salary.

“They [government] gave some allowances to adhoc workers for a few months. So those adhoc workers helped….The state government also provided ambulances, one ambulance to one local government. They gave ambulances and drivers…also provided security…” ( SE ) .

“Health care workers were also provided with the relief materials to also help them continue in their work” ( SE ) .

“Increasing the health workers hazard allowance is something that the government did….” ( SW ) .

“Yes, the government provided allowances to encourage those who were at the frontline to ensure that they [health workers] at least had something reasonable to hold on to while offering their services and apart from that government was coordinating the activities of the various fronts including that of security.” ( SE ) .

“Giving reliefs, packages, and giving us bonus that was all.” ( SW ) .

“Those that took part in surveillance were given certain stipends, those that did case management were given certain stipends, those that took part IPC, risk communication, point of entry was given certain stipends.” ( SS ) .

“Governor continued, was even giving transport stipends to surveillance officers, laboratory personnel, just to encourage them to do the work and so, these things were going on as a kind of stimulant, a kind of motivation to assist in getting the job done. So as at that period those things were not lacking for us, so that is what I can say about that .” ( SS ) .

“The support is the trainings that were done, stipends were paid adequately as at when due and the health workers were happy with that, as they carried out their duties” ( SE ) .

“The hazard allowance was increased, I think to about 15% or thereabout, so all those incentives were there for health workers to actually motivate them to do more, so the State government did that.” ( SW ) .

“Palliatives, all the health workers were given palliatives.” ( SW ) .

“All the health workers were given adequate and reasonable support; number one, in the FCT, they were well paid. Those that were directly involved [in COVID-19 control] were well paid by the honorable minister of the FCT, secondly, they were all provided at any given point in time with PPEs, they were also well trained to monitor patient, and even the family of those who died were given some support, I think some were promised land, I don’t know if they have given them. They were given high level of support.” ( NC ) .

Other support granted by the State to motivate health workers included training, recruitment to support existing staff, provision of security, relief packages and ambulances.

“I know I have talked about redistribution of workers, of course ad-hoc workers for those very few months, then some of the PPEs and some of the security, I think that’s the only thing I can say.” ( SE ) .

“To be sincere we have to appreciate the state government, at that time they even gave us accommodations, food and everything during the first pandemic. They support us with training of case management for us to take care of patients as well as series of other training. We all attended online training on oxygen therapy and it was even paid for” ( NW ) .

“At one point, it was difficult for health workers to move from one point to the other, so government aided the movement of health workers by providing certain things to identify them, also providing ambulances, movement support to enable them move from their homes. They also provided accommodation for health workers at the isolation center.” ( SE ) .

“And also, they bring in special teams to also support the teams on ground.” ( SE ) .

“Well, we did some form of reorganization and that did include the personnel. So, we had to increase the number of personnel in our focal facilities which increase the services” ( SS ) .

“Yes training has always been in existence so they do refresher training but during the COVID it become more intensified because of the session or season we are.” ( SW ) .

Information systems

Respondents mentioned that government engaged in communication/sensitization programmes to improve service utilization using different media including the traditional and social media. The targets of the communication programmes were the community members including religious and ethnic groups. Communities, markets, churches and mosques were some of the places where the health promotion campaigns took place (Table 3 ).

Adaptations of the health systems during COVID-19

Sustainable adaptations.

Table 4 shows the emerging themes on sustainable adaptations done by the health systems. Respondents considered the infection, prevention and control (IPC) infrastructure (taps for running water), the telehealth call center, the IPC protocols and the service reorganization, as sustainable. A respondent mentioned that each health facility had an IPC focal person and also IPC teams which the health system can continually optimize.

Respondents considered that the training programs and capacity building efforts (especially the ‘network electronic platform’), implemented during the pandemic were sustainable. They opined that IPC training should be mainstreamed because the topic was broad and had impact on prevention of other infectious disease areas apart from COVID-19.

Respondents also mentioned that the volunteer groups formed during the pandemic for community sensitization and community engagement, were retained and would be used for other intervention programmes. Health teams have also retained the virtual mode of conducting team meetings.

Unsustainable adaptations

Respondents considered some adaptations in financing, service delivery and supplies, as unsustainable (Table 5 ). The funds that the government mobilized in form of incentives to health workers, stipends for campaigns team members and payment for other ad hoc staff such as town criers, were no longer being provided. The free testing and healthcare for COVID-19 patients which governments implemented was not sustained. The health workers who were redeployed have returned to their pre-pandemic assignments. In addition, all the services rendered to patients at the COVID-19 treatment centers including treatment, accommodation, consumables, were free and therefore, considered unsustainable. This also included the free consumables supplied to the health workers.

Summary of findings

The qualitative study selected senior persons in decision-making positions. Respondents acknowledged a reduction in patients’ attendance at the PHCs and interruption in service delivery. This prominently affected antenatal care attendance by pregnant women and the care for the under-5 children across all regions in the country. There was consensus among the regions that the urban communities had a higher burden of COVID-19 infection making the activities around COVID-19 control more intense in these communities. Unfortunately, this negatively impacted the provision of care in health facilities in these communities, leading to a negative impact on provision of EHS.

The challenges experienced in maintaining essential health services cut across the pillars of the health systems. Resources were reallocated to COVID-19 control activities from other budgetary lines due to insufficient funds to implement control activities. The infrastructure of most of the PHCs could not accommodate changes in service reorganization which was needed to enable physical distancing. It was also challenging to transport referred COVID-19 patients to isolation centers. Patients had misconceptions on the cause and transmission of COVID-19 and were unwilling to adhere to facility protocols. There was severe shortage of human resources which predated and was accentuated by COVID-19 control interventions such as lockdowns, staff redeployment and task shifting. Health workers were reluctant to discharge their duties because of fear of contracting the infection. There was inadequate consumables for use albeit sometimes due to deficient supply chain management.

Several mitigation strategies were implemented to address the challenges encountered. Political will towards improvement of health service projects was increased during the COVID-19 pandemic. This was reflected in government efforts to make health services available, accessible and affordable. Efforts were also made to provide consumables, recruit both permanent and ad-hoc staff, motivate existing health workforce, and redeploy/train health workers. The health infrastructure capacity was also expanded across regions, to free up spaces for provision of EHS by building/renovating COVID-19 isolation and treatment centers. Service delivery was also reorganized by rescheduling appointment for non-emergency to a later date and prioritizing essential services such as immunization, maternal and child care. Health promotion campaigns to groups and communities, were conducted to improve service patronage. Sustainable systems adaptations included IPC and telehealth infrastructure, IPC protocols, IPC teams and focal persons, training and capacity building, virtual meetings and community groups set up for sensitization and engagement. Unsustainable adaptations included funding, free healthcare and consumables, redistribution of staff, and the maintenance of COVID-19 treatment centers.

Results in the context of the literature

The COVID-19 pandemic disrupted EHS in almost all countries of the world and the disruption continued for over two years in more than 90% of countries surveyed by the WHO [ 30 ]. Particularly affected were the maternal and childcare services as corroborated in both quantitative and mixed methods design studies [ 31 , 32 , 33 ]. Our study corroborated findings from surveys among health workers and community members in Burkina Faso, Ethiopia and Nigeria, confirmed partial-to-total interruptions in health services delivery and utilization especially maternal and child health services [ 34 ] due to lockdowns, fear of infection/stigmatization, misconceptions/misinformation about the disease, stockout of drugs, and lack of transportation due to lockdowns [ 35 , 36 ]. As noted in this study, the disruption affected most services to the extent that some PHCs with low capacity were closed down. Studies indicated that disruptions appeared to affect disproportionately maternal and child care including immunization [ 30 ]. As noted in the WHO survey and as corroborated by our study, the major barriers to health service recovery were health systems challenges which predated the COVID-19 pandemic. Very prominent pre-existing health systems deficiencies identified by our study were in the human resources, service delivery and the finance pillars.

The adaptations to service delivery implemented in healthcare facilities were similar across regions in Nigeria and notably, were designed to reduce patient inflow. Non-emergency cases were discouraged from accessing clinics and follow-up appointments were rescheduled because the facilities lacked the capacity to implement the recommended physical distancing between patients. In Ghana [ 37 ] similar adaptations were made to routine healthcare service delivery which also aimed at reducing patient flow to the health facilities. In this study, only clients with extremely important conditions were encouraged to visit the health facilities, appointments were reduced, non-essential medical and surgical procedures were less prioritized.

Although, facility closures occurred in most settings around the world during lockdowns because there was no health manpower to provide services [ 3 ], the telemedicine infrastructure which existed before the pandemic in some settings, were deployed to bridge the gap in consultation demands [ 3 , 38 ]. Nigeria developed a telehealth call center which was mainly for COVID-19 case finding but provides opportunities for general health consultations use.

Also, some health professionals were reassigned to COVID-19 control programmes which ultimately affected services such as home visits, immunization and other community health services [ 37 ]. A study conducted in Lagos, Nigeria highlighted the willingness of community health workers to function as care providers during the pandemic but were challenged by heavy workload and lack of transportation [ 39 ]. These recommendations informed some of the decisions to improve health workforce care packages including financial incentives and employment of additional staff [ 39 ].

Limited evidence exist in the literature on the challenges encountered in maintaining EHS in health systems. In Bangladesh, similar challenges were reported as we found in our study. The demand pull challenges in Bangladesh included fear of COVID-19 infection, difficulty with commuting during lockdown and reduction in health seeking behavior emanating from closure of health facilities without providing alternatives [ 40 ]. Also, as found in our study, health resources were redirected to COVID-19 leaving other important health programmes deprived. Likewise, there were staff shortages which predated COVID-19: Acting in synergy with panic among health workers, more health facilities and programmes were further abandoned as similarly documented in our study.

The literature was richer in terms of mitigation and adaptation strategies implemented to maintain EHS during the COVID-19 pandemic. Kabwama et al., used the same health systems pillar thematic framework to analyze the interventions implemented in maintaining EHS in Uganda [ 41 ]. Prominent in the Uganda analysis was the private sector engagement for public-private partnership in fund mobilization as reported in our Nigeria analysis. Unique adaptation in service provisions in Uganda involved leveraging patient networks to deliver medicine which was not found in our analysis. The Ugandan study appeared to focus more on general interventions that were not specifically directed at challenges in maintaining EHS contrary to what our study did. The mitigation strategies implemented in Bangladesh closely mirrored what our study found such as provision of consumables under the medicines and supply pillar, fund mobilization under the leadership/governance and finances pillars among others [ 40 ].

Perhaps, the most robust survey on service adaptations involved 129 countries and was conducted by the WHO [ 30 ]. It was clear that in all countries, services were shifted off the health facilities and moved to home-based or to tele-infrastructure. Low and middle income countries like Nigeria may benefit from such easily adaptable strategies because creating separate facilities for COVID-19 and EHS delayed implementation as a result of the considerable financial investment required. Policy makers involved in emergency and epidemic preparedness plans may incorporate proactive plans to achieve rapid implementation of similar strategies. Other prominent cross-cutting mitigation strategies reported across countries in the WHO survey included healthcare financing, health workforce training and capacity building, procuring of essential medicines and consumables, risk communications and community engagement.

Implication of findings and lessons learned

The WHO recommends that advanced planning and long-term investments in health systems is important for epidemic preparedness and in safeguarding the continued provision of EHS during a health crisis [ 42 ]. Findings derived from this study are imperative for a robust epidemic preparedness plan. Strategies to maintain supply and demand for EHS should be incorporated as essential elements of epidemic preparedness plans. Response to health crisis require a more holistic and proactive approach at planning. The challenges facing the Nigerian health system are long-term which will require considerable and consistent efforts to resolve. Thus, learnings on mitigation strategies and adaptations during the COVID-19 pandemic would be applicable for future public health emergencies as well as routine health services delivery. The sustainable adaptations can potentially serve as a foundation for a gradual, planned, and intentional investments in the core functions of the Nigerian health system in order to improve its resilience and preparedness. For example, maintaining a pool of potential ad hoc volunteers consisting of retired health workers and community volunteers who can be mobilized at short notice. Also, the partnership built during the COVID-19 pandemic between the government of Nigeria and the private sector could be strengthened and optimized for epidemic preparedness and EHS delivery. The government at all levels received funds and donations from the private sector which was channelled to COVID-19 control and health care service delivery.

Our study also highlights the importance of adequate and timely public health messaging. Misconceptions and misinformation were rife during the COVID-19 pandemic in Nigeria [ 23 ]. Also noted [ 23 ], most of the information provided were technical and focused on prevention of COVID-19, with only minimal messaging on the provision/utilization of EHS. Thus, on the social media, misconceptions festered and was a major cause of demand-pull decline in EHS utilization by communities. Both patients in need of treatment and those who were on follow-up appointments, largely stayed away from the health facilities due to fear of contracting COVID-19. Health facilities were stigmatized, and health providers discriminated against for fear of contracting the virus. Another driver of decline in demand was the fear of testing positive and being isolated [ 43 ]. Although, adaptations to EHS later reduced the need for physical contact with the health facilities, a large proportion of potential clients stayed away from the formal health system. The learnings derived from adaptations during the pandemic could provide opportunities for a transformative evolution of the primary health care system in Nigeria. Before the pandemic, across the country, only about 20% of the PHCs were assessed as functional [ 28 ], resulting in consultation overload of the secondary and tertiary facilities. The participants considered the telehealth call center to be a sustainable innovation. The Nigerian health system could benefit from upgrading and expanding telemedicine infrastructure to shift some of the PHC overload to this platform. This will enhance an elastic, epidemic prepared EHS delivery system.

As confirmed in this study, poor funding was a systemic challenge that predated COVID-19 pandemic. EHS delivery suffered major set-backs partly because the meagre financial resources available for healthcare delivery were diverted to COVID-19 control. The budget for the State PHC Board in a state in North Central zone of Nigeria was reduced by 11.5% in order to secure funds for COVID-19 control activities [ 44 ]. The government was able to raise some funds mainly from the private sector most of which was deployed towards public health measures for COVID-19 control with little investment to strengthen the health system [ 45 ]. Public-private partnership could be strengthened to form an extra-budgetary sovereign wealth fund which will be used for emergency health purpose only and which can be mobilized at short notice. The state governments demonstrated commitment to long-term public health investments and reforms during, and in the immediate post-pandemic period [ 46 ]. A sustained commitment will improve the overall performance of primary healthcare in Nigeria in the near future.

Strengths and limitations of the study

The strength of this study is that participants were actors at the sub-national (state) -level. They were senior personnel who were decision makers in COVID-19 control and provision of EHS. They had good knowledge of activities that transpired in the states during the COVID-19 pandemic. Also, we sampled participants from all geopolitical zones of Nigeria in the interviews, which ensured representativeness. We translated and back-translated tools across zones to ensure accuracy.

The tool was designed using the conceptual framework developed by Kruk et al. [ 16 ], which was not initially based on the health systems pillars. It is possible that data on some health systems pillars exist which were not captured during the interviews. Conceptual framework used in the Kruk’s framework are not strictly health systems pillars or building blocks. Our study recruited mainly senior personnel in the ministries which might skew observations without the views of the junior personnel. Readers should interpret the findings with the view that potential richer health systems context may exist.

This study showed that there were significant challenges in maintaining essential health services delivery and utilization during the COVID-19 pandemic in Nigeria. The maternal and child care services were particularly affected. The core health systems challenges which prevented the maintenance of EHS delivery were mainly in the human resources, service delivery and the financing pillars. The mitigation strategies and adaptations implemented were important contributors to EHS recovery especially in the high resilience LGAs and have implications for future epidemic preparedness plans.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Acknowledgements

The authors appreciate all participants who invested their time in responding to the interviews, colleagues at the FMoH who guided the smooth execution of this work and Hanovia Limited colleagues for the implementation of the qualitative data collection and transcription. Mohammad Tawab Hashemi provided additional support from the Global Financing Facility for Women, Children, and Adolescents.

Funding for this study was provided by Gates Ventures and the Global Financing Facility for Women, Children, and Adolescents. The funders of this study were involved in the study design, data collection, interpretation, and reporting. The views, opinions, and content of this publication are those of the authors and do not necessarily reflect the views, opinions, or policies of the funders.

Author information

Authors and affiliations.

University of Ibadan, Ibadan, Nigeria

Segun Bello, Ayodele S Jegede, Eniola A. Bamgboye, Mobolaji M. Salawu, Rotimi Felix Afolabi & Olufunmilayo I. Fawole

The Global Financing Facility for Women, Children, and Adolescents, 1818 H ST NW, Washington, DC, 204333, USA

Rachel Neill, Munirat Ogunlayi & Saudatu Umma Yaradua

Nigeria Federal Ministry of Health, Federal Secretariat Complex, Phase III, Shehu Shagari Way, Central Business District, Abuja, Nigeria

Charles Nzelu, Ngozi Azodo & Anthony Adoghe

Gate Ventures, Seattle, Washington, USA

William Wang & Anne Liu

You can also search for this author in PubMed Google Scholar

Contributions

SB, RN, ASJ, EAB, MMS, RFA, CN, NA, AA, MO, SUY, WW, AL, & OIF were involved in the conceptualization and/or design of this study. RN, SB, ASJ, NA, CN, AA, MO, SUY, & OIF were involved in data collection and analysis. SB and OIF developed the first draft of the manuscript. SB, RN, ASJ, EAB, MMS, RFA, CN, NA, AA, MO, SUY, WW, AL, & OIF reviewed and revised the manuscript. All coauthors have approved the manuscript for publication.

Corresponding author

Correspondence to Segun Bello .

Ethics declarations

Ethics approval and consent to participate.

The study was conducted in accordance with the Declaration of Helsinki. Written informed consent was obtained from the participants. Ethical clearance was obtained from National Health Research Ethics Committee NHREC/01/01/2007.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Bello, S., Neill, R., Jegede, A.S. et al. Health systems challenges, mitigation strategies and adaptations to maintain essential health services during the COVID-19 pandemic: learnings from the six geopolitical regions in Nigeria. BMC Health Serv Res 24 , 625 (2024). https://doi.org/10.1186/s12913-024-11072-2

Download citation

Received : 09 September 2023

Accepted : 03 May 2024

Published : 14 May 2024

DOI : https://doi.org/10.1186/s12913-024-11072-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Health systems resilience
Essential health services

BMC Health Services Research

ISSN: 1472-6963

General enquiries: [email protected]

2024 Theses Doctoral

Artificial Intelligence vs. Human Coaches: A Mixed Methods Randomized Controlled Experiment on Client Experiences and Outcomes

Barger, Amber

The rise of artificial intelligence (AI) challenges us to explore whether human-to-human relationships can extend to AI, potentially reshaping the future of coaching. The purpose of this study was to examine client perceptions of being coached by a simulated AI coach, who was embodied as a vocally conversational live-motion avatar, compared to client perceptions of a human coach. It explored if and how client ratings of coaching process measures and outcome measures aligned between the two coach treatments. In this mixed methods randomized controlled trial (RCT), 81 graduate students enrolled in the study and identified a personally relevant goal to pursue. The study deployed an alternative-treatments between-subjects design, with one-third of participants receiving coaching from simulated AI coaches, another third engaging with seasoned human coaches, and the rest forming the control group. Both treatment groups had one 60-minute session guided by the CLEAR (contract, listen, explore, action, review) coaching model to support each person to gain clarity about their goal and identify specific behaviors that could help each make progress towards their goal. Quantitative data were captured through three surveys and qualitative input was captured through open-ended survey questions and 27 debrief interviews. The study utilized a Wizard of Oz technique from human-computer interaction research, ingeniously designed to sidestep the rapid obsolescence of technology by simulating an advanced AI coaching experience where participants unknowingly interacted with professional human coaches, enabling the assessment of responses to AI coaching in the absence of fully developed autonomous AI systems. The aim was to glean insights into client reactions to a future, fully autonomous AI with the expert capabilities of a human coach. Contrary to expectations from previous literature, participants did not rate professional human coaches higher than simulated AI coaches in terms of working alliance, session value, or outcomes, which included self-rated competence and goal achievement. In fact, both coached groups made significant progress compared to the control group, with participants convincingly engaging with their respective coaches, as confirmed by a novel believability index. The findings challenge prevailing assumptions about human uniqueness in relation to technology. The rapid advancement of AI suggests a revolutionary shift in coaching, where AI could take on a central and surprisingly effective role, redefining what we thought only human coaches could do and reshaping their role in the age of AI.

Adult education
Artificial intelligence--Educational applications
Graduate students
Educational technology--Evaluation
Education, Higher--Technological innovations
Education, Higher--Effect of technological innovations on

This item is currently under embargo. It will be available starting 2029-05-14.

More About This Work

DOI Copy DOI to clipboard

IMAGES

(PDF) Document Analysis as a Qualitative Research Method
[PDF] Conducting a Qualitative Document Analysis
[PDF] Document Analysis as a Qualitative Research Method
Understanding Qualitative Research: An In-Depth Study Guide
Types Of Qualitative Research Design With Examples
Qualitative Research

VIDEO

Qualitative Research Reporting Standards: How are qualitative articles different from quantitative?
Understanding Quantitative and Qualitative Research Method
Qualitative Research Tools
Document Analysis (የመረጃ ክምችትን ፍተሻ/ምርምራ በማካህድ መረጃን የመሰብሰብ ዘዴ)
Introduction to documentary research
How To do UX Research Using AI. #uiuxdesign #uiuxdesigner #productdesign #uxresearch

COMMENTS

Document Analysis as a Qualitative Research Method
Document analysis is a research method that involves collecting, reviewing, and analyzing written materials such as policies, laws, regulations, reports, memos, and minutes to answer research ...
Document Analysis
As a qualitative method, document analysis is defined as a systematic procedure for reviewing and evaluating documents that entails finding, selecting, appraising (making sense of), and synthesizing data contained within them (Bowen, 2009).Documents are more complex than just being content containers; they are social products of collective, organized action (Prior, 2003).
How to use and assess qualitative research methods
Abstract. This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions ...
Document Analysis as a Qualitative Research Method
This article examines the function of documents as a data source in qualitative research and discusses document analysis procedure in the context of actual research experiences. Targeted to research novices, the article takes a nuts‐and‐bolts approach to document analysis. It describes the nature and forms of documents, outlines the ...
PDF Qualitative Research Journal
In relation to other qualitative research methods, document analysis has both advantages and limitations. Let us look first at the advantages. Efficient method: Document analysis is less time-consuming and therefore more efficient than other research methods. It requires data selection, instead of data collection.
"Conducting a Qualitative Document Analysis" by Hani Morgan
Document analysis has been an underused approach to qualitative research. This approach can be valuable for various reasons. When used to analyze pre-existing texts, this method allows researchers to conduct studies they might otherwise not be able to complete. Some researchers may not have the resources or time needed to do field research. Although videoconferencing technology and other types ...
Document analysis in health policy research: the READ approach
Document analysis (also called document review) is one of the most commonly used methods in health policy research; it is nearly impossible to conduct policy research without it. Writing in early 20th century, Weber (2015) identified the importance of formal, written documents as a key characteristic of the bureaucracies by which modern ...
PDF Evaluation Briefs No 18
This brief describes document review as a data collection method for evaluation. It includes a basic overview of document review; when to use it; how to plan and conduct it; and its advantages and disadvantages. ... Engaging in Qualitative Research Methods: Opportunities for Prevention and Health Promotion. LeConté J. Dill, DrPH, MPH.
Conducting a Qualitative Document Analysis
Document analysis is a valuable research method that has frequently been an underused approach to qualitative research (Morgan 2022). For our study, we used a systematic approach for document ...
(PDF) Learning about Qualitative Document Analysis
Document review is expressed as a meticulous and systematic examination of the content of all documents, including printed and electronic materials (Wach, 2013). Document review carried out in ...
Conducting a Qualitative Document Analysis
document analysis, qualitative inquiry, reflexive thematic analysis. Introduction . Document analysis is a valuable research method that has been used for many years. This method consists of analyzing various types of documents including books, newspaper articles, academic journal articles, and institutional reports. Any document containing ...
The Basics of Document Analysis
Published: Dec. 12, 2023. Document analysis is the process of reviewing or evaluating documents both printed and electronic in a methodical manner. The document analysis method, like many other qualitative research methods, involves examining and interpreting data to uncover meaning, gain understanding, and come to a conclusion.
Document Review as a Qualitative Research Data Collection Method for
This case study is an introduction to the use of a document checklist as part of a qualitative research method of document review. Details are provided on the types of documents that can be utilized in a document review; the advantages and disadvantages of using document review as a research data collection method; description of how to design, implement, analyze, and present documents used to ...
Qualitative document analysis
Guest post by Professional MAXQDA Trainer Dr. Daniel Rasch.. Introduction. Qualitative text or document analysis has evolved into one of the most used qualitative methods across several disciplines (Kuckartz, 2014 & Mayring, 2010).Its straightforward structure and procedure enable the researcher to adapt the method to his or her special case - nearly to every need.
Document Review as a Qualitative Research Data Collection Method for
DOI: 10.4135/9781473957435 Corpus ID: 63132407; Document Review as a Qualitative Research Data Collection Method for Teacher Research @inproceedings{Bretschneider2017DocumentRA, title={Document Review as a Qualitative Research Data Collection Method for Teacher Research}, author={Pamela J. Bretschneider and Stefanie Cirilli and Tracey Jones and Shannon Lynch and Natalie Wilson}, year={2017 ...
Literature review as a research methodology: An ...
As mentioned previously, there are a number of existing guidelines for literature reviews. Depending on the methodology needed to achieve the purpose of the review, all types can be helpful and appropriate to reach a specific goal (for examples, please see Table 1).These approaches can be qualitative, quantitative, or have a mixed design depending on the phase of the review.
SAGE Research Methods: Find resources to answer your research methods
<button>Click to continue</button>
Developing a Feasible and Credible Method for Analyzing Healthcare
Document analysis is a topical method used in health and nursing sciences. Written, audio, and visual healthcare documents are constantly being produced (Bowen, 2009; Coffey, 2014; Gibson & Brown, 2011) and the number of documents is increasing (Olivares Bøgeskov & Grimshaw-Aagaard, 2019), because of wider healthcare regulations and the need to evaluate the effectiveness of care and services.
Increasing rigor and reducing bias in qualitative research: A document
Qualitative research methods have traditionally been criticised for lacking rigor, and impressionistic and biased results. ... Increasing rigor and reducing bias in qualitative research: A document analysis of parliamentary debates using applied thematic analysis ... Cambron C (2013) Examining foundations of qualitative research: A review of ...
What Is Qualitative Research?
Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research. Qualitative research is the opposite of quantitative research, which involves collecting and ...
Document analysis in health policy research: the READ approach
Document analysis (also called document review) is one of the most commonly used methods in health policy research; it is nearly impossible to conduct policy research without it. Writing in early 20th century, Weber (2015) identified the importance of formal, written documents as a key characteristic of the bureaucracies by which modern ...
Planning Qualitative Research: Design and Decision Making for New
While many books and articles guide various qualitative research methods and analyses, there is currently no concise resource that explains and differentiates among the most common qualitative approaches. We believe novice qualitative researchers, students planning the design of a qualitative study or taking an introductory qualitative research course, and faculty teaching such courses can ...
Data collection methods for evaluation: Document review
Data collection methods for evaluation - document review (PDF, 162KB) This resource from the Centers for Disease Control and Prevention (CDC) provides a brief guide to using document review as a data collection method for evaluation. This guide provides an overview of when to use document review, how to plan and conduct it, and its advantages ...
Integrating qualitative research within a clinical trials unit
The value of using qualitative methods within clinical trials is widely recognised. How qualitative research is integrated within trials units to achieve this is less clear. This paper describes the process through which qualitative research has been integrated within Cardiff University's Centre for Trials Research (CTR) in Wales, UK. We highlight facilitators of, and challenges to, integration.
Qualitative Research: Definition, Methodology, Limitation, Examples
Qualitative research is a method focused on understanding human behavior and experiences through non-numerical data. Examples of qualitative research include: One-on-one interviews, Focus groups, Ethnographic research, Case studies, Record keeping, Qualitative observations. In this article, we'll provide tips and tricks on how to use ...
The bright side of sports: a systematic review on well-being, positive
The keywords will be decided by a Delphi Method in two rounds with sport psychology experts. There are no participants in the present research. The main exclusion criteria were: Non-sport thema, sample younger or older than 20-65 years old, qualitative or other methodology studies, COVID-related, journals not exclusively about Psychology.
Health systems challenges, mitigation strategies and adaptations to
The COVID-19 pandemic control strategies disrupted the smooth delivery of essential health services (EHS) globally. Limited evidence exists on the health systems lens approach to analyzing the challenges encountered in maintaining EHS during the COVID-19 pandemic. This study aimed to identify the health system challenges encountered and document the mitigation strategies and adaptations made ...
Young widowhood: A qualitative study of sexuality after partner loss
View PDF View EPUB. Partner loss deprives young widows of physical contact, emotional intimacy, and the fulfillment of sexual desire. Although disenfranchised and oppressed, sexuality is a core piece of women's identity, and sexual bereavement may compel widows to reconstruct their sexual identities. This existential phenomenological study ...
301 Moved Permanently
301 Moved Permanently. nginx
Artificial Intelligence vs. Human Coaches: A Mixed Methods Randomized
In this mixed methods randomized controlled trial (RCT), 81 graduate students enrolled in the study and identified a personally relevant goal to pursue. ... Quantitative data were captured through three surveys and qualitative input was captured through open-ended survey questions and 27 debrief interviews. The study utilized a Wizard of Oz ...

Document Analysis

Cite this chapter

Access this chapter

Similar content being viewed by others

Qualitative Text Analysis: A Systematic Approach

Systematic Reviews and Meta-Analysis: A Guide for Beginners

Qualitative Content Analysis: Theoretical Background and Procedures

Additional Reading

Online Resources

Author information

Corresponding author

Editor information

Rights and permissions

Copyright information

About this chapter

Download citation

Share this chapter

To read this content please select one of the options below:

Related articles

All feedback is valuable

Join us on our journey

Questions & More Information

Article Contents

Document analysis in health policy research: the READ approach

Step 1. Ready your materials

Step 2. Extract data

Step 3. Analyse data

Step 4. Distil your findings

Supplementary data

Affiliations

This Feature Is Available To Subscribers Only

The Basics of Document Analysis

What is Meant by Document Analysis?

How Do You Do Document Analysis?

Why is Document Analysis Useful?

How NVivo Can Help with Document Analysis

A QDA recipe? A ten-step approach for qualitative document analysis using MAXQDA

Introduction

The ten steps for conducting qualitative document analyses using MAXQDA

Step 2: Data collection and data sampling

Step 3: Select and prepare the data

Step 4: Codebook development

Step 5: Unitizing and coding instructions

Step 6: Trial, training, reliability

Step 7: Revision and modification

Step 8: Coding

Step 9: Analyze and compare

Step 10: Interpretation and presentation

Summary and adoption

About the author

MAXQDA Newsletter

Similar Articles

Document analysis in health policy research: the READ approach

Hina Khalid

Shannon A McMahon

Associated Data

Key Messages

Introduction

What is document analysis?

The READ approach

Step 1. Ready your materials

Step 2. Extract data

Step 3. Analyse data

Step 4. Distil your findings

Supplementary Data

Supplementary Material

Data collection methods for evaluation: Document review

'Data collection methods for evaluation: Document review' is referenced in:

Integrating qualitative research within a clinical trials unit: developing strategies and understanding their implementation in contexts

Background/aims

Conclusions

Background to the integration of qualitative research within CTR

Understanding implementation—Normalisation Process Theory (NPT)

Coherence—making sense of qualitative research

Cognitive participation—legitimising qualitative research

Collective action—operationalising qualitative research

Reflexive monitoring—evaluating the impact of qualitative research

Availability of data and materials

Acknowledgements

Author information