cognitive operations in problem solving

• Subject List
• Take a Tour
• For Authors
• Subscriber Services
• Publications
• African American Studies
• African Studies
• American Literature
• Anthropology
• Architecture Planning and Preservation
• Art History
• Atlantic History
• Biblical Studies
• British and Irish Literature
• Childhood Studies
• Chinese Studies
• Cinema and Media Studies
• Communication
• Criminology
• Environmental Science
• Evolutionary Biology
• International Law
• International Relations
• Islamic Studies
• Jewish Studies
• Latin American Studies
• Latino Studies
• Linguistics
• Literary and Critical Theory
• Medieval Studies
• Military History
• Political Science
• Public Health
• Renaissance and Reformation
• Social Work
• Urban Studies
• Victorian Literature
• Browse All Subjects

How to Subscribe

• Free Trials

In This Article Expand or collapse the "in this article" section Problem Solving and Decision Making

Introduction.

• General Approaches to Problem Solving
• Representational Accounts
• Problem Space and Search
• Working Memory and Problem Solving
• Domain-Specific Problem Solving
• The Rational Approach
• Prospect Theory
• Dual-Process Theory
• Cognitive Heuristics and Biases

Related Articles Expand or collapse the "related articles" section about

About related articles close popup.

Lorem Ipsum Sit Dolor Amet

Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aliquam ligula odio, euismod ut aliquam et, vestibulum nec risus. Nulla viverra, arcu et iaculis consequat, justo diam ornare tellus, semper ultrices tellus nunc eu tellus.

• Artificial Intelligence, Machine Learning, and Psychology
• Counterfactual Reasoning
• Critical Thinking
• Heuristics and Biases
• Protocol Analysis
• Psychology and Law

Other Subject Areas

Forthcoming articles expand or collapse the "forthcoming articles" section.

• Data Visualization
• Remote Work
• Workforce Training Evaluation
• Find more forthcoming articles...
• Export Citations
• Share This Facebook LinkedIn Twitter

Problem Solving and Decision Making by Emily G. Nielsen , John Paul Minda LAST REVIEWED: 26 June 2019 LAST MODIFIED: 26 June 2019 DOI: 10.1093/obo/9780199828340-0246

Problem solving and decision making are both examples of complex, higher-order thinking. Both involve the assessment of the environment, the involvement of working memory or short-term memory, reliance on long term memory, effects of knowledge, and the application of heuristics to complete a behavior. A problem can be defined as an impasse or gap between a current state and a desired goal state. Problem solving is the set of cognitive operations that a person engages in to change the current state, to go beyond the impasse, and achieve a desired outcome. Problem solving involves the mental representation of the problem state and the manipulation of this representation in order to move closer to the goal. Problems can vary in complexity, abstraction, and how well defined (or not) the initial state and the goal state are. Research has generally approached problem solving by examining the behaviors and cognitive processes involved, and some work has examined problem solving using computational processes as well. Decision making is the process of selecting and choosing one action or behavior out of several alternatives. Like problem solving, decision making involves the coordination of memories and executive resources. Research on decision making has paid particular attention to the cognitive biases that account for suboptimal decisions and decisions that deviate from rationality. The current bibliography first outlines some general resources on the psychology of problem solving and decision making before examining each of these topics in detail. Specifically, this review covers cognitive, neuroscientific, and computational approaches to problem solving, as well as decision making models and cognitive heuristics and biases.

General Overviews

Current research in the area of problem solving and decision making is published in both general and specialized scientific journals. Theoretical and scholarly work is often summarized and developed in full-length books and chapter. These may focus on the subfields of problem solving and decision making or the larger field of thinking and higher-order cognition.

back to top

Users without a subscription are not able to see the full content on this page. Please subscribe or login .

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here .

• About Psychology »
• Meet the Editorial Board »
• Abnormal Psychology
• Academic Assessment
• Acculturation and Health
• Action Regulation Theory
• Action Research
• Addictive Behavior
• Adolescence
• Adoption, Social, Psychological, and Evolutionary Perspect...
• Advanced Theory of Mind
• Affective Forecasting
• Affirmative Action
• Ageism at Work
• Allport, Gordon
• Alzheimer’s Disease
• Ambulatory Assessment in Behavioral Science
• Analysis of Covariance (ANCOVA)
• Animal Behavior
• Animal Learning
• Anxiety Disorders
• Art and Aesthetics, Psychology of
• Assessment and Clinical Applications of Individual Differe...
• Attachment in Social and Emotional Development across the ...
• Attention-Deficit/Hyperactivity Disorder (ADHD) in Adults
• Attention-Deficit/Hyperactivity Disorder (ADHD) in Childre...
• Attitudinal Ambivalence
• Attraction in Close Relationships
• Attribution Theory
• Authoritarian Personality
• Bayesian Statistical Methods in Psychology
• Behavior Therapy, Rational Emotive
• Behavioral Economics
• Behavioral Genetics
• Belief Perseverance
• Bereavement and Grief
• Biological Psychology
• Birth Order
• Body Image in Men and Women
• Bystander Effect
• Categorical Data Analysis in Psychology
• Childhood and Adolescence, Peer Victimization and Bullying...
• Clark, Mamie Phipps
• Clinical Neuropsychology
• Clinical Psychology
• Cognitive Consistency Theories
• Cognitive Dissonance Theory
• Cognitive Neuroscience
• Communication, Nonverbal Cues and
• Comparative Psychology
• Competence to Stand Trial: Restoration Services
• Competency to Stand Trial
• Computational Psychology
• Conflict Management in the Workplace
• Conformity, Compliance, and Obedience
• Consciousness
• Coping Processes
• Correspondence Analysis in Psychology
• Counseling Psychology
• Creativity at Work
• Cross-Cultural Psychology
• Cultural Psychology
• Daily Life, Research Methods for Studying
• Data Science Methods for Psychology
• Data Sharing in Psychology
• Death and Dying
• Deceiving and Detecting Deceit
• Defensive Processes
• Depressive Disorders
• Development, Prenatal
• Developmental Psychology (Cognitive)
• Developmental Psychology (Social)
• Diagnostic and Statistical Manual of Mental Disorders (DSM...
• Discrimination
• Dissociative Disorders
• Drugs and Behavior
• Eating Disorders
• Ecological Psychology
• Educational Settings, Assessment of Thinking in
• Effect Size
• Embodiment and Embodied Cognition
• Emerging Adulthood
• Emotional Intelligence
• Empathy and Altruism
• Employee Stress and Well-Being
• Environmental Neuroscience and Environmental Psychology
• Ethics in Psychological Practice
• Event Perception
• Evolutionary Psychology
• Expansive Posture
• Experimental Existential Psychology
• Exploratory Data Analysis
• Eyewitness Testimony
• Eysenck, Hans
• Factor Analysis
• Festinger, Leon
• Five-Factor Model of Personality
• Flynn Effect, The
• Forensic Psychology
• Forgiveness
• Friendships, Children's
• Fundamental Attribution Error/Correspondence Bias
• Gambler's Fallacy
• Game Theory and Psychology
• Geropsychology, Clinical
• Global Mental Health
• Habit Formation and Behavior Change
• Health Psychology
• Health Psychology Research and Practice, Measurement in
• Heider, Fritz
• History of Psychology
• Human Factors
• Humanistic Psychology
• Implicit Association Test (IAT)
• Industrial and Organizational Psychology
• Inferential Statistics in Psychology
• Insanity Defense, The
• Intelligence
• Intelligence, Crystallized and Fluid
• Intercultural Psychology
• Intergroup Conflict
• International Classification of Diseases and Related Healt...
• International Psychology
• Interviewing in Forensic Settings
• Intimate Partner Violence, Psychological Perspectives on
• Introversion–Extraversion
• Item Response Theory
• Law, Psychology and
• Lazarus, Richard
• Learned Helplessness
• Learning Theory
• Learning versus Performance
• LGBTQ+ Romantic Relationships
• Lie Detection in a Forensic Context
• Life-Span Development
• Locus of Control
• Loneliness and Health
• Mathematical Psychology
• Meaning in Life
• Mechanisms and Processes of Peer Contagion
• Media Violence, Psychological Perspectives on
• Mediation Analysis
• Memories, Autobiographical
• Memories, Flashbulb
• Memories, Repressed and Recovered
• Memory, False
• Memory, Human
• Memory, Implicit versus Explicit
• Memory in Educational Settings
• Memory, Semantic
• Meta-Analysis
• Metacognition
• Metaphor, Psychological Perspectives on
• Microaggressions
• Military Psychology
• Mindfulness
• Mindfulness and Education
• Minnesota Multiphasic Personality Inventory (MMPI)
• Money, Psychology of
• Moral Conviction
• Moral Development
• Moral Psychology
• Moral Reasoning
• Nature versus Nurture Debate in Psychology
• Neuroscience of Associative Learning
• Nonergodicity in Psychology and Neuroscience
• Nonparametric Statistical Analysis in Psychology
• Observational (Non-Randomized) Studies
• Obsessive-Complusive Disorder (OCD)
• Occupational Health Psychology
• Olfaction, Human
• Operant Conditioning
• Optimism and Pessimism
• Organizational Justice
• Parenting Stress
• Parenting Styles
• Parents' Beliefs about Children
• Path Models
• Peace Psychology
• Perception, Person
• Performance Appraisal
• Personality and Health
• Personality Disorders
• Personality Psychology
• Person-Centered and Experiential Psychotherapies: From Car...
• Phenomenological Psychology
• Placebo Effects in Psychology
• Play Behavior
• Positive Psychological Capital (PsyCap)
• Positive Psychology
• Posttraumatic Stress Disorder (PTSD)
• Prejudice and Stereotyping
• Pretrial Publicity
• Prisoner's Dilemma
• Problem Solving and Decision Making
• Procrastination
• Prosocial Behavior
• Prosocial Spending and Well-Being
• Psycholinguistics
• Psychological Literacy
• Psychological Perspectives on Food and Eating
• Psychology, Political
• Psychoneuroimmunology
• Psychophysics, Visual
• Psychotherapy
• Psychotic Disorders
• Publication Bias in Psychology
• Reasoning, Counterfactual
• Rehabilitation Psychology
• Relationships
• Reliability–Contemporary Psychometric Conceptions
• Religion, Psychology and
• Replication Initiatives in Psychology
• Research Methods
• Risk Taking
• Role of the Expert Witness in Forensic Psychology, The
• Sample Size Planning for Statistical Power and Accurate Es...
• Schizophrenic Disorders
• School Psychology
• School Psychology, Counseling Services in
• Self, Gender and
• Self, Psychology of the
• Self-Construal
• Self-Control
• Self-Deception
• Self-Determination Theory
• Self-Efficacy
• Self-Esteem
• Self-Monitoring
• Self-Regulation in Educational Settings
• Self-Report Tests, Measures, and Inventories in Clinical P...
• Sensation Seeking
• Sex and Gender
• Sexual Minority Parenting
• Sexual Orientation
• Signal Detection Theory and its Applications
• Simpson's Paradox in Psychology
• Single People
• Single-Case Experimental Designs
• Skinner, B.F.
• Sleep and Dreaming
• Small Groups
• Social Class and Social Status
• Social Cognition
• Social Neuroscience
• Social Support
• Social Touch and Massage Therapy Research
• Somatoform Disorders
• Spatial Attention
• Sports Psychology
• Stanford Prison Experiment (SPE): Icon and Controversy
• Stereotype Threat
• Stereotypes
• Stress and Coping, Psychology of
• Student Success in College
• Subjective Wellbeing Homeostasis
• Taste, Psychological Perspectives on
• Teaching of Psychology
• Terror Management Theory
• Testing and Assessment
• The Concept of Validity in Psychological Assessment
• The Neuroscience of Emotion Regulation
• The Reasoned Action Approach and the Theories of Reasoned ...
• The Weapon Focus Effect in Eyewitness Memory
• Theory of Mind
• Therapy, Cognitive-Behavioral
• Thinking Skills in Educational Settings
• Time Perception
• Trait Perspective
• Trauma Psychology
• Twin Studies
• Type A Behavior Pattern (Coronary Prone Personality)
• Unconscious Processes
• Video Games and Violent Content
• Virtues and Character Strengths
• Women and Science, Technology, Engineering, and Math (STEM...
• Women, Psychology of
• Work Well-Being
• Wundt, Wilhelm
• Privacy Policy
• Cookie Policy
• Legal Notice
• Accessibility

Powered by:

• [66.249.64.20|81.177.182.174]
• 81.177.182.174

• Search Menu
• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Urban Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Papyrology
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Evolution
• Language Reference
• Language Acquisition
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Media
• Music and Religion
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Science
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Lifestyle, Home, and Garden
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Neuroanaesthesia
• Clinical Neuroscience
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Ethics
• Business Strategy
• Business History
• Business and Technology
• Business and Government
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic History
• Economic Systems
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Natural Disasters (Environment)
• Social Impact of Environmental Issues (Social Science)
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• International Political Economy
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Behaviour
• Political Economy
• Political Institutions
• Political Sociology
• Political Methodology
• Political Communication
• Political Philosophy
• Political Theory
• Politics and Law
• Public Policy
• Public Administration
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Developmental and Physical Disabilities Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

12 Problem Solving

Stephen K. Reed, Department of Psychology, San Diego State University

• Published: 05 December 2014
• Cite Icon Cite
• Permissions Icon Permissions

Solving a problem results in obtaining a desired goal through the use of higher mental functions, including reasoning and planning. Problems—such as those requiring arrangement, transformation, and inducing structure—can be classified based on the cognitive skills that are required to solve them. Although general heuristics are sufficient for solving knowledge-lean problems, organized knowledge structures (schemas) are needed to solve knowledge-rich problems. Using analogous solutions is often helpful for both types of problems. Mappings across concepts, problem states, and operations relate the structure of analogous problems and of different solutions to the same problem. EUREKA, CLARION, and ACT are examples of cognitive architectures that apply to problem solving. Underinvestigated topics include problems with insufficient information, estimated answers, complex problem solving, and collaborative problem solving.

The APA Dictionary of Psychology ( VandenBoss, 2006 ) defines problem solving as:

The process by which individuals attempt to overcome difficulties, achieve plans that move them from a starting situation to a desired goal, or reach conclusions through the use of higher mental functions such as reasoning and creative thinking.

Reviewing problem-solving research and theories is a challenge because this definition is so inclusive. Our task is made easier, however, because of previous reviews. In particular, I have built on problem-solving chapters by Bassok and Novick (2012) and by VanLehn (1989) . The Bassok and Novick chapter appears in the Oxford Handbook of Thinking and Reasoning and emphasizes research by cognitive psychologists. The VanLehn chapter appears in Foundations of Cognitive Science and includes a computational approach to problem solving. My objective here is to present both research findings and computational models while extending the contributions made in these previous chapters.

There are many different kinds of problems that fit the definition of problem solving in the first paragraph. The first section of this chapter therefore includes a taxonomy that partitions problems into categories based on the skills required to solve them. This section also describes major historical approaches. The second section discusses the role of organized knowledge structures, labeled schemas , in supporting the development of expertise. The third section explores relations between different problems and between different solutions to the same problem. The fourth section illustrates how cognitive architectures have enhanced our understanding through embedding problem solving within broad theoretical frameworks. The final section proposes future directions by identifying underresearched and developing topics.

Kinds of Problems

To make a review of problem solving more manageable, Greeno (1978) divided problems into three categories based on the cognitive skills required to solve them. He labeled the categories arrangement problems, transformation problems, and inducing structure problems. Arrangement problems require rearranging parts to satisfy some criterion, such as creating a word from the letters ARAGMAN. Transformation problems require transforming an initial state into a goal state, such as moving four rings from peg A to peg C under the constraint that a larger ring can not be placed on a smaller ring. The goal state is known in transformation problems: the stack of rings on peg C varies from the largest on the bottom to the smallest on the top. Inducing structure problems require identifying relations among the parts of a problem and then using that structure to produce the solution. Examples include series completion problems such as producing the next four letters in the series r s c d s t d e t u e f . Discovering the relations among parts is crucial for solving both arrangement and inducing structure problems, but parts cannot be rearranged in inducing structure problems.

I begin by discussing problems that fall into each of these three categories because it provides an opportunity to write about two important movements during the 20th-century work on problem solving. Gestalt psychologists focused on arrangement problems during the earlier half of the century because these problems fit into their theoretical framework that a pattern is more than the sum of its parts. Stimulated by applications of computer science to human problem solving ( Newell, Shaw, & Simon, 1958 ), transformation problems began to play an important role in the second half of the century. Inducing structure has a more eclectic history, with psychometric, artificial intelligence, and information processing approaches all making significant contributions.

The classification of problems into one of the three categories should not imply that all problems fit into a single category. Greeno (1978) admitted that complex problems require multiple skills. For instance, playing chess requires arranging chess pieces to accomplish the goal of winning the game, moving (transforming) pieces toward particular arrangements for executing a plan, and inducing structure to analyze the opponent’s plan.

Arrangement Problems and Gestalt Psychology

Gestalt psychologists were primarily interested in problems that required arranging the parts to find new relations that achieved a goal. Kohler (1925) described an early example in his book The Mentality of Apes . A cage of a chimpanzee contained fruit hanging from the top and sticks and crates on the floor. The chimp could obtain the fruit by using a stick after stacking and climbing on the crates. Solving the problem, according to the Gestalt analysis, depended on reorganizing the objects into a new structure.

Another famous example is Duncker’s (1945) radiation problem. A medical procedure required using radiation to destroy a tumor without destroying the healthy tissue that surrounds it. A solution is to divide the radiation into multiple rays that converge on the location of the tumor. Intense radiation occurs only at the point of convergence so does not harm healthy tissue.

Gestalt psychologists used the term insight to describe the sudden discovery of a correct arrangement of parts following a succession of incorrect arrangements ( Kohler, 1947 ). Metcalfe and Wiebe (1987) empirically evaluated this concept by giving students nonroutine problems such as planting four trees exactly the same distance from the others. Every 15 seconds, the participants had to indicate on a 7-point scale how close they believed they were to solving the problem. Although the highest rating was the most frequent rating at the solution, the lowest rating was the most frequent rating 15 seconds before the solution. The findings support the construct of insight in which solutions occur very suddenly following a perceived lack of progress.

One interpretation of such findings is that insight occurs when solvers remove self-imposed constraints ( Knoblich, Ohlsson, Haider, & Rhenius, 1999 ). For example, people typically attempt to solve the four-trees problem in two dimensions although this constraint is not mentioned in the problem. The solution requires a three-dimensional arrangement.

Knoblich and his coauthors evaluated their theory of constraint relaxation by asking participants to rearrange matchsticks, including the ones shown in Figure 12.1 . The objective is to move a single stick to turn an incorrect arithmetic statement into a correct one. The stick cannot be discarded but must occupy a new position in the equation. The findings confirmed the predictions based on constraint relaxation. Type a problems are solved by modifying the numerals (changing IV to VI) and were the easiest. Type b problems are solved by modifying arithmetic operations (moving a match stick from the equals sign to the minus sign). Type c problems are solved by creating more than one equals sign and were the most difficult. The three types of problems became equally easy after participants realized that they could modify and create equal signs.

Matchstick problems.

A question regarding the restructuring that leads to insight is whether restructuring involves controlled search processes or whether it involves an automatic redistribution of activation in long-term memory. Ash and Wiley (2006) investigated this question by determining whether individual differences in working memory span predicted performance on the initial search and restructuring phases. Working memory span did predict success on problems that required both initial search and restructuring but did not predict success on problems that isolated the restructuring phase. The findings are consistent with the interpretation that restructuring involves an automatic redistribution of activation.

Although the Gestalt approach emphasized problem representations, and Newell and Simon (1972) emphasized searching for a solution, both search and representation are important in solving problems ( Bassok & Novick, 2012 ). The next section focuses on the search process.

Transformation Problems and Search

The study of transformation problems—the transformation of an initial problem state into a goal state—became an important area of research as computers became a source of symbol manipulation. Newell, Shaw, and Simon (1958) made the connection between computers and human problem solving in their Psychological Review article “Elements of a Theory of Human Problem Solving.” Their theory proposed (1) a control system consisting of a number of memories containing symbols interconnected by various relations, (2) primitive information processes that operate on information in the memories, and (3) a set of rules for combining these processes into whole programs.

A program constitutes a theory that can make precise predictions. As stated by Newell, Shaw, and Simon:

The ability to specify programs precisely, and to infer accurately the behavior they produce, drives from the use of high-speed digital computers. Each specific theory—each program of information processes that purports to describe some human behavior—is coded for a computer. That is, each primitive information process is coded to be a separate computer routine, and a “master” routine is written that allows these primitive processes to be assembled into any system we wish to specify. Once this has been done, we can find out exactly what behavior the purported theory predicts by having the computer “simulate” the system. (pp. 152–153)

The authors designed the programs to simulate human problem solving by comparing the behavior predicted by the program with actual behavior observed in experimental settings. The promise of the digital computer was that it provided a device for determining what behavior is implied by a program and for subsequently modifying the program if the predictions failed. Programming required a detailed specification of the operations, which enabled theorists to evaluate whether the operations were sufficient to produce the behavior. It thereby avoided the vagueness that limited other theories of higher mental processes ( Newell et al., 1958 ).

The interest in writing simulation programs was accompanied by an interest in writing artificial intelligence (AI) programs to enable computers to produce, rather than simulate, intelligent behavior. Early examples of these activities are provided in the book GPS: A Case Study in Generality and Problem Solving ( Ernst & Newell, 1969 ). The General Problem Solver (GPS) had the objective of using generic principles to solve a variety of problems including Tower of Hanoi, Missionaries and Cannibals, integration, and logical proofs. The GPS was not completely general because it solved only transformation problems by using the means-end analysis heuristic. Means-end analysis attempts to successively eliminate differences between the initial state and the goal state until the program arrives at the goal state. This heuristic is often successful on transformation problems because these problems have a well-defined goal state: all rings are moved from peg A to peg C or all missionaries and cannibals are moved across the river. In contrast, producing the goal state is typically required in arrangement problems such as solving an anagram or Dunker’s radiation problem.

Solution of a logic problem using means-end analysis.

These initial AI programs provided possible theories for how people solve problems. One example is the construction of logical proofs, a task that Newell and Simon (1972) extensively analyzed in their classic book Human Problem Solving . The problem solver was given 12 rules for manipulating letters connected by dots •, wedges v, horseshoes ⊃, and tildes ~. These connectives are used to represent and, or, implies , and no t in logic but were not interpreted for the participants. The 12 rules enable problem solvers to modify logical expressions until they have constructed a proof by transforming the initial state into the goal state. For instance, the initial state could be A ⊃ B and the goal state could be ~ B ⊃ ~ A .

Means-end analysis was implemented in the GPS for logic problems by including a table of connections in the program that showed which of six differences could be modified by each of the 12 rules. Figure 12.2 shows a simplified table of connections consisting of three rules and three differences for solving the A ⊃ B problem.

Transforming the initial state A ⊃ B into the goal state ~ B ⊃ ~ A requires eliminating differences in both the sign and position of the letters. The table of connections reveals that both Rules 2 and 3 can change a sign, but Rule 2 cannot be applied because it has different connectives than the initial state. Application of Rule 3 to the initial state changes the sign of A to negation. However, the resulting problem state, ~ A v B , now differs from the goal state in the sign of B , position of the letters, and connective. The application of Rule 1 to Line 2 produces an expression that can be changed to the goal state through the reapplication of Rule 3. Newell and Simon (1972) asked their participants to verbalize their thoughts as they worked on the problems. Many aspects of their thinking corresponded to means-end analysis used in the GPS.

A difference between arrangement and transformation problems is that solvers of transformation problems should realize that they are making progress as they gradually reduce differences between the current problem state and the goal state. Metcalfe and Wiebe (1987) confirmed this difference by finding higher ratings of approaching the solution as their participants continued to work on the transformation problems. Both arrangement and transformation problems received a high rating at the solution, but only transformation problems received a high rating 15 seconds before the solution.

An important theoretical component of Newell and Simon’s (1972) theory of problem solving is the problem space . The search space specifies the permissible actions (legal moves) at each problem state. Figure 12.3 shows the problem space for the five missionaries-cannibals problem ( Simon & Reed, 1976 ):

Five missionaries and five cannibals who have to cross a river find a boat, but the boat is so small that it can hold no more than three persons. If the cannibals outnumber the missionaries on either bank of the river or in the boat at any time, the missionaries will be eaten. Find the simplest schedule of crossings that will allow everyone to cross safely. At least one person must be in the boat at each crossing.

Problem space for the five Missionaries and Cannibals problems.

Each oval in Figure 12.3 is a problem state of the form MC/MC* in which the first MC is the number of missionaries (M) and cannibals (C) on the initial bank, and the second MC is the number of missionaries and cannibals across the river. The asterisk shows the location of the boat, and the links show the number of missionaries and cannibals in the boat. Solving the problem requires transforming the initial state A into the goal state Z . The problem space reveals a number of important characteristics of the problem, such as there are four legal moves at the initial state, state J is the end of a blind alley that requires reversing the two previous moves, and the minimal solution requires 11 moves.

The search space differs from the problem space because it reveals which moves are considered by the problem solver ( Newell & Simon, 1972 ). For instance, undergraduates required an average of 30 moves to solve the problem without a hint and 20 moves to solve the problem when given a subgoal that at some point there will be 3 cannibals and 0 missionaries across the river without the boat (state L ). Simon and Reed (1976) proposed a strategy-shift model to predict the average number of times students in each group would visit each of the problem states in Figure 12.3 . The model assumes that students begin with a balance strategy in which they attempt to equalize the number of missionaries and cannibals across the river, as in state D . They then switch to a means-end strategy in which they attempt to take as many people across the river as possible (3) and bring back as few as possible (1). The probability of switching strategies is higher for the subgoal group, which helps them avoid the blind alley ending in state J . The strategy-shift model is consistent with the “unbalanced” subgoal—3 cannibals and 0 missionaries across the river.

Inducing Structure and Reasoning

The sections on arrangement and transformation tasks contained research that is typically included in problem-solving chapters. In contrast, tasks that require inducing structure might appear in reasoning chapters. An inclusive definition of problem solving, such as the one at the beginning of this chapter, includes reasoning, but there is a distinction between reasoning and problem solving. Holyoak and Morrison (2012) state that reasoning places an emphasis on drawing inferences (conclusions) from some initial information (premises) and has a foundation in logic. Problem solving involves a course of action to achieve a goal.

I focus on a particular reasoning task (the four-card selection problem) in this section for three reasons. First, the task has been one of the most widely studied tasks in the reasoning literature. Second, it illustrates how inducing structure differs from arranging and transforming components. Inducing structure is similar to arrangement problems because it is necessary to discover the relations among the components of the problem ( Greeno, 1978 ). However, unlike arrangement problems, these components are static and cannot be rearranged. Third, research on this task illustrates the challenge of identifying the extent to which reasoning depends on general knowledge. The arrangement and transformation problems in the previous two sections consisted primarily of puzzles that did require extensive knowledge about a particular domain. The four-card selection problem illustrates how our familiarity with the content of information in rules influences our ability to evaluate those rules.

The four-card selection problem ( Wason & Johnson-Laird, 1972 ) requires deciding which one of four cards needs to be turned over to evaluate a conditional rule; for example, if there is a D on one side of the card, then there is a 3 on the other side. The four cards in this example either display the letter D , the letter K , the number 3 , or the number 7 . The experimenter informs participants that each of the cards contains a letter on one side and a number on the other side. The answer is that it is necessary to turn over the D card and the 7 card but only 5 of 128 participants turned over only the two correct cards ( Wason & Shapiro, 1971 ).

Wason and Shapiro (1971) hypothesized that performance would dramatically improve if the conditional rules had realistic rather than abstract content, a prediction that was confirmed in a letter-sorting task ( Johnson-Laird, Legrenzi, & Legrenzi, 1972 ). The task consisted of four envelopes. Two were face up, revealing either a 50 lira or a 40 lira stamp. Two were face down, revealing either a sealed or an unsealed envelope. Participants were told to imagine that they worked in a post office and had to enforce the rule “If a letter is sealed then it has a 50-lira stamp on it.” Most participants (17 of 24) accurately selected the two envelopes required to enforce the rule.

Although Wason and Shapiro (1971) argued that conditional reasoning is vastly improved with realistic content, Griggs and Cox (1982) questioned whether the letter task required conditional reasoning. Their memory-retrieval explanation proposed that the British participants did the task by recalling their experience in placing more postage on sealed envelopes. Griggs and Cox therefore predicted that their American students, who lacked such experience, would do poorly on the task. As predicted, American students did poorly on the unfamiliar letter task but excelled in evaluating a familiar drinking-age rule “If a person is drinking beer then the person must be over 19 years of age.”

Griggs and Cox’s findings are discouraging because they support the conclusion that people are very limited in evaluating conditional rules unless the rules contain familiar content, in which case reasoning is not required. A more optimistic view of reasoning is that people do well at conditional reasoning if the content is familiar at a general, schematic level. For instance, pragmatic reasoning schemata are organized knowledge structures that enable us to evaluate practical situations such as seeking permission or fulfilling an obligation ( Cheng, Holyoak, Nisbett, & Oliver, 1986 ).

Imagine that you are hired to enforce the rule “If a passenger wishes to enter the country, then he or she must have an inoculation against cholera.” Four cards identify a passenger who wishes to enter, a passenger who does not wish to enter, a passenger who has been inoculated, and a passenger who has not been inoculated. The pragmatic reasoning hypothesis predicts that you can use your schematic knowledge about seeking permission to evaluate this rule even if you have no experience with this particular task. More information is required for the passenger who wishes to enter and for the passenger who has not been inoculated. Research supports the hypothesis that people do much better in evaluating conditional statements involving permission or obligation than in evaluating conditional statements involving arbitrary relations ( Cheng et al., 1986 ).

In summary, the evolution of research on the four-card selection problem reveals the relative influence of concrete and familiar experiences on reasoning. People did very poorly in evaluating the implications of conditional rules involving arbitrary relations between letters and numbers. Performance dramatically improved on concrete versions of the rules but raised the question of whether retrieving experiences from memory removed the need to reason. An intermediate level of abstractness is provided by schemas that generalize the commonality among individual experiences, such as seeking permission or fulfilling an obligation. People can effectively reason about unfamiliar experiences if those experiences can be linked to a familiar schema. Schemas also play an important role in problem solving, as discussed in the next section.

Much of the research on problem solving during the 1970s was influenced by Newell and Simon’s (1972) book in which general strategies (heuristics) such as using means-end analysis or forming subgoals guided the search process. VanLehn (1989) refers to problems such as Missionaries and Cannibals or the Tower of Hanoi as knowledge-lean tasks because they can be solved without prior experience. In contrast, research in the 1980s began to focus on problems from algebra, physics, geometry, and computer programming. These are knowledge-rich tasks that require many hours of instruction ( VanLehn, 1989 ).

Schemas as a Theoretical Construct

Organized knowledge structures called schemas are an effective method for organizing this knowledge. Brewer and Nakamura (1984) described the characteristics of schemas by contrasting them with learning based on stimulus-response (S-R) associations.

S-R learning is based on small units of knowledge. A schema is a larger unit in which knowledge is combined into clusters.

S-R learning requires learning an association between a stimulus and a response. A schema provides a knowledge structure for interpreting and encoding aspects of particular experiences.

S-R learning involves a particular stimulus and response. A schema is more general and represents a variety of experiences.

The association between a stimulus and a response can be learned in a passive manner. Invoking a schema is a more active process in which a particular experience is matched to the schema that best fits the experience.

In her book, Marshall (1995) began by reviewing the historic development of schemas as a theoretical construct by tracing the ideas of Plato, Aristotle, Kant, Bartlett, and Piaget. In her working definition, a schema is a memory organization that can (1) recognize similar experiences; (2) access a general framework that contains essential elements of those experiences; (3) use the framework to draw inferences, create goals, and develop plans; and (4) provide skills and procedures for solving problems in which the framework is relevant.

Marshall then described her research that built on the analysis of addition and subtraction problems. Riley, Greeno, and Heller (1983) had analyzed elementary word problems into change, combine, and compare problems. Kintsch and Greeno (1985) further developed these distinctions as a set schema in which the slots consisted of objects <noun>; quantity <number>; specification <owner>, <location>, <time>; and role <start, transfer, result, superset, subset, largeset, smallset, difference>. Marshall added two additional schematic situations (labeled restate and vary ) and constructed a computer tutor to help students learn to solve multistep arithmetic word problems.

Learning these schematic components is important because they form the building blocks of more complex problems, as in physics ( Sherin, 2001 ) and algebra word problems ( Reed et al., 2012 ). Research shows that algebra word problems are difficult for university students, not only because of algebra, but because students have not adequately learned the change, combine, and compare schema that are the components of both arithmetic and algebra word problems ( Reed et al., 2012 ). Learning these elementary and more advanced schemas supports the development of expertise.

Schemas in Experts

The transition from the study of domain-lean problems in the 1970s to domain-rich problems in the 1980s resulted in investigations of how domain knowledge influenced problem solving. Silver (1981) asked good, average, and poor problem solvers to sort arithmetic word problems into groups based on common solution procedures. The better problem solvers excelled at this assignment, but the weaker problem solvers sorted by story content. For example, they placed problems about hens and rabbits into the same category although the problems required different solutions.

Silver’s finding has been confirmed for many domains and for many levels of expertise. Chi, Glaser, and Reese (1982) asked eight undergraduates and eight advanced physics doctoral students to sort 24 physics problems into categories based on similar solutions. Novices tended to classify problems on the basis of common objects such as inclined planes and springs. Experts tended to classify problems based on physics principles such as the conservation of energy or Newton’s second law (F = MA).

Although such expert-defined schemas are usually very helpful, they can occasionally constrain innovative solutions. Dane (2010) defines cognitive entrenchment as a high level of stability in domain schemas that can cause experts to be inflexible in their thinking. Cognitive entrenchment increases the likelihood of problem-solving fixation and blocks the generation of novel ideas. However, Dane proposes two factors that can reduce cognitive entrenchment. The first is working in a dynamic environment in which one must remain open to a wide range of possibilities and options. The second is focusing attention on outside-domain tasks in which counterexamples and exceptions can increase the flexibility of one’s beliefs.

Schema Abstraction

The ability to see structural commonalities in situations that appear quite different can be very helpful, as illustrated by the use of pragmatic reasoning schema to reason about conditional rules ( Cheng et al., 1986 ); the use of change, combine, and compare schema to classify arithmetic word problems ( Silver, 1981 ); and the use of physics principles to classify physics problems ( Chi et al., 1982 ). All of these situations can be aided by schema abstraction , in which problem solvers focus on the structural relations among the objects (inoculation, cholera, hens, rabbits, springs, inclined planes) rather than on the objects.

A challenge is to encourage noticing these structural relations through schema abstraction—a challenge that was met in a classic study by Gick and Holyoak (1983) . Three years earlier, they published research that demonstrated the difficulty of spontaneously noticing analogous solutions ( Gick & Holyoak, 1980 ). Their goal in this earlier research was to increase the number of convergence solutions to Duncker’s (1945) radiation problem. Participants read an analogous problem in which a general wanted to capture a fortress but could not attack along one road because it was mined. The general therefore divided his army into small groups that simultaneously converged on the fortress from different roads. Very few participants, however, used the analogy unless they were given a hint that the military problem would help them solve the radiation problem.

To spontaneously notice an analogy, people need to think about analogous solutions at a more abstract level so that differences in the objects, such as a fortress and a tumor, would not be a hindrance. Gick and Holyoak (1983) therefore asked participants to compare the similarities between two stories, the military problem and a story about Red Adair whose crew put out fires in oil derricks by using multiple hoses that converged on the site of the fire. Comparing two stories helped participants spontaneously notice the analogy to the radiation problem by creating the more abstract convergence schema shown in Table 12.1 . Simply reading the two stories was insufficient; abstraction depended on the comparison ( Catrambone & Holyoak, 1989 ).

A productive application of this finding occurred in a negotiation training program for management consultants who had approximately 15 years of work experience ( Gentner, Lowenstein, Thompson, & Forbus, 2009 ). The consultants studied two cases of a contingent contract that depended on the outcome of some future event. One group studied the two cases separately, and another group compared the similarities of the two cases. As found in laboratory studies ( Catrambone & Holyoak, 1989 ), the comparison aided schema abstraction. The comparison group was more successful in describing the principles of a contingent contract and in recalling examples of contingent contracts from their own experiences.

Mapping Across Problems and Solutions

Using the solution of the military problem to find a solution to the radiation problem requires finding corresponding objects and relations in the two solutions. As shown in Table 12.1 , the fortress in the military problem corresponds to the tumor in the radiation problem, the large army corresponds to powerful rays, the inability to use a single road corresponds to the inability to use a single pathway, and dividing the army corresponds to dividing the radiation. Establishing these correspondences requires mapping the objects and relations in the military problem to objects and relations in the radiation problem.

Illustration of one-to-one, one-to-many, and partial mappings across knowledge states.

There have been a number of detailed computational models of analogical mappings, including one by Hummel and Holyoak (1997) . Mappings in their model are guided by three constraints:

Structural consistency implies a one-to-one mapping between an element in the source and an element in the target.

Semantic similarity implies that elements with prior semantic similarity (such as joint membership in a taxonomic category) should tend to map to each other.

Pragmatic centrality implies that mappings should give preference to elements that are important for goal attainment.

Structural consistency in the fortress-tumor analogy is illustrated by the one-to-one mapping between objects in the two problems, semantic similarity is illustrated by similar actions (dividing the army and the tumor), and pragmatic centrality is illustrated by the principle of converging forces in both solutions.

Reed (2012) has extended this one-to-one mapping across problems to construct a taxonomy consisting of three types of mappings (one-to-one, partial, and one-to-many, as illustrated in Figure 12.4 ) and four types of situations (problems, solutions, representations, and sociocultural contexts). Mappings across problems and mappings across solutions—different solutions to the same problem—illustrate parts of the taxonomy.

Mapping Across Problems

Most computational models of transfer, including the one proposed by Hummel and Holyoak (1997) , have emphasized one-to-one mappings across isomorphic problems. In contrast, Reed, Ernst, and Banerji (1974) investigated transfer between two problems in which the problem states and moves in one problem had a one-to-many mapping to the problem states and moves in the other problem. One of the problems was the Missionaries and Cannibals (MC) problem, in which three missionaries and three cannibals cross a river using a boat that can hold two people under the constraint that cannibals can never outnumber missionaries. The other problem was the Jealous Husbands (JH) problem:

Three jealous husbands, and their wives, having to cross a river, find a boat. However, the boat is so small that it can hold no more than two persons. Find the simplest schedule of crossings that will permit all six persons to cross the river so that no woman is left in the company of any other woman’s husband unless her own husband is present.

We anticipated, based on our perceived similarity of the two problems, that there would be substantial transfer from one problem to the other. Our first experiment found no transfer, but our second experiment found some transfer when students were informed about the mapping between the two problems: husbands correspond to missionaries and wives correspond to cannibals. However, even with this hint, there was evidence of transfer only from the JH problem to the MC problem. The asymmetrical transfer is consistent with a one-to-many mapping from the MC to the JH problem because moving a missionary does not specify which husband to move and moving a cannibal does not specify which wife to move. Although all missionaries and cannibals are equivalent, all husbands and wives are not because they are paired with each other. For example, the three circles in Figure 12.2 representing one-to-many mappings might represent moving husbands A and B, B and C , and A and C . Each of these moves maps onto moving two missionaries, but moving two missionaries does not specify which two husbands to move. It should therefore be more difficult to map moves from the MC problem to the JH problem because this mapping does not specify a unique move.

Partial mappings are similar to isomorphic mappings because both specify one-to-one mappings between the source and the target. The difference is that isomorphic mappings are sufficient for solving the target problem, whereas partial mappings are not. It may therefore be helpful to use more than one analogy ( Gentner & Gentner, 1983 ).

The Garden Border Problem. From Greeno & van de Sande (2007) .

The Gentners identified two analogies (flowing waters and teeming crowds) for helping students understand electric circuits. They predicted that students who used the flowing waters analogy (pressure of water, flow in a pipe) should do well on questions about voltage and current because serial and parallel reservoirs combine in the same manner as serial and parallel batteries. In contrast, students with the moving crowd model should do better on resistors because of the analogy to gates. The results supported their predictions in the first experiment. In the second experiment, the analogy to flowing waters was not as helpful as expected because students lacked knowledge in this area.

Spiro, Feltovich, Coulson, and Anderson (1989) discuss practical implications of partial mappings. They propose that simple analogies help beginners gain a preliminary understanding of complex concepts but can later block fuller understanding if learners never progress beyond the simple analogy. One consequence is that instructors need to pay closer attention to how analogies can fail. The authors discuss eight possible failures of simple analogies including misleading properties, missing properties, a focus on surfaces descriptions, and wrong grain size. Their remedy is to use multiple analogies to convey the complexity of difficult ideas.

Mapping Across Solutions

Most teachers and researchers are delighted if problem solvers find one solution to a problem. However, Alan Schoenfeld is more demanding. After students in his math classes at Berkeley solve the problem, he asks them to find another solution. Then a third. The reason is that any one of these solutions might prove helpful in solving future problems ( Schoenfeld, 1985 ).

Studying mapping across solutions attempts to establish how one solution to a problem is related to an alternative solution ( Reed, 2012 ). Rittle-Johnson, Star, and Durkin (2009) discovered that asking seventh- and eighth-grade students to compare two solutions for solving the same problem was helpful when they had the appropriate prior knowledge. One of the solutions showed a short-cut method. In the example given here, the first method requires multiplication, subtraction, and division. The second method requires only division and subtraction:

Students who were familiar with one of the two methods typically noticed that one method required fewer steps or was more efficient than the other. Comparing solutions for these students produced flexible knowledge of procedures. In contrast, students who were not familiar with either method benefited more from the sequential presentation of the solutions.

Comparing alternative solutions can be particularly rewarding when the solutions are generated by different people. Greeno and van de Sande’s (2007) analysis of the Garden Border problem in Figure 12.5 illustrates how a shift in a teacher’s perspective helped her understand that a student’s different approach to the problem could provide an alternative solution. The key difference between the two solutions was how the teacher and student used the phrase “an even border of flowers.” The teacher represented the width of this border by the unknown variable (such as w ) and constructed an equation to represent the area of the inner rectangle by multiplying the length of this rectangle by its width:

This equation followed a previous calculation that the area of the inner rectangle is 1,680 square feet. Because the border is even, 2 w can be subtracted from both the length and width of the outer rectangle.

However, this one-to-one mapping from the text to a variable did not occur to a student who represented the border’s width by two variables: w 1 = ( 72 − y ) / 2 with respect to the length of the outer rectangle and w 2 = ( 40 − x ) / 2 with respect to the width of the outer rectangle. Another student understood how this representation could work by using two equations. The teacher then encouraged the class to figure out the values of x and y by generating the two equations. This second solution was not as efficient because it requires two equations to solve for the two unknown variables. However, the teacher not only encouraged the students in their attempts to generate an alternative solution but recognized that the alternative solution provided a learning opportunity for the class to practice solving for two unknown values.

In summary, one-to-one, one-to-many, and partial mappings across knowledge states provides a basis for analyzing both analogical transfer across problems and the relations between different solutions to the same problem.

Cognitive Architectures

Computational models have contributed to our theoretical understanding of topics such as exploring a problem space ( Simon & Reed, 1976 ) and using analogous solutions ( Hummel & Holyoak, 1997 ). Embedding computational models within cognitive architectures increases their generality by modeling a greater range of activities. EUREKA ( Jones & Langley, 2005 ), CLARION ( Heile & Son, 2010 ; Sun & Zhang, 2006 ), and ACT ( Anderson, Byrne, Douglass, Lebeire, & Qin, 2004 ) illustrate how cognitive architectures have been used to model problem solving.

VanLehn (1989) identified 10 robust findings in his problem-solving chapter that provided a test bed for the design and evaluation of a problem-solving architecture called EUREKA ( Jones & Langley, 2005 ). EUREKA attempts to solve all problems by using analogical reasoning. By incorporating human memory constraints, EUREKA strives to qualitatively replicate VanLehn’s (1989) reported findings. To find solutions to problems, analogies are created to map the operators used in a previous problem to the new problem. The degree of mapping can differ, depending on the degree of the match of the two situations and the level of activation of relevant retrieval patterns.

EUREKA uses means-ends analysis ( Newell & Simon, 1972 ) to divide problem solution into two tasks. The transform task transforms the current state into a desired state. The apply task satisfies preconditions of operators. If the current state satisfies the operators’ preconditions, then the operators are applied to generate a desired state. Otherwise, another transform task is necessary to change the current state into a new state that satisfies the preconditions.

The transformations and applications are stored in EUREKA’s long-term memory in the form of a semantic network of concepts and relations. To make helpful retrievals, Jones and Langley (2005) use a spreading activation framework similar to Anderson’s (1983) early ACT models. EUREKA activates links of concepts in proportions to the trace strengths attached to the links. By increasing or decreasing the trace strengths, the retrieval patterns are strengthened or weakened, respectively.

Table 12.2 lists the 10 psychological findings identified in VanLehn’s (1989) literature review. The first three describe practice effects. Item 1 refers to how people automate the problem-solving process with practice. The rate of learning is fastest at the beginning but slows with more practice, as described in items 2–3. To evaluate EUREKA’s practice effects, Jones and Langley (2005) presented the system with Towers of Hanoi and Blocks World problems. Similar to humans, graphs of EUREKA’s performance showed a rapid decrease on several measures (number of attempts, total search effort, and productive search effort) after the first and second trials, and it remained fairly constant for the remaining trials. The data indicate that EUREKA had difficulty solving the problems on the first trial but quickly improved after the first trial. Once productive trace links are strengthened, the problem-solving process becomes more automatic.

Item 4 describes differences in improvement across intradomain problems that vary in complexity. Assuming that difficult problems are a composite of simple problems, transfer can occur from simple problems to difficult problems, but not vice versa. To test this prediction, EUREKA again was given Towers of Hanoi and Blocks World problems that became increasingly difficult to solve. In the control condition, each trial was run separately. In the test condition, trials were run continuously, allowing EUREKA to store information from previous trials. EUREKA struggled to solve the problems as they became more difficult in the control condition. However, for the test condition, the system was able to solve even the most difficult problems by using analogy to previous solutions.

Based on VanLehn (1989) .

Negative transfer, in which previous learning makes new learning more difficult, rarely occurs, as stated in item 5 of Table 12.2 . An exception, however, is the set effect or Einstellung (item 6) demonstrated in Luchins’s (1942) water jug task that requires obtaining a specified amount of water by filling and emptying jugs of varying sizes. Luchins found that when people solved practice problems with complex solutions, they failed to discover simpler solutions for the test problems. Similar to the human subjects, EUREKA failed to find simpler solutions after the system solved more complex problems. The results can be explained by the fact that EUREKA continues to use operators that have been successful.

Analogy can also be used to solve isomorphic problems from different domains. To evaluate how EUREKA represents interdomain transfer, Jones and Langley (2005) gave it Holyoak and Koh’s (1987) radiation and broken-light problems. The test group was presented with Duncker’s (1945) radiation problem in which a patient with a tumor must be saved by using X-rays. The transfer problem consisted of a broken light bulb that had to be repaired by using laser beams. The test group was able to solve the transfer problem more successfully than a control group that did not receive the light bulb problem.

In EUREKA’s simulation, the light bulb problem was given first, and the radiation problem was used as the analogous problem. The results showed that EUREKA successfully solved the radiation problem 50% of the time in the control condition and 80% of the time in the test condition. This improvement suggests the use of analogical reasoning to transfer solutions, similar to intradomain transfer.

Intradomain and interdomain transfer differ in that concepts and relations in the current problem and the analogous problem are semantically further apart for interdomain transfer. Therefore, for interdomain transfer, retrieval may be more difficult because activation of abstract nodes may be necessary to find solutions. This assumption explains why semantically similar isomorphic problems, such as the tumor and light bulb problems, are easier to solve than semantically dissimilar isomorphic problems (items 7 and 8). The more semantically similar the problems are, the more likely the “correct” activation will occur.

Items 9 and 10 in Table 12.2 state that spontaneous retrieval of solutions is rare and usually only occurs when people use analogies based on surface similarities. Although spontaneous noticing of analogies is uncommon, it can occur with hints. To simulate the effect of hints, the semantic network nodes describing the broken bulb were activated before the system attempted to solve the radiation problem. Compared to the previous trials in which the hint was not given, the activation of relevant nodes greatly reduced the number of attempts and search efforts. The strengthened activation of relevant nodes helped improve the system’s problem-solving performance, as stated in items 9 and 10.

CLARION is an integrative cognitive architecture that consists of a top-level explicit representation and a bottom-level implicit representation ( Sun & Zhang, 2006 ). Explicit knowledge is represented by easily interpretable symbols that have clear conceptual meaning. Implicit knowledge is represented by a subsymbolic distributed representation within a back-propagation network. In contrast to an explicit memory that encodes rules as all or none, implicit memory supports a more gradual accumulation of knowledge

Heile and Sun (2010) subsequently developed the explicit-implicit interaction (EII) theory based on CLARION to analyze the four stages of problem solving proposed in Wallas’s (1926) influential book The Art of Thought . Preparation is the initial search for a solution, incubation is a period of inactivity following an impasse, illumination (or insight) is a sudden discovery of a possible solution, and verification is a determination of whether the discovered solution is valid.

The EII theory distinguishes between explicit processing based on well-defined rules and implicit processing based on associations. Most problems elicit both implicit and explicit processing. The integration of conclusions from both types of processing influences an internal confidence level that measures the probability of finding the solution.

The theory postulates that the initial preparation phase is predominately rule-based processing as people respond to verbal instructions, form representation of the problem, and establish goals. In contrast, the second incubation state is predominately implicit processing in which people may not consciously think about the problem. The third stage, insight, occurs when the internal confidence level crosses a threshold that makes the output available for verbal report. The final verification stage, like the initial stage, requires primarily explicit processing to evaluate the potential of the discovered solution.

The importance of implicit processes in solving insight problems is illustrated by the success of solving the following problem from Schooler, Ohlsson, and Brooks (1993) :

A dealer in antique coins got an offer to buy a beautiful bronze coin. The coin had an emperor’s head on one side and the date 544 B.C. stamped on the other. The dealer examined the coin, but instead of buying it, he called the police. Why?

After working on the problem for 2 minutes in Schooler’s experiment, half of the participants verbalized their strategies while the remainder worked on an unrelated task. After returning to the problem, 36% of the former group and 46% of the latter group solved the problem. CLARION simulates these findings by assuming that the explicit process of verbalizing strategies disrupts the implicit process that can result in insight.

The goal of CLARION and EUREKA is to propose computational models that can provide theoretical explanations of research on human problem solving. In contrast, an early objective in the evaluation of ACT was to evaluate its theoretical assumptions by designing cognitive tutors to improve instruction ( Anderson, Boyle, & Reiser, 1985 ). An extensive ongoing project at Carnegie Mellon University has continued to design intelligent tutoring systems for teaching topics such as algebra, high school geometry, genetics, and computer programming ( Koedinger & Corbett, 2006 ).

ACT consists of a set of assumptions about both declarative and procedural knowledge. The assumptions about declarative knowledge emphasize the representation and organization of factual information. The assumptions about procedural knowledge emphasize how people use this knowledge to carry out various tasks. This part of the theory consists of production rules that specify which action should be performed under a particular set of conditions and have the form IF <condition> THEN <action>. The condition typically states a goal, and the action specifies a potential way to achieve the goal. Production rules were formulated by Newell and used in his cognitive architecture SOAR ( Laird, Newell, & Rosenbloom, 1987 ). The goal of the production rules in ACT, however, is to model human cognition.

One of the initial cognitive tutors helped students learn the programming language LISP. The major theoretical assumptions underlying the construction of the LISP tutor include the following ( Anderson, 1990 ):

Production rules . A skill such as programming can be decomposed into a set of production rules.

Skill complexity . Hundreds of production rules are required to learn a complex skill. This assumption is consistent with the domain-specific view of knowledge.

Hierarchical goal organization . All productions are organized by a hierarchical goal structure in which subgoals are helpful in accomplishing goals.

Declarative origins of knowledge . All knowledge begins in some declarative representation, typically acquired from instruction or example. Before people practice solving problems, they are instructed in how to solve problems.

Compilation of procedural knowledge . Solving problems requires more than being told about how to solve problems. Problem solvers have to convert this declarative knowledge into efficient procedures for solving specific problems.

The LISP tutor consisted of 1,200 production rules that model student performances on programming problems. It covered all the basic concepts of LISP during a full-semester, self-paced course at Carnegie Mellon University. Students who worked on problems with the LISP tutor generally received one letter grade higher on exams than did students who had not worked with the tutor.

Both ACT theory ( Anderson, 2007 ) and cognitive tutors have continued to evolve. The most extensive application of the cognitive tutors has been to mathematics classes, and, by 2007, data had been collected from more than 7,000 students in pre-algebra classes ( Ritter, Anderson, Koedinger, & Corbett, 2007 ). The curriculum includes both a textbook and software so students can divide their time between the classroom (typically 3 days a week) and a computer lab (typically 2 days a week).

The primary source of declarative knowledge is worked examples that show problem solutions ( Anderson & Fincham, 1994 ). Although the presentation of worked examples has typically occurred in the classroom rather than in the computer lab, interweaving worked examples with practice problems has been particularly effective ( Pashler et al., 2007 ). This can be achieved by adding worked examples to the cognitive tutor and requiring that students solve a practice problem on the cognitive tutor after studying each worked example ( Reed, Corbett, Hoffman, Wagner, & MacLaren, 2013 ).

Other recent work to improve the cognitive tutor provides support for seeking help. Students can request hints but occasionally either do not take advantage of this feature or exploit it by requesting so many hints that the tutor does most of the problem solving. Ideally, learners should develop strong metacognitive skills in which they become proficient at requesting the appropriate amount of help. Such training is provided by the help tutor , which has been integrated into the geometry cognitive tutor ( Roll, Aleven, McLaren, & Koedinger, 2011 ). Results showed that this additional assistance not only improved help-seeking skills for solving geometry problems but transferred to a different topic a month later ( Roll et al., 2011 ).

Future Directions

The typical research paradigm for studying problem solving requires individuals to find a single solution. We still have much to learn by using this paradigm, but our knowledge of problem solving would be broadened by investigating a greater variety of topics such as the value of multiple solutions to a problem ( Reed, 2012 ; Rittle-Johnson et al., 2009 ; Schoenfeld, 1985 ), including understanding an alternative solution ( Greeno & van de Sande, 2007 ). Alternative solutions to a problem reveal the problem space of possible solutions. Other underinvestigated topics include (1) problems with insufficient information, (2) estimated answers, (3) complex problem solving, and (4) collaborative problem solving.

A useful skill outside the classroom is the ability to identify problems that have missing information required for a solution. Perhaps because students do not expect to be assigned such problems, they require a hint to identify them. The hint helped high math ability students discover the missing information, but students with moderate ability required familiar cover stories ( Rehder, 1999 ). Other problems provide only enough information to constrain correct answers:

Morita has five friends and Takeda has seven friends. They decide to throw a party together and invite all their friends. All friends are present. How many friends are there at the party?

An instructional example was moderately helpful in reducing single answers and increasing the number of two answers or, more appropriately, a range of possible answers ( Kinda, 2012 ).

Some problems provide enough information for an estimate:

An athlete’s best time to run a mile is 4 minutes and 7 seconds. About how long would it take him to run 3 miles? ( Greer, 1993 )

Research on these kinds of problems has focused on the incorrect application of proportional reasoning ( Verschaffel, Greer, & de Corte, 2000 ), but we need more information on how people use proportional reasoning as a first step toward making reasonable estimates. Estimated answers are important because people often base their decisions on estimates rather than on precise calculations. Estimates are also helpful in evaluating whether a calculated answer is correct. Checking a calculation is recommended when the answer appears unreasonable.

We also need more information on how to help people improve their estimates. The animation tutor provides simulations of people’s estimates so they can improve their estimates of the time to fill a tank, paint a fence, or complete a round trip ( Reed, 2005 ). Although both the American Association for the Advancement of Science ( AAAS, 1993 ) and the National Council of Teachers of Mathematics ( NCTM, 2000 ) have stressed the importance of estimated answers, their recommendations have had a limited impact on instruction.

The topic of complex problem solving is slowly becoming integrated with the more mainstream research and theory discussed in this chapter. Complex problem solving emerged approximately 30 years ago in Europe as a new topic of investigation ( Funke, 2010 ). The problems are formulated in computer-simulated microworlds (MicroDYN) that require discovering causal relations between input and output variables. In one application labeled Handball Training, the input variables were three different training procedures, and the output variables were motivation, power of the throw, and exhaustion ( Wustenberg, Greiff, & Funke, 2012 ). Participants attempted to reach specified target goals in the output variables by adjusting the values of the three training procedures. Performance on this task explained variance in grade point average beyond reasoning ability as measured by scores on Raven’s Advanced Progressive Matrices ( Wustenberg et al., 2012 ).

Another topic that is receiving increased attention is collaborative problem solving, in part assisted by the 2006 launch of the International Journal of Computer-supported Collaborative Learning ( Stahl & Hesse, 2006 ). An example article from this journal is the Engelmann and Hesse (2010) study in which three group members, working at separate computers, had to determine which pesticide and fertilizer to use to rescue a spruce forest. Each member of the group was given both relevant and irrelevant information to construct a concept map of shared information. Groups who initially had access to the knowledge of other group members started significantly earlier in discussing the problems and solved the fertilizer problem significantly sooner.

In conclusion, although research on the individual solutions of individual problem solvers will continue to be a major focus, research on multiple solutions, problems with insufficient information, estimated answers, complex problem solving, and collaborative problem solving will expand and enrich our knowledge.

AAAS. ( 1993 ). Benchmarks for science literacy . New York: Oxford University Press.

Google Scholar

Google Preview

Anderson, J. R. ( 1983 ). The architecture of cognition . Cambridge, MA: Harvard University Press.

Anderson, J. R. ( 1990 ). Analysis of student performance with the LISP tutor. In N. Frederiksen , R. Glaser , A. Lesgold , & M. Shafto (Eds.), Diagnostic monitoring of skill and knowledge acquisition (pp. 27–50). Hillsdale, NJ: Erlbaum.

Anderson, J. R. ( 2007 ). How can the human mind occur in the physical universe? Oxford, UK: Oxford University Press.

Anderson, J. R. , Boyle, C. F. , & Reiser, B. J. ( 1985 ). Intelligent tutoring systems.   Science , 228, 456–462.

Anderson, J. R. , Byrne, M. D. , Douglass, S. , Lebeire, C. , & Qin, Y. ( 2004 ). An integrated theory of the mind.   Psychological Review , 4, 1036–1060.

Anderson, J. R. , & Fincham, J. M. ( 1994 ). Acquisition of procedural skills from examples.   Journal of Experimental Psychology: Learning , Memory , and Cognition , 20, 1322–1340.

Ash, I. K. , & Wiley, J. ( 2006 ). The nature of restructuring in insight: An individual-differences approach.   Psychonomic Bulletin & Review , 13, 66–73.

Bassok, M. , & Novick, L. R. ( 2012 ). Problem solving. In K. J. Holyoak & R. G. Morrison (Eds.), Oxford handbook of thinking and reasoning (Chapter 21, pp. 413-422). New York: Oxford University Press.

Brewer, W. R. , & Nakamura, G. V. ( 1984 ). The nature and function of schemas. In R. S. Wyer & T. S. Srull (Eds.), Handbook of social cognition , pp. 119-160. Hillsdale, NJ: Erlbaum.

Catrambone, R. , & Holyoak, K. J. ( 1989 ). Overcoming contextual limitations on problem-solving transfer.   Journal of Experimental Psychology: Learning , Memory and Cognition , 15, 1147–1156.

Cheng, P. W. , Holyoak, K. J. , Nisbett, R. E. , & Oliver, L. M. ( 1986 ). Pragmatic versus syntactic approaches to training deductive reasoning.   Cognitive Psychology , 18, 293–328.

Chi, M. T. H. , Glaser, R. , & Rees, E. ( 1982 ). Expertise in problem solving. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 1 , pp. 7–75). Mahwah, NJ: Erlbaum.

Dane, E. ( 2010 ). Reconsidering the trade-off between expertise and flexibility: A cognitive entrenchment perspective.   Academy of Management Review , 35, 579–603.

Duncker, K. ( 1945 ). On problem solving.   Psychological Monographs , 58 (5, Whole No. 270, pp. i-113).

Engelmann, T. , & Hesse, F. W. ( 2010 ). How digital concept maps about the collaborators’ knowledge and information influence computer-supported collaborative problem solving.   Computer-supportive Collaborative Learning , 5, 299–319.

Ernst, G. W. , & Newell, A. ( 1969 ). GPS: A case study in generality and problem solving. New York: Academic Press.

Funke, J. ( 2010 ). Complex problem solving: A case for complex cognition.   Cognitive Processing , 11, 133–142.

Gentner, D. , & Gentner, D. R. ( 1983 ). Flowing waters or teeming crowds: Mental models of electricity. In D. Gentner & A. L. Stevens (Eds.), Mental models, pp. 99-129. Mahwah, NJ: Lawrence Erlbaum.

Gentner, D. , Lowenstein, J. , Thompson, L. , & Forbus, K. ( 2009 ). Reviving inert knowledge: Analogical encoding supports relational retrieval of past events. Cognitive Science , 33, 1343–1382.

Gick, M. , & Holyoak, K. J. ( 1980 ). Analogical problem solving.   Cognitive Psychology , 15, 1–38.

Gick, M. , & Holyoak, K. J. ( 1983 ). Schema induction and analogical transfer.   Cognitive Psychology , 15, 1–38.

Greeno, J. G. ( 1978 ). Natures of problem solving abilities. In W. K. Estes (Ed.), Handbook of learning and cognition (Vol. 5, pp. 239-270). Hillsdale, NJ: Elbaum.

Greeno, J. G. , & van de Sande, C. ( 2007 ). Perspectival understanding of conceptions and conceptual growth in interaction.   Educational Psychologist , 42, 9–23.

Greer, B. ( 1993 ). The mathematical modeling perspective on wor(l)d problems.   Journal of Mathematical Behavior , 12, 239–250.

Griggs, R. A. , & Cox, J. R. ( 1982 ). The elusive thematic-materials effect in Wason’s selection task.   British Journal of Psychology , 73, 407–420.

Heile, S. , & Son, R. ( 2010 ). Incubation, insight, and creative problem solving: A unified theory and a connectionist model.   Psychological Review , 117, 994–1024.

Holyoak, K. J. , & Koh, K. ( 1987 ). Surface and structural similarity in analogical transfer.   Memory & Cognition , 15, 332–340.

Holyoak, K. J. , & Morrison, R. G. ( 2012 ). Thinking and reasoning: A reader’s guide. In K. J. Holyoak & R. G. Morrison (Eds.), Oxford handbook of thinking and reasoning (pp. 1-7). New York: Oxford University Press.

Hummel, J. E. , & Holyoak, K. J. ( 1997 ). Distributed representations of structure: A theory of analogical access and mapping.   Psychological Review , 104, 427–466.

Johnson-Laird, P. N. , Legrenzi, P. , & Legrenzi, M. S. ( 1972 ). Reasoning and a sense of reality.   British Journal of Psychology , 63, 395–400.

Jones, R. M. , & Langley, P. ( 2005 ). A constrained architecture for learning and problem solving.   Computational Intelligence , 21, 480–502.

Kinda, S. ( 2012 ). Generating multiple answers for a word problem with insufficient information.   Instructional Science, 40, 1021-1031.

Kintsch, W. , & Greeno, J. G. ( 1985 ). Understanding and solving word arithmetic problems.   Psychological Review, 92, 109–129.

Knoblich, G. , Ohlsson, S. , Haider, H. , & Rhenius, D. ( 1999 ). Constraint relaxation and chunk decomposition in insight problem solving.   Journal of Experimental Psychology: Learning , memory and cognition , 25, 1534–1555.

Koedinger, K. R. , & Corbett, A. ( 2006 ). Cognitive tutors: Technology bringing learning sciences to the classroom. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 61–77). New York: Cambridge University Press.

Kohler, W. ( 1925 ). The mentality of apes . New York: Harcourt.

Kohler, W. ( 1947 ). Gestalt psychology . New York: Liveright.

Laird, J. E. , Newell, A. , & Rosenbloom, P. S. ( 1987 ). SOAR: An architecture for general intelligence.   Artificial Intelligence , 33, 1–64.

Luchins, A. S. ( 1942 ). Mechanization in problem solving.   Psychological Monographs , 54(248).

Marshall, S. P. ( 1995 ). Schemas in problem solving . New York: Cambridge University Press.

Metcalfe, J. , & Wiebe, D. ( 1987 ). Intuition in insight and noninsight problem solving.   Memory & Cognition , 15, 238–246.

NCTM. ( 2000 ). Principles and standards for school mathematics . Reston, VA: National Council of the Teacher of Mathematics.

Newell, A. , Shaw, J. C. , & Simon, H. A. ( 1958 ). Elements of a theory of human problem solving.   Psychological Review , 65, 151–166.

Newell, A. , & Simon, H. A. ( 1972 ). Human problem solving . Englewood Cliffs, NJ: Prentice-Hall.

Pashler, H. , Bain, P. , Bottge, B. , Graesser, A. , Koedinger, K. , McDaniel, M. , & Metcalfe, J. ( 2007 ). Organizing instruction and study to improve student learning. Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.

Reed, S. K. ( 2005 ). From research to practice and back: The Animation Tutor project.   Educational Psychology Review , 17, 55–82.

Reed, S. K. ( 2012 ). Learning by mapping across situations.   The Journal of the Learning Sciences, 21, 354-398.

Reed, S. K. , Corbett, A. , Hoffman, B. , Wagner, A. , & MacLaren, B. ( 2013 ). Effect of worked examples and Cognitive Tutor training on constructing equations.   Instructional Science , 41, 1-24.

Reed, S. K. , Ernst, G. W. , & Banerji, R. ( 1974 ). The role of analogy in transfer between similar problem states.   Cognitive Psychology , 6, 436–450.

Reed, S. K. , Stebick, S. , Comey, B. , & Carroll, D. ( 2012 ). Finding similarities and differences in the solutions of word problems.   Journal of Educational Psychology, 104, 636-646.

Rehder, B. ( 1999 ). Detecting unsolvable algebra word problems.   Journal of Educational Psychology , 91, 669–683.

Riley, M. , Greeno, J. G. , & Heller, J. I. ( 1983 ). Development of children’s problem-solving ability in arithmetic. In H. P. Ginsberg (Ed.), The development of mathematical thinking (pp. 153–196). New York: Academic Press.

Ritter, S. , Anderson, J. R. , Koedinger, K. R. , & Corbett, A. ( 2007 ). Cognitive Tutor: Applied research in mathematics education.   Psychonomic Bulletin & Review , 14, 249–255.

Rittle-Johnson, B. , Star, J. R. , & Durkin, K. ( 2009 ). The importance of prior knowledge when comparing examples: Influences on conceptual and procedural knowledge of equation solving.   Journal of Educational Psychology , 101, 836–852.

Roll, I. , Aleven, V. , McLaren, B. M. , & Koedinger, K. R. ( 2011 ). Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system.   Learning and Instruction , 21, 267–280.

Schoenfeld, A. H. ( 1985 ). Mathematical problem solving . Orlando: Academic Press.

Schooler, J. W. , Ohlsson, S. , & Brooks, K. ( 1993 ). Thoughts beyond words: When language overshadows insight.   Journal of Experimental Psychology: General , 122, 166–183.

Sherin, B. L. ( 2001 ). How students understand physics equations.   Cognition and Instruction , 19, 479–541.

Silver, E. ( 1981 ). Recall of mathematical problem information: Solving related problems.   Journal for Research in Mathematics Education , 12, 54–64.

Simon, H. A. , & Reed, S. K. ( 1976 ). Modeling strategy shifts in a problem-solving task.   Cognitive Psychology , 8, 86–97.

Spiro, R. J. , Feltovich, P. J. , Coulson, R. L. , & Anderson, D. ( 1989 ). Multiple analogies for complex concepts: Antidotes for analogy-induced misconception in advanced knowledge acquisition. In S. Vosniadou & A. Ortony (Eds.), Similarity and analogical reasoning (pp. 498–531). Cambridge, MA: Cambridge University Press.

Stahl, G. , & Hesse, F. ( 2006 ). ijCSCL—a journal for research in CSCL.   International Journal of Computer-supported Collaborative Learning , 1, 3–7.

Sun, R. , & Zhang, X. ( 2006 ). Accounting for a variety of reasoning data within a cognitive architecture.   Journal of Experimental & Theoretical Artificial Intelligence, 18, 169–191.

VandenBoss, G. R. (Ed.). ( 2006 ). APA dictionary of psychology . Washington, DC: American Psychological Association.

VanLehn, K. ( 1989 ). Problem solving and cognitive skill acquisition. In M. I. Posner (Ed.), Foundations of cognitive science (pp. 526–579). Cambridge, MA: MIT Press.

Verschaffel, L. , Greer, B. , & de Corte, E. (Eds.). ( 2000 ). Making sense of word problems . Heereweg, The Netherlands: Swets & Zeitlinger.

Wallas, G. ( 1926 ). The art of thought . New York: Harcourt, Brace.

Wason, P. C. , & Johnson-Laird, P. N. ( 1972 ). Psychology of reasoning: Structure and content . Cambridge, MA: Harvard University Press.

Wason, P. C. , & Shapiro, D. ( 1971 ). Natural and contrived experience in a reasoning problem.   Quarterly Journal of Experimental Psychology , 23, 62–71.

Wustenberg, S. , Greiff, S. , & Funke, J. ( 2012 ). Complex problem solving—more than reasoning? Intelligence , 40, 1–14.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 04 February 2019

Intelligent problem-solvers externalize cognitive operations

• Bruno R. Bocanegra   ORCID: orcid.org/0000-0003-0158-2340 1 , 2 ,
• Fenna H. Poletiek 2 , 3 ,
• Bouchra Ftitache 4 &
• Andy Clark 5  

Nature Human Behaviour volume  3 ,  pages 136–142 ( 2019 ) Cite this article

18 Citations

219 Altmetric

Metrics details

• Human behaviour
• Intelligence

Humans are nature’s most intelligent and prolific users of external props and aids (such as written texts, slide-rules and software packages). Here we introduce a method for investigating how people make active use of their task environment during problem-solving and apply this approach to the non-verbal Raven Advanced Progressive Matrices test for fluid intelligence. We designed a click-and-drag version of the Raven test in which participants could create different external spatial configurations while solving the puzzles. In our first study, we observed that the click-and-drag test was better than the conventional static test at predicting academic achievement of university students. This pattern of results was partially replicated in a novel sample. Importantly, environment-altering actions were clustered in between periods of apparent inactivity, suggesting that problem-solvers were delicately balancing the execution of internal and external cognitive operations. We observed a systematic relationship between this critical phasic temporal signature and improved test performance. Our approach is widely applicable and offers an opportunity to quantitatively assess a powerful, although understudied, feature of human intelligence: our ability to use external objects, props and aids to solve complex problems.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Brain activity links performance in science reasoning with conceptual approach

Pathfinder: a gamified measure to integrate general cognitive ability into the biological, medical, and behavioural sciences

Statistical and sequence learning lead to persistent memory in children after a one-year offline period

Code availability.

The routines/code that were used to perform the statistical analyses in this study are available from the corresponding author upon request. For the routine/code that was used for simulating the dual-mode and single-mode problem-solvers see Supplementary Code .

Data availability

The data that support the findings of this study are available from the corresponding author upon request.

Jensen A. R. The G Factor: The Science of Mental Ability. (Praeger, Westport, CT, USA, 1998).

Deary, I. J., Strand, S., Smith, P. & Fernandes, C. Intelligence and educational achievement. Intelligence 35 , 13–21 (2007).

Article   Google Scholar  

Kyllonen, P. C. & Christal, R. E. Reasoning ability is (little more than) working-memory capacity?! Intelligence 14 , 389–433 (1990).

Engle, R. W., Tuholski, S. W., Laughlin, J. E. & Conway, A. R. Working memory, short-term memory, and general fluid intelligence: a latent-variable approach. J. Exp. Psychol. Gen. 128 , 309–331 (1999).

Article   CAS   Google Scholar  

Duncan, J. et al. A neural basis for general intelligence. Science 289 , 457–460 (2000).

Conway, A. R., Cowan, N., Bunting, M. F., Therriault, D. J. & Minkoff, S. R. A latent variable analysis of working memory capacity, short-term memory capacity, processing speed, and general fluid intelligence. Intelligence 30 , 163–183 (2002).

Engle, R. W. Working memory as executive attention. Curr. Dir. Psychol. Sci. 11 , 19–23 (2002).

Kyllonen, P. C. In The General Factor of Intelligence: How General Is It? (eds. Sternberg, R. J. & Gigorenko, E. L.) 415–445 (Erlbaum, Mahwah, NJ, USA, 2002).

Baddeley, A. Working memory: looking back and looking forward. Nat. Rev. Neurosci. 4 , 829–839 (2003).

Colom, R., Flores-Mendoza, C. & Rebollo, I. Working memory and intelligence. Pers. Indiv. Differ. 34 , 33–39 (2003).

Conway, A. R., Kane, M. J. & Engle, R. W. Working memory capacity and its relation to general intelligence. Trends Cogn. Sci. 7 , 547–552 (2003).

Gray, J. R., Chabris, C. F. & Braver, T. S. Neural mechanisms of general fluid intelligence. Nat. Neurosci. 6 , 316–322 (2003).

Olesen, P. J., Westerberg, H. & Klingberg, T. Increased prefrontal and parietal activity after training of working memory. Nat. Neurosci. 7 , 75–79 (2004).

Kane, M. J., Hambrick, D. Z. & Conway, A. R. A. Working memory capacity and fluid intelligence are strongly related constructs. Psychol. Bull. 131 , 66–71 (2005).

Jaeggi, S. M., Buschkuehl, M., Jonides, J. & Perrig, W. J. Improving fluid intelligence with training on working memory. Proc. Natl Acad. Sci. USA 105 , 6829–6833 (2008).

Hutchins, E. Cognition in the Wild . (MIT Press, Cambridge, MA, USA, 1995).

Google Scholar  

Clark, A. & Chalmers, D. The extended mind. Analysis 58 , 7–19 (1998).

Clark, A. An embodied cognitive science? Trends Cogn. Sci. 3 , 345–351 (1999).

Giere, R. in The Cognitive Bases of Science (ed.s Carruthers, P., Stitch, S. & Siegal, M.) 285–299 (Cambridge Univ. Press, Cambridge, UK, 2002).

Clark, A. Supersizing the Mind: Action, Embodiment, and Cognitive Extension . (Oxford Univ. Press, Oxford, 2008).

Rowlands, M. The New Science of the Mind: From Extended Mind to Embodied Phenomenology. (MIT Press, Cambridge, MA, USA, 2010).

Bocanegra, B. R. Troubling anomalies and exciting conjectures: a bipolar model of scientific discovery. Emot. Rev. 9 , 155–162 (2017).

Lee, K., & Karmiloff-Smith, A. In Perceptual and Cognitive Development (eds. Gelman R. et al.) 185–211 (Academic Press, New York, 1996).

Mithen, S. In Evolution and the Human Mind (eds. Carruthers, P. & Chamberlain, A.) 207–217 (Cambridge Univ. Press, Cambridge, UK, 2002).

Clark, A. Natural-born Cyborgs: Minds, Technologies and the Future of Human Intelligence . (Oxford Univ. Press, Oxford, UK, 2003).

Risko, E. F. & Gilbert, S. J. Cognitive offloading. Trends Cogn. Sci. 20 , 676–688 (2016).

Risko, E. F. & Dunn, T. L. Storing information in-the-world: metacognition and cognitive offloading in a short-term memory task. Conscious. Cogn. 36 , 61–74 (2015).

Gilbert, S. J. Strategic offloading of delayed intentions into the external environment. Q. J. Exp. Psychol. 68 , 971–992 (2015).

Vallée-Tourangeau, F., Euden, G. & Hearn, V. Einstellung defused: interactivity and mental set. Q. J. Exp. Psychol. 64 , 1889–1895 (2011).

Vallée-Tourangeau, F., Steffensen, S. V., Vallée-Tourangeau, G. & Sirota, M. Insight with hands and things. Acta Psychol. 170 , 195–205 (2016).

Weller, A., Villejoubert, G. & Vallée-Tourangeau, F. Interactive insight problem solving. Think. Reasoning 17 , 424–439 (2011).

Kirsh, D. & Maglio, P. On distinguishing epistemic from pragmatic action. Cognitive Sci. 18 , 513–549 (1994).

Kirsh, D. Thinking with external representations. AI Soc. 25 , 441–454 (2010).

Duncan, J., Chylinski, D., Mitchell, D. J. & Bhandari, A. Complexity and compositionality in fluid intelligence. Proc. Natl Acad. Sci. USA 114 , 5295–5299 (2017).

Kaplan, R. & Saccuzzo, D. Psychological Testing: Principles, Applications, and Issues. 8th edn. (Cengage, Boston, 2012).

Barabasi, A. L. The origin of bursts and heavy tails in human dynamics. Nature 435 , 207–211 (2005).

Tomasello, M. The Cultural Origins of Human Cognition (Harvard Univ. Press, Cambridge, MA, USA, 2009).

Goodale, M. Thinking outside the box. Nature 457 , 539–539 (2009).

Download references

Acknowledgements

The authors received no specific funding for this work. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and affiliations.

Department of Psychology, Educational, and Child Studies, Erasmus University Rotterdam, Rotterdam, the Netherlands

Bruno R. Bocanegra

Institute of Psychology, Leiden University, Leiden, the Netherlands

Bruno R. Bocanegra & Fenna H. Poletiek

Max Planck Institute for Psycholinguistics, Nijmegen, the Netherlands

Fenna H. Poletiek

Institute for Mental Health Care GGZ Rivierduinen, Leiden, the Netherlands

Bouchra Ftitache

School of Philosophy, Psychology, and Language Sciences, University of Edinburgh, Edinburgh, UK

You can also search for this author in PubMed   Google Scholar

Contributions

B.R.B., F.H.P. and B.F. designed the experiments. B.R.B. carried out the experiments, simulations and statistical analyses. B.R.B., F.H.P., B.F. and A.C. wrote the paper.

Corresponding author

Correspondence to Bruno R. Bocanegra .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information.

Supplementary Methods, Supplementary Results, Supplementary Tables 1 and 2, Supplementary Note, Supplementary Figs 1–10, and Supplementary References

Reporting Summary

An explanation of how the Supplementary Code can be run.

Supplementary Code

An excel file that allows simulations of the data, noted in the SI and described in the SI Guide.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Bocanegra, B.R., Poletiek, F.H., Ftitache, B. et al. Intelligent problem-solvers externalize cognitive operations. Nat Hum Behav 3 , 136–142 (2019). https://doi.org/10.1038/s41562-018-0509-y

Download citation

Received : 28 November 2017

Accepted : 05 December 2018

Published : 04 February 2019

Issue Date : February 2019

DOI : https://doi.org/10.1038/s41562-018-0509-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Extended animal cognition.

• Marco Facchin
• Giulia Leonetti

Synthese (2024)

Extended plant cognition: a critical consideration of the concept

• André Geremia Parise
• Michael Marder

Theoretical and Experimental Plant Physiology (2023)

How can embodied cognition naturalize bounded rationality?

• Enrico Petracca
• James Grayot

Synthese (2023)

Catch that word: interactivity, serendipity and verbal fluency in a word production task

• Frédéric Vallée-Tourangeau

Psychological Research (2021)

Reliability and Individual Specificity of EEG Microstate Characteristics

• Jia-Hong Gao

Brain Topography (2020)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Different cognitive mechanisms for process-open and process-constrained problem solving

• Original Paper
• Published: 28 May 2022
• Volume 54 , pages 529–541, ( 2022 )

Cite this article

• Li Wang 1 , 2 ,
• Jieying Zeng 3 ,
• Xiaomeng Ran 4 ,
• Zhanling Cui 5 &
• Xinlin Zhou   ORCID: orcid.org/0000-0002-3530-0922 1 , 2  

3 Citations

Explore all metrics

Mathematical problems can be divided into two types, namely, process-open and process-constrained problems. Solving these two types of problems may require different cognitive mechanisms. However, there has been only one study that investigated the differences of the cognitive abilities in process-open and process-constrained problem solving, and these researchers did not consider the influence of other closely related cognitive abilities. Therefore, in the current study we aimed to investigate the different cognitive mechanisms for process-open and process-constrained problem solving while controlling for related cognitive abilities. For this study we recruited 336 fifth grade students (mean age = 10.73 ± 0.32 years old). The results showed that after controlling for age, gender, and other cognitive abilities, inductive reasoning (measured by nonverbal matrix reasoning) contributes only to process-open problem solving; language comprehension (measured by sentence comprehension) contributes only to process-constrained problem solving; spatial ability (measured by paper folding) contributes to both process-open and process-constrained problem solving; and executive function (measured by the Wisconsin card sort test) contributes to neither process-open nor process-constrained problem solving. These results suggest that cognitive abilities related to generating new information (e.g., inductive reasoning, spatial ability) might play an important role in process-open problem solving. In contrast, cognitive abilities related to retrieval prior knowledge (e.g., language comprehension, spatial ability) might play an important role in process-constrained problem solving. The current findings could promote the efficiency of learning and teaching by enabling the designing of different instruction for process-open and process-constrained problem solving according to students’ cognitive characteristics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Mathematical problem posing of elementary school students: the impact of task format and its relationship to problem solving

Problem-solving support and instructional sequence: impact on cognitive load and student performance

Exploring problem conceptualization and performance in STEM problem solving contexts

Availability of data and material.

Data are available upon reasonable request.

Code availability

Codes are available upon reasonable request.

Abedi, J., & Lord, C. (2001). The language factor in mathematics tests. Applied Measurement in Education, 14 (3), 219–234.

Article   Google Scholar  

Agustini, R. Y., Suryadi, D., & Jupri, A. (2017). Construction of open-ended problems for assessing elementary student mathematical connection ability on plane geometry. In International conference on mathematics and science education (Icmsce) (p. 895).

Andersson, U. (2008). Working memory as a predictor of written arithmetical skills in children: The importance of central executive functions. British Journal of Educational Psychology, 78 , 181–203.

Ardila, A., Pineda, D., & Rosselli, M. (2000). Correlation between intelligence test scores and executive function measures. Archives of Clinical Neuropsychology, 15 (1), 31–36.

Bahar, A., & Maker, C. J. (2015). Cognitive backgrounds of problem solving: A comparison of open-ended vs. closed mathematics problems. Eurasia Journal of Mathematics Science and Technology Education, 11 (6), 1531–1546.

Beller, M., & Gafni, N. (2000). Can item format (multiple choice vs. open-ended) account for gender differences in mathematics achievement? Sex Roles, 42 (1–2), 1–21.

Bennett, R. E., Steffen, M., Singley, M. K., Morley, M., & Jacquemin, D. (1997). Evaluating an automatically scorable, open-ended response type for measuring mathematical reasoning in computer-adaptive tests. Journal of Educational Measurement, 34 (2), 162–176.

Boaler, J. (1998). Open and closed mathematics: Student experiences and understandings. Journal for Research in Mathematics Education, 29 (1), 41–62.

Boonen, A. J., Reed, H. C., Schoonenboom, J., & Jolles, J. (2016). It’s not a math lesson—We’re learning to draw! Teachers’ use of visual representations in instructing word problem solving in sixth grade of elementary school. Frontline Learning Research, 4 (5), 55–82.

Cai, J. (2000). Mathematical thinking involved in U.S. and Chinese students’ solving of process-constrained and process-open problems. Mathematical Thinking and Learning, 2 (4), 309–340.

Cai, J., & Cifarelli, V. (2005). Exploring mathematical exploration: How two college students formulated and solved their own mathematical problems? Focus on Learning Problems in Mathematics, 27 (3), 43–72.

Google Scholar  

Cheng, D. Z., Yan, X. X., Gao, Z. J., Xu, K. M., Zhou, X. L., & Chen, Q. (2017). Neurocognitive profiles in childhood absence epilepsy: A focus on cognitive dysfunction associated with the frontal lobe. Journal of Child Neurology, 32 (1), 46–52.

Cifarelli, V., & Cai, J. (2005). The evolution of mathematical explorations in open-ended problem-solving situations. The Journal of Mathematical Behavior, 24 (3–4), 302–324.

Cirino, P. T. (2011). The interrelationships of mathematical precursors in kindergarten. Journal of Experimental Child Psychology, 108 (4), 713–733.

Clarke, D., & Roche, A. (2018). Using contextualized tasks to engage students in meaningful and worthwhile mathematics learning. Journal of Mathematical Behavior, 51 , 95–108.

Clauser, B. E., Harik, P., Margolis, M. J., McManus, I. C., Mollon, J., Chis, L., & Williams, S. (2009). An empirical examination of the impact of group discussion and examinee performance information on judgments made in the Angoff standard-setting procedure. Applied Measurement in Education, 22 (1), 1–21.

Cui, J., Zhang, Y., Cheng, D., Li, D., & Zhou, X. (2017). Visual form perception can be a cognitive correlate of lower level math categories for teenagers. Frontiers in Psychology, 8 , 1336.

Cui, J., Zhang, Y., Wan, S., Chen, C., Zeng, J., & Zhou, X. (2019). Visual form perception is fundamental for both reading comprehension and arithmetic computation. Cognition, 189 , 141–154.

De Smedt, B., & Boets, B. (2010). Phonological processing and arithmetic fact retrieval: Evidence from developmental dyslexia. Neuropsychologia, 48 (14), 3973–3981.

Ekstrom, R. B., Dermen, D., & Harman, H. H. (1976). Manual for kit of factor-referenced cognitive tests (Vol. 102). Educational Testing Service.

Fuchs, L. S., Fuchs, D., Seethaler, P. M., & Craddock, C. (2020). Improving language comprehension to enhance word-problem solving. Reading & Writing Quarterly, 36 (2), 142–156.

Gilbert, S. J., & Burgess, P. W. (2008). Executive function. Current Biology, 18 (3), 110–114.

Hawes, Z., & Ansari, D. (2020). What explains the relationship between spatial and mathematical skills? A review of evidence from brain and behavior. Psychonomic Bulletin and Review, 27 (1), 465–482.

Hegarty, M., & Kozhevnikov, M. (1999). Types of visual–spatial representations and mathematical problem solving. Journal of Educational Psychology., 91 (4), 684–689.

Jimerson, S., Egeland, B., & Teo, A. (1999). A longitudinal study of achievement trajectories: Factors associated with change. Journal of Educational Psychology, 91 (1), 116–126.

Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness (Vol. 6). Harvard University Press.

Kapa, E. (2007). Transfer from structured to open-ended problem solving in a computerized metacognitive environment. Learning and Instruction, 17 (6), 688–707.

Kaplinsky, R. (2019). Open middle math: Problems that unlock student thinking (pp. 6–12). Stenhouse Publishers.

Lin, X. (2020). Investigating the unique predictors of word-problem solving using meta-analytic structural equation modeling. Educational Psychology Review., 33 (1), 1097–1124.

Linn, M. C., & Petersen, A. C. (1985). Emergence and characterization of sex-differences in spatial ability—A meta-analysis. Child Development, 56 (6), 1479–1498.

Lohman, D. F. (1996). Spatial ability and g human abilities: Their nature and measurement (Vol. 97, no. 116, p. 1)

Markina, P. N., & Vladimirov, I. Y. (2019). Executive function role on a stage of impasse in insight problem solving. Psychology-Journal of the Higher School of Economics, 16 (3), 562–570.

Mix, K., Levine, C., Cheng, Y.-L., Young, C., Hambrick, D., Ping, R., & Konstantopoulos, S. (2016). Separate but correlated: The latent structure of space and mathematics across development. Journal of Experimental Psychology. General., 145 (9), 1206.

Molnar, G., Greiff, S., & Csapo, B. (2013). Inductive reasoning, domain specific and complex problem solving: Relations and development. Thinking Skills and Creativity, 9 , 35–45.

Moreno, R., & Mayer, R. E. (1999). Gender differences in responding to open-ended problem-solving questions. Learning and Individual Differences, 11 (4), 355–364.

O’donnell, A. M., Dansereau, D. F., & Hall, R. H. (2002). Knowledge maps as scaffolds for cognitive processing. Educational Psychology Review, 14 (1), 71–86.

Oller, J. W., Jr. (1981). Language as intelligence? Language Learning, 31 (2), 465–492.

Pascual, A. C., Moyano, N., & Robres, A. Q. (2019). The relationship between executive functions and academic performance in primary education: Review and meta-analysis. Frontiers in Psychology, 10 , 1582.

Peng, P., Wang, T. F., Wang, C. C., & Lin, X. (2019). A meta-analysis on the relation between fluid intelligence and reading/mathematics: Effects of tasks, age, and social economics status. Psychological Bulletin, 145 (2), 189–236.

Rasmussen, D., & Eliasmith, C. (2011). A neural model of rule generation in inductive reasoning. Topics in Cognitive Science, 3 (1), 140–153.

Raven, J. C. (1983). Manual for Raven’s progressive matrices and vocabulary scales. Standard progressive matrices . HK Lewis.

Reilly, D., & Neumann, D. L. (2013). Gender-role differences in spatial ability: A meta-analytic review. Sex Roles, 68 (9–10), 521–535.

Resnick, L., & Glaser, R. (1975). Problem solving and intelligence (pp. 205–230). Lawrence Erlbaum Associates.

Romine, C. B., Lee, D., Wolfe, M. E., Homack, S., George, C., & Riccio, C. A. (2004). Wisconsin Card Sorting Test with children: A meta-analytic study of sensitivity and specificity. Archives of Clinical Neuropsychology, 19 (8), 1027–1041.

Rourke, B. P. (1993). Arithmetic disabilities, specific and otherwise. Journal of Learning Disabilities, 26 (4), 214–226.

Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8 (4), 350.

Shi, Z. (2021). Intelligence science: Leading the age of intelligence . Elsevier.

Book   Google Scholar  

Sternberg, R. (1982). Reasoning, problem solving, and intelligence (pp. 225–307). Cambridge University Press.

Stevenson, H., Lee, S., & Stigler, J. (1986). Mathematics achievement of Chinese, Japanese, and American children. Science, 231 (4739), 693–699.

Sung, J., & Wickrama, K. A. S. (2018). Longitudinal relationship between early academic achievement and executive function: Mediating role of approaches to learning. Contemporary Educational Psychology, 54 , 171–183.

Taasoobshirazi, G., & Wang, S. (2016). The performance of the SRMR, RMSEA, CFI, and TLI: An examination of sample size, path size, and degrees of freedom. Journal of Applied Quantitative Methods, 11 (3), 31–39.

Traff, U. (2013). The contribution of general cognitive abilities and number abilities to different aspects of mathematics in children. Journal of Experimental Child Psychology, 116 (2), 139–156.

Van Dyke, J. A., & Johns, C. L. (2012). Memory interference as a determinant of language comprehension. Language and Linguistics Compass, 6 (4), 193–211.

Villeneuve, E. F., Hajovsky, D. B., Mason, B. A., & Lewno, B. M. (2019). Cognitive ability and math computation developmental relations with math problem solving: An integrated, multigroup approach. School Psychological Quarterly, 34 (1), 96–108.

Wai, J., Lubinski, D., & Benbow, C. P. (2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101 (4), 817–835.

Wang, L., Cao, C., Zhou, X., & Qi, C. (2022). Spatial abilities associated with open math problem solving. Applied Cognitive Psychology., 36 (2), 306–317.

Wang, S. H., Lin, M. C., & Liao, C. W. (2014). A virtual experiential learning and students’ ill-structured problem-solving ability. Interacting with Computers, 26 (4), 334–347.

Wei, W., Yuan, H., Chen, C., & Zhou, X. (2012). Cognitive correlates of performance in advanced mathematics. British Journal of Educational Psychology, 82 (1), 157–181.

Williams, B. A., & Pearlberg, S. L. (2006). Learning of three-term contingencies correlates with Raven scores, but not with measures of cognitive processing. Intelligence, 34 , 177–191.

Xie, F., Zhang, L., Chen, X., & Xin, Z. (2020). Is spatial ability related to mathematical ability: A meta-analysis. Educational Psychology Review, 32 (1), 113–155.

Xin, Z. Q., & Zhang, L. (2009). Cognitive holding power, fluid intelligence, and mathematical achievement as predictors of children’s realistic problem solving. Learning and Individual Differences, 19 (1), 124–129.

Yeniad, N., Malda, M., Mesman, J., van Ijzendoorn, M. H., & Pieper, S. (2013). Shifting ability predicts math and reading performance in children: A meta-analytical study. Learning and Individual Differences, 23 , 1–9.

Download references

Acknowledgements

We thank Professor Jinfa Cai for his suggestions in designing experiment and writing manuscript.

This research was supported by a grant from Natural Science Foundation of China (31671151), the 111 Project (BP0719032), and a grant from the Advanced Innovation Center for Future Education (27900-110631111).

Author information

Authors and affiliations.

State Key Laboratory of Cognitive Neuroscience and Learning, Faculty of Psychology, Beijing Normal University, No. 19, Xinjiekouwai St, Haidian District, Beijing, 100875, China

Li Wang & Xinlin Zhou

Advanced Innovation Center for Future Education, Beijing Normal University, Beijing, China

Business School, Beijing Wuzi University, Beijing, China

Jieying Zeng

Dalu Beikou Central Primary School, Tianjin, China

Xiaomeng Ran

College of Education, Hebei Normal University, Shijiazhuang, China

Zhanling Cui

You can also search for this author in PubMed   Google Scholar

Contributions

XZ and JZ designed experiment and revised the manuscript. XR and ZC collected data and analysed data. LW analysed data and wrote the manuscript.

Corresponding author

Correspondence to Xinlin Zhou .

Ethics declarations

Conflict of interest.

We have no known conflict of interest to disclose.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 492 KB)

Rights and permissions.

Reprints and permissions

About this article

Wang, L., Zeng, J., Ran, X. et al. Different cognitive mechanisms for process-open and process-constrained problem solving. ZDM Mathematics Education 54 , 529–541 (2022). https://doi.org/10.1007/s11858-022-01373-3

Download citation

Accepted : 24 April 2022

Published : 28 May 2022

Issue Date : June 2022

DOI : https://doi.org/10.1007/s11858-022-01373-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Inductive reasoning
• Spatial ability
• Language comprehension
• Execution function
• Process-open problems
• Mathematical problem solving
• Find a journal
• Publish with us
• Track your research

Intelligent problem-solvers externalize cognitive operations

Affiliations.

• 1 Department of Psychology, Educational, and Child Studies, Erasmus University Rotterdam, Rotterdam, the Netherlands. [email protected].
• 2 Institute of Psychology, Leiden University, Leiden, the Netherlands. [email protected].
• 3 Institute of Psychology, Leiden University, Leiden, the Netherlands.
• 4 Max Planck Institute for Psycholinguistics, Nijmegen, the Netherlands.
• 5 Institute for Mental Health Care GGZ Rivierduinen, Leiden, the Netherlands.
• 6 School of Philosophy, Psychology, and Language Sciences, University of Edinburgh, Edinburgh, UK.
• PMID: 30944451
• DOI: 10.1038/s41562-018-0509-y

Humans are nature's most intelligent and prolific users of external props and aids (such as written texts, slide-rules and software packages). Here we introduce a method for investigating how people make active use of their task environment during problem-solving and apply this approach to the non-verbal Raven Advanced Progressive Matrices test for fluid intelligence. We designed a click-and-drag version of the Raven test in which participants could create different external spatial configurations while solving the puzzles. In our first study, we observed that the click-and-drag test was better than the conventional static test at predicting academic achievement of university students. This pattern of results was partially replicated in a novel sample. Importantly, environment-altering actions were clustered in between periods of apparent inactivity, suggesting that problem-solvers were delicately balancing the execution of internal and external cognitive operations. We observed a systematic relationship between this critical phasic temporal signature and improved test performance. Our approach is widely applicable and offers an opportunity to quantitatively assess a powerful, although understudied, feature of human intelligence: our ability to use external objects, props and aids to solve complex problems.

Publication types

• Academic Success
• Intelligence / physiology*
• Intelligence Tests
• Neuropsychological Tests
• Problem Solving / physiology*
• Psychomotor Performance / physiology*
• Universities
• Young Adult

Self-regulated learning (SRL) is a critical component of mathematics problem solving. Students skilled in SRL are more likely to effectively set goals, search for information, and direct their attention and cognitive process so that they align their efforts with their objectives. An influential framework for SRL, the SMART model, proposes that five cognitive operations (i.e., searching, monitoring, assembling, rehearsing, and translating) play a key role in SRL. However, these categories encompass a wide range of behaviors, making measurement challenging – often involving observing individual students and recording their think-aloud activities or asking students to complete labor-intensive tagging activities as they work. In the current study, we develop machine-learned indicators of SMART operations, in order to achieve better scalability than other measurement approaches. We analyzed student’s textual responses and interaction data collected from a mathematical learning platform where students are asked to thoroughly explain their solutions and are scaffolded in communicating their problem-solving process to their peers and teachers. We built detectors of four indicators of SMART operations (namely, assembling and translating operations). Our detectors are found to be reliable and generalizable, with AUC ROCs ranging from .76-.89. When applied to the full test set, the detectors are robust against algorithmic bias, performing well across different student populations.

INTRODUCTION

Work over the last two decades has developed automated detectors of a range of behaviors and constructs in students’ interaction with computer-based learning environments [12]. These detectors utilize log data collected from these learning environments to infer the presence or absence of a complex behavior or a construct in student learning. For example, detectors have been built to identify student affect (e.g., [7, 21, 31]), engagement (e.g.,[6, 37]), and problem-solving strategies [43]. Such detectors can be split into two broad categories: detectors for post-hoc analysis and detectors for real-time adaptation. Post-hoc analysis allows researchers to detect constructs retrospectively and subsequently understand their prevalence (e.g., [32]) and conduct further analysis (e.g., [17, 41]). Detectors designed for real-time use facilitate adaptive experiences and real-time feedback [48], as well as reports to teachers[1].

In particular, considerable work has been devoted to detecting and understanding behaviors and strategies involved in self-regulated learning (SRL). By examining student behavior patterns, automated detectors have been developed for a range of SRL related constructs, including help avoidance [3], gaming the system [7], setting goals [5, 14], and planning and tracking progress [14]. However, these constructs have not been clearly linked to any of the growing number of theoretical models of SRL (although [29] is an exception to this) and have mostly been operationalized in terms of high-level strategies that combine several of behaviors treated as separate in SRL theories, rather than the finer-grained behaviors used in those theories [cf. 46, 48]. Capturing fine-grained indicators of key aspects of SRL in terms of these theoretical models may yield a better understanding of the process of SRL and help EDM research make more direct theoretical contributions.

Self-regulation is a critical component of learning, and has been positively associated with learning outcomes [20, 34, 57]. In mathematics problem-solving, students who are skilled in SRL are able to effectively set goals, search for information, and direct their attention and cognitive resources to align their efforts with their objectives [56]. As a result, SRL facilitates the successful problem-solving process [20, 34, 57] and enables students to acquire a deep and conceptual understanding of the embedded knowledge [31]. Given its benefits, theory-based interventions have been developed to promote SRL [21]. However, current SRL assessments, such as self-reports and think-aloud activities, are not sufficient to provide measurement at scale; at the same time, existing scalable SRL assessments based on automated detection in log data are typically not connected back to theory, making it difficult to use them in theory-driven interventions. SRL assessments based on automated detectors have therefore been used in more ad-hoc, system-specific interventions, often with unintended consequences or unexpected patterns of findings [2, 35]

In the current study, we develop automated detectors that identify fine-grained evidence of SRL constructs drawn from theory. This study does so in the context of CueThink, a digital learning application that focuses on enhancing middle school student mathematics problem-solving skills. Through the lens of the SMART model of SRL (described in greater detail below) [53], we identify and operationalize five SRL indicators: numerical representation, contextual representation, strategy orientation, outcome orientation, and data transformation. We then build automated detectors for each indicator, evaluate their performance, and check them for algorithmic bias.

SRL and the SMART Model

Grounded in information processing theory, Winne and Hadwin [55] characterize the process of SRL as four interdependent and recursive stages, in which learners: 1) define the task, 2) set goals and form plans, 3) enact the plans, and 4) reflect and adapt strategies when goals are not met.

The SMART model of SRL [53] was later proposed to further elucidate the processes involved in these four tasks. Specifically, the model separates the “cognitive and behavioral actions applied to perform the task” into five categories: searching , monitoring , assembling , rehearsing , and translating . Each operation describes a way that learners cognitively engage and interact with information. For example, when working on a task, learners direct their attention to particular information (searching) and compare the information with a standard (monitoring), evaluating the relevance or the importance of the information. When relevant information is identified, students relate pieces of information to one another (assembling), in order to create a comprehensive understanding of the problem. When information does not fit into the current problem representation, learners manipulate the ways information is presented in order to find a solution (translating). Throughout the process, working memory is used to actively maintain and reinstate information (rehearsing).

These cognitive operations are an integral aspect of self-regulation: they help determine student success at completing each of the four SRL tasks, which, in turn, influences the progression of the problem-solving process [52]. However, despite the SMART categories' importance for SRL, they are often difficult to observe or measure, as most learning activities (whether online or offline) do not fully reify the cognitive process involved in their learning tasks. Further, these operations may occur non-linearly, and multiple operations can be employed when completing the same task– making the measurement of these constructs challenging.

Challenges in SRL Measurements

SRL has typically been measured using three common approaches: self-reports, think-aloud activities, and log data collected in computer-based learning environments [51]. With traditional self-report studies, students are asked about their SRL process either outside of a task (i.e., before or after completing a task) or while working on a task. In decontextualized self-report (outside of a task), students report on the SRL strategies they plan to use or recall on the strategies they used, using a pre- or post-task survey. Even though this approach is widely used, the nature of surveying cognitive processes outside of the task may lead to inaccuracies in the representation of cognition [51]. For example, when recalling a cognitive process retrospectively, students may aggregate the out-of-context, self-reported experience across numerous tasks, failing to demonstrate the relationship between the task and the corresponding SRL strategies. For this reason, several studies have adopted in-context self-report (e.g., [42, 47, 54]), in which students are asked to tag their SRL strategies as they occur.

Other research has leveraged think-aloud activities that ask students to verbalize their cognitive processes when solving a problem [28]. As with in-context self-reports, think-alouds give researchers an opportunity to identify processes that are contextualized in the problem-solving activity and are approximately concurrent with their occurrences. However, this process can suffer from an observation effect. Students being prompted to discuss their thinking process in real-time may alter that process and not provide an accurate representation of the processes they would engage in naturally [16, 44]. This, in turn, calls into question the validity of findings obtained using this type of measurement and whether they are generalizable to new students and contexts.

Use Log Data to Measure SRL in Computer-Based Learning Environments

Both self-report and think-aloud approaches are labor-intensive and time-consuming, which make them difficult to scale. As such, a third approach, analyzing log data collected from computer-based learning environments, has emerged as a promising way to measure SRL.

Aleven and colleagues [3] designed an exhaustive set of production rules to represent help-seeking behaviors within a geometry learning system, and then compared these rules to student problem-solving steps to determine whether those steps were warranted by the current situation. Similarly, Biswas et al. [14] used sequences of student behaviors to model a range of SRL behaviors, such as monitoring through explanation, self-assessment, tracking progress, and setting learning goals. Additionally, Segedy et al. [45] utilized log data and coherence analysis to assess students’ ability to seek out, interpret, and apply information in an open-ended learning environment, examining if a student’s subsequent action is coherent based on the information presented.

Researchers have also used textual responses within dialogue-based learning systems to measure SRL. Graesser and colleagues [27] used latent semantic analysis to study student conversations with animated pedagogical agents to assess and support SRL. Students who frequently use questions in a conversation can be interpreted as showing initiative, and engagement in monitoring can be inferred when students demonstrate in their responses that they feel they know the answer [26].

However, often the log data collected does not straightforwardly reflect a SRL construct [4]. Researchers must decide what data to use, what constructs to measure, and how to operationalize the constructs with the existing data [30]. SRL, as a process, covers a range of behaviors and strategies, so the constructs can vary depending on how SRL is conceptualized and also based on the design of the activity the learner is participating in. To ensure the validity of the operationalization, it is recommended that the operationalization should be conceptualized in terms of a SRL model and contextualized in the learning environment where the data is generated [51].

Algorithmic Bias

In order to use detectors at scale, we must ensure that they will be valid for the entire populations they are scaled to rather than only subgroups of students. Recent evidence suggests that many published detectors are prone to algorithmic bias, functioning better for some populations of learners than others [11]. However, there has been limited attention to algorithmic bias within the field of educational data mining, where analyses of algorithmic bias are rare and even overall population demographics are only reported in 15% of publications [38]. Verifying detector fairness is particularly pertinent to our study given the diverse student population who use CueThink, making it important to evaluate detector effectiveness across demographic groups before deploying and using the detectors at scale.

Current Study

In the current study, we address the challenges noted above by building automated detectors of SRL constructs from a theory-driven lens. Using a dataset of 79 students as they interacted with the online math learning platform CueThink, we first examine the learning environment understanding how students interact with the platform and the context of where the log data is generated. Based on the context and the log data available, we identify relevant theoretical constructs grounded in the SMART model. In particular, five SRL indicators relating to cognitive operations in the SMART model (in this case, all either assembling or translating ) are identified for investigation.

We use text replay to code student interactions for each defined indicator. These labels are then used as ground truth for machine learning. We distill a variety of features from the log data to represent multiple aspects of a student’s interaction, including the number of responses and the content in the responses. The ground truth and the features are then input into a machine learning process, training a model to emulate human coders’ judgement, making predictions on the presence or absence of a SRL indicator.

We demonstrate that trained detectors provide accurate detection, suitable for real-time use. Finally, we also evaluate, through slicing analysis, the performance of our models across different demographic groups.

Learning Environment

CueThink is a digital learning application that focuses on enhancing middle school student math problem-solving skills, by encouraging students to engage in self-regulated learning and to develop math language to communicate problem-solving processes. CueThink asks students both to solve a math problem and to create a shareable screen-cast video that provides the student’s answer and also demonstrates their problem-solving process. As Figure 1 shows, CueThink structures a problem into a Thinklet , a process that includes four phases— Understand, Plan, Solve, and Review —that closely align with Winne & Hadwin’s model of SRL [55].

Each phase of the Thinklet (outlined in Table 1 and described in more detail below), asks students to focus on a different part of the problem-solving process. While working on a Thinklet , students can move freely across the four phases, including going back to a previous phase or skipping phases.

Starting with the Understand phase , students read a problem and provide text-based responses to three questions: (1) “What do you notice?” (2) “What do you wonder?” and (3) “What is your estimated answer to the problem?” This phase encourages students to actively look for information in the problem and create a representation of the problem space. Thus, students demonstrate their understanding of what they know and what they need to know at this phase.

In the Plan phase, students build on what they have established in the Understand phase by planning how they will solve the problem. Students are first prompted to select what strategies they will use to solve the problem. They may choose from a predefined strategy list (i.e., draw a picture, model with an equation, work backwards, etc.) or define their own strategies. Once the student has selected which strategies they will use, the student is prompted to write a plan on how they will use the strategies to solve the problem.

In the Solve phase, students explain and present their answer . Specifically, they create a screen cast video using an interface that provides them with a whiteboard and mathematical tools (i.e., number lines, ruler, etc.).

In the Review phase, students provide the final answer to the math problem, but also reflect on whether the answer makes sense and whether their communication is clear, using checklists to scaffold their reflection.

Once students have completed the problem, they share their screencast explanation for Peer Review . In this phase, teachers and peers annotate both the textual responses and video, often asking the student for their underlying reasoning or why the student picked specific methods. These annotations are then sent back to the video’s author for possible revision.

* Student activity in the Solve phase is not used in this paper’s analyses

Student Demographics

In this study, 79 students in grade 6 and 7 at a suburban school in the southwestern U.S. used CueThink during the 2020-21 school year. The school contains a diverse student population with around 40% Hispanic or Latino, 40% White, 15% African American, and 5% Asian students. Students’ self-reported demographic information on gender and race/ethnicity was collected. For gender, students could choose to identify as male, female, non-binary, or leave the question blank. For race/ethnicity, options included African America, Hispanic/Latinx, White, Asian, Native American, two or more races, other, or prefer not to say. Students reporting “other” for their race/ethnicity were provided the option to give detail.

CueThink was implemented in six classrooms over multiple weeks, with teachers assigning problems for students to complete in the application. We collected log files that reflect how students use the application and their problem-solving process. On average, students spent 5.2 hours in CueThink and 1.8 hours working on each Thinklet . Specifically, for each problem, encapsulated in a Thinklet , data generated during the problem-solving process that includes the questions students answered and their textual responses at each phase were collected. In this study, we analyzed textual and click-stream data, but did not analyze data from the videos. In total, we collected 349 Thinklets from 79 students working on 24 different problems. Of those 349 Thinklets , not all were first attempts. Students have the opportunity to revise their work, which creates another Thinklet . In those cases, it is possible that students do not go through the entire problem-solving process. Of the total number of Thinklets , 146 were duplicate attempts.

Building Detectors

Building automated detectors of self-regulated behaviors was a multi-step process (detailed in the following subsections). First, we distilled human-readable text replays from log data. Using these text replays, we identified and operationalized qualitative categories that corresponded with SRL constructs, grounding the operationalization in Winne’s SMART model. We then labeled the self-regulated behaviors, generating ground truth data. Feature engineering and feature distillation were conducted and used to train the predictive models. Lastly, we evaluated model performance and checked for algorithmic bias.

Text Replays of Interaction Logs

To facilitate inspection and exploration of the data, we used text replays. This method presents segments of interaction data (referred to as clips) in a human-readable presentation. This process facilitates both initial exploration of the data (such as in section 4.2) along with the final coding process (section 4.3). Clips are then viewed by human coders who label them accordingly [8]. Previous studies have used text replay coding to label student affect, disengagement, and learning strategies, such as gaming the system [10], confrustion [32], player goals [22], and SRL strategies such as whether a student is using a table to plan their analyses [43].This approach demonstrates a similar level of reliability as classroom observations and is 2-6 times faster compared to other methods of generating labels, such as classroom observations, screen replay, and retrospective think/emote-aloud protocols [8].

The length and the grain-size of text replay clips can vary depending on both the available data and the granularity of the predictions the researcher intends to make. Because this study seeks to detect cognitive SRL operations in the problem-solving process, which requires a comprehensive examination across questions and phases, the log files were delineated into clips on the level of entire Thinklets . Each clip contains a student's actions and text-based responses that were submitted as that student worked through the four phases to produce a single Thinklet . (Note that because video data was not available in these replays, our coders did not see information from the Solve phase.) The clips were distilled from log files and presented using a Python window, shown in Figure 2.

Construct Operationalization

To identify constructs to detect, we first examined the clips containing student responses in Thinklets and coded student responses for indicators of SRL—qualitative categories that correspond with SRL constructs (we discuss the details of exactly how the data was coded in section 4.3). The definitions of the indicators we coded were developed through dialogue between the research team and system developers. This process followed the recursive, iterative process used in [49] that includes seven stages: conceptualization of codes, generation of codes, refinement of the first coding system, generation of the first codebook, continued revision and feedback, coding implementation, and continued revision of the codes [49]. The conceptualization of codes included a review of related literature, including several theoretical frameworks and perspectives [13, 15, 24], primarily focusing on the SMART model [53]. Using grounded theory [18], we identified common behaviors that were (1) indicative of SRL as characterized by Winne’s SMART model [53] and (2) salient in the log files. A draft lexicon and multiple criteria were generated for a coding system to help identify these constructs.

Given the learning environment's design and the available data, our efforts focused on defining behaviors related to two categories of cognitive operations (the assembling and translating operations from the SMART model) as they are frequently employed in the initial stages in SRL that help learners define tasks and set goals. Following the process used in [49], two coders (the first and second authors) coded a set of clips together, identified five SRL indicators (i.e., numerical representation, contextual representation, strategy orientation, outcome orientation, and data transformation), outlined the criteria for each indicator, and created a rubric.

The draft coding manual was discussed with all members of the research team and developers and designers at CueThink to build a common understanding of the criteria and constructs being examined as well as the features of the system to gain feedback for further refinement. This process was repeated until the entire team had reached a shared understanding of the criteria and constructs being examined by the codebook. The SRL indicators identified, the criteria, and alignment with the SMART model are included in Table 2.

Numerical and contextual representation consider a learner’s process of creating a problem representation, which often occurs in the initial stage in the problem-solving process (i.e., define the task), outlined in the four-phase model of SRL [55]. In problem representation, learners create a problem space by identifying information they know and information they need to know. The two SRL indicators encode how learners represent and process information in math problems, denoting if numerical components and/or contextual details are noted. We consider both of these processes to reflect assembling in the SMART model as students are creating their representation of the data from the information provided. There may also be overlap with translating in some cases, especially if the question provides a different representation to the one the students use. However, as this is not always the case, we primarily consider both indicators to reflect assembling actions and tag translating actions in a different code (see below).

Strategy and outcome orientation also reflect student assembling behaviors. Both indicators consider how students set their goals and form plans for the problem-solving process. These two indicators demonstrate a difference in focuses (process vs. output).

Lastly, data transformation reflects behaviors that are associated with the translating operation, in which the learner manipulates the ways information is represented to them in the problem to find a solution.

Coding the Data

After constructs were operationalized and defined, we proceeded to code the remainder of the data. Two coders, (the same as in the previous section), completed the text replay coding in three phases: preliminary coding (discussed above), separate coding (two coders per clip; for establishing inter-rater reliability), and individual coding (one coder per clip; for completeness).

The two coders each used the codebook/rubric to code the same set of clips separately. They then compared the labels and computed the inter-rater reliability (IRR) kappa. For constructs with low kappa, the two coders discussed their differences in labeling and conducted another round of coding. This step of separate coding and comparing is repeated until an acceptable reliability is established. After two rounds of coding, as shown in Table 3, the two coders reached an acceptable IRR above 0.60 for all five SRL indicators (M=0.75).

Once the reliability was established, the coders moved on to the individual coding where they split the rest of the clips and coded them individually.

In total, the two coders coded 349 clips. However, in order to consistently examine the entire problem-solving process, 167 clips that were marked incomplete because students stopped before completing the entire problem were excluded. Of the remaining 182 clips, coding resulted in the following distribution of labels: 64% numerical representation, 77% contextual representation, 8% strategy orientation, 72% outcome orientation, and 73% data transformation. These were produced by 72 students, who, on average, each contributed 3 clips (max=4, min=1, median=3).

Feature Distillation

Two sets of features were distilled to build the detectors. Both sets of features consist solely of features that can be extracted and used in real-time. The first set of features were designed to provide an overview of a Thinklet by examining the number of responses in a Thinklet . These features (N = 10) were distilled at the Thinklet level. For example, we distilled the number of questions students answered in a Thinklet and the number of responses in each phase. To understand the strategies that students select in the Plan phase, we also created a feature that counts the number of strategies a student selects among the top two strategies used by peers for the same problem.

The second set of features were designed to examine the content and the linguistic features of students’ text-based responses. These features (N = 90) were first extracted at the response level and then aggregated to the phase level. These aggregations were calculated for the Understand , Plan , and Review phases. (No textual data was extracted from the Solve phase as there was no textual input in this phase.)

Specifically, we distilled whether each response: 1) contains a numerical value, 2) consists of only numerical values, 3) has mathematical operation signs, 4) contains a question (if it contains a question mark or uses keywords such as “wonder”, “why”, etc.), 5) uses language that indicates the formation of a plan (e.g., the use of keywords like “plan”, “I will”, “going to”, etc.) , and 6) is the exact repetition of a previous answer. These criteria generate a set of binary variables for each response. We averaged these binary variables across the responses within a phase, creating 18 features for each Thinklet .

Additionally, in each response, we counted the number of 7) characters, 8) words, 9) numerical values, 10) verbs, 11) nouns, and 12) pronouns. Features 10-12 were counted using Udpipe, a natural language processing toolkit [50]. We also 13) counted the number of keywords used from a predefined list that provides the context of each problem; and 14) computed how similar each response is to the problem item using the Smith-Waterman algorithm [46]. For these continuous variables, we computed the mean, standard deviation, and max of the values for each phase, creating 72 features.

Features distilled from the two sets were combined. In total, 100 features were extracted from each Thinklet process and were then used to construct the automated detectors. Note that we did not extract any features from the video that students make in the current work. Similarly, we did not use any of the audio from the video (or transcription thereof) for any features.

Machine Learning Algorithms

We used the scikit-learn library [40] to implement commonly-used models, including Logistic Regression, Lasso, Decision Tree, Random Forest as well as Extreme Gradient Boosting (XGBoost) as implemented in the XGBoost library [19]. XGBoost outperformed other algorithms in all cases; we therefore only discuss the XGBoost results below.

XGBoost uses an ensemble technique that trains an initial, weak decision tree and calculates its prediction errors. It then iteratively trains subsequent decision trees to predict the error of the previous decision tree, with the final prediction representing the sum of the predictions of all the trees in the set. We tested the detectors with 10-fold student-level cross-validation. For this approach, the dataset was split into 10 student-level folds, meaning that in cases where students had multiple Thinklets, all of their data would be contained within the same fold and at no time could data from a student be included in both the training and testing set. Nine folds were used to train the model, and the trained model was used to make predictions for the 10th fold. Each fold acted as the test set once. Student-level cross-validation was conducted to verify generalizability to new students.

Models were evaluated using the area under the Receiver Operating Characteristic curve (AUC ROC), which indicates the probability that the model can correctly distinguish between an example of each class. An AUC ROC of 0.5 represents chance classification, while an AUC ROC of 1 represents perfect classification. Results were calculated for each fold and averaged to yield one AUC ROC score per detector.

Model Performance

Due to the rarity of strategy orientation (only 14 clips were labeled with this construct), a detector could not be built for this construct. Automated detectors were built for the other four constructs. As shown in Table 4, the average AUC ROC derived from 10-fold student-level cross-validation is 0.894 for numerical representation (NR), 0.813 for contextual representation (CR), 0.761 for outcome orientation (OO), and 0.815 for data transformation (DT). These findings suggest that the detectors were generally successful at capturing these four SRL constructs. We also calculated the standard deviations (SD) of the AUC ROCs across the 10 folds for each detector.

Feature Importance

To better understand the detectors as well as to inform our understanding of how these features relate to the constructs, the SHapley Additive exPlanations (SHAP) [33] value, which reflects feature importance, was calculated for each feature within each test set.

These values were then averaged across the 10 testing sets and ranked based on their absolute values. Of the 100 features used, Table 5 reports the top five features with the highest absolute SHAP values for each detector. To understand the directionality, we examined the average SHAP values of the features listed. A positive average SHAP value was found for all the features listed (except for one, as indicated in the Contextual Representation section of Table 5). The positive values indicate that the features are positive predictors of the SRL indicators, suggesting that the higher the values in each feature, the more likely the model is to infer the presence of a SRL indicator.

We note that of the 20 features listed in Table 5, 11 are from the Understand phase, 5 are from the Plan Phase and 3 are from the Review phase. In other words, behaviors in the early phases contributed more heavily to the predictions. This finding aligns with how the Thinklets were initially coded. Specifically, the coders primarily examined student responses in the Understand phase for numerical and contextual representation as this phase contains information demonstrating how student assemble information and create a problem representation; the coders examined the Thinklet more broadly when coding for other SRL indicators, as they encompass behaviors that span across phases.

Algorithmic bias occurs when model performance is substantially better or worse across mutually exclusive groups separated by generally non-malleable factors [11]. To validate our detectors, we tested the model performance in different student populations, based on gender and race/ethnicity using slicing analysis [25]. Specifically, utilizing the predictions made in the testing sets, AUC was computed for each subgroup of students in the data for which we received data on group membership. However, due to sample size, comparisons were not possible for gender non-binary students (N=2), Asian students (N=2), or Native American students (N=0).

As Table 6 shows, the difference in model performance measured by AUC between male and female students is small, ranging from 0.01-0.11 for the four detectors. The detectors for numerical representation and contextual representation performed somewhat better for female students (AUC NR = .93, AUC CR = .75) than for male students (AUC NR = .82, AUC CR = .74), while detectors for outcome orientation and data transformation preformed somewhat better for male students (AUC OO = .78, AUC DT = .88) than for female students (AUC OO = .74, AUC DT = .87).

Table 6 also shows the analysis of algorithmic bias in terms of race/ethnicity, comparing the AUC between student racial/ethnic subgroups that had more than 5 students in our sample: African American, Hispanic/Latinx, and White. Small to moderate differences were observed across the three groups, though the differences were not consistent (i.e., no racial/ethnic group consistently had the best-performing detectors). However, performance remained acceptable for all four detectors across all groups. When detecting numerical representation and contextual representations, the detectors performed somewhat better for White students (AUC NR = 0.96, AUC CR = 0.80), than for African American (AUC NR = 0.92, AUC CR = 0.75) and Hispanic/Latinx (AUC NR = 0.88, AUC CR = 0.72) students. However, the outcome orientation detector had somewhat higher performance for Hispanic/Latinx students (AUC OO = 0.81), than for White (AUC OO = 0.80) and African American (AUC OO = 0.71) students. The data transformation detector performed better for African American students (AUC DT = 0.92) than for Hispanic/Latinx (AUC DT = 0.91) and White (AUC DT = 0.83) students.

Performance was substantially lower for two constructs/group combinations: detecting contextual representation for students who identify race as other (AUC CR = 0.65) and detecting outcome orientation for students who identify as belonging to two or more races (AUC OO = 0.46). These more substantial differences may be due to the small sample size of these constructs within these subgroups; in future work, larger samples will be collected in order to validate performance in these groups.

Given the data collected, we noticed a considerable number of students who declined to report gender (N = 9) and race (N = 19). Both groups performed close to the average model performance, across groups and contexts.

Conclusion and discussion

Main findings.

Given the importance of self-regulation in learning, specifically in the problem-solving process, an increasing number of studies have looked into ways to promote self-regulated learning. This first requires the ability to accurately measure SRL, so that interventions can be introduced to encourage and guide students to self-regulate effectively. However, the most common ways of measuring SRL in a fine-grained fashion – either through self-report and think-aloud protocols –are difficult to automate and scale, and they can also interrupt or interfere with the learning task. Log data collected from computer-based learning environments offers an unobtrusive and potentially scalable solution to help understand when and how students self-regulate within the problem-solving process, in order to inform decisions on intervention (e.g., [3]). However, previous automated detection of SRL constructs using log data has mostly not been explicitly connected to SRL theory. In the current work, we explored the possibility of detecting SRL constructs at a fine-grained level, focusing on detecting cognitive operations (i.e., assembling and translating), outlined in the SMART model [53]. Specifically, we detected the presence of four self-regulation indicators related to two categories of operations: numerical representation, contextual representation, outcome orientation, and data transformation. Evaluated using 10-fold student-level cross-validation, our detectors were found to be accurate and valid across demographic groups, with AUC ROC ranging from .76-.89.

To understand the detectors, feature importance was examined using SHAP values. The top five features with the highest absolute SHAP values were identified for each detector. With the features identified, we find that except for outcome orientation, the detectors primarily rely on features extracted from the Understand and Plan phases of the learning activity, the two phases where students assemble information and make plans. In particular, the numerical representation detector mainly relies on features that examine the numerical values used in the Understand phase as well as features that compare the similarity between student responses and the problem item. The numerical value feature makes sense, as the detector is operationalized to identify if numerical components are processed and represented when students assemble information.

However, the maximum similarity feature, a feature that takes both numerical values and text into account, also contributes to the NR indicator. This finding suggests that the NR detector not only examines if numbers are used in responses, but also how they are used in relation to the problem. As such, this finding validates the operationalization of this indicator, showing that the learner demonstrates a level of understanding of how numerical values are used in math problems, creating a representation of the problem space utilizing numbers.

The contextual representation detector looks at the keywords used in student responses in the Understand phase and the length of the responses in the Plan phase, which indicates the relationship that the longer the responses are when a student is forming a plan, the more likely it is for the student to contextually representing the problem. When predicting the presence of outcome orientation, the model utilizes features extracted in the Understand and the Review phases, understanding students’ use of keywords, nouns, and numerical values in these two phases. At last, the data transformation detector checks the number of top strategies students select as well as the length and the variation in the length of the responses in the Understand and Plan phase.

Additionally, we examined model performance on different demographic subgroups of students, both in terms of gender and racial/ethnic groups, to verify their fairness and lack of algorithmic biases. Relatively small differences were observed in each comparison, and no student group (either gender or racial/ethnic group) consistently had the best-performing detectors.

Applications

The detectors built in the current study provide two advantages over previous SRL detectors. First, previous SRL detectors generally identify higher-level strategies and are not typically linked to theory; in contrast, we specifically based our detectors on a SRL model in order to identify theoretically-grounded SRL constructs at a finer-grain size. Having developed these fine-grained models of behavior associated with the assembling and translating operations of SMART, we can conduct analyses to further our understanding of the role that cognitive operations play in the broader process of SRL. For example, we can investigate questions about how often students use these cognitive operations in each of the four tasks outlined in the Winne and Hadwin’s four-stage model, and how the engagement and the frequency of the engagement in these cognitive operations contributes to the success of completing the tasks. Results from future analyses will help expand the current theoretical understanding on SRL, adding specificity to the still high-level processes represented in contemporary SRL theory.

Second, given that most previous detectors are not connected back to SRL theory, it has been difficult to use them with theory-driven interventions. The detectors proposed in the current study are developed based on a theoretical model of SRL [46, 48] and are operationalized to capture key aspects of the cognitive operations in the model. These detectors can therefore be used to facilitate the development of adaptive learning environments that respond to student SRL, in a fashion connected to theory. For instance, a student demonstrating an outcome orientation could be encouraged to reflect further on their strategy.

Similarly, these detectors could also provide theory-grounded information to teachers (e.g., through a dashboard), providing insight on how students are approaching problems. This data can inform teachers as they create and refine their problems, as well as informing how they support their students. As with any application of this nature, careful attention will be needed in design to ensure that data is presented in the most useful form for teachers and appropriately represents the uncertainty in the model (i.e., false positives or false negatives).

Limitations and Future Work

This work has two principal limitations that should be addressed in future work. First, when validating the fairness of the models, the sample size is small (less than five students) for several student groups. Reliable comparison of the model performance for these groups of students is therefore not possible. In future work, larger and more representative samples should be collected in order to validate model performance for a broader range of student groups.

Second, although our detectors are based on a theoretical model of SRL, the operationalization of our constructs is contextualized in the current learning environment, so our detectors may be platform-specific. Future work should study the transferability of the current detectors by examining their applicability and predictive performance, and explore how they can be adapted for use in other learning environments. To the extent that some of our detectors (such as the data transformation detector) apply across learning environments, we can investigate their performance within those contexts to evaluate their transferability (see, for instance, [39]).

Additionally, since the detectors are trained primarily relying on complete Thinklets , when being implemented in a learning platform, the detectors will make predictions after a student has solved a problem, reflecting the student’s use or a lack of use of assembling and translating in the problem-solving process. Though these detectors may not provide real-time detection when students are working through a problem, they can be used to inform and direct following practices and instructions. Future studies may examine ways of incorporating incomplete Thinklets in order to provide detection during the problem-solving process, enabling real-time interventions.

As an explorative study, five constructs concerning two cognitive operations are identified in the current study. However, SRL as a process, covers a range of behaviors and strategies that elicit the use of various cognitive operations. Therefore, in future studies, it is our goal to continuously understand and detect the use of cognitive operations throughout the four stages of self-regulated learning in the context of problem-solving.

Conclusions

To better understand and facilitate the use of self-regulation in problem-solving, the current study tested the possibility of scaling up SRL measurement by leveraging machine learning to automatically detect individual SRL indicators through the lens of the SMART model. We built automated detectors that identify four commonly used strategies in math problem solving, indicating assembling and translating operations. Our detectors were found to be reliable and generalizable. Additionally, the detectors were also tested on different student populations to verify their fairness and lack of algorithmic bias, addressing a previously overlooked issue in the field of educational data mining. Given these properties, we anticipate implementing the detectors in the learning environment to collect more fine-grained data and to leverage the detection to inform interventions, creating more positive experiences in mathematical problem-solving.

Achknowledgements

The research reported here was supported by the EF+Math Program of the Advanced Education Research and Development Program (AERDF) through funds provided to the University of Pennsylvania, University of New Hampshire, and CueThink. The opinions expressed are those of the authors and do not represent views of the EF+Math Program or AERDF

• Aguilar, S.J. et al. 2021. Associations between learning analytics dashboard exposure and motivation and self-regulated learning. Computers & Education . 162, (2021). DOI:https://doi.org/10.1016/j.compedu.2020.104085.
• Aleven, V. et al. 2016. Help Helps, But Only So Much: Research on Help Seeking with Intelligent Tutoring Systems. International Journal of Artificial Intelligence in Education . 26, 1 (2016), 205–223. DOI:https://doi.org/10.1007/s40593-015-0089-1.
• Aleven, V. et al. 2006. Toward Meta-cognitive Tutoring: A Model of Help Seeking with a Cognitive Tutor. International Journal of Artificial Intelligence in Education . 16, 2 (2006), 101–128.
• Azevedo, R. et al. 2017. Understanding and Reasoning about Real-Time Cognitive, Affective, and Metacognitive Processes to Foster Self-Regulation with Advanced Learning Technologies. Handbook of Self-Regulation of Learning and Performance . (2017), 254–270. DOI:https://doi.org/10.4324/9781315697048-17.
• Azevedo, R. et al. 2011. Use of hypermedia to assess and convey self-regulated learning. Handbook of Self-Regulation of Learning and Performance . 32, (2011), 102–121. DOI:https://doi.org/10.4324/9780203839010.ch7.
• Baker, R.S. et al. 2010. Detecting Gaming the System in Constraint-Based Tutors. Proceedings of the 18th Annual Conference on User Modeling, Adaptation, and Personalization . (2010), 267–278. DOI:https://doi.org/10.1007/978-3-642-13470-8_25.
• Baker, R.S. et al. 2004. Detecting student misuse of intelligent tutoring systems. International Conference on Intelligent Tutoring Systems . (2004), 531–540. DOI:https://doi.org/10.1007/978-3-540-30139-4_50.
• Baker, R.S. et al. 2006. Human Classification of Low-Fidelity Replays of Student Actions. Proceedings of the Educational Data Mining Workshop at the 8th International Conference on Intelligent Tutoring Systems . (2006), 29–36.
• Baker, R.S. et al. 2012. Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra. Proceedings of the 5th International Conference on Educational Data Mining . (2012), 126–133.
• Baker, R.S. and Carvalho, A.M.J.A. 2008. Labeling student behavior faster and more precisely with text replays. Proceedings of the 1st International Conference on Educational Data Mining . (2008), 38–47.
• Baker, R.S. and Hawn, A. 2021. Algorithmic Bias in Education. International Journal of Artificial Intelligence in Education . (2021), 1–41. DOI:https://doi.org/10.35542/osf.io/pbmvz.
• Baker, R.S. and Yacef, K. 2009. The State of Educational Data Mining in 2009: A Review and Future Visions. Journal of Educational Data Mining . 1, 1 (2009), 3–17. DOI:https://doi.org/10.5281/zenodo.3554657.
• Bandura, A. 1986. Social foundations of thought and action: A social cognitive theory .
• Biswas, G. et al. 2010. Measuring Self-regulated Learning Skills Through Social Interactions in a Teachable Agent. Research and Practice in Technology Enhanced Learning . 05, 02 (Jul. 2010), 123–152. DOI:https://doi.org/10.1142/S1793206810000839.
• Boekaerts, M. 1999. Self-regulated learning: where we are today. International Journal of Educational Research . 31, 6 (1999), 445–457. DOI:https://doi.org/10.1016/S0883-0355(99)00014-2.
• Bosch, N. et al. 2021. Students’ Verbalized Metacognition During Computerized Learning. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (May 2021), 1–12.
• Botelho, A.F. et al. 2018. Studying Affect Dynamics and Chronometry Using Sensor-Free Detectors. Proceedings of the 11th International Conference on Educational Data Mining . (2018), 157–166.
• Charmaz, K. 1983. The grounded theory method: An explication and interpretation. Contemporary Field Research . (1983), 109–126.
• Chen, T. and Guestrin, C. 2016. XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco California USA, Aug. 2016), 785–794.
• Cleary, T.J. and Chen, P.P. 2009. Self-regulation, motivation, and math achievement in middle school: Variations across grade level and math context. Journal of School Psychology . 47, 5 (Oct. 2009), 291–314. DOI:https://doi.org/10.1016/j.jsp.2009.04.002.
• Devolder, A. et al. 2012. Supporting self‐regulated learning in computer‐based learning environments: systematic review of effects of scaffolding in the domain of science education. Journal of Computer Assisted Learning . 28, 6 (Dec. 2012), 557–573. DOI:https://doi.org/10.1111/j.1365-2729.2011.00476.x.
• DiCerbo, K.E. and Kidwai, K. 2013. Detecting player goals from game log files. Presented at the 6th International Conference on Educational Data Mining (2013).
• D’Mello, S.K. et al. 2008. Automatic detection of learner’s affect from conversational cues. User Modeling and User-Adapted Interaction . 18, 1–2 (Feb. 2008), 45–80. DOI:https://doi.org/10.1007/s11257-007-9037-6.
• Efklides, A. 2011. Interactions of Metacognition with Motivation and Affect in Self-Regulated Learning: The MASRL Model. Educational Psychologist . 46, 1 (Jan. 2011), 6–25. DOI:https://doi.org/10.1080/00461520.2011.538645.
• Gardner, J. et al. 2019. Evaluating the Fairness of Predictive Student Models Through Slicing Analysis. Proceedings of the 9th International Conference on Learning Analytics & Knowledge (Tempe AZ USA, Mar. 2019), 225–234.
• Graesser, A. and McNamara, D. 2010. Self-Regulated Learning in Learning Environments with Pedagogical Agents That Interact in Natural Language. Educational Psychologist . 45, 4 (Oct. 2010), 234–244. DOI:https://doi.org/10.1080/00461520.2010.515933.
• Graesser, A.C. et al. 2007. Using LSA in AutoTutor: Learning through mixed initiative dialogue in natural language. Handbook of Latent Semantic Analysis . 243–262.
• Greene, J.A. et al. 2017. Capturing and modeling self-regulated learning using think-aloud protocols. Handbook of Self-Regulation of Learning and Performance . 323–337.
• Hutt, S. et al. 2021. Investigating SMART Models of Self-Regulation and their Impact on Learning. Proceedings of the 14th International Conference on Educational Data Mining . (2021), 580–587.
• Kovanovic, V. et al. 2016. Does Time-on-task Estimation Matter? Implications on Validity of Learning Analytics Findings. Journal of Learning Analytics . 2, 3 (Feb. 2016), 81–110. DOI:https://doi.org/10.18608/jla.2015.23.6.
• Labuhn, A.S. et al. 2010. Enhancing students’ self-regulation and mathematics performance: the influence of feedback and self-evaluative standards. Metacognition and Learning . 5, 2 (Aug. 2010), 173–194. DOI:https://doi.org/10.1007/s11409-010-9056-2.
• Lee, D.M.C. et al. 2011. Exploring the Relationship between Novice Programmer Confusion and Achievement. International Conference on Affective Computing and Intelligent Interaction . (2011), 175–184. DOI:https://doi.org/10.1007/978-3-642-24600-5_21.
• Lundberg, S.M. et al. 2019. Consistent Individualized Feature Attribution for Tree Ensembles. arXiv:1802.03888 . (2019). DOI:https://doi.org/10.48550/arXiv.1802.03888.
• Nota, L. et al. 2004. Self-regulation and academic achievement and resilience: A longitudinal study. International Journal of Educational Research . 41, 3 (Jan. 2004), 198–215. DOI:https://doi.org/10.1016/j.ijer.2005.07.001.
• Ocumpaugh, J. et al. 2021. Using Qualitative Data from Targeted Interviews to Inform Rapid AIED Development. Proceedings of the 29th International Conference on Computers in Education . (2021), 69–74.
• Paquette, L. et al. 2014. Sensor-Free Affect Detection for a Simulation-Based Science Inquiry Learning Environment. International Conference on Intelligent Tutoring Systems . (2014), 1–10. DOI:https://doi.org/10.1007/978-3-319-07221-0_1.
• Paquette, L. et al. 2014. Towards Understanding Expert Coding of Student Disengagement in Online Learning. Proceedings of the 36th annual cognitive science conference (2014), 1126–1131.
• Paquette, L. et al. 2020. Who’s learning? Using demographics in EDM research. Journal of Educational Data Mining . 12, 3 (2020), 1–30. DOI:https://doi.org/10.5281/zenodo.4143612.
• Paquette, L. and Baker, R.S. 2017. Variations of Gaming Behaviors Across Populations of Students and Across Learning Environments. Proceedings of the 18th International Conference on Artificial Intelligence in Education . (2017), 274–286. DOI:https://doi.org/10.1007/978-3-319-61425-0_23.
• Pedregosa, F. et al. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research . 12, (2011), 2825–2830. DOI:https://doi.org/10.1007/s13398-014-0173-7.2.
• Richey, J.E. et al. 2021. Gaming and Confrustion Explain Learning Advantages for a Math Digital Learning Game. Artificial Intelligence in Education . (2021), 342–355. DOI:https://doi.org/10.1007/978-3-030-78292-4_28.
• Sabourin, J. et al. 2012. Predicting Student Self-regulation Strategies in Game-Based Learning Environments. Intelligent Tutoring Systems . (2012), 141–150. DOI:https://doi.org/10.1007/978-3-642-30950-2_19.
• Sao Pedro, M.A. et al. 2013. Leveraging machine-learned detectors of systematic inquiry behavior to estimate and predict transfer of inquiry skill. User Modeling and User-Adapted Interaction . 23, 1 (Mar. 2013), 1–39. DOI:https://doi.org/10.1007/s11257-011-9101-0.
• Schooler, J.W. et al. 1993. Thoughts Beyond Words: When Language Overshadows Insight. Journal of Experimental Psychology: General . 122, 2 (1993), 166–183. DOI:https://doi.org/10.1037/0096-3445.122.2.166.
• Segedy, J.R. et al. 2015. Using Coherence Analysis to Characterize Self-Regulated Learning Behaviours in Open-Ended Learning Environments. Journal of Learning Analytics . 2, 1 (May 2015), 13–48. DOI:https://doi.org/10.18608/jla.2015.21.3.
• Smith, T.F. and Waterman, M.S. 1981. Identification of common molecular subsequences. Journal of Molecular Biology . 147, 1 (Mar. 1981), 195–197. DOI:https://doi.org/10.1016/0022-2836(81)90087-5.
• Taub, M. et al. 2014. Can the use of cognitive and metacognitive self-regulated learning strategies be predicted by learners’ levels of prior knowledge in hypermedia-learning environments? Computers in Human Behavior . 39, (Oct. 2014), 356–367. DOI:https://doi.org/10.1016/j.chb.2014.07.018.
• Walonoski, J.A. and Heffernan, N.T. 2006. Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems. International Conference on Intelligent Tutoring Systems (2006), 722–724.
• Weston, C. et al. 2001. Analyzing Interview Data: The Development and Evolution of a Coding System. Qualitative Sociology . 24, 3 (2001), 381–400. DOI:https://doi.org/10.1023/A:1010690908200.
• Wijffels, J. et al. 2017. Package ‘udpipe’.
• Winne, P.H. 2010. Improving Measurements of Self-Regulated Learning. Educational Psychologist . 45, 4 (Oct. 2010), 267–276. DOI:https://doi.org/10.1080/00461520.2010.517150.
• Winne, P.H. 2005. Key Issues in Modeling and Applying Research on Self‐Regulated Learning. Applied Psychology . 54, 2 (2005), 232–238. DOI:https://doi.org/10.1111/j.1464-0597.2005.00206.x.
• Winne, P.H. 2017. Learning Analytics for Self-Regulated Learning. Handbook of Learning Analytics . (2017), 531–566. DOI:https://doi.org/10.18608/hla17.021.
• Winne, P.H. et al. 2019. nStudy: Software for Learning Analytics about Processes for Self-Regulated Learning. Journal of Learning Analytics . 6, 2 (Jul. 2019), 95–106. DOI:https://doi.org/10.18608/jla.2019.62.7.
• Winne, P.H. and Hadwin, A.F. 1998. Studying as Self-Regulated Learning. Metacognition in Educational Theory and Practice . (1998), 277–304.
• Zimmerman, B.J. 2000. Attaining self-regulation: A social cognitive perspective. Handbook of Self-Regulation . (2000), 13–39. DOI:https://doi.org/10.1016/B978-012109890-2/50031-7.
• Zimmerman, B.J. 1990. Self-Regulated Learning and Academic Achievement: An Overview. Educational Psychologist . 25, 1 (Jan. 1990), 3–17. DOI:https://doi.org/10.1207/s15326985ep2501_2.

© 2022 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International ( CC BY-NC-ND 4.0 ) license.

Cognitive Behavioral Therapy

Solving problems the cognitive-behavioral way, problem solving is another part of behavioral therapy..

Posted February 2, 2022 | Reviewed by Ekua Hagan

• What Is Cognitive Behavioral Therapy?
• Find a therapist who practices CBT
• Problem-solving is one technique used on the behavioral side of cognitive-behavioral therapy.
• The problem-solving technique is an iterative, five-step process that requires one to identify the problem and test different solutions.
• The technique differs from ad-hoc problem-solving in its suspension of judgment and evaluation of each solution.

As I have mentioned in previous posts, cognitive behavioral therapy is more than challenging negative, automatic thoughts. There is a whole behavioral piece of this therapy that focuses on what people do and how to change their actions to support their mental health. In this post, I’ll talk about the problem-solving technique from cognitive behavioral therapy and what makes it unique.

The problem-solving technique

While there are many different variations of this technique, I am going to describe the version I typically use, and which includes the main components of the technique:

The first step is to clearly define the problem. Sometimes, this includes answering a series of questions to make sure the problem is described in detail. Sometimes, the client is able to define the problem pretty clearly on their own. Sometimes, a discussion is needed to clearly outline the problem.

The next step is generating solutions without judgment. The "without judgment" part is crucial: Often when people are solving problems on their own, they will reject each potential solution as soon as they or someone else suggests it. This can lead to feeling helpless and also discarding solutions that would work.

The third step is evaluating the advantages and disadvantages of each solution. This is the step where judgment comes back.

Fourth, the client picks the most feasible solution that is most likely to work and they try it out.

The fifth step is evaluating whether the chosen solution worked, and if not, going back to step two or three to find another option. For step five, enough time has to pass for the solution to have made a difference.

This process is iterative, meaning the client and therapist always go back to the beginning to make sure the problem is resolved and if not, identify what needs to change.

Advantages of the problem-solving technique

The problem-solving technique might differ from ad hoc problem-solving in several ways. The most obvious is the suspension of judgment when coming up with solutions. We sometimes need to withhold judgment and see the solution (or problem) from a different perspective. Deliberately deciding not to judge solutions until later can help trigger that mindset change.

Another difference is the explicit evaluation of whether the solution worked. When people usually try to solve problems, they don’t go back and check whether the solution worked. It’s only if something goes very wrong that they try again. The problem-solving technique specifically includes evaluating the solution.

Lastly, the problem-solving technique starts with a specific definition of the problem instead of just jumping to solutions. To figure out where you are going, you have to know where you are.

One benefit of the cognitive behavioral therapy approach is the behavioral side. The behavioral part of therapy is a wide umbrella that includes problem-solving techniques among other techniques. Accessing multiple techniques means one is more likely to address the client’s main concern.

Salene M. W. Jones, Ph.D., is a clinical psychologist in Washington State.

• Find a Therapist
• Find a Treatment Center
• Find a Psychiatrist
• Find a Support Group
• Find Online Therapy
• United States
• Brooklyn, NY
• Chicago, IL
• Houston, TX
• Los Angeles, CA
• New York, NY
• Portland, OR
• San Diego, CA
• San Francisco, CA
• Seattle, WA
• Washington, DC
• Asperger's
• Bipolar Disorder
• Chronic Pain
• Eating Disorders
• Passive Aggression
• Personality
• Goal Setting
• Positive Psychology
• Stopping Smoking
• Low Sexual Desire
• Relationships
• Child Development
• Self Tests NEW
• Therapy Center
• Diagnosis Dictionary
• Types of Therapy

At any moment, someone’s aggravating behavior or our own bad luck can set us off on an emotional spiral that threatens to derail our entire day. Here’s how we can face our triggers with less reactivity so that we can get on with our lives.

• Emotional Intelligence
• Gaslighting
• Affective Forecasting
• Neuroscience

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support
• Piaget's Theory
• Sensorimotor Stage
• Preoperational Stage
• Concrete Operational Stage
• Formal Operational Stage
• Support and Criticism

The Formal Operational Stage of Cognitive Development

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Carly Snyder, MD is a reproductive and perinatal psychiatrist who combines traditional psychiatry with integrative medicine-based treatments.

Aaron Johnson is a fact checker and expert on qualitative research design and methodology. 

• Conceptualizing Balance
• Abstraction of Ideas
• Next in Stages of Cognitive Development Guide Support and Criticism of Piaget's Stage Theory

The formal operational stage is the fourth and final stage of Jean Piaget's theory of cognitive development . It begins at approximately age 12 and lasts into adulthood.

In the formal operational stage, children's thinking becomes much more sophisticated and advanced. Kids can think about abstract and theoretical concepts and use logic to come up with creative solutions to problems. Skills such as logical thought, deductive reasoning, and systematic planning also emerge during this stage.

Piaget tested formal operational thought in a few different ways. Two of the better-known tests explored physical conceptualization and the abstraction of thought.

Balance in the Formal Operational Stage

One task involved having children of different ages balance a scale by hooking weights on each end. To balance the scale, the children needed to understand that both the heaviness of the weights and the distance from the center played a role.

• Younger children around the ages of 3 and 5 were unable to complete the task because they did not understand the concept of balance.
• Seven-year-olds knew that they could adjust the scale by placing weights on each end, but failed to understand that where they put the weights was also important.
• By age 10, the kids considered location as well as weight but had to arrive at the correct answer using trial and error.

It wasn't until around age 13 that children could use logic to form a hypothesis about where to place the weights to balance the scale and then complete the task.

Abstraction in the Formal Operational Stage

In another experiment on formal operational thought, Piaget asked children to imagine where they would want to place a third eye if they had one.

Younger children said that they would put the imagined third eye in the middle of their forehead. Older children, however, were able to come up with a variety of creative ideas about where to place this hypothetical eye and various ways the eye could be used.

For example, an eye in the middle of one's hand would be useful for looking around corners. An eye at the back of one's head could be helpful for seeing what is happening in the background.

Creative ideas represent the use of abstract and hypothetical thinking, both important indicators of formal operational thought.

Formal Operational Stage Skills

Important skills that emerge during the formal operational stage include the following:

Deductive Logic

Piaget believed that deductive reasoning becomes necessary during the formal operational stage. Deductive logic requires the ability to use a general principle to determine a particular outcome. Science and mathematics often require this type of thinking about hypothetical situations and concepts.

Abstract Thought

While children tend to think very concretely and specifically in earlier stages, the ability to think about abstract concepts emerges during the formal operational stage. Instead of relying solely on previous experiences, children begin to consider possible outcomes and consequences of actions. This type of thinking is important in long-term planning.

Problem-Solving

In earlier stages, children used trial-and-error to solve problems . During the formal operational stage, the ability to systematically solve a problem in a logical and methodical way emerges. Children at the formal operational stage of cognitive development are often able to plan quickly an organized approach to solving a problem.

Hypothetical-Deductive Reasoning

Piaget believed that what he referred to as "hypothetical-deductive reasoning" was essential at this stage of intellectual development. At this point, teens become capable of thinking about abstract and hypothetical ideas. They often ponder "what-if" type situations and questions and can think about multiple solutions or possible outcomes.

While kids in the previous stage ( concrete operations ) are very particular in their thoughts, kids in the formal operational stage become increasingly abstract in their thinking.

As children gain greater awareness and understanding of their own thought processes, they develop what is known as metacognition, or the ability to think about their thoughts as well as the ideas of others.

Criticisms of the Formal Operational Stage

Some researchers have noted that while Piaget's theory indicates there are four stages of cognitive development, there is also evidence that indicates that not all adolescents reach the formal operational stage.

The formal operational stage hinges on the emergence of critical thinking skills. Depending on factors such as education, parenting, and cultural influences, some children do not necessarily develop the requisite thinking skills to fully approach this stage.

It has also been noted that formal operational thought may, in some cases, be domain specific. A trained engineer may be able to engage in formal operational thought with regard to their profession, but they may lack the ability to apply similar skills in domains such as economics, politics, or social science.

Babakr ZH, Mohamedamin P, Kakamad K. Piaget’s cognitive developmental theory: Critical review . Education Quarterly Reviews . 2019;2(3).

Malik F, Marwaha R. Cognitive development . PubMed.

Inhelder B, Piaget J. The Growth of Logical Thinking: From Childhood to Adolescence . Basic Books; 1958. doi:10.1037/10034-000

Sortino DP. Brain Changers: Major Advances in Children’s Learning and Intelligence . Rowman & Littlefield; 2020.

Jeon M, Draney K, Wilson M, Sun Y. Investigation of adolescents’ developmental stages in deductive reasoning: An application of a specialized confirmatory mixture IRT approach . Behavior Research Methods . 2019;52. doi:10.3758/s13428-019-01221-5

Wright L, Kutcher S. Adolescent brain development . Colloquium Series on The Developing Brain . 2016;5(1). doi:10.4199/c00133ed1v01y201602dbr012

Senan D. Enhancing educational effectiveness in physics through cognitive development model: An investigation in formal operational stage . The International Journal Of Engineering And Science (IJES) . 2013;2(8).

Padmanabha CH. Metacognition: Conceptual framework . i-Manager’s Journal on Educational Psychology . 2020;14(1). doi:10.26634/jpsy.14.1.16710

Piaget, J. (1977). Gruber, H.E.; Voneche, J.J. eds. The essential Piaget. New York: Basic Books.

Piaget, J. (1983). Piaget's theory. In P. Mussen (ed). Handbook of Child Psychology. 4th edition. Vol. 1. New York: Wiley.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Detecting SMART Model Cognitive Operations in Mathematical Problem-Solving Process

Research output : Chapter in Book/Report/Conference proceeding › Conference contribution

Self-regulated learning (SRL) is a critical component of mathematics problem solving. Students skilled in SRL are more likely to effectively set goals, search for information, and direct their attention and cognitive process so that they align their efforts with their objectives. An influential framework for SRL, the SMART model, proposes that five cognitive operations (i.e., searching, monitoring, assembling, rehearsing, and translating) play a key role in SRL. However, these categories encompass a wide range of behaviors, making measurement challenging – often involving observing individual students and recording their think-aloud activities or asking students to complete labor-intensive tagging activities as they work. In the current study, we develop machine-learned indicators of SMART operations, in order to achieve better scalability than other measurement approaches. We analyzed student’s textual responses and interaction data collected from a mathematical learning platform where students are asked to thoroughly explain their solutions and are scaffolded in communicating their problem-solving process to their peers and teachers. We built detectors of four indicators of SMART operations (namely, assembling and translating operations). Our detectors are found to be reliable and generalizable, with AUC ROCs ranging from .76-.89. When applied to the full test set, the detectors are robust against algorithmic bias, performing well across different student populations.

Publication series

Bibliographical note.

• automated detectors
• self-regulated learning
• SMART model

Publisher link

• 10.5281/zenodo.6853161

Other files and links

• Link to publication in Scopus
• Link to the citations in Scopus

Fingerprint

• Mathematical Problem Solving Keyphrases 100%
• Self-regulated Learning Keyphrases 100%
• Problem-solving Process Keyphrases 100%
• Cognitive Operations Keyphrases 100%
• Regulated Learning Computer Science 100%
• Cognitive Operation Computer Science 100%
• Mathematical Problem Computer Science 100%
• Solving Process Computer Science 100%

T1 - Detecting SMART Model Cognitive Operations in Mathematical Problem-Solving Process

AU - Zhang, Jiayi

AU - Andres, Juliana Ma Alexandra L.

AU - Hutt, Stephen

AU - Baker, Ryan S.

AU - Ocumpaugh, Jaclyn

AU - Mills, Caitlin

AU - Brooks, Jamiella

AU - Sethuraman, Sheela

AU - Young, Tyron

N1 - Publisher Copyright: © 2022 Copyright is held by the author(s).

N2 - Self-regulated learning (SRL) is a critical component of mathematics problem solving. Students skilled in SRL are more likely to effectively set goals, search for information, and direct their attention and cognitive process so that they align their efforts with their objectives. An influential framework for SRL, the SMART model, proposes that five cognitive operations (i.e., searching, monitoring, assembling, rehearsing, and translating) play a key role in SRL. However, these categories encompass a wide range of behaviors, making measurement challenging – often involving observing individual students and recording their think-aloud activities or asking students to complete labor-intensive tagging activities as they work. In the current study, we develop machine-learned indicators of SMART operations, in order to achieve better scalability than other measurement approaches. We analyzed student’s textual responses and interaction data collected from a mathematical learning platform where students are asked to thoroughly explain their solutions and are scaffolded in communicating their problem-solving process to their peers and teachers. We built detectors of four indicators of SMART operations (namely, assembling and translating operations). Our detectors are found to be reliable and generalizable, with AUC ROCs ranging from .76-.89. When applied to the full test set, the detectors are robust against algorithmic bias, performing well across different student populations.

AB - Self-regulated learning (SRL) is a critical component of mathematics problem solving. Students skilled in SRL are more likely to effectively set goals, search for information, and direct their attention and cognitive process so that they align their efforts with their objectives. An influential framework for SRL, the SMART model, proposes that five cognitive operations (i.e., searching, monitoring, assembling, rehearsing, and translating) play a key role in SRL. However, these categories encompass a wide range of behaviors, making measurement challenging – often involving observing individual students and recording their think-aloud activities or asking students to complete labor-intensive tagging activities as they work. In the current study, we develop machine-learned indicators of SMART operations, in order to achieve better scalability than other measurement approaches. We analyzed student’s textual responses and interaction data collected from a mathematical learning platform where students are asked to thoroughly explain their solutions and are scaffolded in communicating their problem-solving process to their peers and teachers. We built detectors of four indicators of SMART operations (namely, assembling and translating operations). Our detectors are found to be reliable and generalizable, with AUC ROCs ranging from .76-.89. When applied to the full test set, the detectors are robust against algorithmic bias, performing well across different student populations.

KW - automated detectors

KW - self-regulated learning

KW - SMART model

UR - http://www.scopus.com/inward/record.url?scp=85160840528&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85160840528&partnerID=8YFLogxK

U2 - 10.5281/zenodo.6853161

DO - 10.5281/zenodo.6853161

M3 - Conference contribution

AN - SCOPUS:85160840528

T3 - Proceedings of the 15th International Conference on Educational Data Mining, EDM 2022

BT - Proceedings of the 15th International Conference on Educational Data Mining, EDM 2022

PB - International Educational Data Mining Society

T2 - 15th International Conference on Educational Data Mining, EDM 2022

Y2 - 24 July 2022 through 27 July 2022

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychiatry

Problem Solving Therapy Improves Effortful Cognition in Major Depression

Chenguang jiang.

1 Wuxi Mental Health Center Affiliated to Nanjing Medical University, Wuxi, China

Hongliang Zhou

2 Nanjing Brain Hospital Affiliated to Nanjing Medical University, Nanjing, China

Zhenhe Zhou

Associated data.

The datasets generated for this study are available on request to the corresponding author.

Background: Effortful cognition processing is an intentionally initiated sequence of cognitive activities, which may supply top-down and goal-oriented reassessment of specific stimuli to regulate specific state-driven responses contextually, whereas automatic cognitive processing is a sequence of cognitive activities that is automatically initiated in response to an input configuration. The effortful–automatic perspective has implications for understanding the nature of the clinical features of major depressions. The aim of this study was to investigate the influence of problem solving therapy (PST) on effortful cognition in major depression (MD).

Methods: The participants included an antidepressant treatment (AT) group ( n = 31) or the combined antidepressant treatment and PST (CATP) group ( n = 32) and healthy controls (HCs) ( n = 30). Hamilton Depression Rating Scale (HAMD, 17-item version) and the face–vignette task (FVT) were measured for AT group and CATP group at baseline (before the first intervention) and after 12 weeks of interventions. The HC group was assessed with the FVT only once. At baseline, both patients and HCs were required to complete the basic facial emotion identification test (BFEIT).

Results: The emotion identification accuracy of the HC group was higher than that of the patient group when they performed BFEIT; patients with MD present poor FVT performances; compared to the antidepressant treatment, PST plus antidepressant treatment decreased HAMD scores and improved FVT performances in patients with MD.

Conclusions: Patients with MD present effortful cognition dysfunction, and PST can improve effortful cognitive dysfunction. These findings suggest that the measurement of effortful cognition might be one of the indexes for the therapeutic effect of PST in MD.

Introduction

Major depression (MD) is a common mental disorder with a higher disability rate, affecting 10–15% of the worldwide population every year. To date, some antidepressants, including several typical antidepressants and several atypical antidepressants, have been used to treat major depression; however, only 60–70% of patients respond to antidepressant treatment. Furthermore, 10–30% of these patients exhibit treatment-resistant symptoms such as suicidal thought, a low mood, a decline in interest, and a loss of happiness ( 1 ).

To improve the symptoms of MD, several treatment options have been developed, such as switching therapies, augmentation, combination, optimization, psychotherapies, modified electro-convulsive therapy (MECT), repetitive transcranial magnetic stimulation therapies, deep brain stimulation therapies, vagal nerve stimulation therapies, light-based therapies, acupuncture treatment, and yoga; these approaches have been considered and tailored for individual patients ( 2 – 4 ). Most important for the improvement of depressed patients' symptoms, many studies had reported that physical activity interventions are helpful to improve major depressive disorders because physical activity is associated with many mental health benefits ( 5 – 11 ). Assessments to determine symptom improvement for patients with MD often depend on decreased total Hamilton Depression Rating Scale (HAMD, 17 or 24 items) scores.

Problem solving therapy (PST) belongs to a type of cognitive behavioral therapy that mainly concentrates on training in appropriate problem-solving notions as well as skills. PST has been used for major depression ( 12 – 15 ). It has been confirmed that, in the depressed patient group, PST was equally effective as antidepressant treatments and more effective than no treatment and support or attention control patients ( 16 ). In clinical practice, the effective treatment program of PST in MD includes three aspects: [1] training in a positive problem orientation, [2] training in problem definition and formulation, the generation of alternatives, decision making, and solution implementation and verification, and [3] training in problem orientation plus problem definition and formulation, the generation of alternatives, decision making, and solution implementation and verification ( 16 ).

Cognitive function refers to mental processes involved in working memory, problem-solving, decision-making, the acquisition of knowledge, regulation of information, and reasoning. As a major symptom, cognitive function impairment is acknowledged as a clinical characteristic of major depression. Additionally, many studies of major depression have suggested a role for cognitive measures in predicting those at risk for poor outcomes ( 17 ). A previous study indicated that patients with major depression present negatively valanced emotional symptoms that are accompanied by cognitive deficits, and the emotional processing dysfunctions of the prefrontal cortex might lead to cognitive deficits in patients with MD ( 18 ). Adaptive emotional responding relies on both effortful cognition processing and automatic cognition processing. Effortful cognition processing is a controlled process and refers to an intentionally initiated sequence of cognitive activities, which may supply top-down as well as goal-oriented reassessment of emotional stimuli to regulate emotion-driven responses contextually ( 19 ). Effortful cognition was measured by the face–vignette task (FVT) ( 19 ). Relative to effortful cognitive processing, automatic cognitive processing is a sequence of cognitive activities that is automatically initiated in response to an input configuration ( 20 ). Automatic cognition processing requires near-zero attention for the task at hand and, in many instances, is executed in response to a specific stimulus.

Previous studies have shown that patients with MD present effortful cognitive dysfunction. For example, a previous study reported that, when patients with MD performed two contrasting cognitive tasks ( i.e ., one requiring sustained effort and information processing and the other requiring only superficial information processing that could be accomplished automatically), only the effort-demanding cognitive task was performed poorly ( 21 ). Additionally, two previous studies investigated the functions of automatic and effortful information processing in a visual search paradigm, and the results showed that the patients with MD exhibited longer reaction times on the tasks requiring more effortful information processing than the controls. However, there were no differences on tasks requiring automatic information processing ( 22 , 23 ).

Since cognitive function impairment plays a critical role in MD, the assessment of cognitive function is a better way to determine the treatment effect for MD. The effortful–automatic perspective has implications for understanding the nature of the clinical features of MD. Furthermore, the investigation of the influence of PST on effortful cognition in MD is helpful for improving the present understanding of the therapeutic mechanism and assess the therapeutic effect of PST. To date, no studies of PST on effortful cognition in MD have been reported. In this study, the participants included patients with MD and healthy controls (HCs). The MD group was treated with antidepressants or the combination of antidepressants with PST, and effortful cognition was rated by the FVT. The hypothesis of this study is that depressed patients display poor effortful cognition performance, and PST can improve effortful cognitive dysfunctions. The aim of this study was to investigate the effect of PST on effortful cognition in MD.

Materials and Methods

Time and setting.

This study was conducted in Wuxi Mental Health Center Affiliated to Nanjing Medical University, No. 156 Qianrong Road, Rongxiang Street, Binhu District, Wuxi City, P.R. China, from February 1, 2016 to February 27, 2020.

Diagnostic Approaches and Subjects

A total of 80 patients meeting the American Psychiatric Association's fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) criteria for major depression were recruited as the research group. The MD patients were randomly assigned to the antidepressant treatment (AT) group or the combined antidepressant treatment and PST (CATP) group. The allocation schedule was generated by using a list of random numbers. Thirty healthy persons were admitted to the HC group. All HCs had no personal history of mental disorders. Patients with MD were selected from Wuxi Mental Health Center Affiliated to Nanjing Medical University, No. 156 Qianrong Road, Rongxiang Street, Binhu District, Wuxi City, P.R. China; the normal controls were citizens of Wuxi City, Jiangsu Province, P.R. China, recruited by online and local community advertisements. Patients with MD and HC subjects were excluded from the study if they had been diagnosed with nicotine addiction or other psychoactive substance dependence, had suffered any systemic disease that may affect the central nervous system, or had received electroconvulsive therapy (including MECT) in the past 24 weeks. All patients and HC subjects were Chinese. All patients and HC subjects were paid 42.12 Euros plus travel costs.

Seven subjects in AT group and five subjects in CATP group were all diagnosed with bipolar disorder in the follow-up survey, and they were ultimately excluded from this study. Two subjects in AT group and three subjects in CATP group were also excluded from this study because they could not finish the follow-up assessment. Finally, the data from 31 subjects in AT group and 32 subjects in CATP group were used in the statistical analyses.

Measurements of Automatic and Effortful Cognition

Basic facial emotion identification test.

The basic facial emotion identification test (BFEIT) consists of eight examples of each of the seven basic facial emotions, e.g ., happy, angry, sad, fear, surprise, disgust, and calm, which were taken from the Chinese affective picture system ( 24 ). Male and female face pictures were balanced across each emotion category.

Face–Vignette Task

FVT was designed based on an effortful cognitive task that was used in the study on effortful vs . automatic emotional processing in patients with schizophrenia by Patrick et al. ( 19 ). E-Prime 2.0 software (Psychology software tools, INC, USA) was used to implement the experimental procedure. The face pictures were white and black photographs and included six emotional expressions, i.e ., happy, angry, sad, fear, surprise, and disgust, which were taken from the Chinese affective picture system ( 24 ). In each emotion, the male and female faces were equal. Within a given emotion category, the same identity was used only once. The situational vignettes communicated the six special emotions, i.e ., guilty, smug, hopeful, insulted, pain, and determined. Before the experiment, the intended emotion for each story (vignette) was verified by seven undergraduates, and the mean accuracy was 0.91 [standard deviation (SD) = 0.08], and the observed inter-rater reliability κ value was 0.75. The face–story pairs were matched such that each story was inconsistent with the facial expression according to the specially appointed emotional category ( e.g ., a happy facial expression paired with a smug story). Each specific emotion category depended on the situational context (see the listed example in Figure 1 ). The specially appointed face–story pairs included sad vs . guilty, happy vs . smug, fearful vs . painful, angry vs . determined, disgusted vs . insulted, and surprised vs . hopeful. During the FVT, the participants viewed a series of 24 face–story (vignette) pairs and were informed that each facial expression represented the subject of the vignette. The faces and vignettes were presented simultaneously. All participants were required to read the vignettes aloud. In each trial, all participants answered the question accompanied by face–vignette pairs through a specially appointed keypad in a multiple choice pattern. The 13 obtainable choices for each trial were as follows: angry, happy, sad, fearful, disgusted, surprised, smug, guilty, hopeful, determined, pain, insulted as well as no emotion.

Example of a trial on the face–vignette task. The situational vignettes in English are as follows: This is a story about a girl's birthday. The girl stayed in her room. She received a call from her beloved boyfriend: “You're waiting for me at home. I'll bring your favorite flowers to your birthday!” Several minutes later, she heard the knock of her boyfriend's arrival. The question was “What emotion is the person feeling?” Responding with “surprise” will be recorded as a face response and responding with “hopeful” will be recorded as a vignette response. Additionally, any other response will be recorded as a random response.

On the FVT, the responses of the participants were labeled as face responses, vignette responses, and random responses. The response data were converted to proportions, which were used for statistical analysis.

Problem Solving Therapy Procedure

The PST was performed as described in a previous study ( 25 ). All the patients with MD were scheduled for PST, which consists of six sessions administered every other week. The treatment sessions were conducted at the psychological therapy room of the Psychiatry Department. The PST was conducted by six psychotherapists, and visits were conducted by two psychiatric resident physicians. All the psychotherapists owned a therapy handbook and underwent training, including a short theoretical course, role playing in a clinical background as well as watching a training videotape. The PST includes three steps: [1] the patient's symptoms are linked with their problems in daily living, [2] the problems are defined and clarified, and [3] an attempt is made to solve the problems in a structured way. The sessions lasted 1 h for the first visit and half an hour for the subsequent visits.

Clinical Interventions and Clinical Assessment

Two psychiatric residents examined all the participants to confirm or exclude a major depression diagnosis based on DSM-5 criteria and to collect medication and sociodemographic data. A HAMD (17-item version) was applied to assess the depressive severity for patients. A decrease of more than 50% in HAMD (17-item version) scores from baseline to follow-up was defined as a treatment response, and HAMD (17-item version) scores <7 at follow-up were defined as clinical remission.

HAMD (17-item version) and the FVT data were measured for the AT group and CATP group at baseline (before the first intervention, time 1) and after 12 weeks of interventions (time 2). The HC group was assessed using the face–vignette task only once. At baseline, both patients and HCs were required to complete the BFEIT.

Statistical Analysis

Data are presented as mean (SD), and all data were analyzed with Statistical Product and Service Solution 18.0 statistical software (SPSS 18.0, International Business Machines Corporation). Comparisons of the demographic data, basic facial emotion identification test scores, face response proportions, vignette response proportions, and random response proportions at baseline among patients and healthy controls were conducted using the method of one-way analysis of variance (ANOVA) or the chi-square test. Comparisons of HAMD (17-item version) scores, face response proportions, vignette response proportions, and random response proportions between baseline (time 1) and after 12 weeks of interventions (time 2) in the patient group were performed using 2 × 2 repeated-measures ANOVA. In this study, all alpha values of 0.05 were considered as statistically significant throughout. Cohen's d effect sizes were used for t -tests. The cutoff values for Cohen's d 's were defined as trivial effect size when d < 0.19, small effect size when 0.2 < d < 0.49, medium effect size when 0.5 < d < 0.79, and large effect size when d > 0.8. Partial eta-square (η p 2 ) effect sizes were used for F -tests. Similarly, the cutoff values for η p 2 were set as trivial effect size when η p 2 < 0.019, small effect size when 0.02 < η p 2 < 0.059, medium effect size when 0.06 < η p 2 < 0.139, and large effect size when η p 2 > 0.14. Phi (ϕ) effect sizes were used for chi-square test. The cutoff values for ϕ were set as trivial effect size when ϕ < 0.09, small effect size when 0.10 < ϕ < 0.29, medium effect size when 0.30 < ϕ < 0.49, and large effect size when ϕ > 0.50.

The Demographic Data of All Participants

The demographic data of the participants are described in Table 1 . No significant differences were observed in sex ratio, mean age, age range, or mean education years among the AT group, CATP group, and HC group.

Demographic characteristics and clinical data of all participants.

AT, antidepressant treatment; CATP, the combination of antidepressant treatment and PST; HC, healthy control; SD, standard deviation; η p 2 , partial eta-square .

Antidepressant Treatments

In the AT group, 20 patients with MD were antidepressant-naïve, and 11 patients with MD were antidepressant-free (six for at least 24 weeks and five for at least 4 weeks); patients with MD received fluoxetine ( n = 8), paroxetine ( n = 7), fluvoxamine ( n = 7), sertraline ( n = 6), or escitalopram ( n = 3). The mean fluoxetine-equivalent dose was 30.5 (8.8) mg/day. In the CATP group, 19 patients with MD were antidepressant-naïve, and 13 patients with MD were antidepressant-free (eight for at least 24 weeks and five for at least 4 weeks); patients with MD received fluoxetine ( n = 9), paroxetine ( n = 8), fluvoxamine ( n = 8), sertraline ( n = 3), or escitalopram ( n = 4). According to a previous report ( 26 ), the mean fluoxetine-equivalent dose was 30.1 (7.9) mg/day. Neither of the patient groups used concomitant medications.

Comparisons of BFEIT Performance Among the AT Group, CATP Group, and HC Group

As shown in Figure 2 , one-way ANOVA revealed that there were significant differences in BFEIT performance (emotion identification accuracy) among the AT group, CATP group, and HC group ( F 2,90 = 27.729, df = 2, η p 2 = 0.33, p = 0.000). Least square difference tests were performed as post hoc analyses and showed significant differences between the HC group, AT group, and CATP group (all p = 0.000). The emotion identification accuracy of the HC group was higher than that of the AT group or CATP group. However, no significant difference was observed between the AT group and the CATP group ( p = 0.951).

Comparisons of BFEIT performance among the AT group, CATP group, and HC group. BFEIT, basic facial emotion identification test; ATG, antidepressant treatment group; CATPG, the combination of antidepressant treatment and PST group; HC, healthy control; SD, standard deviation.

Comparisons of HAMD (17-Item Version) Scores Before and After Clinical Interventions

As shown in Figure 3 , using HAMD (17-item version) scores as dependent variables, a 2 × 2 repeated-measures ANOVA with group (AT group vs . CATP group) as a between-subjects factor and time point (time 1 vs . time 2) as a within-subjects factor revealed that the interaction effect for group × time point was not significant ( F 1,61 = 1.697, η p 2 = 0.003, p = 0.198); however, the main effect for time point was significant ( F 1,61 = 206.419, η p 2 = 0.35, p = 0.000), and the main effect for group was significant ( F 1,61 = 170.914, η p 2 = 0.18, p = 0.038). The 12-week interventions decreased HAMD (17-item version) scores in the two patient groups.

Comparisons of HAMD scores before and after clinical interventions between the AT group and CATP group. HAMD, Hamilton Depression Rating Scale (17-item version); ATG, antidepressant treatment group; CATPG, the combination of antidepressant treatment and PST group; time 1, baseline; time 2, after 12 weeks of intervention; SD, standard deviation.

There were significant differences in the remission rate between the CATP group (19/32) and the AT group (14/31); the remission rate in the CATP group was higher than that of the AT group (χ 2 = 6.123, ϕ = 0.29, p = 0.028). There were significant differences in the treatment response rate between the CATP group (25/32) and AT group (18/31); the treatment response rate in the CATP group was higher than that of the AT group (χ 2 = 4.370, ϕ = 0.26, p = 0.035).

Comparisons of FVT Performance Among the AT Group, CATP Group, and HC Group

Baseline level.

As shown in Table 2 , one-way ANOVA revealed that there were significant differences in face response proportions and vignette response proportions among the AT group, CATP group, and HC group ( F 2,90 = 27.861, 18.234, all df = 2; η p 2 = 0.32, 0.36, all p = 0.000). Least square difference tests were performed as post hoc analyses and showed significant differences between the HC group and AT group or between the HC group and the CATP group (all p = 0.000). The face response proportions of the HC group were lower than those of the AT group and CATP group, and the vignette response proportions of the HC group were higher than those of the AT group and CATP group. For the above-mentioned two variables, no differences between the AT group and CATP group were observed ( p = 0.951, 0.913).

Face–vignette task performances (%, SD) among the AT group, CATP group, and healthy control group.

AT, antidepressant treatment; CATP, the combination of antidepressant treatment and PST; Time 1, baseline; Time 2, after 12 weeks of interventions; F, face response proportions; V, vignette response proportions; R, random response proportions .

However, there were no significant differences in random response proportions among the AT group, CATP group, and HC group ( F 2,90 = 0.979, df = 2, η p 2 = 0.006, p = 0.380).

Before and After Interventions

As shown in Table 2 , using face response proportions, vignette response proportions, and random response proportions as dependent variables, a 2 × 2 repeated-measures ANOVA with group (AT group vs . CATP group) as the between-subjects factor and time point (time 1 vs . time 2) as the within-subjects factor was performed.

Face Response Proportions

The interaction effect for group × time point was significant ( F 1,61 =25.174, df =1, η p 2 = 0.30, p = 0.000), the main effect for time point was significant ( F 1,61 = 138.086, df = 1, η p 2 = 0.32, p = 0.000), and the main effect for group was significant ( F 1,61 = 4.853, df = 1, η p 2 = 0.24, p = 0.031).

Vignette Response Proportions

The interaction effect for group × time point was significant ( F 1,61 = 29.450, df = 1, η p 2 = 0.31, p = 0.000), the main effect for time point was significant ( F 1,61 = 144.130, df = 1, η p 2 = 0.32, p = 0.000), and the main effect for group was significant ( F 1,61 = 3.083, df = 1, η p 2 = 0.18, p = 0.041).

Random Response Proportions

The interaction effect for group × time point was not significant ( F 1,61 = 1.003, df = 1, η p 2 = 0.001, p = 0.320), the main effect for time point was not significant ( F 1,61 = 1.519, df = 1, η p 2 = 0.001, p = 0.223), and the main effect for group was not significant ( F 1,61 = 0.017, df = 1, η p 2 = 0.000, p = 0.897).

This study is the first to survey the effect of problem-solving therapy on effortful cognition in MD using FVT; measurements of the basic facial emotion identification were also conducted. Our data showed that the emotion identification accuracy of HCs was higher than that of patients with MD; patients with MD exhibited poor FVT performance. Compared to antidepressant treatment, PST plus antidepressant treatment resulted in lower HAMD (17-item version) scores and better FVT performance.

This study also investigated the ability of patients with MD to employ contextual information when determining the intended or expressed or signified message of facial emotional expressions. In the FVT, target facial emotional expressions are preceded by stories describing situational messages which are discrepant in affective valence. What both patients with MD and HCs had judged reflects either the dominance of the emotional context or the facial emotional expression. Many studies on cognitive processing by patients with MD reported that depressive symptoms interfere with effortful processing, and the degree of interference is determined by the degree of effort required for the task, the severity of depression, and the valence of the stimulus material to be processed. However, depressive symptoms only interfere minimally with automatic processes ( 27 ).

Consistent with the findings of previous studies ( 21 – 23 ), our results showed that patients with MD could not utilize contextual information for specific face–vignette pairs. However, HCs more extensively made good judgments on emotion in line with contextual information, which indicates that patients with MD display poor effortful cognition performance. Cognition dysfunctions in MD include impairments of social cognition and neurocognition ( 28 , 29 ). Social cognition refers to a process or a function for an individual's mental operations underlying social behavior, while neurocognition refers to those basic information processing functions such as attention and executive processes. Effortful cognitive processing was involved in either social cognition or neurocognition. We verified our hypothesis, i.e ., patients with MD present effortful cognitive dysfunction.

In this study, we confirmed that PST plus antidepressant treatments leads to a greater reduction of depressive symptoms, a greater response rate, and a greater remission rate over a period of 12 weeks than antidepressant treatments only in patients with MD. We also indirectly verified our previous hypothesis, i.e ., PST can improve effortful cognitive dysfunction, namely, PST improved the severity of MD by improving effortful cognition. Our data provide supporting evidence for the conclusion that the facial affect processing ability could be a valuable predictor of successful social context integration in FVT in MD.

Conclusions

In conclusion, patients with MD present effortful cognitive dysfunction, and PST can improve effortful cognitive dysfunction. The measurement of effortful cognition might be one of the indexes for the therapeutic effect of PST in MD.

There are some limitations in the study. First, the findings must be considered preliminary due to the small sample size. Second, healthy controls were assessed with the FVT only once; therefore, the results of the FVT would be influenced by the practice effect in patients with MD. Future studies should augment the sample size and eliminate the practice effect to further confirm the relationship between effortful cognition and PST in MD. Finally, this study investigated the effect of PST plus antidepressant treatment on effortful cognition in MD. Therefore, no outcome of the pure PST effect on effortful cognition was obtained. The examination of the pure PST effect on effortful cognition in MD is necessary in a future study.

Data Availability Statement

Ethics statement.

The studies involving human participants were reviewed and approved by Affiliated Wuxi Mental Health Center of Nanjing Medical University. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

CJ, HZ, and ZZ designed the study and wrote the paper. CJ, HZ, LC, and ZZ acquired and analyzed the data. All authors reviewed the content and approved the final version for publication.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank the Key Medical Talent Training Project of Jiangsu Province for providing support (project Grant No. ZDRCC2016019) for this research.

Funding. This research was supported by the Wuxi Taihu Talent Project (No. WXTTP2020008) and the Key Medical Talent Training Project of Jiangsu Province (No. ZDRCC2016019).