16.422 Human Supervisory Control Naturalistic Decision Making Massachusetts Institute of Technology
16.422 Human Supervisory Control Naturalistic Decision Making
Naturalistic Decision Making 16.422 How experts make decisions in the real world Descriptive method Stress, time-pressure, dynamic conditions, ambiguous information, and ill-defined goals Heuristics may introduce bias but can be very powerful Other related research areas Behavioral decision theory Judgment decision making Organizational decision making Group of behavioral scientists who shared common theme of "the importance of (D)time pressure, (2)uncertainty, (3)ill-defined goals, (4) high personal stakes, and(5)other complexities that characterize decision making in real-world settings. nd theme was to study people with expertise during the decision making process 3rd theme was that people appeared to size up situations with more importance than they would select between courses of action
Naturalistic Decision Making 16.422 • How experts make decisions in the real world – Descriptive method – Stress, time-pressure, dynamic conditions, ambiguous information, and ill-defined goals • Heuristics may introduce bias but can be very powerful. • Other related research areas: – Behavioral decision theory – Judgment decision making – Organizational decision making Group of behavioral scientists who shared com mon theme of “the importance of (1) time pressure, (2) uncertainty, (3) ill-defined goals, (4) high personal stakes, and (5) other complexities that characterize decision making in real-world settings.” 2nd theme was to study people with expertise during the decision making process 3rd theme was that people appeared to size up situations with more importance than they would select between courses of action
NDM VS Classic Decision Theory 16.422 People are not always rational decision makers But people are proficient Situation-action matching decision rules As opposed to choosing an alternate Context-bound informal modeling as opposed to context-free formal modeling 1g Process orientation as opposed to prediction of outcomes Empirical-based prescriptions
NDM vs. Classic Decision Theory 16.422 • People are not always “rational” decision makers – But people are proficient • Situation-action matching decision rules – As opposed to choosing an alternate • Context-bound informal modeling as opposed to context-free formal modeling • Process orientation as opposed to prediction of outcomes • Empirical-based prescriptions
Forms of NdM 16.422 Recognition primed decision making Related theories/methods Critical decision method Explanation based theory Image theor
Forms of NDM 16.422 • Recognition primed decision making • Related theories/methods – Critical decision method – Explanation based theory – Image theory
Recognition Primed Decision Making (RPD) 16.422 Intuitive form of diagnosis and prediction Pattern matching Multiple cues. highly correlated ey observations Requires learning and expertise Experts choose feasible course of action without analyzing all or even many options Decision making under uncertainty, time-pressure, stress Military commanders firefighters What biases might we see in RPD?
Recognition Primed Decision Making (RPD) 16.422 • Intuitive form of diagnosis and prediction – Pattern matching • Multiple cues, highly correlated – Key observations • Requires learning and expertise – Experts choose feasible course of action without analyzing all or even many options • Decision making under uncertainty, time-pressure, & stress – Military commanders & firefighters • What biases might we see in RPD?
RPD Process 16.422 Boundary conditions: expertise, time-pressure uncertainty/ill-defined goals Recognition Situation Understanding plausible goals critical cues expectancies typical actions · Serial evaluation Mental simulation
RPD Process 16.422 • Boundary conditions: expertise, time-pressure, uncertainty/ill-defined goals • Recognition • Situation Understanding – plausible goals – critical cues – expectancies – typical actions • Serial Evaluation • Mental Simulation
Coping With Uncertainty 16.422 Inadequate understanding, lack of information conflicted alternatives RAWFS heuristic Reducing uncertainty Gathering more information Assumption-based reasoning Filling in gaps Weighing pros and cons Forestalling Anticipate undesirable consequences Suppressing uncertainty Rationalization
Coping With Uncertainty • Inadequate understanding, lack of information, conflicted alternatives • RAWFS heuristic – Reducing uncertainty • Gathering more information – Assumption-based reasoning • Filling in gaps – Weighing pros and cons – Forestalling • Anticipate undesirable consequences – Suppressing uncertainty • Rationalization 16.422
Recognition/Meta-cognition Model 16.422 Another model (framework? for how decision makers cope with uncertainty Pattern matching critical What happens when recognition fails Decision makers revert to assumption-based reasoning and meta-cognitive processes Attempt to find flaws weaknesses in evaluation STEP: Construct a story, test, evaluate, lan A prescriptive approach
Recognition/Meta-cognition Model 16.422 • Another model (framework?) for how decision makers cope with uncertainty • Pattern matching critical • What happens when recognition fails? – Decision makers revert to assumption-based reasoning and meta-cognitive processes • Attempt to find flaws & weaknesses in evaluation • STEP: Construct a story, test, evaluate, & plan – A prescri ptive approach
NdM and Teams 16.422 How does team decision making differ from individual? Team sa and shared mental models Studying teams in their natural environment Real teams performing real tasks in real settings Contextual focus as opposed to more general and abstract Team research is not easy
NDM and Teams 16.422 • How does team decision making differ from individual? • Team SA and shared mental models • Studying teams in their natural environment – Real teams performing real tasks in real settings – Contextual focus as opposed to more general and abstract • Team research is not easy
CTA Methods for ndm research COGNITIVE TASK ANALYSIS Exploring the Current World Exploring the Envisioned World Understanding the way people operate aeowerng support for how peop e in their world w Operae ther wond rstand/model expertise, knowledge, strategies, and error Techniques thnographic Observational investigations Decision Method Structured Interview Techniques CTA Scratch odel stand/model complexities, mands, variability, and complicating Functional/Means-ends Analysis ethnographic /Observational investigations unctional Task and Workflow Modeling Discovering how to support the way the world Understanding the way the world works will work nege,c critical decision method in which participants are asked to describe a specific decision-making incident in detail and then to respond to probes seeking elaboration of important aspects of the decision sequences Semantic mapping(ak a, mind-mapping, idea mapping, word webbing, etc. )is a term which describes a variety of strategies designed to show how key words or concepts are related to one another through graphic representations. Mapping is an effective technique for teaching vocabulary and textual patterns of organization, and it is also effective for improving note taking and creative
CTA Methods for NDM Research • critical decision method in which participants are asked to describe a specific decision-making incident in detail and then to respond to probes seeking elaboration of important aspects of the decision sequences. • Semantic mapping (a.k.a., mind-mapping, idea mapping, word webbing, etc.) is a term which describes a variety of strategies designed to show how key words or concepts are related to one another through graphic representations. Mapping is an effective technique for teaching vocabulary and textual patterns of organization; and it is also effective for improving note taking and creative thinking skills. Goal : Understand/model expertise, knowledge, strategies, and error Dis c o v ering sup p ort for ho w p e o ple will o p erate in their w orld Techniques : Ethnographic / Observational investigations Critical Incident Technique Critical Decision Method Structured Interview Techniques Goal : Understand/model complexities, demands, variability, and complicating factors Techniques : Scratch C T A Model Growth of Understanding Time Time Dis c o v ering ho w to sup p ort the w a y the w orld Und erstanding the w a y the w orld w orks will w ork Design Basis Und erstanding the w a y p e o ple o p erate in their w orld E x plorin g th e C urre nt W orld Practitioner(s) Field of Practice The Domain CTA Representation Prototype Representation E x plorin g th e E n visio n e d W orld C O G NITIV E T A S K A N A L Y SIS Adapted from Carnegie Group, Inc. Semantic Mapping Functional/Means-ends Analysis Ethnographic / Observational investigations Functional Task and Workflow Modeling Structured Interview Techniques
