ORCID
Sydney Rothstein: 0009-0009-7406-9063
Kristen Patel: 0009-0005-0189-6325
Arthur Paul Pedersen: 0000-0002-2164-6404
Document Type
Article
Date
6-1-2026
Keywords
Intelligence Community, Analysis, Structured Analytic Techniques, Analysis of Competing Hypotheses
Language
english
Disciplines
Cognitive Psychology | Law | Psychology | Public Affairs, Public Policy and Public Administration
Description/Abstract
The present paper argues that decades of debate have not established whether Analysis of Competing Hypotheses (ACH) works in intelligence analysis. Reviewing more than fifty English-language studies and commentaries from 2025 – 2026, it identifies only eight empirical, direct tests of ACH relevant to intelligence work, none of which demonstrates success or failure at improving analytic decision-making. It traces a conceptual error in these otherwise methodologically sound experiments to Richards J. Heuer Jr.’s 1999 Psychology of Intelligence Analysis: researchers conflated Heuer’s account of satisficing with confirmation bias and evaluated individual accuracy, belief change, or hypothesis ranking rather than whether ACH’s matrix-based procedure improves the analytic process. Cataloged are five recurring design flaws — oversimplified scenarios; omission of unreliable or missing information; exclusion of hypothesis revision; limited transfer from non-US analytic cultures; and inadequate training. These flaws, rather than being evidence of ineffectiveness, have sustained skepticism about ACH. The paper recommends within-subjects designs, verified training, practitioner participants, complex real-world scenarios, tests of the matrix format, and closer collaboration between the intelligence community and academic researchers as the IC incorporates artificial intelligence into analytic workflows. It also proposes grouping structured analytic techniques by temporal orientation rather than cafeteria-style selection.
Recommended Citation
Rothstein, Sydney Rose; Patel, Kristen; and Pedersen, Arthur Paul, "Lessons Learned: Helping Researchers Help the Intelligence Community" (2026). Public Administration - All Scholarship. 8.
https://surface.syr.edu/ppa/8
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Included in
Cognitive Psychology Commons, Law Commons, Public Affairs, Public Policy and Public Administration Commons
Additional Information
This draft article has been submitted to the CIA's quarterly journal, Studies in Intelligence.