Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs
A survey paper maps the overlooked territory of 'abductive reasoning' in AI — inferring the best explanation — and finds current large language models are surprisingly weak at it.
The Thesis
Abductive reasoning is the kind of thinking a doctor uses when they see symptoms and infer the most likely diagnosis — not just following a rule (deduction) or spotting a pattern (induction), but generating and selecting the most plausible explanation for incomplete evidence. This survey is the first to systematically map how well large language models (LLMs) handle this capability, and the short answer is: not well, and the field doesn't even agree on how to measure it. The authors propose a two-stage framework — 'Hypothesis Generation' followed by 'Hypothesis Selection' — to bring conceptual order to a scattered literature. This matters because abductive reasoning underlies diagnosis, scientific discovery, legal inference, and debugging, all areas where AI is being deployed. The catch is that this paper is a survey and benchmark study, not a new algorithm — it identifies the problem more than it solves it.
Catalyst
LLMs have matured enough to be seriously evaluated on complex reasoning tasks beyond simple question-answering, making a systematic audit both possible and overdue. At the same time, commercial deployment of LLMs in high-stakes domains like medicine, law, and software development has accelerated pressure to understand exactly where these models fail — abductive gaps are a plausible explanation for hallucinations and confident-but-wrong outputs. The authors also note that benchmark saturation on deductive and inductive tasks has pushed researchers to look harder at what remains unsolved.
What's New
Prior work on LLM reasoning largely focused on deductive tasks (e.g., math proofs, logical entailment) and inductive tasks (e.g., pattern completion), with abductive reasoning treated as an afterthought or conflated with the others. Earlier abductive benchmarks — such as αNLI (Abductive Natural Language Inference) and ART (Abductive Reasoning in Narrative Text) — tackled narrow slices of the problem without a unifying framework. This paper synthesizes those scattered efforts under a single two-part taxonomy and adds a comparative benchmark study across model sizes and families, enabling apples-to-apples comparisons for the first time.
The Counter
This is a survey paper with a compact benchmark study attached — it classifies and measures a problem rather than advancing the capability. The two-stage framework (Hypothesis Generation vs. Hypothesis Selection) is conceptually tidy, but it is not obvious that this taxonomy will stick any better than the 'disjointed' prior definitions the authors criticize. The benchmark study is described as 'compact,' which likely means limited model coverage and narrow task diversity — not enough to draw strong quantitative conclusions. More fundamentally, whether abductive reasoning is genuinely distinct from well-calibrated deductive and inductive reasoning combined is an open philosophical question that the paper doesn't resolve. Practitioners deploying LLMs in medical or legal settings are unlikely to change their evaluation pipelines based on a survey paper; they need new training methods or architectures, neither of which this paper provides. The modest cultural momentum score (0.5) suggests the research community itself hasn't yet rallied around this framing.
Longs
- MSFT — Azure OpenAI deployments in clinical and legal reasoning
- C3.ai (AI) — enterprise AI applications where explainability is a selling point
- CDNS — electronic design automation tools that need fault-hypothesis reasoning
- GOOGL — DeepMind and Gemini research on reasoning benchmarks
- ARKG (ARK Genomic Revolution ETF) — genomics AI platforms that need diagnostic inference
Shorts
- Vendors selling LLM-based diagnostic or legal inference tools who have not stress-tested abductive failure modes — this paper provides a roadmap for regulators and buyers to probe those gaps
- Benchmark-as-a-service startups whose static test sets are specifically called out as inadequate for measuring abductive reasoning
Enablers (Picks & Shovels)
- Hugging Face — open model hub used for benchmarking across model families
- EleutherAI lm-evaluation-harness — open-source framework for standardized LLM benchmarking
- AllenAI — original creators of αNLI and ARC benchmarks foundational to abductive NLP tasks
- arXiv — rapid dissemination of reasoning benchmark papers that this survey synthesizes
Private Watchlist
- Cohere — enterprise LLM provider focused on business reasoning tasks
- Ought (now Elicit) — AI research assistant built on structured reasoning over evidence
- Imbue — AI lab explicitly focused on reasoning and agency in language models
- Contextual AI — retrieval-augmented generation for enterprise reasoning applications
Resources
The Paper
Regardless of its foundational role in human discovery and sense-making, abductive reasoning--the inference of the most plausible explanation for an observation--has been relatively underexplored in Large Language Models (LLMs). Despite the rapid advancement of LLMs, the exploration of abductive reasoning and its diverse facets has thus far been disjointed rather than cohesive. This paper presents the first survey of abductive reasoning in LLMs, tracing its trajectory from philosophical foundations to contemporary AI implementations. To address the widespread conceptual confusion and disjointed task definitions prevalent in the field, we establish a unified two-stage definition that formally categorizes prior work. This definition disentangles abduction into \textit{Hypothesis Generation}, where models bridge epistemic gaps to produce candidate explanations, and \textit{Hypothesis Selection}, where the generated candidates are evaluated and the most plausible explanation is chosen. Building upon this foundation, we present a comprehensive taxonomy of the literature, categorizing prior work based on their abductive tasks, datasets, underlying methodologies, and evaluation strategies. In order to ground our framework empirically, we conduct a compact benchmark study of current LLMs on abductive tasks, together with targeted comparative analyses across model sizes, model families, evaluation styles, and the distinct generation-versus-selection task typologies. Moreover, by synthesizing recent empirical results, we examine how LLM performance on abductive reasoning relates to deductive and inductive tasks, providing insights into their broader reasoning capabilities. Our analysis reveals critical gaps in current approaches--from static benchmark design and narrow domain coverage to narrow training frameworks and limited mechanistic understanding of abductive processes...