ASTRA: Adaptive Semantic Tree Reasoning Architecture for Complex Table Question Answering
A new LLM reasoning architecture claims state-of-the-art accuracy on complex table question answering by converting tables into searchable logic trees.
The Thesis
Most large language models (LLMs) struggle to answer questions about complex tables — spreadsheets with nested headers, merged cells, or multi-level hierarchies — because they flatten everything into plain text and lose the structure. ASTRA tries to fix this by first converting a table into a 'Logical Semantic Tree,' a hierarchical representation that preserves parent-child relationships between cells. A second component called DuTR (Dual-mode Tree Reasoning) then answers questions by combining two strategies: navigating the tree as text and running symbolic code to verify numeric results. The appeal is that structured reasoning over explicit hierarchies should catch errors that pure text-based approaches miss. The catch is that the benchmarks are academic, the method requires LLM calls to build the tree itself, and real enterprise tables can be far messier than anything in a dataset.
Catalyst
LLMs capable of generating structured, hierarchically valid representations — a prerequisite for this approach — have only become reliably good enough in the past year or two, driven by models like GPT-4 and its successors. Benchmark datasets for complex table QA (such as HiTab and WikiTableQuestions) have also matured, giving researchers reproducible targets. The convergence of capable code-execution sandboxes (used here for symbolic verification) with strong semantic models made the dual-mode approach tractable now.
What's New
Earlier table QA methods either serialize tables as flat markdown or comma-separated text — discarding hierarchy — or use rigid template-based trees that cannot adapt when a table changes in size or structure. Systems like TAPAS (Google's table-parsing model) and DITTO used learned encoders but lacked explicit symbolic verification. ASTRA's claimed advance is an adaptive construction step: the LLM chooses a different tree-building strategy depending on the table's scale, then validates numeric answers with code execution rather than pattern-matching alone.
The Counter
The paper claims state-of-the-art results, but the benchmarks it targets — HiTab, WikiTableQuestions, and similar academic datasets — contain tables that are clean, well-formatted, and annotated by humans. Real enterprise tables are none of those things: they have inconsistent units, missing values, multi-sheet dependencies, and undocumented abbreviations. The Logical Semantic Tree construction is itself an LLM call, which means errors compound — a wrong tree leads to a wrong answer with no obvious failure signal. The dual-mode framework (text navigation plus code execution) also adds latency and cost over simpler approaches, which matters a lot for production deployments. There is no ablation showing the tree construction step is actually responsible for the gains versus simply using a stronger base model or better prompting. Competitors like Microsoft's TAPEX and newer GPT-4-native table-reasoning prompts are strong baselines that the paper may not have fully stress-tested. Until these results are replicated on messy real-world enterprise data with independent evaluation, the claimed gains should be treated as promising but preliminary.
Longs
- MSFT — Copilot for Excel and Power BI directly benefit from better table reasoning in LLMs
- CRM — Salesforce Einstein Analytics tables are a natural deployment surface
- SSNC — SS&C Technologies processes complex financial tables at scale; improved QA reduces analyst overhead
- BBAI — BigBear.ai focuses on enterprise AI analytics where structured data reasoning matters
- BOTZ (robotics/automation ETF) — indirect exposure to enterprise AI productivity gains
Shorts
- TAPAS-style flat-encoder table models — their fixed-schema approach is the direct foil ASTRA argues against; startups or teams invested in that paradigm face displacement if ASTRA's gains hold
- Spreadsheet BI vendors using naive LLM integrations — products that rely on simple cell-dump serialization for natural-language queries would look worse in direct comparisons
Enablers (Picks & Shovels)
- OpenAI API / GPT-4 class models — the tree construction step in ASTRA depends on a capable base LLM
- Python code execution sandboxes (e.g., E2B, Modal) — DuTR's symbolic verification requires safe runtime environments
- HiTab and WikiTableQuestions benchmark datasets — open evaluation infrastructure that made reproducible comparison possible
- LangChain / LlamaIndex — orchestration frameworks that would be used to productionize tree-search reasoning pipelines
Private Watchlist
- Glean — enterprise search startup that frequently ingests tabular data from internal systems
- Hebbia — AI research platform built around complex document and table analysis for finance and law
- Vectara — semantic search and RAG infrastructure that would need better table handling to serve enterprise clients
- Aeye Analytics (private) — focused on automated financial document parsing
Resources
The Paper
Table serialization remains a critical bottleneck for Large Language Models (LLMs) in complex table question answering, hindered by challenges such as structural neglect, representation gaps, and reasoning opacity. Existing serialization methods fail to capture explicit hierarchies and lack schema flexibility, while current tree-based approaches suffer from limited semantic adaptability. To address these limitations, we propose ASTRA (Adaptive Semantic Tree Reasoning Architecture) including two main modules, AdaSTR and DuTR. First, we introduce AdaSTR, which leverages the global semantic awareness of LLMs to reconstruct tables into Logical Semantic Trees. This serialization explicitly models hierarchical dependencies and employs an adaptive mechanism to optimize construction strategies based on table scale. Second, building on this structure, we present DuTR, a dual-mode reasoning framework that integrates tree-search-based textual navigation for linguistic alignment and symbolic code execution for precise verification. Experiments on complex table benchmarks demonstrate that our method achieves state-of-the-art (SOTA) performance.