Why Large Language Models Fail at Tabular Prediction
Authors: Marta Garnelo, Wojciech M. Czarnecki
Paper: https://arxiv.org/abs/2608.02412v1
Code: N/A
Model: N/A
TL;DR
WHAT was done? The authors systematically isolate the precise cause of large language model (LLM) failure on tabular classification by evaluating five common folklore hypotheses in a pure inference regime. Using a rigorous data-hygiene memorisation probe, controlled feature manipulations, and random-projection dimensionality sweeps across benchmark and synthetic datasets, the study falsifies four popular explanations—class overlap, serialized CSV formatting, numeric tokenization, and per-query test load—and identifies input dimensionality as the sole factor driving the performance collapse of models like claude-opus-4-6.
WHY it matters? This work provides the first causal explanation for the persistent performance gap between general-purpose language models and classical machine learning baselines on tabular data. By demonstrating that LLM capabilities collapse as feature dimensions grow—even when information content is held strictly constant—it establishes clear architectural limits for in-context tabular reasoning, refutes naive prompt-engineering fixes, and provides strong theoretical justification for dedicated tabular foundation models (see TabFM btw).
Executive summary: Applied machine learning practitioners frequently observe that frontier language models lose to fifty-year-old algorithms on simple numeric tables. Rather than adding complex prompt scaffolding or agent loops, this paper probes the raw model in pure in-context inference mode to understand why. The findings reveal that claims of LLM failure due to awkward CSV text formatting, tokenization of numbers, or noisy class boundaries are completely unfounded. Instead, LLMs act as effective local distance-based classifiers in two-dimensional space but experience a catastrophic breakdown as feature count increases beyond low dimensions. Consequently, attempting to fix high-dimensional tabular prediction through prompt adjustments is fundamentally misguided, confirming the necessity of specialized tabular architectures for tabular workloads.
Details
The Tabular Anomaly in In-Context Learning
Large language models have demonstrated impressive capabilities across natural language processing, code synthesis, and multi-step reasoning tasks. However, predictive analytics over tabular data remains a glaring exception where general LLMs systematically underperform compared to traditional tree ensembles and simple distance-based baselines, as illustrated in Figure 1.
While the prevailing response across the machine learning community has been to layer external software harnesses—such as retrieval-augmented generation, automated feature engineering agents, and multi-turn loops—around the core model, this approach obscures whether the failure stems from the model itself or the surrounding infrastructure. To address this fundamental question, the investigation evaluates the core model under a pure inference regime: a single user turn containing the complete training set and test queries without system prompts, external tools, code execution, or task-specific fine-tuning.
The underlying conflict in tabular learning lies in the stark contrast between human natural language processing and structured data matrices. Standard tabular classifiers, such as gradient boosted decision trees or nearest-neighbor algorithms, explicitly exploit vertical column structures, invariant feature coordinate systems, and continuous metric spaces. In contrast, standard autoregressive transformers operate on linear token sequences where spatial tabular topology is flattened into text strings. The study aims to delineate whether the LLM’s failure on tabular data reflects an intrinsic limitation of sequence models in high-dimensional feature spaces or merely secondary issues related to text serialization, numeric token precision, or dataset noise.
