Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning
Authors: Yinghui He, Ling Yang, Jiarui Liu, Yongjin Yang, Lechen Zhang, Yingcheng Wu, Zhenfei Yin, Mengdi Wang, Sanjeev Arora
Affiliations: Princeton University, Carnegie Mellon University, University of Toronto, University of Illinois Urbana-Champaign, Stanford University, University of Oxford
Paper: https://arxiv.org/abs/2608.05139v1
Code: https://github.com/Gen-Verse/Skill-Entropy-RL
Dataset: https://huggingface.co/datasets/Gen-Verse/Skill2-Bench
TL;DR
WHAT was done? The authors introduce Skill Entropy, a directional metric quantifying the difficulty large language models experience when switching between distinct reasoning skills within a continuous reasoning chain. Using this metric, they construct Skill²-Bench, a benchmark spanning 558 skills across 9 verifiable and open-ended domains, and propose Skill-Entropy RL, a reinforcement learning framework that penalizes skill-switching misalignments while rewarding answer correctness.
WHY it matters? While modern frontier LLMs perform exceptionally well on isolated domain benchmarks, their accuracy degrades significantly when compelled to transition between different cognitive skills in multi-step tasks. Skill-Entropy RL demonstrates that explicitly supervising and rewarding structured skill transitions allows smaller open-source models (such as Qwen3-4B-Instruct) to close this cross-skill gap, outperforming standard outcome-based RL methods like GRPO and transferring effectively to off-the-shelf datasets like OpenR1-Math.
Executive summary: Current evaluations measure model competence in isolated silos such as code generation or symbolic math. However, real-world agentic execution requires fluidly chaining diverse skills within a single context. This paper identifies a fundamental failure mode in LLMs: models suffer a “cognitive inertia,” persisting with previous solution patterns rather than adapting to the skill required by the next step. By formalizing this switching friction as “Skill Entropy” and transforming it into an explicit reinforcement learning reward, the authors provide both an evaluation diagnostic and a post-training objective that significantly improves long-horizon, multi-domain reasoning performance.
Details
The Cross-Skill Transition Bottleneck in Long-Horizon Reasoning
Long-horizon autonomous execution demands that models execute complex chains of reasoning spanning diverse cognitive domains. A real-world agentic pipeline might require executing a mathematical calculation, translating that output into a logistical constraint, and ultimately incorporating those parameters into a qualitative textual report. While frontier systems demonstrate near-saturation performance on single-domain benchmarks like MMLU-Pro or LiveCodeBench, they exhibit brittle behavior when forced to execute multi-step chains requiring distinct reasoning modalities.
Previous evaluations fail to isolate why this breakdown occurs, treating long-horizon failure as a monolithic issue of context window degradation or cumulative error propagation. The central insight of this work is that domain competence in isolation does not imply smooth transition competence between domains. Models suffer from a specific form of cognitive inertia, carrying over the reasoning style, output format, and problem-solving strategy of step i−1 into step i, even when step i demands an entirely different skill set. This performance delta between isolated skill execution and cross-skill execution remains invisible to conventional benchmarks, creating a critical bottleneck for deploying agents in compositional, real-world workflows.
Directed Skill Entropy First Principles: Formalizing Transition Friction
To rigorously quantify the friction of switching cognitive contexts, the authors formalize the concept of pairwise directed Skill Entropy. Let S denote a finite set of fine-grained skills derived across domains D. Given a target model or reference model, let Accuracy(sa) represent the baseline accuracy when solving single-step tasks using skill sa∈S in isolation. Let Accuracy(sa,sb) represent the multi-step accuracy on a sequential task where step 1 demands skill sa and step 2 demands skill sb.
The pairwise directional Skill Entropy SkE(sa,sb) is defined as a smoothed ratio comparing isolated skill competence against chained transition competence:
Here, α=0.1 is a Laplace smoothing parameter ensuring numerical stability. When SkE(sa,sb)≈1, chaining skill sa to sb adds negligible cognitive friction over their standalone baselines. Conversely, values significantly greater than 1 signal high transition difficulty. Crucially, this quantity is asymmetric: SkE(sa,sb)≠SkE(sb,sa), reflecting the directional asymmetry of cognitive switching (e.g., transitioning from symbolic code execution to natural language summarizing imposes different constraints than the inverse).
To evaluate an extended multi-step task τ with an associated skill sequence μ(τ)=(s1,s2,…,sL) of length L∈[2,10], the overall task-level skill entropy SkE(τ) is computed by aggregating pairwise directional transition costs along the skill sequence trajectory:
To make pairwise matrix generation computationally tractable across ∣S∣=558 skills—which would otherwise require evaluating ∣S∣2≈3.1×105 ordered pairs—the framework factorizes skill-to-skill entropy through domain-level transitions. Pairwise skill entropy is approximated as SkE(sa,sb)≈SkE(sa,ds_b)⋅SkE(ds_a,sb), where ds represents the source domain of skill s. This reduces evaluation complexity from O(∣S∣2) down to O(∣S∣⋅∣D∣).
Chaining Heterogeneous Cognition: From Problem Definition to Execution
The Skill²-Bench dataset operationalizes these theoretical definitions into a benchmark suite spanning 558 skills across 9 distinct domains (Math, Science, Coding, Logic, Information Extraction, Planning, Creative Writing, Context Retrieval, and Instruction Following), as illustrated in Figure 1. Tasks are explicitly synthesized at controlled low, medium, and high task-level skill entropy thresholds, derived under a strong reference model (Claude-opus-4.7).
To understand how an input traverses a cross-skill long-horizon task, consider the case study featured in Figure 3. The system receives a unifying scenario σ: analyzing spatial patterns in a museum’s right-triangle-themed paintings. The task requires executing a two-step skill chain μ(τ)=(Geometric Calculation,Theme Creation). Step 1 prompts the model to compute the physical area of a specified painting. Step 2 requires using the contextual state and result from Step 1 to draft a qualitative exhibition theme.
When evaluated on standard base models, a severe failure mode occurs during Step 2. As visualized in Figure 3 and analyzed in Figure 5, the unaligned model suffers from modality persistence. Having executed a geometric formula in Step 1, the model emits a short numeric response for Step 2 (”Painting R”) rather than executing the requested creative text generation. The model fails to adjust its internal solution template to match the required skill shift.
As demonstrated in Figure 2, per-domain skill entropy is decoupled from single-domain accuracy; for instance, science tasks yield high single-skill accuracy yet exhibit severe skill-switching friction when combined with adjacent domains.
Optimization Strategy and Implementation Mechanics
To address this failure mode, the authors convert Skill Entropy from an offline evaluation diagnostic into an online reinforcement learning signal via Skill-Entropy RL. The core architecture relies on an explicit model commitment mechanism during rollout generation. Rather than directly generating solutions, the policy network is structured to emit explicit domain and skill tags prior to generating each step’s solution trace. The formatted rollout follows a structured template:
Optimization is conducted using Group Relative Policy Optimization (GRPO). Given a prompt and task τ, the policy emits a predicted skill sequence μ̑(τ)=(s̑1,…,s̑L) alongside step answers (ȃ1,…,ȃL). The scalar reward function r combines step-level correctness ransrans with a structural skill-entropy reward rent:
r=λansrans+λentrent
The answer reward
computes the mean per-step accuracy across domain-specific verifiers (e.g., symbolic equivalence for mathematics, sandboxed unit test execution for coding, or constraint satisfaction for planning tasks). The skill-entropy reward rentrent assesses whether the model’s self-predicted skill trajectory reflects the actual cognitive difficulty of the task.
Defining
as the empirical CDF rank transformations of predicted and gold task-level skill entropies over the training set, the entropy reward is formulated as:
Using CDF rank-normalization ensures that rent∈[0,1] remains scale-free across task difficulty distributions. To resolve lexical discrepancies when a model emits skill tags outside the canonical dictionary (e.g., predicting “integer_arithmetic” instead of “number_theory”), the training pipeline embeds generated tags using Qwen3-Embedding-0.6B and performs cosine similarity matching against the canonical skill bank. Emitted tags below a 0.5 cosine similarity threshold are categorized as out-of-bank and receive zero entropy reward.
Training hyperparameters are rigorously documented: RL policy training uses AdamW with an actor learning rate of 1×10−6, a fixed KL coefficient of 1×10−3, PPO clip ratio of 0.2, GRPO group size of 8, and a prompt batch size of 256. The reward weights are balanced at λans=0.7 and λent=0.3. Experiments are conducted on hardware consisting of an 8×H100 GPU cluster.
Empirical Validation: Uncovering and Closing the Transition Gap
The evaluation across 8 frontier models (including Claude-opus-4.7, GPT-5.5, and Gemini-3.1-pro) and 4 open-source baselines on Skill²-Bench confirms that task difficulty scales directly with task-level skill entropy (Table 2).
Across all models, accuracy decreases monotonically as task-level skill entropy transitions from low to high. Even high-performing frontier models show accuracy drops of −4% to −10% when a skill is embedded within a multi-step cross-skill task compared to its standalone single-skill performance.
As detailed in Table 3, applying Skill-Entropy RL to Qwen3-4B-Instruct achieves an overall Skill²-Bench score of 68.4%, significantly outperforming standard GRPO (58.8%), as well as skill-aware baselines such as STAT (61.4%), SkillRL (59.3%), and Skill-Distill (58.1%). Similar relative gains are recorded on Qwen3-1.7B, improving from a base performance of 14.6% up to 40.1%.
As shown in Table 14, the cross-skill performance of Skill-Entropy RL (68.4%) exceeds even the single-skill oracle score (62.2%) where steps are evaluated in isolation without surrounding scenario context.
Ablation experiments reported in Table 13 justify the choice of reward weights λans=0.7,λent=0.3. Removing the skill-entropy reward entirely (λent=0.0, corresponding to standard GRPO) causes a 9.6 percentage point drop in performance. Conversely, over-indexing on skill entropy (λent=0.7) degrades performance to 48.6%, confirming that outcome correctness must remain the primary optimization target while skill entropy serves as a structural regularizer.
Furthermore, the authors demonstrate the reusability of this signal on off-the-shelf datasets. When applied to OpenR1-Math reasoning traces (by segmenting traces and labeling steps via an automated Qwen3-8B annotator), Skill-Entropy RL prevents training reward saturation (Figure 4) and achieves superior downstream generalization across six mathematical benchmarks (Table 11), outperforming standard GRPO by +1.9% on average.
Placing Skill Entropy in the Post-Training Landscape
This work bridges three distinct subfields in post-training research: skill-aware data composition, agentic long-horizon evaluation, and dense reinforcement learning supervision.
Prior skill-aware frameworks—such as STAT, Instruct-SkillMix, and Symbolic MoE—leveraged skill tags primarily at the data filtering, routing, or prompt synthesis layers. Similarly, agentic benchmarks like GAIA and AgentBench measured multi-step performance, but conflated environmental tool interaction with underlying cognitive switching capabilities. Meanwhile, dense reward methods like Process Reward Models (PRMs) and self-revision rewards (DeepSeek-R1) focused heavily on densifying feedback within a single reasoning step.
Skill-Entropy RL differentiates itself by targeting transition difficulty between steps. Rather than altering routing or context construction, it injects structural transition difficulty directly into the policy optimization objective. This approach acts orthogonally to PRMs: while PRMs verify intra-step mathematical or logical correctness, Skill Entropy enforces inter-step structural adaptability.
Critical Assessment and Structural Limitations
Despite its strong empirical performance, the proposed methodology presents several trade-offs that warrant critical examination:
Reference Model Dependence: The derivation of pairwise Skill Entropy SkE(sa,sb) relies on offline execution logs from a fixed reference model (Claude-opus-4.7). Although the authors perform sensitivity analyses showing partition overlap remains above 80% when swapping reference models to Gemini-3.1-pro or GPT-5.5 (Table 7), the absolute entropy values remain inherently bounded by the capabilities of the chosen reference architecture.
Computational Overhead of Skill Factorization: Computing matrix estimates across fine-grained skills requires thousands of Monte Carlo sampling passes under strict temperature constraints. Although the domain factorization trick reduces complexity from O(∣S∣2) to O(∣S∣⋅∣D∣), scaling this approach to dynamic or open-ended skill libraries during continuous pre-training remains computationally expensive.
Open-Ended Evaluation Variance: Evaluating open-ended domains (Creative Writing, Context Retrieval, Instruction Following) relies on an LLM judge (Claude-opus-4.7). While the authors validate judge alignment against human annotators (Pearson r=0.88, Table 10), automated evaluation of qualitative outputs introduces potential judge biases that outcome-verifiable domains (such as sandboxed code execution) do not suffer from.
Strategic Takeaways and Future Trajectory
The formulation of Skill Entropy provides a valuable conceptual framework for understanding multi-step LLM failures. It demonstrates that long-horizon degradation is not merely a function of sequence length or memory loss, but is driven by transition friction between distinct cognitive domains.
For post-training research teams, the takeaways are immediate: outcome-only RL objectives (GRPO) leave substantial performance on the table by ignoring step-to-step structural transitions. Incorporating explicit skill-planning tags and rewarding task-level entropy alignment offers an efficient method to improve compositional reasoning without modifying data pipelines or inference architectures. Expanding Skill-Entropy supervision to tool-use environments, multi-modal task switching, and real-time agentic execution represents a promising direction for future research.
