[ICML 2026] Position: The Alignment Community is Unintentionally Building a Censor’s Toolkit
Authors: Sarah Ball, Phil Hackemann
Paper: https://openreview.net/forum?id=dy2HwmOvFX
Site: https://s-ball-10.github.io/censors-toolkit/
Code: N/A
Model: N/A
This paper has the ICML 2026 Outstanding Position Paper Award.
TL;DR
WHAT was done? The authors present a highly systematic socio-technical analysis mapping modern AI alignment techniques—specifically across data curation, preference learning, and inference filtering—to their dual-use potential for state-level censorship and informational manipulation.
WHY it matters? This work fundamentally challenges the assumption that AI safety research is inherently benevolent, demonstrating that the exact technical mechanisms engineered to protect humanity can be easily repurposed for authoritarian informational control. This crucial paradigm shift earned the paper the prestigious ICML 2026 Outstanding Position Paper Award.
Details
The Convergence of Safety and Suppression
The rapid integration of large language models (LLMs) into the global information infrastructure has altered how humanity accesses knowledge. To prevent these systems from generating toxic, deceptive, or outright dangerous outputs, the research community has focused heavily on refining alignment techniques. However, this pursuit has proceeded under the implicit, comfortable assumption that alignment is a value-neutral, unilaterally positive endeavor. The authors challenge this orthodoxy by establishing a critical delta from prior literature: instead of treating censorship as an isolated regional failure or debating AI risks from a speculative, high-level perspective, they frame the technical alignment stack itself as an inherently dual-use technology. They argue that the optimization of safety safeguards is functionally equivalent to the optimization of an extremely efficient, low-cost censor’s toolkit. This realization is particularly urgent given the current geopolitical landscape, which is defined by an oligopolistic LLM market and a global trend toward digital authoritarianism.
Defining the Vectors of Informational Control
To analyze this risk from first principles, it is necessary to define the core technical and ethical substrates of the problem. Mathematically, a language model acts as a parameterized conditional probability distribution Pθ(y∣x), mapping a user prompt x to a generated sequence y. Alignment is the process of modifying Pθ so that the generated outputs conform to a specific set of human values. However, because these values must be explicitly defined by whoever controls the training process, the system is fundamentally purpose-agnostic. The authors define “misuse” strictly through the lens of international human rights, particularly the freedom of thought and expression. Under this framework, informational control manifests in two primary vectors: censorship and manipulation. Censorship is the complete suppression of information, technically achieved by forcing Pθ(y∣x)≈0 for any response y containing target information. Manipulation is more subtle, involving the systematic biasing of the distribution to steer the expected properties of the output y toward a specific ideological viewpoint without the user’s explicit awareness.
Anatomy of the Three-Stage Control Stack
To understand how these vectors are realized, we can examine how a single sensitive prompt flows through the modern three-stage control stack, as detailed in Table 1. Consider a user querying a model about a politically sensitive historical event, such as the 1989 Tiananmen Square massacre.
The workflow begins with pre-training data filtering, where heuristic keyword lists or model-based classifiers are used to clean the training corpus. If the state or provider completely excises all documents containing the target keywords, the base model’s parameters never learn the probability distribution of those historical facts. Consequently, the model cannot generate information about the event because it literally does not know it exists.
Second, if some information survives pre-training, the model undergoes post-training preference alignment. Here, methods like Reinforcement Learning from Human Feedback (RLHF) or Constitutional AI are applied to optimize the policy. If the preference dataset is curated to penalize discussions of the historical event, the model learns to systematically refuse the query, redirection-steering its parameters away from the sensitive knowledge space.
Third, if both prior defenses fail, inference-time controls act as a final runtime guardrail before output generation. The model provider can prepend a hidden system prompt instructing the model to decline political discussion, or deploy independent safety classifiers like Llama Guard to scan the generated tokens in real time. If a trigger phrase is detected, the generation is abruptly terminated and replaced with a generic refusal message.
Technical Barriers and Modification Depth
The engineering characteristics of these three alignment stages reveal a stark trade-off between the depth of modification and the ease of execution, as summarized in Table 1. Pre-training data filtering requires direct control over the data pipeline and massive computational resources to train a model from scratch, but it results in a fundamental and highly persistent modification of the model’s capabilities. Post-training alignment, which can be executed via Direct Preference Optimization (DPO), represents a moderate-to-high technical and computational barrier. The DPO objective is formulated as:
In this objective, θ represents the parameters of the target policy πθ, πref is the reference base model, D is the dataset of pairwise preferences containing a preferred aligned response yw and a dispreferred response yl for a given prompt x, β is a hyperparameter scaling the constraint, and σ is the standard sigmoid function. By substituting state-approved narratives as yw and neutral historical facts as yl, malicious actors can directly align the model parameters to enforce an ideology. Conversely, inference-time interventions require negligible computational resources and can be modified instantaneously. While superficial and easily bypassed by sophisticated jailbreaks, inference-time controls provide maximum operational flexibility for real-time censorship updates.
Empirical Realities of Alignment Weaponization
The paper substantiates its conceptual framework with compelling real-world evidence showing that this weaponization is actively occurring. In China, the Cyberspace Administration (CAC) mandates that generative models adhere to “socialist core values,” requiring model providers to maintain extensive refusal datasets. Consequently, commercial Chinese models like Baidu’s Ernie Bot and DeepSeek systematically refuse queries regarding state-disapproved historical events while actively amplifying official government narratives. This manipulation has led to systemic “preemptive obedience,” where Western LLMs trained on filtered global datasets show self-censorship when queried in Simplified Chinese. Furthermore, the threat is not limited to state actors; powerful individuals also abuse these tools. For example, the alignment of Grok was unilaterally shifted via system prompt changes to match specific political perspectives, which unexpectedly triggered highly biased outputs, including the generation of antisemitic content and Holocaust denial. Additionally, real-time inference filtering was documented in the model Yi-large, which abruptly substituted a critical response about political leadership with a refusal mid-generation, exposing the active role of runtime output classifiers.
Positioning in the Broader AI Safety Narrative
This position paper stands in contrast to the dominant literature in AI safety, which predominantly focuses on protecting models from adversarial exploitation, such as JailbreakBench. While existing works have evaluated localized bias or documented regional chatbot censorship, the authors synthesize these phenomena into a unified socio-technical theory. They build directly upon recent developments in PRISM alignment dataset curation and studies on model representation, but invert the narrative. Instead of viewing alignment purely as an defense against harms, they draw parallels to the history of nuclear physics to illustrate how safety-critical optimizations naturally yield weapons of mass informational control.
The Feasibility Gap in Countermeasures
Despite the strength of its diagnosis, the paper’s proposed mitigations face substantial real-world implementation challenges. The authors advocate for “Neutrality Through Diversity” by fostering model pluralism to prevent monopolistic concentrations of power. However, this proposal struggles against the economic realities of frontier model training, where astronomical compute costs and data access barriers naturally drive oligopolistic consolidation. Additionally, while the authors suggest that model jailbreaks can serve as a form of “freedom insurance” to bypass restrictive systems, this viewpoint overlooks the intense security trade-offs. The same bypasses that allow citizens in authoritarian regimes to access censored history also enable malicious actors to exploit models for cyber-attacks or automated propaganda campaigns. Finally, building globally accepted, objective benchmarks to distinguish between legitimate moderation and malicious censorship remains an open, deeply subjective sociopolitical challenge.
Strategic Outlook and Verdict
This award-winning paper acts as a necessary and highly professional wake-up call for the AI safety community. By reframing alignment technologies as dual-use instruments of power, it forces researchers to look past the technical mechanics of optimization and confront the political reality of who defines the alignment objectives. The recognition of this paper at ICML 2026 highlights a growing institutional awareness that safety cannot be developed in a geopolitical vacuum. Moving forward, the research community must transition away from pursuing a singular, top-down “perfectly aligned” system. Instead, the strategic priority must shift toward developing open, verifiable alignment auditing standards and ensuring a diverse ecosystem of models that protect intellectual freedom and democratic self-determination.
