CLM-8B: Open System One Model Scores Agent Actions at High Speed

The research organization Contrastive-LM has unveiled CLM-8B, marking the introduction of an open model in a category known as Contrastive Language Models (CLMs). Unlike traditional autoregressive large language models that generate text token by token, CLM-8B does not produce text. Instead, it evaluates a declared set of candidate actions against the current state of an environment and outputs a probability distribution across those options. This architectural shift specifically targets decision-making and routing steps within autonomous agent loops, where generation overhead often introduces substantial latency.

The primary baseline for CLM-8B is Jev, a proprietary System One model developed by TypeSafe AI that entered limited early access on September 15, 2026. Like Jev, CLM-8B is engineered to deliver structured categorical choices rather than generative prose. To ensure straightforward integration, the CLM repository serves the model through a TypeSafe-compatible interface. This design supports three specific query formats: Noul, which computes the probability of a statement being true; Choice, which selects a single candidate from an explicit list with assigned probabilities; and Score, which rates an input against an ordered rubric.

Dual Encoders and Vector Caching in Agent Loops

At the architectural level, CLM-8B uses a dual-encoder design comprising a state encoder and an action encoder. Both encoders share a frozen Qwen3-8B backbone paired with a trainable 20-million-parameter projection head. Training optimizes a bidirectional InfoNCE loss objective, which pulls the embedding of a given state toward the embedding of the action that was actually executed while pushing it away from unselected candidate actions. During inference, candidate actions are ranked via the dot product of their embeddings with the state embedding, followed by a softmax function to determine the probability distribution.

This separation between state and action representations allows for compute optimizations. In practical agent workflows, the available action set often remains static while the environment state changes at every turn. Through its dedicated serving component, clm-serve, the system reserves a dedicated slab of GPU memory to cache precomputed action vectors, drawing architectural inspiration from standard key-value caching mechanisms. When tested on a single NVIDIA RTX 4090 GPU with three candidate actions, reusing cached vectors reduced latency from 1.7 milliseconds to 0.6 milliseconds for revisited states. On large option sets containing approximately 1,000 candidate actions, the model card reports inference speeds up to 13 times faster than Jev.

The training pipeline for CLM-8B relies on a three-stage sequential recipe designed to refine ranking accuracy without destabilizing the frozen base encoder:

• Pre-training on approximately 60 million Nemotron DQA question-and-answer pairs, achieving 52.1% top-1 accuracy on a held-out test suite of 100,000 questions.
• Mid-training on approximately 30 million synthetic hard negative examples generated with Gemini 2.5 Flash-Lite, which increases top-1 accuracy to 69.2%.
• Post-training on approximately 1 million agent trajectories sourced from the Agent Data Protocol, Endless-Terminals, and LiteCoder-Terminal-SFT datasets.

Contrastive-LM researchers noted that attempting to train on hard negatives from the initial pre-training stage caused performance to peak prematurely at 62.4% accuracy before encountering severe overfitting, underscoring the necessity of their staged curriculum.

Zero-Shot Evaluations and Verifier Performance

In zero-shot comparative evaluations, CLM-8B demonstrated significant throughput advantages over Jev. The headline figure of a ninefold latency reduction was recorded during tests on the T-Rex game, where candidates repeatedly recur across successive environmental states. Across broader evaluations, CLM-8B matched Jev in performance on T-Rex and Super Mario environments, while trailing Jev on tool-calling evaluations and WikiRacing tasks, though executing with lower latency across all evaluated scenarios.

Beyond reactive game control, CLM-8B was evaluated as a verifier model for software engineering agents. In this workflow, a generative language model samples multiple candidate solutions, and the verifier ranks the options to select the optimal completion. On 38 held-out tasks from DeepSWE using best-of-4 candidate sets generated by Opus 5, a lightweight fine-tuned head on CLM-8B attained an 81.6% success rate. On 30 held-out tasks from Terminal-Bench 2.1 using best-of-5 candidates from Fable 5, the model reached 87.6% accuracy.

Benchmark measurements conducted on an NVIDIA H100 system showed that CLM-8B operated between 4.1 and 5.7 times faster than Jev during verification. The research team highlighted that Jev scored below the pass@1 baseline on both verification suites, meaning that selecting with Jev yielded worse outcomes than picking a single random sample. However, the authors explicitly clarified that these verifier figures reflect specialized fine-tuned heads on held-out subsets rather than zero-shot checkpoint performance or full leaderboard submissions.

Technical Limitations and Enterprise Deployment

Despite its operational efficiency, CLM-8B operates under specific functional constraints. The projection heads are locked to Qwen3-8B last-token-pooled embeddings, preventing their use with other base architectures. Furthermore, the model cannot generate novel responses; it only calculates relative probabilities across an externally supplied candidate set. Contrastive-LM has indicated that broader generalization capabilities will be explored in an upcoming multimodal CLM-35B model planned for release in early October.

For enterprise development teams building autonomous agents, coding assistants, or structured routing pipelines, CLM-8B offers a practical alternative to slow and expensive generative LLM calls. The entire system is licensed under Apache-2.0, with the projection head weighing just 75 megabytes and running on a single NVIDIA GPU alongside vLLM. Organizations can self-host high-throughput action ranking and tool routing internally, eliminating external API dependencies while slashing latency in complex multi-step workflows.