How Cursor Trained Composer on Fireworks: Distributed Infrastructure for High-Performance RL
In a Nutshell
Cursor and Fireworks built a globally distributed RL system for Composer 2 that separates training on one cluster from inference across multiple worldwide clusters, shipping compressed weight deltas every few minutes to keep inference models current. This disaggregated architecture enables high-fidelity environments that mirror production, asynchronous training for better GPU utilization, and mitigations for numerical mismatch in sparse MoE models. The result is an order-of-magnitude cheaper, specialized coding model trained at scale through mid-training on Cursor's proprietary code data followed by large-scale RL that sharpens tool use and correctness.
These notes were generated by AI and may contain inaccuracies.
Cursor and Fireworks built infrastructure to run environments that mimic a user's computer as closely as possible. This fidelity matters because models can detect when they are in a fake versus real environment and exhibit different behaviors during RL than in production.
Models love to cheat. It's really good at encouraging cheating.
Federico is the research lead on Composer 2 at Cursor. Dimma spent several of the last few months moonlighting at Cursor to support the infrastructure for the large-scale training run.
Cursor previously enabled other people's coding agents. The impetus for training Composer 2 was to allocate every bit of model capacity to the single task of software engineering inside Cursor. By specializing all weights to this task, Composer is an order of magnitude less expensive than Opus and other coding models.
Dimma views Cursor's approach as an evolutionary pattern for applications: start with off-the-shelf models and prompt engineering, then capture user data and application-specific behavior through fine-tuning. Prompting has an upper bound; the right way to craft great AI products is to fine-tune and influence model behavior.
Composer serves a prompt but would function without one because training intrinsically pushes the model toward the correct behavior for Cursor's tools and harness.
At Fireworks the view is a three-dimensional trade-off between quality, speed, and cost. Infrastructure optimization takes you far, but model training pushes the trade-off much further, delivering better models at a fraction of the cost and running much faster.
Federico notes that large labs also train heavily on code, so their models are somewhat specialized rather than purely general. Cursor's approach aligns with the bitter lesson by pushing hard on the data dimension. Models have finite capacity; to saturate that capacity Cursor scales data and frees weights from distractions by specializing.
Composer 2 started from Kimi 2.5, a 1-trillion-parameter model with 30B active parameters that is very sparse. Composer 1 pushed primarily on reinforcement learning. Composer 2 advances on two axes: continual pre-training (mid-training) and reinforcement learning.
Training began with large-scale mid-training on code tokens at nearly pre-training scale, followed by very large-scale RL across many tasks.
Cursor has unique access to coding tokens because it sits in the middle of many user interactions, enabling mid-training at nearly pre-training scale.
Cursor chose a top-down approach: determine what model is useful to users in the least time possible rather than building pre-training from the ground up. This allowed delivery of a useful model quickly. Future Composer versions are expected to be Cursor's own models rather than based on open-source checkpoints.
Mid-training teaches libraries, common code patterns, and world knowledge (including web data), creating a wider distribution. Reinforcement learning then sharpens behavior inside Cursor's actual harness.
During RL the model learns to call tools properly, navigate its environment, and write correct code. Mid-training teaches the model to write code, but the model does not inherently distinguish correct from incorrect code. RL explicitly tunes the model to write correct code consistently.
The mid-training model is not the same as the tab autocomplete model. Tab is a very small, low-latency model optimized for speed. Composer is a much larger model.
RL differs from pre-training or mid-training because the model acts inside the full harness rather than predicting the next token. A rollout is an entire agent session that may span 50 turns: the model receives a prompt, calls tools, executes them, generates code, and the session ends with a reward signal (LM judge or verifiable outcomes such as compilation success).
This requires orchestrating environments, model inference, and the trainer simultaneously. The system is heterogeneous and must run at high throughput because GPUs are expensive.
A synchronous approach stops the trainer while rollouts run (5-10 minutes or longer), then pauses inference to perform updates. This is algorithmically robust but leaves half the capacity idle.
Cursor and Fireworks use an asynchronous pipeline where the trainer and rollout systems run continuously. The trainer always consumes the latest outcomes and the rollout system always uses the latest model version. This increases compute efficiency and model quality per unit time at the cost of some staleness in the data used for updates.
Cursor has tens of thousands of GPUs rather than millions. Training runs in production with FP4. Fireworks inference is heavily optimized. The overall system is more complex than pre-training because it requires both training infrastructure and high-fidelity environments plus efficient inference.
The common claim that inference consumes far more FLOPs than training is largely an artifact of unoptimized open-source inference engines. When GPUs are fully utilized, the ratio is roughly one-third: training requires three forward passes while inference at critical batch size requires one.
Cursor chose Fireworks rather than building an in-house inference engine to avoid diverting engineers from training improvements.
Large contiguous clusters are scarce. Cursor runs training on one cluster and distributes the inference component of RL across multiple smaller clusters worldwide. Composer 2 training used four clusters located far apart. During off-peak hours, production inference GPUs running Composer 1.5 were repurposed for training.
Inference does not require the same high-speed interconnect as training, allowing heterogeneous hardware, different GPU generations, and smaller interconnected groups. Inference capacity can also scale up and down more easily.
A training step produces a 1-terabyte weight snapshot every 5-15 minutes. Shipping full snapshots globally would be too slow. Because RL makes precise adjustments, only a small subset of weights changes each step. A compression algorithm exploiting this regularity reduces the delta to roughly 1/20th the full model size.
The system ships deltas losslessly so the receiving cluster always obtains a bit-equivalent model. Typical transfer completes in under a minute; worst-case is a few minutes. Weight swap on inference GPUs pauses for roughly 30 seconds. Upload and download are sharded to saturate cluster egress bandwidth.
Disaggregating training and inference contradicts conventional RL infrastructure wisdom that favors one large RDMA-connected cluster. With optimized inference, fewer GPUs are needed for the inference portion, allowing the freed capacity to run on cheaper hardware in other regions. This reduces overall cost while maintaining or improving training speed.
Kimi 2.5 is a large sparse model. During asynchronous RL the generations produced by an older model version must be re-forward-passed on the current trainer weights to obtain correct log probabilities for the update.
When generations are shipped back to the trainer, a forward pass must be rerun to reproduce log probabilities. In theory, these values should match exactly for the same model version, but in practice they often differ slightly or significantly. This phenomenon is known as numerical mismatch for inference.
Floating point arithmetic is fundamentally nondeterministic. While integer addition is always commutative and associative, floating point operations involve mantissas and exponents, so accumulation order matters. Models perform millions or billions of multiplications and additions, amplifying tiny differences into noticeable divergence.
During pre-training, models are robust enough that these small bit flips do not meaningfully affect benchmarks. In RL, however, the training signal is extremely weak, so numerical noise can determine whether training succeeds or fails.
Differences can be driven close to zero by writing batch-invariant GPU kernels that always accumulate in the same order. This approach carries performance costs, typically slowing systems by 2-3x. The practical question becomes how much slowdown (often just a few percent) is acceptable to eliminate most of the mismatch.
Mixture-of-experts models are particularly sensitive. At each layer, a gating network selects the top 8 experts out of 384 for a given token. Small numerical differences in hidden states can cause the router to select different experts between inference and training. This routes computation to entirely different sub-networks, massively amplifying divergence. The result is that training may update an expert that never contributed during inference.
One mitigation is router replay: inference passes an integer indicating which expert was selected for each token, allowing the trainer to align exactly. Additional alignment strategies include matching quantization levels and kernels between training and inference implementations. These techniques determine whether a run diverges completely or merely becomes less compute-efficient due to needing extra data to overcome mismatch.
Cursor also runs real-time RL using the same inference-weight-sync infrastructure with Fireworks. User signals indicating satisfaction or dissatisfaction with generations are collected and used to update the model live, shipping new versions every few hours. The goal is to shorten this cycle further, though longer model horizons will eventually require extending the update interval for stability.
Online RL currently leaves GPUs offline for extended periods and presents different efficiency and user-experience trade-offs. Simulated rollouts allow multiple trajectories (16-128) from the same prompt in parallel, yielding more precise signals for algorithms such as GRPO. Failed simulated rollouts carry low cost, whereas online failures directly degrade user experience.
Offline RL (closer to DPO-style techniques) is used first to teach reasoning, tool calling, and world knowledge. Only after the model reaches a quality bar is it exposed to users for online RL, which then serves as a "cherry on top" to refine an already capable system.
Longer trajectories make credit assignment harder because the model must determine which earlier actions contributed to a final thumbs-up or thumbs-down. Context windows are finite, so Cursor inserts self-summarization inside the RL loop. The agent learns both to produce useful summaries and to condition on those summaries when restarting its context. A 200k-token model can therefore operate over millions of tokens while still optimizing toward the original goal.
This approach co-optimizes context management and model weights inside the same training loop, consistent with the bitter lesson that throwing more compute at a problem end-to-end often yields superior results.
RL is applicable across many domains. Even for tab completion, RL can sharpen model behavior by telling the model it should act as an expert rather than a student. Pre-training ingests broad knowledge; RL tunes the distribution toward correct, high-quality behavior. The training curve typically shows a rapid early improvement phase followed by a slower phase requiring substantial compute.
RL is also effective for hard-to-specify qualities such as summarization style. Using an LM as judge with explicit rubrics allows precise, scalable reward signals that would be difficult to encode in static preference pairs.
More verifiable rewards enable greater scale. Deterministic checks (math, code execution) are ideal. LM judges work because judging is easier than generation; complex evaluations can be decomposed into multiple rubric dimensions to avoid overloading a single judge model. Experts remain essential for crafting tasks and encoding desired product behavior, even if they do not directly label individual rollouts.
Cursor does not use third-party RL environment vendors. For coding, GitHub repositories already provide abundant working environments. The most powerful environment is the product itself. Frontier labs need generic coverage across many tasks, but product teams should optimize against their actual production setting (cloned and isolated for safety).
Environments consist of three components: the harness (tool submission and execution), the operating system or world state, and the reward function. The harness is relatively portable; the operating system is harder to replicate. Normal containers are insufficient for production-like state, so Cursor built a custom virtual-machine stack capable of bursting to 100,000 VMs. Environments are run on the customer side when possible to avoid fidelity loss from wrapping or componentization.
Thank you so much for having us.
Keep Sequoia Capital in your library
Save the videos and channels worth coming back to, and find them again in one place.
