I've been talking to a lot of people working on RL lately, and I've noticed something interesting — whenever the conversation turns to RL Infra, it almost always gravitates toward one topic: train-inference alignment. How to keep the training and inference policies consistent. How to control off-policy degree. How to handle log prob diff after introducing async. These are all important questions, no doubt. But I'm increasingly convinced that RL Infra is suffering from a significant misallocation of attention. Borrowing a framing from a recent discussion with a colleague, I call this the Barrel Effect of RL Infra. A barrel holds only as much water as its shortest stave. The throughput and correctness of an RL training system work the same way — they're not determined by the module you've optimized the most, but by the one you've neglected the most. Train-inference alignment might be the stave you've sanded and polished to perfection. But if your sandbox stability is a disaster, your reward pipeline stalls constantly, and your end-to-end observability is virtually nonexistent — what good is perfect alignment? The system's capacity is already capped by every other weak link. This is fundamentally different from how inference system optimization works. As an inference engine, SGLang has an enormous strategy space for optimization, but its pipeline is relatively linear — process request, prefill, decode. You can isolate bottlenecks module by module, and the coupling between components is manageable. RL training is a different beast entirely — a nightmarishly complex multi-system loop: rollout generation depends on the inference engine, reward computation may depend on external environments, policy updates depend on the training framework, and the next round of rollouts depends on the updated policy. If any single link breaks, the entire loop collapses. Unfortunately, from what I've seen over the past year, there are still many severely underestimated weak points: Agent Sandbox Reliability. This is probably the dirtiest, most grueling, and least academically glamorous work in RL Infra today. Agent-based RL needs a reliable execution sandbox for rollouts — sounds simple, turns out to be a nightmare. Container stability, cold start latency, resource isolation reliability, sandbox state management — these things appear decoupled on paper, but the sandbox products available on the market consistently underperform expectations. Agent sandboxing isn't an algorithm problem, but it directly determines your data generation efficiency, which in turn determines your training speed. Observability. Debugging pretraining is relatively straightforward — watch the loss curve, check the gradient norm, and you can usually pinpoint the issue. But debugging RL requires end-to-end tracing capabilities: rollout quality distributions, reward statistics, off-policy degree, policy update magnitudes, and even attribution of logprob diff (is the diff coming from the inference side, or from async training's version lag?). Unfortunately, most teams I've encountered are essentially flying blind on these dimensions. This leads to an awkward situation — when training results are poor, you don't even know which module to blame. The Scale Dilemma. Many RL Infra optimizations only show measurable impact at sufficient scale. Small-scale experiments often reveal no meaningful difference — not because the optimization is useless, but because the noise is too high and the step count too low for the signal to surface. Yet large-scale experiments are prohibitively expensive. This creates a vicious cycle: you can't prove your optimization works at small scale, so you can't secure the resources for large-scale experiments; and without large-scale validation, your optimization is forever stuck at "theoretically it should help." The industry's investment in RL Infra is severely mismatched with its actual complexity. Most teams treat it as a patch job on top of pretraining infra — grab an off-the-shelf training framework, bolt on an inference engine, glue them together with scripts, and call it RL Infra. But the system complexity of RL training and pretraining aren't even in the same league. Pretraining pipelines are linear, homogeneous, and have virtually zero external dependencies. RL training pipelines are cyclical, heterogeneous, and heavily dependent on external environments. Applying the architectural mindset of the former to the latter is guaranteed to hit a wall at scale. The real difficulty in systems engineering isn't about pushing any single module to its extreme — it's about understanding the coupling between modules and the global trade-off space. This is true for inference systems, and even more so for RL Infra, where the coupling dimensions are greater, the feedback loops are longer, and the information density for debugging is far lower. I want to close with two questions I've been mulling over, and I'd love to hear from others working in this space: Where exactly do the marginal returns of train-inference alignment start to diminish? Once async is introduced, the off-policy degree is already substantial. On that baseline, is the incremental gain from further alignment actually higher-ROI than investing the same engineering effort into sandbox stability, reward pipeline optimization, or observability infrastructure? I have my own tentative answer, but I think this question deserves serious thought from more people — rather than defaulting to alignment as the top priority simply because it's the most visible topic. And there's a reason it's the most visible: train-inference alignment has clean mathematical formalization and produces elegant ablations — it's a natural fit for papers. But how do you write a paper about sandbox stability? How do you frame container orchestration reliability as an academic story? You can't, really. So these problems get collectively ignored. Even if an RL Infra system achieves bit-level train-inference alignment, overall efficiency can still be dismal — because the bottleneck moved somewhere else a long time ago. To what extent can RL Infra be standardized? Inference systems have relatively well-defined benchmark metrics — TTFT, TBT, Throughput. These objective indicators let us clearly evaluate the impact of optimizations. But what are the evaluation standards for RL Infra? Training throughput? Sample efficiency? End-to-end wall-clock time? The optimal architecture might vary dramatically across scenarios (code generation vs. agent vs. reasoning). If we don't even have consensus on what "good RL Infra" looks like, then engineering knowledge in this field will be extremely difficult to accumulate and reuse. Whether RL is the critical path for improving model capabilities — that judgment is still evolving. But if the answer is yes, then Infra is the most underestimated bottleneck on that path. Not because nobody is working on it, but because collective attention is misallocated. The cruelty of the Barrel Effect is this: no matter how tall your tallest stave, it can't save the system. RL Infra is not a secondary concern. It is an independent, high-complexity systems engineering domain. Only by treating it as a first-class citizen will we have any chance of making RL scale.