Lots of chatter about agentic/RL simulation environments recently! Some key misconceptions (slightly caricatured): >> Building RL envs is easy, because you just code up a verifier quickly, and let the model do the tough data generation on its own! - Usually, this boils down to over-indexing on environments where verification is easy. - For example: you might need a chess expert to generate realistic expert gameplay traces, but anyone with a basic chess rulebook could verify a win easily. - However: there are many, many settings where verification is not at all trivial. The simplest examples are settings with nuanced, domain-specific evaluation rubrics (e.g. most real world enterprise settings). An extreme example being: verify whether a program will halt :) >> Building RL envs will get commoditized as the "standard" environments get rapidly solved. - RL environments effectively encode a complete product spec - including unique tools, data resources, constraints, rubrics/verifiers, and human/agent simulators - and as such, are as diverse as the space of all possible AI products. - Yes, certain generic RL envs will rapidly commoditize ('web browsing', 'computer OS') - but these are not the useful ones anyway! - The useful RL envs will be deeply domain- and product-specific – and will require corresponding human expertise and customization to build and evolve over time. >> RL (and RL envs) will be all that you need! - Current evidence suggests that RL / RL envs will be one part of the overall AI development loop- which will continue to require golden human annotations/traces for initial SFT; ongoing human evals; and more - Just like trial-and-error based learning is only one part of human learning, RL will likely be one tool/phase of many. In summary: - (1) Building the components of an RL environment is usually highly non-trivial. - (2) RL envs effectively describe a product spec - there will be a wide range of unique ones, requiring deep product/domain expertise. - (3) RL (and RL envs) will be one component of a rich ecosystem of tools for model learning, including human data, rubrics, evals, and more. If interested in some of the work the @SnorkelAI team is doing in partnership with leading LLM developers here- shoot us a note! It's an exciting time to build in this space :)

Starbucks using Splunk ES for years and together with Phantom over 2 Years! Please continue to protect my coffee ☕️ yummy waiting for a break to get one #splunkconf18 #siem #protectthebusiness