Replying to @sebuzdugan
Fair point. Frozen weights only rule out overfitting through gradients. The harness itself (prompts, tools, lessons) can still overfit the bench. So Propose and Probe only ever see a dev split. Each adopted version gets scored once on a held-out set of new images it never touched, and that's the number we'll report. Lessons are kept as general cue → action rules (low_light → boost_contrast), not per-image fixes. Held-out results ship with the repo.
Replying to @ziziran2
Of course, that includes the data for the entire benchmark, which will be stored in the repository.
Replying to @thsottiaux @xiangning0
Tibo Why am I burning through my quota 100x faster than with Claude?
Replying to @typesafeai
We plugged Google Research’s Harness² into JEV 1.14.
Zero weights touched, yet driving massive gains on vision benchmarks round after round.
Under the hood, the system:
⚡️ Rewrites its own prompts
🛠️ Builds its own tools
🧠 Distills its own experience
xitter.cf/openmys/status/2112024…


