Which model names (-m) can you use in Harbor? Now there's one answer: LiteLLM's. harbortoolkit.com/model

Agents that can edit files and run commands can also game their own evals. Harbor + Docker Sandboxes fix that: every eval task runs in an isolated microVM, with network policies for fair, reproducible runs. bit.ly/4B8xf9Z

Introducing Blender Bench v1 (BBv1): a benchmark for frontier LLMs on 20 realistic 3D production workflows including animation, modeling, rigging, cloth & surfacing. GPT-6 Astra leads overall; Claude Opus 5.5 wins modeling & cloth simulation.

Full results, harbor traces and implementation details in our blog - unreallabs.ai/blog/long-hori…

New blog: Is human taste overrated in harness engineering? An agent harness is the system around a model. It decides which tools the model can call, what evidence it sees, and what it remembers between steps. Most harnesses are still designed by human taste. Run the agent, read where it failed, change the harness, try again. That loop is increasingly being automated, with a strong model rewriting the harness itself. So we asked how far that can go. We ran Qwen on three drug design tasks from SMDD-Bench and compared a manually redesigned harness against one produced by automated harness search with Claude in the loop. No single approach won everywhere. The manual harness pulled clearly ahead when the fix was changing what the agent sees and remembers. Automated search did better once that was in place and the remaining problem was how to search the chemistry itself. The hard part was rarely implementing the fix. It was diagnosing what kind of failure we were looking at, and that is where human taste still mattered. Work with @KevinH1119568 , @aviral_kumar2 and @niloofar_mire Read more 👇

Jev-as-a-Judge in Harbor Rewardkit is pretty amazing. It agrees with Fable and GPT and is very fast. When we test it on a real eval that uses an LLM judge, such as the @harvey LAB benchmark, Jev achieves perfect agreement but is 50x faster.

In my view, ATLAS Finance is the closest benchmark yet to real end-to-end professional financial work. Code to run the benchmark as a @harborframework task suite: github.com/Handshake-AI-Rese… Data/environments: huggingface.co/datasets/hand…

New Harbor docs! We've released many exciting features that were previously undocumented like simulating users, streaming, and regrading trials. Check them out or give your coding agent the MCP. docs.harborframework.com/

We are releasing AutoResearchExam, a benchmark on open-ended machine learning and engineering tasks. Our benchmark covers seven research areas including model training, data curation, AI safety and interpretability. benchmarks.bespokelabs.ai/au… In each task, we give agents 24 hours with a CPU or GPU machine to develop and improve their solutions through experiments and feedback. We measure both speed and quality with a combined score. Our benchmark has a unique feature: testing if agents create improvements that hold up on data they never see. We find that AI research agents often overfit as they try to improve. We see an interesting head-to-head comparison at the frontier: Astra starts the strongest and holds the lead for up to 19 hours but Fable 5.1 catches up and gets the top performing spot in the final hours. Qwen3.8 Max, Gemini 3.8 Flash and Grok 4.6 all sit on the cost-performance Pareto frontier, giving strong options at lower API budgets. Anthropic's Opus and Fable retain nearly all their validation performance on hidden tests, with gaps of 1.1% and 2.9%. Astra's improvement over Sol extends to generalization too, with that gap falling from 6.9% to 1.7%. (1/n)