Harbor ❤️ Sidecars Multi-container environments are a popular way to mock external services. Harbor now allows you to collect artifacts from sidecars, run verification in a sidecar, and execute pre-verification commands in any service.

Starting today, run any dataset or task at any commit in any git repo from any git platform at any path in one line of code:

Save money running Harbor rollouts ‼️ Sometimes cost is more important that reliability or reproducibility when running rollouts (e.g. during rapid iteration). Now in Harbor you can configure resource enforcement policies to save money.

🚨 stop zipping job results 🚨 ... upload results to Harbor Hub instead The hub makes it easy to share results with team members, customers, or simply save for later in a centralized place. Example of a TB2.1 job in 🧵

We built Harbor to evaluate agents. But why limit ourselves to just agents? Today we're adding first-class support for evaluating skills, MCPs, prompts, and services. Ablate your agents.

Agents building agents! Just put everything into a filesystem (agent code and evaluation results - trajectories and rewards generated by harbor) and let the Meta-Harness iterate.

We've been using Harbor skills to build tasks and integrate new benchmarks way faster. Now you can too:

Benchmarking skills has been a common Harbor use case (e.g. skillsbench.ai). Harbor now has first-class support for skills. Agents receive skills_dir in their __init__ method and can choose to register the skills in their setup or run methods. Typically, this means copying the skills directory to the expected location, e.g. ~/.claude/skills.