🚀 Sharing some exciting new results: Harness optimization got open-weight agents to frontier-level performance at ~3% of the inference cost.
Open-weight models offer better economics, more control over deployment and data, and less dependence on a small number of providers. But they can also be more sensitive to how the agent around them is designed.
We used RELAI to optimize the harnesses for three open-weight agents on AutomationBench and found that each was failing in a completely different way—and each needed a different fix.
‼️ Three models. Three failure modes. Three different fixes.
With the right harness, the open-weight agents reached comparable performance to frontier agents at a fraction of the inference cost.
Read more in the article below 👇
And if you already have an agent built on a frontier model and want to explore an optimized open-weight version, lets connect and chat.
RELAI retweeted
🚀 Harness optimization got open-weight agents to frontier-level performance at ~3% of the inference cost.
Three models. Three failure modes. Three different fixes.
The harness mattered more than we expected.
Read our latest results on joint quality-cost optimization of AI agents below. You can try it at: relai.ai
RELAI retweeted
Can harness optimization make AI agents better and cheaper?
Using RELAI across GPT-5.5, Claude Opus 5, and Gemini 3.5 Flash, we improved task quality while reducing inference cost.
Full results in the article ↓
RELAI retweeted
Sharing a short recording of my talk from the Berkeley RDI Agentic AI Summit 2026 on verifiable continual learning for AI agents.
I cover what continual learning is, why it is becoming a new frontier for agentic AI, and how RELAI helps agents learn continuously and verifiably across prompts, tools, memory, workflows, and models.
As AI agents move from demos to production systems, the key challenge is no longer whether they can solve a task once, but whether they can keep improving from failures and feedback without introducing regressions.
Learn more about RELAI: relai.ai
RELAI retweeted
The RELAI team and I are excited to attend the Berkeley RDI Agentic AI Summit 2026 this week!
We’re honored that RELAI has been selected as a featured startup in the Berkeley Xcelerator cohort.
Looking forward to an incredible lineup of speakers from OpenAI, Google DeepMind, NVIDIA, Amazon, Databricks, Microsoft AI, and many others, including Andrew Ng, Wojciech Zaremba, Ali Ghodsi, Dawn Song, Sergey Levine, and Peter DeSantis: rdi.berkeley.edu/events/agen…
If you’ll be at Berkeley and are interested in continual learning for AI agents, stop by and say hello. We’d love to show you how RELAI helps agents continuously improve from evaluation and production failures while preserving what already works.
Looking forward to seeing many of you there!
RELAI retweeted
Current benchmarks miss the central challenge in agent optimization.
Most evaluations test one-shot improvement, while real-world use requires recursive optimization as new tasks continue to arrive.
To study this, we built a two-phase continual-learning evaluation on hard Terminal-Bench 2.0 tasks, comparing GEPA, Meta Harness, and RELAI.
The methods behaved very differently:
- GEPA improved initially but transferred negatively to unseen tasks, indicating overfitting.
- Meta Harness transferred well but could not continue improving.
- RELAI was the only method that both generalized to unseen tasks and improved again without regressing prior performance.
Full results are in the article below.
You can access RELAI’s continual-learning engine to run similar optimization studies and apply it to your own agents at relai.ai
I'm very excited to attend the AI Engineer World’s Fair this week in San Fransisco @aiDotEngineer: ai.engineer/worldsfair/2026
If you’re interested in continual learning for AI agents, want to see a live demo of how agents can learn continuously and verifiably, or want a chance to win a MacBook Pro (!), stop by the RELAI booth.
Looking forward to seeing many of you there!
Come work with us on exciting problems in continual learning for AI agents.
🚀 We’re hiring at RELAI.
We have an opening for an AI Research / Engineering role focused on continual learning for AI agents.
Requirements:
- PhD in CS, ECE, or a related field
- At least two high-impact publications
- Located in the U.S. or Canada
Details:
- Full-time
- Remote or DMV area
- Immediate start
Apply here: forms.gle/JFWTd4t1h6QvP15E8
Please feel free to share with strong candidates in your network.
Excited to share that RELAI will be presenting at the Berkeley Xcelerator Spring Cohort Showcase and VC Mixer on June 18 at UC Berkeley.
We’ll be demonstrating the RELAI's Verifiable Continual Learning Engine—our approach to making AI agents self-improving through learning that is replayable, holistic, lifelong, and efficient. RELAI turns an agent’s real-world feedback, failures, and signals into testable learning environments, improves the right layer of the agent, and controls regressions over time.
Event details: luma.com/xuwozgpu
RELAI retweeted
Thank you to @thecubetv for covering the launch of our continual learning platform! Sign up for early access at: relai.ai
siliconangle.com/2026/06/10/…
RELAI retweeted
Today, we’re launching RELAI’s verifiable continual learning platform for AI agents and announcing $6.9 million in total funding, led by @406Ventures.
Future agents will get better through experience. But today, most of that learning happens through vibes—not verification.
RELAI’s continual learning engine is built around four principles: make every failure replayable, improve without forgetting what already works, apply feedback to the right layer of the agent stack—prompt, tool, memory, workflow, model, or code—and keep the entire loop efficient enough to run continuously.
🚀 The best news: you can now add RELAI’s continual learning loop to your agents in just two commands.
👉 Now available in limited access. Join the waitlist at relai.ai.
🚀 Today, we’re launching RELAI’s continual learning platform for AI agents and announcing $6.9M in funding led by @406Ventures.
As enterprises deploy more AI agents, a new challenge is emerging: keeping them reliable over time. Agents fail. Teams patch prompts. Re-run evals. Fix one issue and silently create another.
RELAI breaks that cycle.
Our platform turns failures, traces, evaluations, and human feedback into replayable learning environments, identifies root causes across the agent stack, and continuously improves agent behavior with built-in regression control.
What excites us most is seeing this work on real and complex enterprise workloads:
“At C3 AI, we’re delivering production-grade agents on the C3 Agentic AI Platform that take on complex, mission-critical workflows for enterprises. As these agents take on harder problems, the ability to evaluate and improve them on realistic edge cases becomes critical, and RELAI has helped us turn hard use cases into evals, and evals into measurable improvements in the agents we ship to customers.”
— Nikhil Krishnan, CTO & Chief AI Officer, @C3_AI
We believe the next generation of agents won’t just be prompted better.
They’ll learn from experience.
Limited-access onboarding is now open.
👉 Join the waitlist now: relai.ai
RELAI retweeted
Our hiring agent has been at work 24/7. If you have not applied yet, now is the time. Please submit by April 30 for best consideration: relai.ai/careers
At RELAI, we built a self-improving hiring agent that talks with applicants to understand their background, past work and fit for our three open roles. Early on, it struggled a bit in both the infra and its “brain” but it kept getting better through feedback from applicants.
Its work does not stop there. After the conversation, the agent continues evaluating applications using submitted information and public sources. It then synthesizes everything to assess and explain each candidate’s fit for our roles.
So far, we have received more than 300 applications and the agent has preliminarily identified 100 strong-fits. Human interviews start soon.
We are building a star team to work on some of the most exciting and challenging problems in agent learning. Come build with us.
We are hiring! Come build with us 🚀
Apply using RELAI-built hiring AI agent: relai.ai/careers
🚀 We’re hiring at RELAI @ReliableAI for three full-time roles:
- AI Research Engineer
- Field AI Engineer
- Backend Engineer
Apply here through our hiring AI agent: relai.ai/careers
It takes about 5–10 mins, only requires your CV and helps us quickly understand your background.
Apply by April 30 for best consideration.
If you’re excited to work on some of the hardest and most exciting problems in self-improving AI agents in production, come build with us.
🚀🚀🚀
I’m excited to share that I've been promoted to Head of Engineering @ReliableAI. Over the past year, we’ve been building our core platform for reliable AI systems. I’m looking forward to continuing that work as we scale both the technology and the team.
More to share soon.
A notebook showing how to optimize AI agents using the user feedback:
How to use expert feedback to optimize AI agents?
In many real-world applications, there is no clear ground truth label for what a “good” agent response is. Often, all we have is user feedback and preferences (“this is wrong”, “missing context”, “too verbose”, etc.). This feedback is an extremely valuable supervision signal, but turning it into effective optimization of agent behavior is not straightforward:
Stochasticity & replay
To learn from feedback, we often need to “replay” the original sample or trace. But agentic systems (with tools, RAG, branching, etc.) are stochastic, so re-running the same input may not reproduce the same trajectory or output.
Linking feedback to replays
Even if we can approximate the original run, evaluating a new or re-played trace against the old feedback is non-trivial. The feedback is textual, often high-level and contextual, not a simple scalar reward.
Optimizing config and structure
Finally, we want to optimize both the agent configuration (prompts, hyperparameters, tools, thresholds) and the agent graph/structure (which nodes, in what order, with what routing). Jointly optimizing these under noisy, text-based feedback is a challenging learning and search problem.
In this notebook, using an agentic RAG example, we show how to operationalize this:
📝 Convert user feedback on agentic runs into an annotation benchmark on RELAI
🎯 Use the Maestro agent optimizer to consume that benchmark and automatically improve both the config and the graph of the agent
🔁 Close the loop from user preference → benchmark → optimization → better agent in a reproducible, data-driven way
🔗 Notebook: colab.research.google.com/dr…
Powered by @ReliableAI (relai.ai)
One notebook to: "build -> simulate -> evaluate -> optimize" your agentic RAG!
🔗 Notebook: colab.research.google.com/dr…
🚀 Sharing a Colab notebook with a complete learning loop for reliable agentic RAG:
🧱 Build agentic RAG on top of your own data
🎭 Simulate with persona-based runs to stress-test your agent
🧑⚖️ Evaluate quality automatically with Critico (LLM-as-a-judge)
🎛️ Optimize configs & structure with Maestro for better performance
All in a single notebook. Works with any model; just drop in your API key.
🔗 Notebook: colab.research.google.com/dr…
Powered by RELAI (relai.ai)
