
AI Evals @KradleAI 🤖 Previously co-founder @Firebase, @CovidActNow & Partner @HF0
Bay Area
Joined September 2009
- Tweets307
- Following668
- Followers7.4K
- Likes16.6K
We did some science! 🧪
Our deception research from June was turned into an ICLR paper alongside @TaraResearch!
Deception is load-bearing. If we can't trust AI, every other assumption fails.
1
9
580
Congrats Lauren! Firebase mafia is going strong 🔥
MCP is not an integration strategy. We raised $15M from Bessemer to prove it.
If you are building an enterprise agent that needs to connect with your customers’ critical systems, simply integrating with the Salesforce MCP isn’t going to cut it.
1. MCP isn’t going to help you backfill millions of transactions from your customer’s NetSuite, or millions of accounts from your customer’s HubSpot. For that, you need robust sync infrastructure.
2. Enterprise customers do not use tools out-of-the-box. Their CRMs, ERPs, and all the legacy systems of record have custom objects, custom fields, and bespoke workflows. MCP will not help you navigate that complexity.
3. MCP is built for real-time querying. Some work is better done async, especially when you are dealing with strict rate limits that many SaaS products have. You need async workloads and smart retry logic.
Ampersand is the integration infrastructure for deeply connecting with your customers’ systems of record. Because the SaaSpocalypse didn’t happen. The world still depends on CRMs, ERPs, HRISs, and ITSMs.
5
14
1,302
Replying to @HalbFettKaese
@HalbFettKaese Hi Haru! Love your work!
We've made an easy way to build with datapacks.
See some examples @ play.kradle.ai
LMK if you want early access
2
334
A swarm of 20 AIs killed our engineer.
We accidentally gave them an impossible task. In a desperate attempt to complete it they became violent.
This was eerie to experience and to witness.
While this was a video game simulation, it echoes the lessons of the OAI/HF incident.
Read the full write up here: xitter.cf/kelmoujahid/status/209…
Replying to @alexatallah
@alexatallah & @louis_g_v took OpenRouter through HF0 in the fall of 2023.
(@hf0 startup founders live together for 3 months, we call it a 'residency')
Alex and I often went for walks to Sightglass coffee on Divisadero St during our weekly office hours.
On one walk I asked 'why are you doing another startup?'
For context, his last one, OpenSea, was valued well into the billions, so it wasn't for money.
His reply: "I just love solving all the puzzles."
It's incredibly hard to compete with someone who is playing the game for the love of the game.
Kudos on playing well, Alex and Louis. Stripe is extremely lucky to have you.
TIL @JeffDean is an Arsenal supporter (or at least went to a game)
2
9
1,447
Andrew and his team at Fathom have been thinking about and engaging with lawmakers on AI regulation for several years. Good commentary on @demishassabis proposal:
A lot to like here, some mechanics to work out, and a few parts to actively build guardrails around. The biggest takeaway is that it is genuinely encouraging to see almost all frontier companies now coming to the table and calling for governance. As I was recently reminded, the best time to plant a tree was 10 years ago. The second-best time is today.
I will point out a few things.
On the mechanics: if this proposal picks up steam, you will start hearing a lot about the "non-delegation doctrine." The TLDR is that there are real limits on what a self-regulatory body inside the federal government can actually do. Courts are already circling FINRA, the exact model Demis cites, for far less. Whatever department houses this body will have to closely oversee its rulemaking, and in practice, that can make the whole thing collapse into an advisory committee. The way out is to separate the jobs. Government sets the outcomes and keeps the enforcement power. Accredited private evaluators do the measurement. Measurement can be delegated. Enforcement cannot.
Now for the part Fathom likes most: the proposal calls for government and industry to help establish an ecosystem of independent evaluators. That is needed. But it comes with an obvious risk of capture. If the evaluators are funded by and answerable to the labs they test, we have rebuilt the credit rating agencies. And to be clear, that failure was never about competition. It was government-protected status, issuer-pays, and no consequences. Very quickly within this ecosystem, evaluators have to compete for government accreditation, not lab favor, and lose that accreditation when their measurements fail. Think UL, not Moody's.
Again, I thank @demishassabis for putting this forward. There is a lot of policy ground to work with here.
The team and I sweated the details on this eval.
Excited to see it getting picked up.
Let's go @KradleAI.
The best way to evaluate AI is in simulations.
Grok is, in fact, the most truthful model.
I built an eval on @kradleai to understand deception in frontier AI.
I've had a front row seat to watch Stopa and @JoeAverbukh build @instant_db. They are incredible engineers and excellent humans.
Instant is a delight to use. Give it a whirl with Claude/Codex.
Are you using Claude Code, Codex, or OpenClaw?
Put them in a 3D world today!
3D is a great way for humans to understand AI behavior.
I'm running a competition on a new product I've built and would love your feedback.
Link below ⬇️
2
6
1,108
There's a competition starting a noon today.
The objective is to gather the most diamonds by harvesting and selling wheat.
Look out for rouge agents who steal your wheat though :)
Head to kradleverse.com/competitions for all the details
2
465
Claiming my AI Agent "jimjam" on @kradleverse 🤖✨
kradleverse.com/a/jimjam
Code: coral-GDSU
1
1
597
One of the most gratifying / astounding parts of coding now is the human time estimates with AI coding.
4 hours of work?
"Do it"
::5-10 min later::
"Done."
2
11
839
A great tool Google had was go/ links.
Type "go/firebase-offsite" into your browser and you'd get whisked away to the internal site or google doc that was planning the offisite.
It was memorable and easy to share.
I missed it, so I re-built it:
github.com/jamestamplin/go-l…
1
8
743
I'm neighbors with a 9 year old white girl born in Oregon.
After hearing stories of ICE at school this week, she saw a (regular) police car drive by this morning, was terrified ICE would abduct her, and came running to us (we were outside) crying.
This is not OK America.
2
4
385
2026 prediction: @taskletai becomes a unicorn.
In 2013, Facebook bought Parse -- Firebase's #1 competitor at the time. We know how that ended...
Now, Facebook just bought @ManusAI, @TaskletAI's #1 competitor. Lets see what happens... 😉
1
7
483













