
Husband to @pearlsesq, Software engineer @databricks, French Canadian (i.e. likes poutine and hockey), previously @stripe, @SlackHQ, @Foursquare, @Google
New York, NY
Joined March 2007
- Tweets215
- Following544
- Followers932
- Likes6.8K
Simon FL retweeted
This is one of those unintuitive things. Agents that cook longer are often worse. Genie just gets to the results faster. Ontology will be key to getting these agents the context they need to get the answers right quickly.
We found that improving data agent quality also improves efficiency. Sometimes more is less: agents that take long, exploratory random walks are often less likely to arrive at the right answer.
We put Genie Code head-to-head against three leading general-purpose coding agents on 400+ real user data tasks.
Result below 👇
Simon FL retweeted
We just published OfficeQA Pro - a set of 133 challenging questions from the original OfficeQA benchmark. Even the best frontier agents still struggle on OfficeQA Pro with common issues stemming from errors in parsing, retrieval, and visual reasoning.
Simon FL retweeted
Most AI benchmarks test reasoning in isolation.
Real enterprise tasks require grounded reasoning:
1️⃣ Find the right documents
2️⃣ Extract the right values
3️⃣ Perform analyses
OfficeQA Pro evaluates this end-to-end. Frontier agents still score <50%.
🧵Paper & details below!
Simon FL retweeted
Today we’re releasing OfficeQA — a new benchmark for end-to-end grounded reasoning that reflects the real work enterprises need AI agents to do.
More details below 👇
Today we’re introducing OfficeQA, a new benchmark grounded in ~89,000 pages of U.S. Treasury Bulletins that reflects the complex, document-heavy tasks enterprises actually face.
Unlike existing benchmarks, OfficeQA measures economically valuable, real-world reasoning: parsing dense tables, navigating scanned PDFs, and retrieving facts across decades of documents.
Even strong agents reach only ~45% accuracy, showing how far the field has to go. The benchmark is now open to the community, and the Databricks Grounded Reasoning Cup in Spring 2026 will challenge teams to push these capabilities forward. databricks.com/blog/introduc…
Simon FL retweeted
Since joining @databricks, our research team has been hard at work on Agent Bricks, a new product that helps enterprises develop state-of-the-art domain-specific agents. We are now releasing a research blog about Agent Learning from Human Feedback (ALHF) databricks.com/blog/agent-le…
Simon FL retweeted
RLVR isn't just for math and coding! At @databricks, it's impacting products and users across domains. One example: SQL Q&A. We hit the top of the BIRD single-model single-generation leaderboard with our standard TAO+RLVR recipe - the one rolling out in our Agent Bricks product.
Simon FL retweeted
Replying to @GergelyOrosz
I always keep in mind the following: When you talk to someone you have the responsibility to do your best to adapt to the other person skill level. If you don't then you are disrespecting the other person and you are losing your time.
Simon FL retweeted
"I chatted on the phone with a friend for an hour about their frustrations with their boss."
If you don't work at the same company, that is being a friend.
If you do work at the same company, that is career development and is formal responsibility of e.g. senior ICs to company.
Simon FL retweeted
learned this lesson very quickly when i tried to introduce daily standup to motivate a low performer to stop slacking and a high performer (ver politely) gave feedback it was breaking up their focused work time. immediately undid the standup. also, got rid of the low performer.
Simon FL retweeted
I know quite a few people who do this kind of optimization work and the fixes to drive "time to first X", fast response, etc. for apps tends to be straightforward compared to spinning up a bunch of regions.
Companies almost always underinvest in simple high-ROI performance work.
Simon FL retweeted
A day and a half of debugging later, you figure out that "[object Object]" is the actual, original, correct text which someone entered
Simon FL retweeted
Replying to @chrisalbon @schloksbagels
The Patagonia jacket attaches itself to you after you write your first angel check, ala the Venom symbiote
Simon FL retweeted
One of the best pieces of #leadership advice I ever got was from @royrapoport.
It has been ringing in my ears for years.
Roy was talking about the spectrum of "principled" to "practical" #management.
Simon FL retweeted
And then the interview circuit really does “If you’re capable of discovering the rules of the game, and then decide to play suboptimally, we would prefer to find a different teammate. If you’re not capable of discovering the rules of the game… indistinguishable for our purposes”

























