
Husband to @pearlsesq, Software engineer @databricks, French Canadian (i.e. likes poutine and hockey), previously @stripe, @SlackHQ, @Foursquare, @Google
New York, NY
Joined March 2007
- Tweets215
- Following544
- Followers932
- Likes6.8K
Simon FL retweeted
This is one of those unintuitive things. Agents that cook longer are often worse. Genie just gets to the results faster. Ontology will be key to getting these agents the context they need to get the answers right quickly.
We found that improving data agent quality also improves efficiency. Sometimes more is less: agents that take long, exploratory random walks are often less likely to arrive at the right answer.
We put Genie Code head-to-head against three leading general-purpose coding agents on 400+ real user data tasks.
Result below 👇
Simon FL retweeted
We just published OfficeQA Pro - a set of 133 challenging questions from the original OfficeQA benchmark. Even the best frontier agents still struggle on OfficeQA Pro with common issues stemming from errors in parsing, retrieval, and visual reasoning.
Simon FL retweeted
Most AI benchmarks test reasoning in isolation.
Real enterprise tasks require grounded reasoning:
1️⃣ Find the right documents
2️⃣ Extract the right values
3️⃣ Perform analyses
OfficeQA Pro evaluates this end-to-end. Frontier agents still score <50%.
🧵Paper & details below!
Simon FL retweeted
Today we’re releasing OfficeQA — a new benchmark for end-to-end grounded reasoning that reflects the real work enterprises need AI agents to do.
More details below 👇
Today we’re introducing OfficeQA, a new benchmark grounded in ~89,000 pages of U.S. Treasury Bulletins that reflects the complex, document-heavy tasks enterprises actually face.
Unlike existing benchmarks, OfficeQA measures economically valuable, real-world reasoning: parsing dense tables, navigating scanned PDFs, and retrieving facts across decades of documents.
Even strong agents reach only ~45% accuracy, showing how far the field has to go. The benchmark is now open to the community, and the Databricks Grounded Reasoning Cup in Spring 2026 will challenge teams to push these capabilities forward. databricks.com/blog/introduc…
Why is it "two buck chuck" and not "two bucks chuck"? Is this like the "maple leafs" vs "maple leaves"? English plurals are insane
1
3
194
Simon FL retweeted
Since joining @databricks, our research team has been hard at work on Agent Bricks, a new product that helps enterprises develop state-of-the-art domain-specific agents. We are now releasing a research blog about Agent Learning from Human Feedback (ALHF) databricks.com/blog/agent-le…
Simon FL retweeted
RLVR isn't just for math and coding! At @databricks, it's impacting products and users across domains. One example: SQL Q&A. We hit the top of the BIRD single-model single-generation leaderboard with our standard TAO+RLVR recipe - the one rolling out in our Agent Bricks product.
Hey @minimax_ai, I'm trying to serve M1-80k on vLLM. Your docs say "a server with 8 H800s can process inputs up to 2 million tokens" but then recommend --max_model_len 4096. What settings did you use for 2M tokens? I'm trying this on 8 H100s.
First day on the job at @databricks and we're already making some big moves. Exciting times ahead!
Big news: we've agreed to acquire @MosaicML, a leading generative AI platform. I couldn’t be more excited to join forces once the deal closes. databricks.com/mosaic-news
2
346
Simon FL retweeted
Replying to @GergelyOrosz
I always keep in mind the following: When you talk to someone you have the responsibility to do your best to adapt to the other person skill level. If you don't then you are disrespecting the other person and you are losing your time.
There's nothing quite like seeing "thought leaders" write about a topic you're very familiar with to disabuse you of the notion that they have any idea what they're talking about.
9
571
A thousand times this. Hell, some of the best people I've worked with barely even have a LinkedIn profile.
Replying to @pulidili
High competence will attract the right network, automatically. Brand (if by that you mean public brand) is highly overrated for high paying professions such as prod mgmt, eng, design, data sci, etc. Majority of highly competent people have relatively small social media following.
1
1
373




















