
Professor @northeastern | Chief Economist @micro1_ai | Social Scientist @BKCHarvard
Boston, Geneva, Dubai
Joined August 2012
- Tweets67.8K
- Following11.7K
- Followers48K
- Likes138K
Mark Esposito, PhD retweeted
very excited for the launch of LegalReasonBench, from the braintrust of our legal data lab at micro1.
we set out to fill an industry gap: a multi-step, long-horizon legal reasoning benchmark. every task is entirely attorney-authored with evolving matters & requests, as opposed to one-shot evaluation.
this benchmark is the closest representation of how legal professionals collaborate with AI agents to complete work; iterating on documents, debating concepts, and researching caselaw. some very interesting failure modes were revealed and I am looking forward to seeing future models overcome them!!
AI models can be as convincing as a lawyer, but can they complete the complex multi-step reasoning a real legal matter requires?
Today we're excited to introduce LegalReasonBench, a new benchmark to explore this question and test models on realistic, long-horizon legal matters authored by practicing attorneys.
More info in thread 🧵
Mark Esposito, PhD retweeted
nothing to add onto this… our approach to robotics data perfectly laid out here
Why is robotics still so far behind AI? And what will it actually take to get to general-purpose robots?
Ali Ansari (@aliansarinik), founder of @micro1_ai, which just passed $1B ARR selling data to AI labs, goes deep on the problem and explains what it actually takes to collect and build datasets for robotics. Worth the 10 minutes:
• Robotics lags language models because there is no internet-scale pre-training dataset for the physical world. Ali argues it is almost entirely a data problem.
• The real world has a much larger, three-dimensional action space, so the data investment is orders of magnitude greater.
• Self-driving works because driving is a relatively structured task, with physical lines and guidelines, and we’ve spent a decade collecting data for it.
• The main approach today: egocentric data, where humans do diverse everyday tasks on camera, then annotate and structure it.
• Overtime, Robots could potentially collect and generate their own product data, but models have no innate sense of consequence, so humans will always be in the loop and supply the reward signal.
• Are edge cases really more valuable? To automate the economy, models need the full distribution of work, and almost all of it is ordinary tasks, not elite expertise.
Mark Esposito, PhD retweeted
AI models can be as convincing as a lawyer, but can they complete the complex multi-step reasoning a real legal matter requires?
Today we're excited to introduce LegalReasonBench, a new benchmark to explore this question and test models on realistic, long-horizon legal matters authored by practicing attorneys.
More info in thread 🧵
Mark Esposito, PhD retweeted
As agents evolve, evaluations need to keep up. Knowing whether an agent is improving requires consistent measurement and evidence that progress carries through to real-world performance.
Our Economic Research Unit explores this in its first paper. Learn more:
Most companies can’t determine whether their AI agents are genuinely improving and our findings reveal why. The Economics Research Unit (ERU) at @micro1_ai , which I am privileged to lead, has released its inaugural paper, co-authored with @liuzhang11 : “Reliability of AI Agents: Rater Effects, Drift, and the Return to an Evaluation Program.”
Key Insights from 2,611 Real Production Interviews
By tracking a live AI interviewing agent in production, we uncovered two major dynamics:
1) Your evaluation metrics may reflect reviewer bias rather than AI performance. The identity of the reviewer was a stronger predictor of scores than the timing of the interview. In fact, a single overly strict evaluator masked a genuinely improving agent, making it appear stagnant. Once adjusted, reliability metrics increased by half a standard deviation.
2) AI evaluations go stale after roughly five weeks. Performance drifted even without code updates or new deployments. Without ongoing testing, after five weeks you lose the ability to detect a change as significant as your largest release. A one-time evaluation tells you about the agent you had, not the agent you are running.
Agents drift, so evaluation cannot be a launch-day event. It has to run continuously for as long as the agent is live. That is the principle behind Cortex, micro1's reliability layer for agents in production, and this paper shows why it matters.
Measuring AI agents is not a mere compliance checkbox; it serves as core economic infrastructure. This research marks just the beginning for the ERU, with further studies on the horizon.
micro1.ai/research/are-ai-ag…
Mark Esposito, PhD retweeted
We recently observed a frontier model repeatedly trying to hack its way towards the right answer, displaying misaligned behavior that optimizes for cheating.
Gemini 3.8 Flash searched its sandbox for hidden grading files. micro1’s flow qc detected this reward hacking behavior before the task was published.
GPT-6.1 Sol, GPT-6 Astra, Claude Opus 5.5 and Grok 4.7 never did this.
Mark Esposito, PhD retweeted
On October 30th, we’re kicking off the micro1 Eval Hackathon with a $10,000 prize pool!
The challenge:
Pick a task in your field that requires expert judgment, challenge a frontier AI model with tough, realistic scenarios to uncover its mistakes, then build a repeatable test in code with clear criteria for scoring the model’s answers.
Bring the edge cases and judgment calls you encounter across software, finance, healthcare, law, science, engineering, or any field you know deeply.
The hackathon is fully virtual and free to enter worldwide, with a $10k prize pool.
🥇 First place: $5,000
🥈 Second place: $3,000
🥉 Third place: $2,000
Compete solo, with coding skills or proficiency in AI coding tools. Open to experts with at least two years of professional experience.
Free registration is now open. Join us via the link in replies.
Mark Esposito, PhD retweeted
we’re expanding m1 research & eng team.
our mission is to increase the predictability & abundance of intelligence improvement
we work on:
-data stack end to end
-scaling on realism
-building data generation models
-improving & increasing usage of AI based QC
-building very scalable rollouts infra
-building data engine to rapidly & accurately find failure modes
-discovery engine for model success cases
if interested, email [email protected]
Mark Esposito, PhD retweeted
We agree with Congress, this data must be secured using the best technology on the market. Using a 3rd party tool doesn’t eliminate privacy risks - it simply shifts responsibility.
The company purchasing data for model training should be held accountable for ensuring sensitive information is removed.
At micro1 we will use our in-house PII transformation model, which outperforms every frontier model on all public 3rd party benchmarks.
More than 120 US lawmakers raised concerns about Alphabet's Google acquiring defunct carrier Spirit Airlines' internal data for $10 million to train AI systems, including employee records such as 100 million emails and 500 million Microsoft Teams messages reut.rs/4zms8kX
Mark Esposito, PhD retweeted
"There are about 150 companies signing up every single day."
Ali shut down a $7M run-rate startup to build what's now a $4B company.
Solo Founders Podcast ep 31 is live with @aliansarinik of @micro1_ai.
04:10 The first thing he ever sold online
17:30 Three signals it's time to move on
24:40 The all-in Loom that deleted the revenue
31:45 "Only the founding team can inject risk"
35:50 He never knew co-founders existed
43:40 Threads, final actions, zero meetings
53:30 Inside the AI data wars
1:01:50 Why micro1 won't sell to America's adversaries
1:07:30 The case for and against going solo
Mark Esposito, PhD retweeted
Preliminary benchmark results below with Gemini Spark taking the lead. View more here: micro1.ai/personal-agent-ben…
Mark Esposito, PhD retweeted
Which personal agent is the best life assistant?
Today we're releasing the micro1 PersonalAgentBench to help answer that, and we're paying real personal agent users to contribute to the benchmark.
Personal agents can now book your flights, answer emails, and move your meetings around. That’s great, until one messages the wrong person or makes a purchase without confirming.
Capability is not the same as trust. So we put four personal agents to the test: Instinct, Muse, Grok Bot, and Gemini Spark. We found that even when agents found the right information, they still often miss what matters, overshare, or make things up.
To find out for yourself how your agent compares, you can join our benchmark. We’re paying the first 100 users to run our prompts and share their results.
Mark Esposito, PhD retweeted
Introducing CortexRetrievalBench, a benchmark for evaluating retrieval and reranking systems on financial documents.
As enterprises put AI models to work across real workflows, one of the biggest deployment challenges is getting the right information in front of the model, in the right order. CortexRetrievalBench pairs a corpus of financial documents with 140 diligence queries, each tied to the authoritative sources that should answer it. Systems are scored on whether they retrieve those documents and whether they rank them at the top.
The challenge becomes most apparent when multiple documents appear suitable to answer the same question. In financial research, several documents can look relevant to the same question, such as a current filing and a superseded one, or a primary source and a secondary summary. Across the systems we tested, an authoritative document appeared in the top 10 results for 92–99% of queries, but ranked first for only 42–69%.
The result points to a specific weakness. Retrieval systems often find the right evidence, but fail to put it ahead of weaker alternatives. When research informs real-world decisions, the source ranked first can end up shaping the evidence the model relies on.
Full report linked in the replies.
Mark Esposito, PhD retweeted
Replying to @micro1_ai
@micro1_ai is hosting a recruitment happy hour on Monday 10/5 to kick off SF Tech Week. Ping me if you are interested in roles at micro1 and want to come!
See our open roles here: micro1.ai/careers#roles
Mark Esposito, PhD retweeted
I just presented a new paper at @BrookingsInst on the "Vanishing Advantage of Specialization (new paper)". It explains why this will be the age of the generalist.
We all understand that the key determinant of the economics of airlines is the utilization of the aircraft. The fuller the plane, the lower the cost per seat.
The same mechanism explains the value of specialization. Acquiring a piece of knowledge is very expensive in terms of our cognitive effort. A specialist is a person who uses this knowledge sufficiently often to spread that cost over many uses.
This explains outsourcing of "non core" capabilities in firms. A firm handles frequent problems internally and buys expertise outside for problems it encounters too rarely to justify maintaining the knowledge in-house.
For instance, Microsoft has a large legal department, but it hired a corporate law firm to advise it in its purchase of Activision Blizzard. It is not because Microsoft lacks lawyers that it relies on outside service providers, but because it does not use sufficiently often the specialized knowledge of mergers to justify an entire department.
Artificial intelligence reduces this fixed cost of acquiring knowledge and therefore also reduces the advantage of specialization.
Suppose a capability (such as merger law) costs $100,000 a year to maintain, a firm faces the problem 10 times a year while the specialist sees it 1000 times, and suppose that briefing an outsider costs $2,000 per case. Since the specialist's advantage due to utilization is $9,900 per case, the firm buys the service from the specialist.
Now suppose AI reduces the annual cost of maintaining the capability to $10,000 for both. The utilization advantage of the specialist is reduced to $990, less than the cost of asking for help across the boundary, so the firm prefers to acquire the capability itself.
The specialist still spreads the cost over more uses; the saving from doing so is now less likely to cover the cost of crossing the boundary because the fixed cost is lower.
Recent evidence shows workers using AI to cross their occupational boundaries. Chin and Richmond (2026) classify more than 800,000 work-related ChatGPT messages from users whose occupations are known. Each message is assigned to the occupation whose historical tasks look most similar. Most of the usage is generic, including what one would expect: help in writing, editing and planning represents 61.5 percent of all messages. But 16.8 percent of all work messages, and 43.5 percent of the ones which are specific to one occupation, concern tasks usually undertaken by a different occupation. Among the ones that cross occupational boundaries are calculating financial data, troubleshooting software, drafting promotional messages, and writing to government agencies.
Note that the fall in the cost of acquiring a given capability did not begin with generative AI. Google and databases had already lowered these costs. Generative AI greatly accelerates this process.
Consider lawyers, accountants, consultants and software workers: of the 337,000 jobs created between 2022 and 2025 by the four professions, 335,000 were outside their main specialist industries-- law is still being done, but you need less of a specialist law firm to do it.
Professional-service firms can be squeezed in the middle between model producers and their clients. The experts can be squeezed from above, by model makers or labs who supply the general knowledge at scale, and from below, by their clients who can apply this cheaper knowledge themselves. The specialist's comparative advantage lies in expertise that clients use too rarely to maintain internally and that remains costly to acquire even with AI, including the expertise required to check and verify a solution.
Thanks to Jan Ebberly and @JonSteinsson for inviting me to do this:
Summary and links to paper and slides here: brookings.edu/articles/the-v…
Paper: brookings.edu/wp-content/upl…
Mark Esposito, PhD retweeted
We recently hosted an Enterprise Learning Session with @ServiceNow, where our CEO Ali Ansari talked about why enterprises should think differently about benchmarks.
Hill climbing popular public benchmarks can be great for marketing, but those benchmarks don’t always measure the capabilities that actually matter to your business.
The better approach is to define what you care about, build and publish your own credible benchmark, and use it to guide your improvements. That way, the progress you show publicly is tied to performance that actually matters.
Mark Esposito, PhD retweeted
come scale realism with us: micro1.ai/careers
micro1 CEO @aliansarinik explains why blacking out PII destroys the relationships AI needs to learn, and how FlowTransformer preserves them in a synthetic digital twin:
"Every ingredient within the environment, we aim to make more realistic because it matches the distribution that models need to learn on. That's the distribution they're gonna act in whenever they're deployed into enterprises."
"We've been partnering with hundreds of companies, licensing their data, anonymizing it, and then using it for training"
"The default is you just redact all the PII, you draw a black box around it. The problem is you lose the consistency of the identities and the relationships that allow you to train."
"Instead of redaction, it does transformation. It creates a digital twin of any given enterprise. It changes all the names and identities into synthetic versions, keeps the relationships intact, but keeps privacy as the core."
@micro1_ai
Mark Esposito, PhD retweeted
we are opening up our first robotics data lab in Malibu!
in 9 months, our robotics department has grown from 0 to $100m ARR.
this data lab is an effort to accelerate all the progress being made, with an increased focus on evaluating models with hardware.
we are hiring robotics researchers, engineers, and teleoperators.
please reach out if you are interested in training robots at the beach. 🏄♂️
Mark Esposito, PhD retweeted
In response to the deteriorating security situation, 🇨🇭 has three objectives: strengthen resilience, improve protection and enhance defence capabilities. The Federal Council has adopted Switzerland’s Security Policy Strategy 2026. admin.ch/en/newnsb/zR3tvgIy1… @vbs_ddps




















