personal assistants are great, especially if you are opinionated about the way you want to have a thing done a certain way. the busier you get, the more value you see in "outsourcing" brainpower/memory. some things I use it for
- tracking travel
- shopping/grocery/wishlists
- workout planning/reminders
it falls squarely in the category of "things I don't want to manage myself but don't want to bother folks with my issues" category. the only thing that folks might have an opinion on is verbosity/style, which is ultimately discretionary to the user/product and can be tweaked to a certain degree, but ultimately great value add to someone's daily drive.
crazy work team! congrats!!
AI models can be as convincing as a lawyer, but can they complete the complex multi-step reasoning a real legal matter requires?
Today we're excited to introduce LegalReasonBench, a new benchmark to explore this question and test models on realistic, long-horizon legal matters authored by practicing attorneys.
More info in thread 🧵
Jacky retweeted
Most companies can’t determine whether their AI agents are genuinely improving and our findings reveal why. The Economics Research Unit (ERU) at @micro1_ai , which I am privileged to lead, has released its inaugural paper, co-authored with @liuzhang11 : “Reliability of AI Agents: Rater Effects, Drift, and the Return to an Evaluation Program.”
Key Insights from 2,611 Real Production Interviews
By tracking a live AI interviewing agent in production, we uncovered two major dynamics:
1) Your evaluation metrics may reflect reviewer bias rather than AI performance. The identity of the reviewer was a stronger predictor of scores than the timing of the interview. In fact, a single overly strict evaluator masked a genuinely improving agent, making it appear stagnant. Once adjusted, reliability metrics increased by half a standard deviation.
2) AI evaluations go stale after roughly five weeks. Performance drifted even without code updates or new deployments. Without ongoing testing, after five weeks you lose the ability to detect a change as significant as your largest release. A one-time evaluation tells you about the agent you had, not the agent you are running.
Agents drift, so evaluation cannot be a launch-day event. It has to run continuously for as long as the agent is live. That is the principle behind Cortex, micro1's reliability layer for agents in production, and this paper shows why it matters.
Measuring AI agents is not a mere compliance checkbox; it serves as core economic infrastructure. This research marks just the beginning for the ERU, with further studies on the horizon.
micro1.ai/research/are-ai-ag…
Jacky retweeted
AI was supposed to make us work less
I've never been this tired in my life
most days I have up to 10 agents running at the same time. one fixing a bug, one writing pages, one refactoring something I already forgot I asked for, 3 other I have no idea what they are doing but i suppose i ask it?
and I jump from conversation to conversation all day
compared to 5 years ago, I probably context switch 100x more
by the end of the day my brain is fried. completely toasted. nothing left, not even for the people I love
I call it the fried brain effect
thought it was just me but almost every founder I talk to lately says the exact same thing
we got superpowers, and somehow the pressure only went up. if you can ship 10x more, you feel like you have to ship 10x more. every single day
I don't have a solution
honestly I have no idea where this goes from here
anyone else feeling this?
Jacky retweeted
We recently observed a frontier model repeatedly trying to hack its way towards the right answer, displaying misaligned behavior that optimizes for cheating.
Gemini 3.8 Flash searched its sandbox for hidden grading files. micro1’s flow qc detected this reward hacking behavior before the task was published.
GPT-6.1 Sol, GPT-6 Astra, Claude Opus 5.5 and Grok 4.7 never did this.
Jacky retweeted
On October 30th, we’re kicking off the micro1 Eval Hackathon with a $10,000 prize pool!
The challenge:
Pick a task in your field that requires expert judgment, challenge a frontier AI model with tough, realistic scenarios to uncover its mistakes, then build a repeatable test in code with clear criteria for scoring the model’s answers.
Bring the edge cases and judgment calls you encounter across software, finance, healthcare, law, science, engineering, or any field you know deeply.
The hackathon is fully virtual and free to enter worldwide, with a $10k prize pool.
🥇 First place: $5,000
🥈 Second place: $3,000
🥉 Third place: $2,000
Compete solo, with coding skills or proficiency in AI coding tools. Open to experts with at least two years of professional experience.
Free registration is now open. Join us via the link in replies.
incredible from @micro1_ai, @aliansarinik, and the entire team. at a loss for words. next milestones will only become more vertical 🚀
about 3 years ago, micro1 lost 60% of its revenue overnight and almost went bankrupt.
today, i'm proud to share we've surpassed $1B annual run rate while remaining historically profitable.
it’s been a special journey. we’re grateful for every struggle, moment, and person who has been part of it.
we've taken a very untraditional path to get here. our core team is only ~180 people, spanning 14+ countries, and fully remote.
many of our decisions sacrifice hundreds of millions of revenue and slow us down short term. but we believe they compound into advantages that are impossible to replicate long term.
the long term is now.
these are the 8 principles that made micro1 a leader in the data industry and will soon make us number one:
1.) human 1st AI: treat the tens of thousands of experts helping train models with absolute best-in-class care. our goal is to make AI training one of the largest, highest paying, and most enjoyable job sectors in the world.
2.) U.S. first: we refuse to work with adversarial nations, or export data that improves models against the interests of Americans.
3.) long-term research over short-term ARR: invest in deep research, products, and infrastructure for years from now, not just the next customer pipeline.
4.) proper incentives for intensely hard work: no arbitrary requirements like 996. inspire people to work extremely hard with outlier incentive structures instead of forcing weekend work. we’ve maintained close to 100% core team retention as a result.
5.) stay lean and global: we’ve reached $1B run rate with ~180 people, and intend to slow team growth significantly from here. exceptional talent is found around the world, and pedigree is the wrong thing to over-index on.
6.) execution alignment: company to retain a streamlined board so we can optimize for the next several decades, remain light-weight and dynamic, and take bold & calculated risks.
7.) abundant intelligence improvement: invest in building data generation models to make high-quality datasets and RL environments dramatically less expensive and more abundant.
8.) scale data on realism and horizon: realism brings the complexity of the real world into training models, while horizon extends how long models can work and how much complexity they can handle. both have enormous room to scale, and we’re pushing the limits of both.
additionally: after surpassing $500M run rate this summer, we priced our Series B at a $4B valuation, alongside a ~$25M tender to give our early employees liquidity as we build for the long term. the round was led by Dream Ventures, a longtime supporter and one of our earliest investors, alongside Microsoft, Javelin, Flybridge, G Squared, Up Partners, and LG.
we’re also grateful to be backed by incredible industry leaders, including Guodong Zhang (co-founder of xAI), Andy Fang (CTO of DoorDash), Igor Babuschkin (co-founder of xAI), Behnam Neyshabur (founder of Mirendil), Jerry Tworek (ex-VP of Research at OpenAI), and many more.
data is on its way to becoming the largest industry humanity has ever seen, and we're certain this is just the beginning.
best,
Ali
Jacky retweeted
about 3 years ago, micro1 lost 60% of its revenue overnight and almost went bankrupt.
today, i'm proud to share we've surpassed $1B annual run rate while remaining historically profitable.
it’s been a special journey. we’re grateful for every struggle, moment, and person who has been part of it.
we've taken a very untraditional path to get here. our core team is only ~180 people, spanning 14+ countries, and fully remote.
many of our decisions sacrifice hundreds of millions of revenue and slow us down short term. but we believe they compound into advantages that are impossible to replicate long term.
the long term is now.
these are the 8 principles that made micro1 a leader in the data industry and will soon make us number one:
1.) human 1st AI: treat the tens of thousands of experts helping train models with absolute best-in-class care. our goal is to make AI training one of the largest, highest paying, and most enjoyable job sectors in the world.
2.) U.S. first: we refuse to work with adversarial nations, or export data that improves models against the interests of Americans.
3.) long-term research over short-term ARR: invest in deep research, products, and infrastructure for years from now, not just the next customer pipeline.
4.) proper incentives for intensely hard work: no arbitrary requirements like 996. inspire people to work extremely hard with outlier incentive structures instead of forcing weekend work. we’ve maintained close to 100% core team retention as a result.
5.) stay lean and global: we’ve reached $1B run rate with ~180 people, and intend to slow team growth significantly from here. exceptional talent is found around the world, and pedigree is the wrong thing to over-index on.
6.) execution alignment: company to retain a streamlined board so we can optimize for the next several decades, remain light-weight and dynamic, and take bold & calculated risks.
7.) abundant intelligence improvement: invest in building data generation models to make high-quality datasets and RL environments dramatically less expensive and more abundant.
8.) scale data on realism and horizon: realism brings the complexity of the real world into training models, while horizon extends how long models can work and how much complexity they can handle. both have enormous room to scale, and we’re pushing the limits of both.
additionally: after surpassing $500M run rate this summer, we priced our Series B at a $4B valuation, alongside a ~$25M tender to give our early employees liquidity as we build for the long term. the round was led by Dream Ventures, a longtime supporter and one of our earliest investors, alongside Microsoft, Javelin, Flybridge, G Squared, Up Partners, and LG.
we’re also grateful to be backed by incredible industry leaders, including Guodong Zhang (co-founder of xAI), Andy Fang (CTO of DoorDash), Igor Babuschkin (co-founder of xAI), Behnam Neyshabur (founder of Mirendil), Jerry Tworek (ex-VP of Research at OpenAI), and many more.
data is on its way to becoming the largest industry humanity has ever seen, and we're certain this is just the beginning.
best,
Ali
Jacky retweeted
We have two additions to PersonalAgentBench: @hark_labs and @OpenAI's Dots!
Dots has taken the lead, with Hark close behind while Gemini Spark moves to third, followed by Instinct, Grok Bot, and Muse.
We also doubled the number of tasks this week, adding research, image understanding, editing, and everyday communication. The wider set changed the picture. Gemini Spark was strongest on research but weaker on image and editing tasks. Hark and Gemini tied on tasks involving user authority. Dots was the most consistent across task types, although it was weaker when it had to infer personal context. No assistant led every category; the overall ranking rewarded consistent performance across the full set.
This is still an early snapshot, and we’re excited to see how results evolve as we add more agents, expand the tasks, and re-evaluate after product updates.
If you’re using any of these six personal agents, follow along by visiting the link in the original post.
Which personal agent is the best life assistant?
Today we're releasing the micro1 PersonalAgentBench to help answer that, and we're paying real personal agent users to contribute to the benchmark.
Personal agents can now book your flights, answer emails, and move your meetings around. That’s great, until one messages the wrong person or makes a purchase without confirming.
Capability is not the same as trust. So we put four personal agents to the test: Instinct, Muse, Grok Bot, and Gemini Spark. We found that even when agents found the right information, they still often miss what matters, overshare, or make things up.
To find out for yourself how your agent compares, you can join our benchmark. We’re paying the first 100 users to run our prompts and share their results.
Jacky retweeted
Today micro1 is announcing a commitment to spend $1B over the next 12 months on the acquisition & licensing of enterprise data for AI training, with financial backing from Citi and Hercules.
micro1.ai/blog/micro1-commit…
Jacky retweeted
Which personal agent is the best life assistant?
Today we're releasing the micro1 PersonalAgentBench to help answer that, and we're paying real personal agent users to contribute to the benchmark.
Personal agents can now book your flights, answer emails, and move your meetings around. That’s great, until one messages the wrong person or makes a purchase without confirming.
Capability is not the same as trust. So we put four personal agents to the test: Instinct, Muse, Grok Bot, and Gemini Spark. We found that even when agents found the right information, they still often miss what matters, overshare, or make things up.
To find out for yourself how your agent compares, you can join our benchmark. We’re paying the first 100 users to run our prompts and share their results.
Jacky retweeted
Introducing CortexRetrievalBench, a benchmark for evaluating retrieval and reranking systems on financial documents.
As enterprises put AI models to work across real workflows, one of the biggest deployment challenges is getting the right information in front of the model, in the right order. CortexRetrievalBench pairs a corpus of financial documents with 140 diligence queries, each tied to the authoritative sources that should answer it. Systems are scored on whether they retrieve those documents and whether they rank them at the top.
The challenge becomes most apparent when multiple documents appear suitable to answer the same question. In financial research, several documents can look relevant to the same question, such as a current filing and a superseded one, or a primary source and a secondary summary. Across the systems we tested, an authoritative document appeared in the top 10 results for 92–99% of queries, but ranked first for only 42–69%.
The result points to a specific weakness. Retrieval systems often find the right evidence, but fail to put it ahead of weaker alternatives. When research informs real-world decisions, the source ranked first can end up shaping the evidence the model relies on.
Full report linked in the replies.
Jacky retweeted
@micro1_ai is hosting a recruitment happy hour on Monday 10/5 to kick off SF Tech Week. Ping me if you are interested in roles at micro1 and want to come!
See our open roles here: micro1.ai/careers#roles
Jacky retweeted
We’re excited to share flow-transform 1.0, our new PII transformation model.
It replaces real-world identities with consistent synthetic counterparts across enterprise datasets while preserving the context and relationships that make the data valuable for AI training.
Across our evaluations, it delivers frontier-level performance in detection, identity synthesis, and full-corpus transformation.
Alongside the model, we’re introducing the Enterprise De-Identification Benchmark and TQI to measure transformation quality beyond detection alone.
Learn more:
Today we’re launching micro1’s PII transformation model, flow-transform 1.0, delivering frontier-level performance across detection, identity synthesis, and transformation of personally identifiable information.
On PrivacyBench, our model reaches 96.0% F1, outperforming every detection baseline we tested, including Tonic Textual, Claude Opus 4.8, Sonnet 4.6, Microsoft Presidio, Haiku 4.5 and GLiNER2.
Some of the most valuable training data for frontier AI models lives inside fully functioning companies. It captures years of real work across decisions, communications, tools, handoffs, exceptions and the relationships connecting them.
The problem is that this data is also full of PII.
Traditional redaction makes the data safe, but it also destroys the very workflows and relationships frontier models need to learn from.
flow-transform 1.0 solves this by turning enterprise operational data into high-fidelity training data for frontier models by replacing real-world identities without flattening the reality the data captures.
Jacky retweeted
Today we’re launching micro1’s PII transformation model, flow-transform 1.0, delivering frontier-level performance across detection, identity synthesis, and transformation of personally identifiable information.
On PrivacyBench, our model reaches 96.0% F1, outperforming every detection baseline we tested, including Tonic Textual, Claude Opus 4.8, Sonnet 4.6, Microsoft Presidio, Haiku 4.5 and GLiNER2.
Some of the most valuable training data for frontier AI models lives inside fully functioning companies. It captures years of real work across decisions, communications, tools, handoffs, exceptions and the relationships connecting them.
The problem is that this data is also full of PII.
Traditional redaction makes the data safe, but it also destroys the very workflows and relationships frontier models need to learn from.
flow-transform 1.0 solves this by turning enterprise operational data into high-fidelity training data for frontier models by replacing real-world identities without flattening the reality the data captures.
Jacky retweeted
Why we bid $12.5 million for Spirit Airlines’s data:
As you may have seen in the news, we are aiming to acquire the Spirit operational data. We'd like to transparently & directly explain why. The future of AI is a bet on two things: the messiness of the real world and the brilliance of the humans working inside it.
The combination of these two things results in a new dimension of scale called realism. Realism is the most significant thing that has ever happened to model training. It’s what lets RL environments and tasks match the conditions a model is deployed into.
The limit is the real world itself. But the real world is also dynamic. It grows with a company and shifts as operations evolve. There is not a binary scaling of entering the real world. Realism keeps scaling as long as real work keeps happening.
As a data lab, we want this data to benefit the entire AI ecosystem, while ensuring labs that we partner with for this data, agree to strict privacy obligations. This includes keeping the data away from labs outside of U.S. and U.S-allied countries. Our mission is to push the frontier in novel ways, with a human & privacy first approach.
To do that, we've bid on a portion of Spirit’s Airline’s non-sensitive operational data.
Effectively scaling realism requires de-identification as a core competency. Very few organizations can do this. We believe the few that can have an obligation to act transparently. That’s why we’ve made binding commitments. We will prohibit re-associating the data and will not profile any former employee. We’re paying for an independent data ombudsman to review the data and how we use it.
Furthermore, our offer aims to create net new opportunities for as many as we can from the 17,000+ former Spirit employees. We’re providing former Spirit employees preferential access to paid human data roles on our platform, along with free access to our AI training and reskilling curriculum.
These commitments bind us to spending at least $1M. That is just the minimum. It’s entirely possible we spend orders of magnitude more than $1M working with and training former Spirit employees to help define the frontier.
We think this transaction will set the terms for how data like this changes hands from here. Those terms should be worth copying.