@leonhoi
iAccount based inCanada
About this account
- Account based in
- Canada
- Connected via
- Canada App Store
Account-level information from X, not a live location or the device used for a specific post.
Building https://nitter.cf/t.co/sDo9bfyNqB, the open-source agent runtime I run https://nitter.cf/t.co/H7ukD0W9yj on solo. I build and run AI operating layers inside founder-led companies.
Greater Vancouver
Joined January 2007
- Tweets7.2K
- Following680
- Followers3.3K
- Likes941
A team of people used to run LifeHack. Today it runs on 129 agents, and I'm the only human left in the loop.
This week I audited the whole fleet. Every agent had settled into one of five shapes. Not roughly five. Five.
GPT-6 Sol's hallucination rate drops because it declines to answer more often. It attempts 83% of the questions where GPT-5.6 attempted 99%, and accuracy falls from 59% to 54%.
If your eval only counts wrong answers, a model that answers less looks like progress.
GPT-6 Sol and Luna push the cost efficiency frontier by halving cost relative to GPT-5.6 Sol and Luna. Intelligence Index and Coding Agent Index scores remain level with GPT-5.6, with progress in some evaluations and regressions in others
Pricing is approximately half that of GPT-5.6: Sol drops from $4/$20 to $2/$10 per million input/output tokens, and Luna from $0.20/$1.20 to $0.10/$0.50, with the same 90% discount for cache reads and 25% premium for cache writes.
Key takeaways:
➤ Halves Cost per Task: GPT-6 Sol (max) costs $1.06 per task to run the Artificial Analysis Intelligence Index, ~50% less than GPT-5.6 Sol (max) at $1.99. GPT-6 Luna (max) costs $0.07 per task, ~60% less than GPT-5.6 Luna (max) at $0.18. This is driven by the price cut, as both models use slightly more output tokens per task (31k vs 29k for Sol, and 51k vs 41k for Luna). These two releases allow OpenAI to capture a significant portion of the cost efficiency Pareto frontier.
➤ In the Coding Agent Index, Sol improves but Luna regresses: In OpenAI's Codex harness, GPT-6 Sol (max) scores 57 in the Artificial Analysis Coding Agent Index, up 2 points from GPT-5.6 Sol (max), with gains in Terminal-Bench 4.0 (43% vs 37%) and SWE-Atlas-QnA (58% vs 54%). At $2.99 per task it costs ~50% less than GPT-5.6 Sol (max) and sits on the Pareto frontier of Coding Agent Index vs Cost per Task. GPT-6 Luna (max) scores 41, down 2 points from GPT-5.6 Luna (max), with lower scores in SWE-Atlas-QnA (44% vs 49%) and DeepSWE v1.1 (64% vs 66%), at ~60% lower cost per task.
➤ Significant reduction in hallucination: Both models hallucinate less in AA-Omniscience, our knowledge and hallucination benchmark. GPT-6 Sol (max) cuts its hallucination rate from 92% to 60% and GPT-6 Luna (max) from 93% to 77%. Sol achieves this by declining to answer more often: it attempts 83% of questions vs 99% for GPT-5.6 Sol (max), which cuts wrong answers by about a quarter but also lowers accuracy 5 points from 59% to 54%. Luna's accuracy is broadly unchanged at 44% vs 43% while it answers fewer questions. On the AA-Omniscience Index, Sol improves from 22 to 27 and Luna from -10 to 1.
➤ Mix of improvement and regression across evals: Beyond AA-Omniscience, both models improve in AutomationBench-AA (Sol 62% vs 60%, Luna 53% vs 50%) and Terminal-Bench 4.0 (Sol 44% vs 40%, Luna 13% vs 12%). However, we observe regressions in two key knowledge work evaluations. In GDPval-AA v2.1, our benchmark adapted from OpenAI's dataset of economically valuable tasks across 44 occupations, Sol drops ~100 Elo points and Luna ~75. Luna also drops ~45 Elo points in AA-Briefcase v1.1, while Sol is level. AA-Briefcase v1.1 is a private evaluation across multi-week knowledge work projects, with thousands of input files. Our team has manually inspected hundreds of model outputs: the regressions tend to be driven by reduced presentation quality and deliverables that omit rubric elements.
Congratulations @OpenAI and @sama on the launch!
so is opus 5.5 high the best value?
The consent design here is good: tab borrowing is controlled by a browser setting, not a command-line flag.
That approval moves a logged-in tab into the Agent Window. The docs show no further approval for each click or submission, so the tab's existing authority still matters.
We open sourced BrowserSkill, a bridge between your agent and your actual browser.
most tools give the agent a blank browser. We let it borrow a tab from yours, then hand it back.
> login state is already there, it just works where you're signed in
> captchas and confirmation dialogs come back to you, then it continues
> it's a CLI, not an MCP server => any agent that can run a shell can use it, and you see every call it makes
one thing that's easy to miss: the agent asks before borrowing a tab, and that switch lives in your browser settings, not in a flag, so it can't be talked around.
one line to install, works with Cursor, Claude Code, Codex, Hermes, Openclaw, CodeBuddy, WorkBuddy. Everything runs locally, MIT.
github.com/Tencent/BrowserSk…
Claude Code shipped a plugin eval last week: run your cases with a skill loaded, then again with nothing, and read the difference. A skills folder is something you can measure now instead of argue about. This result is self-reported without numbers. Yours doesn't have to be.
Astra gives long Codex sessions a searchable work history.
Codex can keep notes across context windows and search earlier messages and tool outputs, including details the notes missed. OpenAI says the feature is experimental and off by default for now.
What I found when I timed the rest of my loops: aishapedcompany.com/p/ai-mad…
Local AI becomes useful when it stops slowing down the agent around it.
In Perplexity's benchmark, Lily reached 186.6 decode tokens per second on an M5 Max. That is fast enough for a practical split: private files on the Mac, heavier reasoning in the cloud.
Today we’re open-sourcing Lily, the local inference engine we built for hybrid compute in Perplexity Computer.
Lily is specialized for Qwen3.6-35B-A3B on Apple silicon, built so on-device compute doesn’t bottleneck Computer tasks.
Read more: perplexity.ai/hub/blog/optim…
Harvey's new legal model came out of 1,750 tasks where experts wrote out what a good answer must contain, around 50 pass/fail checks each, tied to specific files.
Those checks let you grade an agent now and train one later. Worth writing for one workflow you already hand off.
Update on @harvey’s model training effort.
We post-trained a model we are calling Tenet:
- Achieves SOTA on LAB
- Generalizes 3rd party legal benchmarks
- Uses sub-agents for domain specific capabilities
Tenet uses Kimi K3 as base and was post-trained in collaboration with @FireworksAI_HQ:
- Rank-64 LoRA over the full network
- GSPO with importance-ratio masking
- 134 B300 GPUs for 2 months
Despite not being trained on 3rd party legal datasets we found improvements on:
- @mercor’s Apex Agents - Corporate Law
- @crosbylegal’s Redline Bench
- LegalBench
Tenet also learned how to use domain-specific subagents (separate post-trained models) for complex tasks:
- M&A Diligence: training in an RLM harness for long-horizon tasks (with @baseten)
- Review Table: specialist models for high-volume structured data extraction (with @appliedcompute)
- Firm Knowledge: parametric memory and structured notes for more efficient enterprise search (with @engram)
These results suggest we can significantly scale training and we plan to:
- Scale both human and synthetic data significantly and scale training to 1K and then 10K GPUs
- This scale will let us move to full parameter fine-tuning and larger models
- We are now starting to post train models in our production harnesses
- Post-train other open-source base models to provide customers with model choice
If these problems sound interesting we are hiring for our post-training team
a very big deal… Moderna uses AI to study each patient’s tumor and create a personalized mRNA treatment just for them, and it just succeeded in a Phase 3 melanoma trial.
BREAKING: Moderna stock, $MRNA, extends gains to +212% on the day in after hours trading following positive phase 3 results of first the mRNA treatment to prevent cancer.
The stock has now added over +$50 billion in market cap today.
I leave Codex on the default context and never think about the window. It compacts itself and keeps going. The million-token setting is for the rare session that has to span a whole big repo, and turning it on eats your usage faster.
If you run agents on your own hardware: dots3-note ships an FP8 build that serves on a single 8-GPU node. That puts a 280B model for long agent tasks inside one box you control, instead of behind someone's API.
Introducing dots3-note preview — a small but mighty step toward long-horizon agency in real life.
🔹 280B MoE with 16B active parameters, a 512K context window, and multimodal understanding across text, vision, and audio
🔹 Introduces TEMPO, a new RL approach for long-horizon agent training through self-critiquing and test-time-scaled value estimation
🔹 Built to reason, explore unfamiliar environments, update memory over time, and combine multimodal perception with coding and tool use to solve complex tasks
🔹 Open weights on Hugging Face, alongside two open benchmarks for real-life agents: VibeSearchBench and VibeLifeBench
Competitive with much larger models across reasoning, agentic, and multimodal evaluations.
🔗 Tech blog: studio.dots.ai/dots/dots3-en…
🔗 Model weights: huggingface.co/dots-studio/d…
🔗 Github: github.com/studio-dots-ai/do…