In the land of the blind the one-eyed man is king
Joined February 2020
- Tweets4.7K
- Following1.3K
- Followers1.5K
- Likes7.7K
Astra level for 20% of the cost! The price of tokens are a race towards the bottom as compute steadily rises. This phenomenon is rapidly unfolding so will be interesting to see second order effects
Jack retweeted
Claude Opus 5.5 takes the top spot on the Artificial Analysis Intelligence Index, along with a 20% price cut and larger cache hit discount
Claude Opus 5.5 brings Anthropic to parity with GPT-6 Astra on evaluations like Terminal-Bench 4.0 and AutomationBench-AA, while extending Anthropic’s lead in agentic knowledge work.
At max effort it scores 58 on the Artificial Analysis Intelligence Index, the highest score we have measured by several points. Anthropic has cut Opus pricing to $4/$20 per 1M input/output tokens (Opus 5: $5/$25) and cache reads from $0.50 to $0.20.
Key takeaways:
➤ Consistent strong performance, with leading scores on six of the ten Intelligence Index evaluations: Humanity's Last Exam 61.4% (previous best 59.1%, Claude Fable 5.1), SciCode 66.9% (63.1%, Fable 5.1), GDPval-AA v2.1, AA-Briefcase v1.1, AA-Omniscience and AutomationBench-AA. On Terminal-Bench 4.0 it scores 59.6%, level with the leader GPT-6 Astra (xhigh) and +11 points over Opus 5. It remains slightly behind on CritPt, AA-LCR, and GDP.pdf
➤ Leads in agentic knowledge work: On AA-Briefcase, our private frontier knowledge work evaluation, it reaches an Elo of 1822. This is +143 over Fable 5.1, ahead on both analytical quality and presentation, and is the first time Anthropic has reached presentation quality surpassing GPT-5.6 Sol. This evaluation tests whether models can produce accurate and well-presented professional outputs using our open source reference agent harness, Stirrup
➤ Level with Opus 5 on cost per task despite 1.6x the output tokens: Opus 5.5 (max) uses ~119k output tokens per Intelligence Index task, against ~73k for Opus 5 (max), ~78k for Fable 5.1 (max) and ~27k for GPT-6 Astra (max)
➤ Four of five effort levels sit on the Intelligence vs Cost per Task frontier: Opus 5.5 max, xhigh, high, and medium all sit on the Pareto frontier, costing less or outperforming other models scoring 50+ (GPT-6 Astra, Claude Fable 5.1, and Claude Opus 5)
Other model details:
➤ Context window: 1 million token context with image and text input support, unchanged from Opus 5
➤ Pricing: $4/$20 per 1M input/output tokens, down 20% from $5/$25 for Opus 5. Cache writes $5 per 1M tokens for the 5 minute TTL, down from $6.25. Cache reads have been further discounted to $0.20 per 1M tokens, down 60% from Opus 5’s $0.50. This is a 95% discount compared to uncached input pricing, up from 90% on previous Opus models
➤ Effort settings: Five effort settings (low, medium, high, xhigh, and max). Intelligence Index evaluations were run at all five with Anthropic's default fallback enabled
So cool
Nearly half a year of silence. We spent it studying one problem: how far RL can scale.
MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks.
Streaming the run: mimo.xiaomi.com/rl/
Jack retweeted
JUST IN: Anthropic says they’re highly profitable if you take out some of their biggest expenses.
Jack retweeted
Probably the hackiest "bypasses" I've ever pulled off in botting:
The Shopify REST API was for a long time the best and fastest way to checkout on Shopify. High rate limits, no captchas, no checkpoint and easy to work with. This is what gave Cyber the edge at the very start in 2018 (Dashe days)
The Shopify team (with @jmwind at the helm) spent years plugging the holes to make sure it wasn't used during drops.
Around 2020, Shopify released Shopify Pay, which allowed you to expedite checkouts on all stores by linking your details to a phone number. Turned out the payment tokens generated through there allowed you to use the payments API as you could 2 years back.
We generated around 3k Shop Pay accounts with randomised details, load balanced them across Cyber tasks (as rate limits were per account) and dominated every single release (...until it got patched).
We called it Fast2.
3 years later I noticed that the payments API could still be used with Shopify mobile apps using Apple Pay. Turns out you could still send the users' tokenised credit card data in the request and that would take precedence - and as long as a valid signed Apple Pay token generated in the last 2 minutes was also in the request data, the payment request would go through and bypass captcha and checkpoint.
Generating those Apple Pay tokens was the challenge, and reversing iOS frameworks was not an option. So I repurposed an old iPhone with a custom sideloaded Apple Pay app that would submit the token to our servers when the flow completed. And by combining accessibility macros, developer sandbox mode to prevent using real credit cards, a voice activation trigger and a old pair of earphones taped onto the microphone - a makeshift Apple Pay token server was born and Fast2 (or was it Fast3?) was back.
@jmwind thank you for providing me with the longest, most multi faceted puzzle ever.
Wow. Magic is back
Frontier pretraining is said to be a big-lab-only game. We don’t have 100k chips yet, so there’s only one way: algorithmic efficiency. Our new recipe matches DeepSeek V4 Pro’s pretrain using 50x less compute – that’s roughly half the FLOPs used for GPT3, or ~$0.5M on GB200.
magic.dev/blog/pretraining
Jack retweeted
Excited for the new release of #HuggingFace durant.ly/huggingface - proud investor! durant.ly/hfrelease
Jack retweeted
Guy on the left can’t even keep a straight face thinking about how hard all the $PURR in his port must be ripping
So cool. Towards self accelerating ai r&d
Grok 4.6 was the first model trained on internal model-development tasks.
My team built the training and evaluation stack that enabled Grok to learn from work that accelerates model development itself, including production inference and kernel optimization.
Grok 4.6 now leads our internal MTS Eval and InferenceEval. In one experiment, we gave an earlier checkpoint our production inference codebase in a fully automated environment. It explored 297 optimization ideas and shipped 3 changes on top of an already human-optimized stack, improving prefill throughput by 3.1% and decode throughput by 1.5%.
Our work is featured in the R&D Enablement section of the Grok 4.6 model card.
