low latency guy

San Francisco, CA
Joined June 2009
Code and sleep retweeted
🚨 @NVIDIAAI just quietly dropped its own local inference server... No announcement. Single-digit stars It's called Flare and Apache 2.0 Built for SM120/SM121 GPUs RTX PRO 6000 Blackwell and DGX Spark >PyTorch + FlashInfer at build time >Rust runtime at serve time. No Python. >CUDA graphs captured before serving >FP8 KV cache by default >Recipes for Qwen Flash, GLM and DeepSeek github.com/NVIDIA/flare-loca…
36
79
23
627
53,691
Коллеги из Смартли некоторое время использовали домен smartly.af (читается «smartly as fuck», «дохуя умный») для внутреннего artifactory, но потом решили перестать поддерживать экономику Афганистана :-)
Every once in a while, some unsuspecting country's domain economy goes vertical 😂 .𝚌𝚘 → Colombia .𝚒𝚘 → I/O → Island Ocean .𝚖𝚍 → Markdown → Moldova .𝚛𝚜 → Rust → Serbia .𝚊𝚒 → AI → Anguilla .𝚜𝚒 → SI → Slovenia
2
1
107
10,439
Code and sleep retweeted
I gave a talk on data parallelism this week, and I discussed 'runtime dispatching'. It is a trick whereas you check what the CPU supports (at runtime) and selects the right code path. The engineer asked 'what happens if the processor speculates wrong?' Indeed, whenever there is a branch, a compiler might try to evaluate the wrong branch. If you try to execute an instruction that your processor does not support, you typically get a crash. So it might be possible that you would check what the CPU supports, and somehow end up on a code path with unsupported instructions due to speculation. So would it crash if it speculated wrong? I said that I expected that there should be no crash, and no side-effect. I am happy to report that my answer was correct. Civilization saved, once again.
11
8
1
139
8,711
Code and sleep retweeted
Opus 5.5 is the result of your feedback. It communicates clearly, it's cheaper per token than Opus 5.0 with the intelligence of Fable 5.1, it’s very token efficient and works across every effort level. We're also increasing 5h rate limits & giving you a banked reset.
Introducing Claude Opus 5.5, the first model in our new Claude 5.5 family. It performs at the level of Claude Fable 5.1 for most tasks, and costs 40% less to run than Opus 5.
760
413
169
12,112
686,140
Code and sleep retweeted
I used Opus 5.5 to formally verify the Claude Agent SDK using Lean. A couple short prompts = 16 PRs fixing various bugs and race conditions. Video attached. TLA+ also works well. I sometimes combine Lean and TLA+ to look for issues around data flow, concurrency, and state mgmt. I don't know either language well, but Claude is excellent at both. This approach is super useful for formally modeling your code and finding bugs that a human probably wouldn't have spotted. Is formal verification the future of coding (or at least, bug finding)?
489
341
301
6,289
2,147,191
Code and sleep retweeted
Replying to @arvidkahl
Perfectly fine in good tools
4
1
5
695
Code and sleep retweeted
It snowed in San Francisco in 1976
Where has snow fallen at any point since 1940?
57
144
23
5,903
605,651
Code and sleep retweeted
Deploying agent is not a trivial task, especially for team: you have to configure inference tokens, git, deploy somewhere, configure authentication. Also how to invite your whole slack to it? Now you can ask Happy Agent to install itself anywhere and invite people via emails. It will do autonomously and deploy to any VM you have access to. In this video i am using my exe.dev account. Tested with AWS. You can use virtually any place you wish. It doesnt neeed public ports, special networking, explicit authentication configuration - everything is handled automatically. Networking is done via tailcat. It will install, configure and invite people all by itself autonomously in any environment.
1
1
473
Code and sleep retweeted
A 35B language model running on an iPhone using only 1–2.5 GB of peak memory. No cloud. No remote server. No desktop GPU. Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device.
391
830
151
10,233
726,335
Claude Fable 5.1 tops the Artificial Analysis Intelligence Index but costs 20% more per task than Fable 5 despite a 75% cache read price cut We supported @AnthropicAI with pre-release evaluation of Claude Fable 5.1. At max effort it scores 66 on the Artificial Analysis Intelligence Index, the highest score we have measured, ahead of Claude Opus 5 (max, 63), Claude Fable 5 (max, 62), GPT-5.6 Sol (max, 61) and Grok 4.6 (high, 61). We evaluated the model with Anthropic's ‘default’ server-side fallback, which routes safety-flagged requests to Claude Opus 4.8 or Claude Opus 5; fallback served ~4% of output tokens across the Intelligence Index. Key takeaways ➤ Frontier Intelligence with improvements across benchmarks: Fable 5.1 gains +4 points on the Intelligence Index over Fable 5. On HLE, Fable 5.1 scores 59.1%, ahead of the previous best of 55.5% from Claude Fable 5. It posts the narrowly highest scores we’ve seen on Terminal-Bench v2.1 (91.4%) and SciCode (62.0%), and on τ³-Banking it gains 9 points over Fable 5 ➤ 75% cache read price cut, but Fable 5.1 still costs more per task: Anthropic has cut the cache read price from $1 to $0.25 per 1M cached input tokens, with standard pricing unchanged at $10/$50 per 1M input/output tokens. Fable 5.1 (max) costs $3.76 per Intelligence Index task, 20% more than Fable 5 (max), because it uses ~1.7x the output tokens. The cache cut saves ~$1.40 per task, concentrated in the agentic evaluations where the majority of input tokens are cache reads. At xhigh effort Fable 5.1 scores 65 at $2.72 per task, $1.04 less than max, but still above Claude Opus 5 (max, 63) at $2.34 ➤ Claude Fable 5.1 holds the upper end of the Intelligence vs Output Tokens per Task Pareto frontier: every model variant scoring higher than GPT-5.6 Sol (medium) on the Intelligence Index is matched or beaten by a Fable 5.1 effort level on both intelligence and token usage ➤ Highest scores on agentic work tasks, but effectively tied with Opus 5: Fable 5.1 sets the highest scores we have measured on GDPval-AA v2 (1,853 Elo, +130 over Fable 5) and AA-Briefcase (1,694 Elo, +122 over Fable 5), our agentic knowledge work evaluations. Against Claude Opus 5 the GDPval-AA v2 lead is within the confidence interval and AA-Briefcase (1,685) is effectively tied, with Fable 5.1 ahead on analytical quality and rubric correctness, but behind on presentation Other model details: ➤ Context window: 1 million tokens, supporting image and text inputs as with Anthropic’s other recent launches ➤ Pricing: Fable 5.1 retains the $10/$50/$12.5 input, output, and cache write prices per million tokens from Fable 5, but cache hits have been reduced to $0.25 per million tokens, a 75% relative reduction from before that will materially reduce agentic workload costs
110
191
120
2,081
506,950
Code and sleep retweeted
We’re introducing Claude Fable 5.1 and Claude Mythos 5.1. They're the world’s most advanced models for coding and knowledge work.
2,874
7,298
4,815
65,407
21,086,805
Code and sleep retweeted
Sun Microsystems again decades ahead of its time.
Edge compute startup @armada_ai unveiled Orion, the latest (and greatest, capacity-wise) in its Galleon data-centers-at-the-edge family. It's Armada's "largest and most powerful Galleon yet,” CEO @danwrightSF told Tectonic. 🔗Read the full story at the link below
30
21
2
231
14,242
Code and sleep retweeted
Crémieux
@cremieuxrecueilAug 28
Mr. President, I know exactly where to put this: Space Academy should be built in the Presidio, in San Francisco.
BREAKING: President Donald J. Trump just signed an Executive Order establishing a new federal academy, THE UNITED STATES SPACE ACADEMY. 🚀🇺🇸
195
120
23
3,580
211,698
Code and sleep retweeted
Claude Code being closed source may have done more damage than good. There was no secret sauce worth protecting — just a long tail of important implementation details about how to use Anthropic's models effectively. Keeping those details closed slowed everyone down, without creating a durable advantage. Now the knowledge is escaping anyway, and coding agents are moving to the cloud.
1
3
264
Code and sleep retweeted
OpenAI
@OpenAIAug 29
We’re ending our partnership with Cursor following its acquisition by SpaceX. Under our proposal, Cursor’s direct access to our models would end on November 12. We know that the people most affected by this decision are the developers who rely on OpenAI models in Cursor. We care about their experience in this transition and we’re ready to go above and beyond to support them. openai.com/index/our-decisio…
3,207
2,314
3,584
22,295
15,998,937
Code and sleep retweeted
These SFO airport signs are such a lie. I looked carefully and there was only 737 and A320.
216
955
97
14,069
276,517
Code and sleep retweeted
25ms for an S3 lookup is "slow", but latency is about budgeting. There's a latency threshold above which product experience degrades. Everything below that is a cost/capacity optimization. Good engineering always considers the product requirements.
2
4
57
2,577
Code and sleep retweeted
Paul Graham
@paulgAug 26
An email I sent to the summer 2007 YC batch to encourage them before Demo Day. It's all still true today.
104
349
64
5,440
437,562
Code and sleep retweeted
Well this is damning and explains SO MUCH about why Meta is losing its best engineers; why morale is rock-bottom; and why an engineering culture built up in 20 years was destroyed overnight. Zuck + execs wanted to lay off ~60% of staff. When the business does better than ever.
121
187
59
3,224
448,699
Code and sleep retweeted
Even Dolly Parton’s philanthropy was good and efficient. Her Imagination Library mails a free book every month to kids from birth to age 5. Instead of just writing checks, she built infrastructure, kind of like what Andrew Carnegie did with libraries. Her foundation negotiates huge bulk discounts, selects the books, and runs the distribution system, and governments, nonprofits, and local donors help cover the marginal costs. So a book that retails for ~$10-15 can be bought and mailed to a child for roughly $2–3. Good model: use philanthropy to build the machine, then let others fund it at a tiny cost per child. A recent study found participants entered kindergarten about 0.13 SD higher on literacy and were 16–24% less likely to be severely behind... But families self-select into the program, so much of the effect is probably selection bias. I made a donation today: donate.imaginationlibrary.co…
9
49
1
489
20,837