Founder https://xitter.cf/t.co/XvGjzwaBya, ex-Red Hat, ex-VA LNX Cares deeply. Parent, survivor (again), & esp your brother. He/Him/ร‰l/They. All copyright works CC BY SA 4.0

Santa Cruz, CA
Joined March 2007
Donald Leon Wade, 23 Jan 1942 - 21 Dec 2021 Of his four sons, I had the fewest formative years w/him. But being his one biological son, his effect on me is as profound as for my brothers K*, T*, & A*. A little ๐Ÿงตabout Dad: scientist, storyteller, home chef, minister's hubby ...
3
1
11
See y'all there!
Open Source AI Week starts today in the Bay Area and runs to 25 October. ๐Ÿ“… If you come for PyTorch Conference at the San Jose Convention Center on 20 and 21 October, stay one more day. On Thursday 22 October, the LMCache hands-on workshop runs 1:30 to 5:30 PM Pacific at Samsung, 3655 N First St, San Jose. ๐Ÿ“ Four hours with a live LMCache environment, engineering sessions, and a reception. Free, registration required: na2.hubs.ly/H08k5yg0
26
Karsten Wade [email protected] @quaid.bsky.social retweeted
After the workshop on Thursday 22 October, the mini-lmcache test environment stays open for your own experiments. ๐Ÿ› ๏ธ So what would you test first? A long shared prompt? A document that many requests read again? Reply with one line. ๐Ÿ’ฌ The hands-on deep dive is part of a free afternoon, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. Registration is required: na2.hubs.ly/H08k5xM0
1
1
1
122
My #Enterprise friends, we have a bouncing new #KVCache product! ๐Ÿ‘‡๐Ÿป Do you run your own inference using Open Weights models? Have you been managing KV cache natively at your inference engine layer? Tensormesh Platform works with all of that stuff & extends it into the future:
Today weโ€™ve announced to the world that ๐“๐ž๐ง๐ฌ๐จ๐ซ๐ฆ๐ž๐ฌ๐ก ๐๐ฅ๐š๐ญ๐Ÿ๐จ๐ซ๐ฆ, the ๐ฆ๐š๐ง๐š๐ ๐ž๐ฆ๐ž๐ง๐ญ ๐ฅ๐š๐ฒ๐ž๐ซ ๐Ÿ๐จ๐ซ ๐€๐ˆ ๐ข๐ง๐Ÿ๐ž๐ซ๐ž๐ง๐œ๐ž ๐๐š๐ญ๐š, is officially live and enterprise-ready ๐Ÿš€ Read the full announcement ๐Ÿ‘‡ na2.hubs.ly/H08nsMN0
1
1
50
Karsten Wade [email protected] @quaid.bsky.social retweeted
CVE-2026-105192 (CVSS 3.1 score 9.8): remote code execution in LMCache multiprocess (MP) mode. The fix is merged into the dev branch and will ship in 0.5.6, expected this week. Affected: 0.3.9 through 0.5.5, the 0.5.6 release candidates, and dev branches before the fix. You are exposed if the ZeroMQ transport (port 5555 by default) binds a routable address, or if you started from our example Kubernetes deployment. Before you upgrade: move workers and servers to 0.5.6 together. The wire format has changed. Interim steps and compromise checks are in the update. Thanks to Yuval Moravchick of JFrog Security Research. blog.lmcache.ai/en/2026/10/1โ€ฆ
1
178
Karsten Wade [email protected] @quaid.bsky.social retweeted
If you could ask the LMCache maintainers for one thing, what would it be? ๐Ÿ’ฌ A feature, a fix, a docs page, or a config example. Leave it in the replies. It is also a question on the registration form for our hands-on workshop, where you can ask in person: Thursday 22 October, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. Free. ๐Ÿค na2.hubs.ly/H08k5mp0
1
2
231
Today I learned: Where a facial wash product for women "gently cleanses and moisturizes ..." A face wash for men "cleans thoroughly and fights dryness ..." ๐Ÿซฏ๐Ÿšฑ
40
Karsten Wade [email protected] @quaid.bsky.social retweeted
Day 1 of The AI Conference SF is in the books! ๐ŸŽ‰ Come say hi to the team at Booth #214. We're excited to show how Tensormesh Platform can speed up AI inference and cut GPU costs with intelligent KV caching. See you on Day 2! #TheAIConference #AIInfrastructure #LLMInference #KVCache
1
4
8
189
If you are at this event, drop by and say howdy. I'll be representing @lmcache and chilling at the Tensormesh booth. Come ask your questions and tell us your thoughts on this new AI-native data layer.
SEE YOU THERE! We're at Booth #214 at T๐™๐™๐™š ๐˜ผ๐™„ ๐˜พ๐™ค๐™ฃ๐™›๐™š๐™ง๐™š๐™ฃ๐™˜๐™š 2026, Sept 30 to Oct 1 at Pier 48 in San Francisco. Come talk KV cache, GPU memory pressure, or anything else in your inference stack. See you there.
1
1
57
Karsten Wade [email protected] @quaid.bsky.social retweeted
What is a meetupโ€ขworkshop? Well, at this one it means a live LMCache environment is stood up before you and made ready for your own experiments. That is the plan for our hands-on meetupโ€ขworkshop in San Jose on Tuesday 6 October. You move from listening to testing: you explore live deployments, test your understanding of LMCache in the environment, and talk directly with experts and peers. ๐Ÿ› ๏ธ A laptop is recommended for the hands-on part. โฐ 1:30 to 5:30 PM Pacific time, at Samsung, 3655 N First St. Free, registration required. luma.com/cm7eip1b
1
1
3
183
Karsten Wade [email protected] @quaid.bsky.social retweeted
What changes in LLM serving when you keep the KV cache? As the PyPI summary makes clear, LMcache is "An LLM serving engine extension to reduce TTFT and increase throughput, especially under long-context scenarios." TTFT is time-to-first-token. ๐Ÿง  In San Jose on Tuesday 6 October, you can watch the same workload run with and without LMCache, and judge the difference yourself. No local install is planned. Free hands-on workshop, 1:30 to 5:30 PM Pacific time. Registration is required. โฐ luma.com/cm7eip1b
3
1
11
484
Karsten Wade [email protected] @quaid.bsky.social retweeted
One question from our workshop registration form, for anyone who runs LLM inference: What would stop you from putting LMCache into production next month? One sentence is enough. If you are new to LMCache, tell us what you would need to know first. ๐Ÿ’ฌ Reply here. The same question is on the form for our free, hands-on LMCache workshop in San Jose on Tuesday 6 October, 1:30 to 5:30 PM Pacific time. ๐ŸŽŸ๏ธ Registration is required: luma.com/cm7eip1b
1
164
Karsten Wade [email protected] @quaid.bsky.social retweeted
Which inference engine do you use, and what kind of workloads do you run? Tell us in one line each. ๐Ÿ’ฌ Brevity is our friend, and fewer details is easier to parse: the engine on the first line, the workloads on the second. We ask the same two questions on the registration form for our free, hands-on LMCache workshop in San Jose. It is on Tuesday 6 October, 1:30 to 5:30 PM Pacific time, at Samsung, 3655 N First St. Registration is required. ๐Ÿ› ๏ธ luma.com/cm7eip1b
1
1
3
320
๐Ÿง ๐Ÿคฏ ๐Ÿ˜‰
The lmcache package on PyPI has passed 1,000,000 all-time downloads, per pepy.tech (CI traffic included). It stood at 1,002,795 on 2026-09-24. LMCache is an LLM serving engine extension that reduces time-to-first-token (TTFT) and increases throughput, especially in long-context scenarios. Current release: 0.5.5. Right now LMCache is a popular way to save money and reduce resource use for GPUs. If this doesn't matter to you directly, it probably matters to your customers. On GitHub, the project has 350+ contributors and about 11.9k stars. Thank you to everyone who has sent a pull request, reported an issue, or reviewed code. โญ github.com/LMCache/LMCache
2
52
Concretely, this tweet is load-bearing.
1
33
And there we have it, a brand-new bouncing baby product, downstream and enterprise-ready!
๐˜๐จ๐ฎ๐ซ ๐ข๐ง๐Ÿ๐ž๐ซ๐ž๐ง๐œ๐ž ๐ฌ๐ญ๐š๐œ๐ค ๐ก๐š๐ฌ ๐š๐ฆ๐ง๐ž๐ฌ๐ข๐š. Before a model can answer, it has to read your prompt, and that reading is most of the GPU work in the request. Almost every setup throws the result away the moment it sends the response, so the next request reads it all over again. Same system prompt, same retrieved documents, same conversation history. That discarded work is the KV cache. Today we're launching ๐“๐ž๐ง๐ฌ๐จ๐ซ๐ฆ๐ž๐ฌ๐ก ๐๐ฅ๐š๐ญ๐Ÿ๐จ๐ซ๐ฆ, a KV cache that remembers. It tiers cached context across GPU memory, host RAM, and storage you already run, so when memory fills up or a pod restarts, state moves down a tier instead of vanishing. ย  From the team behind @LMCache, the open-source KV caching engine, now past 10,000 stars on GitHub. ย Runs on your own GPUs, in your own environment. Tensormesh Platform serves vLLM on Kubernetes, with more engines and storage backends on the way. Full write-up on the ๐›๐ฅ๐จ๐ :๐Ÿ‘‡ na2.hubs.ly/H07PyVf0
74
Karsten Wade [email protected] @quaid.bsky.social retweeted
It's 3am on a holiday weekend. GPUs look healthy. Benchmarks say you're fine. Latency is climbing anyway. We talked to Qian Cao, Founding Engineer @Tensormesh, about why benchmarks don't predict production AI performance โ€” and what actually breaks at hyperscale. tensormesh.ai/blog-posts/buiโ€ฆ
1
3
120
Karsten Wade [email protected] @quaid.bsky.social retweeted
๐Ÿš€ ๐—Ÿ๐— ๐—–๐—ฎ๐—ฐ๐—ต๐—ฒ ๐˜ƒ๐Ÿฌ.๐Ÿฑ.๐Ÿฏ ๐—ถ๐˜€ ๐—ผ๐˜‚๐˜! โœจ New model support โ€ข Kimi K3 (day-0, incl. MLA-layer KV shapes) โ€ข MiniMax M3 and other Mamba models fixed for vLLM 0.26 ๐Ÿ—‚๏ธ Fleet-wide cache directory (MP mode) โ€ข MP servers stream cache events into a shared key + token directory, with a query API, quota accounting, and global L1 token matching for CacheBlend ๐Ÿ”ง Highlights โ€ข Hot-plug Device-DAX L1 devices at runtime โ€” no restart โ€” plus batched DAX L2 loads โ€ข Faster CacheBlend retrieve: native plan executor with kernel fusion, new ๐™ฝ๐™ป_๐š‡_๐™ฝ๐™ฑ_๐™ฑ๐š‚๐š…_๐™ฑ๐š‚๐š‚ layout โ€ข AES-GCM encryption for KV cache at rest, configurable from the operator โ€ข Multimodal support in MP mode (no encoder cache yet) and non-blocking ๐šœ๐š๐š˜๐š›๐šŽ_๐š”๐šŸ_๐šŠ๐šœ๐šข๐š—๐šŒ for SGLang โ€ข Prebuilt ROCm wheels (gfx942/gfx950) and NVMe FDP placement for RawBlock ๐—ง๐—ต๐—ฎ๐—ป๐—ธ๐˜€ ๐˜๐—ผ ๐—ผ๐˜‚๐—ฟ ๐Ÿญ๐Ÿฑ ๐—ป๐—ฒ๐˜„ ๐—ฐ๐—ผ๐—ป๐˜๐—ฟ๐—ถ๐—ฏ๐˜‚๐˜๐—ผ๐—ฟ๐˜€! ๐Ÿ™Œ Full changelog: github.com/LMCache/LMCache/cโ€ฆ #LMCache #LLMInference #OpenSource #KVCache #ArtificialIntelligence
๐Ÿค– Made with AI
4
11
491
Been using #2FA on this site for a while, lately for some reason it keeps being the same code: '696 969' Before that it was '158 008' for a long time and the app kept suggesting that I turn my device upside-down while entering it ...๐Ÿค”
70
Karsten Wade [email protected] @quaid.bsky.social retweeted
We're excited to see Tensormesh featured in @BusinessInsider discussing one of enterprise AI's biggest challenges: reducing inference costs while maximizing ROI. As our CEO & Co-Founder, @JunchenJiang, notes: "KV cache never appears in people's token bills, but it certainly affects their costs." At Tensormesh, we're helping organizations serve more AI workloads on fewer GPUs through intelligent KV cache and model weight management. Read more: businessinsider.com/ai-cost-โ€ฆ #AI #GenAI #LLM #AIInfrastructure #Inference #GPU #EnterpriseAI
3
4
139