@lmcache

🧪 Open Source Team that maintains LMCache and Production Stack 🤖 Democratizing AI by providing efficient LLM serving for ALL

 Github, Online
lmcache.ai/
Joined September 2024
  • Tweets302
  • Following56
  • Followers2K
146Media
What is the hardest KV cache question in your stack right now? 🧠 Bring it to the LMCache booth at PyTorch Conference, today or tomorrow, at the San Jose Convention Center. If it needs more time, bring it to our hands-on workshop on Thursday, and test your ideas against a live LMCache environment. Thursday 22 October, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. Free. 📍 na2.hubs.ly/H08k5_-0
Dark card, at the LMCache booth: What is your hardest KV cache question? Bring it to the booth at PyTorch Conference, or to the workshop. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

ALT Dark card, at the LMCache booth: What is your hardest KV cache question? Bring it to the booth at PyTorch Conference, or to the workshop. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

1
2
169
Here's a question from our ecosystem of LMCache integrators: When you run several vLLM processes, does each one keep LMCache inside its own process, or can all of them share one LMCache service? Here's one way: In multi-process mode, you start lmcache server with CPU memory as L1 and FlashBlade over GPUDirect Storage (GDS) as L2. Each vLLM process connects to the shared server through LMCacheMPConnector. Josh Lay and Jean-Baptiste Thomas published the full server and vLLM configuration, pinned to lmcache==0.5.3 and vllm==0.24.0, on the Everpure blog: na2.hubs.ly/H08nWCR0 #LMCache #KVCache

How to Integrate LMCache with FlashBlade for Faster LLM Inference

Explore how LMCache and Everpure FlashBlade can enable persistent KV caching to help reduce time to first token and increase LLM inference throughput.

blog.everpuredata.com
1
3
210
PyTorch Conference day 1 starts this morning at the San Jose Convention Center, and LMCache has a booth. 📍 Find us to talk about KV cache management, and register at the booth for our hands-on workshop. The workshop is in 2 days: Thursday 22 October, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. Free. ⏰ What should we show you at the booth? 💬 na2.hubs.ly/H08k5_K0
Large orange 2, days to go. PyTorch Conference, day 1: We are at the LMCache booth, San Jose Convention Center. Sign up there for the workshop, Thu 22 Oct, San Jose. luma.com/cm7eip1b

ALT Large orange 2, days to go. PyTorch Conference, day 1: We are at the LMCache booth, San Jose Convention Center. Sign up there for the workshop, Thu 22 Oct, San Jose. luma.com/cm7eip1b

109
Tag one teammate who should spend Thursday afternoon with LMCache. 🔁 22 October, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose: a live LMCache environment, a hands-on deep dive with mini-lmcache, engineering sessions, and a reception. A laptop is recommended. 🤝 It is free. Each of you registers here: na2.hubs.ly/H08k5PT0
Light card, 3 days to go, with @teammate in large type. Tag the person who should come with you. Free, registration required. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

ALT Light card, 3 days to go, with @teammate in large type. Tag the person who should come with you. Free, registration required. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

2
148
Going to PyTorch Conference this week? Find the LMCache booth at the San Jose Convention Center on 20 and 21 October. 📍 You can register at the booth for Thursday's LMCache hands-on workshop. It is the day after the conference ends: 22 October, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. Free. 🎟️ Ask us about KV cache management while you are there. Or register now: na2.hubs.ly/H08k5Pw0
At PyTorch Conference? Find the LMCache booth. Tue 20 to Wed 21 Oct, San Jose Convention Center. Sign up there for the hands-on workshop, Thu 22 Oct, Samsung, San Jose. luma.com/cm7eip1b

ALT At PyTorch Conference? Find the LMCache booth. Tue 20 to Wed 21 Oct, San Jose Convention Center. Sign up there for the hands-on workshop, Thu 22 Oct, Samsung, San Jose. luma.com/cm7eip1b

1
110
In 3 days, LMCache runs a hands-on workshop in San Jose. Is it for you? 🧠 It is for engineers and researchers who work on KV cache management and LLM efficiency. If you run LLM inference and care about time-to-first-token (TTFT) and throughput, the afternoon is built for you. Thursday 22 October, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. Free, registration required. ⏰ na2.hubs.ly/H08k5M40
Large teal 3 with the words days to go. Built for people who work on KV cache management and LLM efficiency. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

ALT Large teal 3 with the words days to go. Built for people who work on KV cache management and LLM efficiency. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

1
143
The second half of the LMCache workshop on Thursday 22 October has two Samsung speakers, one after the other. 🤝 4:00 PM: Luis Chamberlain (mcgrof) on asymmetric quantization, dma-buf with io_uring, and learned prefix caching. 4:30 PM: Dongjoo Seo, "What we should consider for KV Cache system deployment." 5:00 PM: closing remarks, then a reception until 5:30 PM. Before that: the Introduction to LMCache and a 45-minute hands-on deep dive with mini-lmcache. 🛠️ Free, registration required: na2.hubs.ly/H08rNNB0
Large orange 4, days to go. The second half, Thu 22 Oct: 4:00 Luis Chamberlain, Samsung; 4:30 Dongjoo Seo, Samsung; 5:00 closing remarks; 5:05 reception. San Jose. luma.com/cm7eip1b

ALT Large orange 4, days to go. The second half, Thu 22 Oct: 4:00 Luis Chamberlain, Samsung; 4:30 Dongjoo Seo, Samsung; 5:00 closing remarks; 5:05 reception. San Jose. luma.com/cm7eip1b

1
148
Which is harder for your stack: long context, or many users at once? 💬 Reply with one word. In 4 days you can bring that answer to a live LMCache environment, and to a room of people who run inference. The LMCache hands-on workshop is on Thursday 22 October, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. Free, registration required. ⏰ na2.hubs.ly/H08k5K50
Large orange 4, days to go. Question: Which hurts more in your stack? Answer chips: Long context, or Many users at once. Thu 22 Oct, San Jose. luma.com/cm7eip1b

ALT Large orange 4, days to go. Question: Which hurts more in your stack? Answer chips: Long context, or Many users at once. Thu 22 Oct, San Jose. luma.com/cm7eip1b

1
117
The agenda for Thursday 22 October has three blocks for people: networking at 1:30 PM, a networking break at 3:30 PM, and a reception from 5:05 PM until 5:30 PM. 🤝 Bring a problem you have not solved in your own stack. Someone at Samsung in San Jose may run a similar setup. Which inference problem would you most like to discuss with peers? 💬 The LMCache hands-on workshop is free, and refreshments are provided. 📍 Registration is required: na2.hubs.ly/H08k5JR0
The people part: 1:30 PM registration and networking, 3:30 PM networking break, 5:05 PM networking reception until 5:30. Thu 22 Oct, Samsung, San Jose. luma.com/cm7eip1b

ALT The people part: 1:30 PM registration and networking, 3:30 PM networking break, 5:05 PM networking reception until 5:30. Thu 22 Oct, Samsung, San Jose. luma.com/cm7eip1b

3
248
45 minutes, one live deployment, and your laptop: that is the deep dive Samuel Shen of Tensormesh leads at 2:45 PM on Thursday 22 October. 🛠️ First, the mechanics of a public server deployment. Then hands-on work against the mini-lmcache API endpoints to benchmark and optimize KV cache management. The test environment stays open after the session. It is part of the free LMCache workshop, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. 📍 Registration is required: na2.hubs.ly/H08p4vJ0
Deep dive with Samuel Shen, 2:45 PM: public server deployment, then hands-on against the API endpoints. Illustrated terminal. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

ALT Deep dive with Samuel Shen, 2:45 PM: public server deployment, then hands-on against the API endpoints. Illustrated terminal. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

1
112
Never run LMCache before? Start at 2:10 PM on Thursday 22 October. 🧠 Karsten Wade gives the Introduction to LMCache: what the KV cache layer does, and where it is useful. Its PyPI summary says it in one line: "A LLM serving engine extension to reduce TTFT and increase throughput, especially under long-context scenarios." TTFT means time-to-first-token. Then you try it yourself in the hands-on deep dive. 1:30 to 5:30 PM Pacific, Samsung, 3655 N First St, San Jose. Free. 📍 na2.hubs.ly/H08k5Dt0
Light card: New to LMCache? Start with the introduction: what the KV cache layer does, and where it helps. 2:10 PM, Karsten Wade. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

ALT Light card: New to LMCache? Start with the introduction: what the KV cache layer does, and where it helps. 2:10 PM, Karsten Wade. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

113
Open Source AI Week starts today in the Bay Area and runs to 25 October. 📅 If you come for PyTorch Conference at the San Jose Convention Center on 20 and 21 October, stay one more day. On Thursday 22 October, the LMCache hands-on workshop runs 1:30 to 5:30 PM Pacific at Samsung, 3655 N First St, San Jose. 📍 Four hours with a live LMCache environment, engineering sessions, and a reception. Free, registration required: na2.hubs.ly/H08k5yg0
Open Source AI Week starts today. Timeline: Fri 16 Oct, the week starts; Tue 20 to Wed 21 Oct, PyTorch Conference, San Jose Convention Center; Thu 22 Oct, LMCache workshop, Samsung, San Jose. luma.com/cm7eip1b

ALT Open Source AI Week starts today. Timeline: Fri 16 Oct, the week starts; Tue 20 to Wed 21 Oct, PyTorch Conference, San Jose Convention Center; Thu 22 Oct, LMCache workshop, Samsung, San Jose. luma.com/cm7eip1b

1
2
173
New speaker for the LMCache workshop: Luis Chamberlain (mcgrof) of Samsung takes the 4:00 PM session on Thursday 22 October. 🧠 His topics: asymmetric quantization, dma-buf support with io_uring, and learned prefix caching with Cartridges and beyond. Dongjoo Seo of Samsung speaks next, at 4:30 PM. 1:30 to 5:30 PM Pacific, Samsung, 3655 N First St, San Jose. Free. 📍 na2.hubs.ly/H08p3xz0
Speaker card: new speaker Luis Chamberlain, Samsung, mcgrof. Technical session: asymmetric quantization; dma-buf support with io_uring; learned prefix caching, Cartridges and beyond. Thu 22 Oct, 4:00 PM, Samsung, San Jose. Free. luma.com/cm7eip1b

ALT Speaker card: new speaker Luis Chamberlain, Samsung, mcgrof. Technical session: asymmetric quantization; dma-buf support with io_uring; learned prefix caching, Cartridges and beyond. Thu 22 Oct, 4:00 PM, Samsung, San Jose. Free. luma.com/cm7eip1b

1
136
After the workshop on Thursday 22 October, the mini-lmcache test environment stays open for your own experiments. 🛠️ So what would you test first? A long shared prompt? A document that many requests read again? Reply with one line. 💬 The hands-on deep dive is part of a free afternoon, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. Registration is required: na2.hubs.ly/H08k5xM0
The test environment stays open after the session. Large question: What would you test first? Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

ALT The test environment stays open after the session. Large question: What would you test first? Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

1
1
1
123
Ecosystem check-in from our community of LMCache integrators: At 128K tokens of context, time to first token went from 33.5 s with no cache to 1.9 s with LMCache storing the KV cache on FlashBlade over GPUDirect Storage (GDS). Josh Lay and Jean-Baptiste Thomas ran the test on NVIDIA L40S with openai/gpt-oss-20b, vllm==0.24.0, and lmcache==0.5.3. The baseline is a cold prefill with no cache. How do you connect vLLM, LMCache, and FlashBlade to get this result? Their post on the Everpure blog has the configs: na2.hubs.ly/H08nWlC0 #LMCache #KVCache

How to Integrate LMCache with FlashBlade for Faster LLM Inference

Explore how LMCache and Everpure FlashBlade can enable persistent KV caching to help reduce time to first token and increase LLM inference throughput.

blog.everpuredata.com
1
1
4
243
One week from today, Dongjoo Seo of Samsung speaks on deployment experience at scale at the LMCache hands-on workshop. ⏰ The session is at 4:30 PM on Thursday 22 October, at Samsung, 3655 N First St, San Jose. The full workshop runs 1:30 to 5:30 PM Pacific and includes hands-on work with a live LMCache environment. Samsung and Tensormesh sponsor the afternoon. 📍 What would you ask about running LLM inference at scale? Bring your question. Free, registration required: na2.hubs.ly/H08k5wk0
Speaker card: Dongjoo Seo, Samsung. Technical session, deployment experience at scale. Thu 22 Oct, 4:30 PM, Samsung, San Jose. Free. Sponsored by Samsung and Tensormesh. luma.com/cm7eip1b

ALT Speaker card: Dongjoo Seo, Samsung. Technical session, deployment experience at scale. Thu 22 Oct, 4:30 PM, Samsung, San Jose. Free. Sponsored by Samsung and Tensormesh. luma.com/cm7eip1b

4
138
Who on your team owns LLM inference? Send this post to them. 🔁 On Thursday 22 October, 1:30 to 5:30 PM Pacific, they can spend the afternoon at Samsung, 3655 N First St, San Jose. They work with a live LMCache environment, join a 45-minute hands-on deep dive with mini-lmcache, hear engineering sessions, and meet other people who run inference. 🤝 It is free. Registration is required: na2.hubs.ly/H08k5td0
Bring the person on your team who owns inference. Chips read You plus The person who owns inference. Free, registration required. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

ALT Bring the person on your team who owns inference. Chips read You plus The person who owns inference. Free, registration required. Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

147
Here is the full afternoon for Thursday 22 October at Samsung, San Jose. All times Pacific. 📅 1:30 PM Registration and networking, with setup help 2:00 PM Host overview, Karsten Wade (Tensormesh) 2:10 PM Introduction to LMCache, Karsten Wade 2:30 PM Break 2:45 PM Deep dive with mini-lmcache, Samuel Shen (LMCache) 3:30 PM Networking break 4:00 PM Technical session: a surprise LMCache contributor 4:30 PM Technical session: deployment experience at scale, Dongjoo Seo (Samsung) 5:00 PM Closing remarks 5:05 PM Reception, until 5:30 PM Which session would you come for? 💬 Free, registration required: na2.hubs.ly/H08k5pq0
Agenda, 8 days to go: 1:30 registration, 2:10 intro (Karsten Wade), 2:45 deep dive (Samuel Shen), 4:00 surprise LMCache contributor, 4:30 Dongjoo Seo, 5:05 reception. Thu 22 Oct, Samsung, San Jose.

ALT Agenda, 8 days to go: 1:30 registration, 2:10 intro (Karsten Wade), 2:45 deep dive (Samuel Shen), 4:00 surprise LMCache contributor, 4:30 Dongjoo Seo, 5:05 reception. Thu 22 Oct, Samsung, San Jose.

1
129
If you could ask the LMCache maintainers for one thing, what would it be? 💬 A feature, a fix, a docs page, or a config example. Leave it in the replies. It is also a question on the registration form for our hands-on workshop, where you can ask in person: Thursday 22 October, 1:30 to 5:30 PM Pacific, at Samsung, 3655 N First St, San Jose. Free. 🤝 na2.hubs.ly/H08k5mp0
Light card with orange quote marks. From the registration form: If you could ask the maintainers for one thing, what would it be? Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

ALT Light card with orange quote marks. From the registration form: If you could ask the maintainers for one thing, what would it be? Thu 22 Oct, 1:30 to 5:30 PM, Samsung, San Jose. luma.com/cm7eip1b

1
2
232
CVE-2026-105192 (CVSS 3.1 score 9.8): remote code execution in LMCache multiprocess (MP) mode. The fix is merged into the dev branch and will ship in 0.5.6, expected this week. Affected: 0.3.9 through 0.5.5, the 0.5.6 release candidates, and dev branches before the fix. You are exposed if the ZeroMQ transport (port 5555 by default) binds a routable address, or if you started from our example Kubernetes deployment. Before you upgrade: move workers and servers to 0.5.6 together. The wire format has changed. Interim steps and compromise checks are in the update. Thanks to Yuval Moravchick of JFrog Security Research. blog.lmcache.ai/en/2026/10/1…

An Update on CVE-2026-105192

Device IPC deserialization fix merged into the dev branch On October 7, 2026, JFrog Security Research publicly disclosed a deserialization remote code execution vulnerability in LMCache’s mul…

blog.lmcache.ai
1
179
Load more