Engineer
Tiny red dot in apac
Joined May 2011
- Tweets825
- Following41
- Followers24
- Likes38
Tsai Li Ming retweeted
vLLM Conference is next week, and we have a packed schedule 🎊 📅. Here's the full list of events you should know:
Mon:
🔷 4–6PM vLLM × Ray × Google Cloud Happy Hour: rsvp.withgoogle.com/events/r…
🔷 6–9PM vLLM × Dynamo Meetup: luma.com/r8o604o0
Tue:
🔷 11–11:30AM vLLM Keynote from @simon_mo_
🔷 12–5PM vLLM Track Day 1
🔷 6–9PM vLLM × AMD Happy Hour: luma.com/6nb8coax
Wed:
🔷 12–5PM vLLM Track Day 2
🔷 6–8:30PM vLLM x DigitalOcean × NVIDIA Happy Hour: luma.com/fufg0bg5
No ticket needed for the happy hours and meetups, but space is limited. To join the full event, register here: vllm.ai/events/vllm-conferen…
Tsai Li Ming retweeted
Define your #AI evaluation strategy with precision. EvalHub collections let you set benchmarks, weights, and thresholds that match your deployment context, not generic leaderboard scores.
🔨 Build measurement criteria that answer your actual questions.
red.ht/4vtUV5k
Tsai Li Ming retweeted
Speculators v0.5.0 ships DFlash training support. The Red Hat AI team trained a Gemma 4 31B speculator with it. It beats Eagle 3 and a standalone FP8 quantized verifier on inter-token latency. Combine DFlash with FP8 and the gains stack.
Serve it:
vllm serve -tp 2 RedHatAI/gemma-4-31B-it-speculator.dflash
developers.redhat.com/articl…
Tsai Li Ming retweeted
This one has been in the works for a while. @cedricclyburn teaching LLM inference, compression, and benchmarking with @vllm_project -- free course with @DeepLearningAI. Proud of this one.
New short course: Fast & Efficient LLM Inference with vLLM, built in partnership with @RedHat and taught by @cedricclyburn.
Learn to quantize an open-source LLM, serve it with vLLM, and benchmark your deployment across speed, cost, and accuracy.
Free to enroll: hubs.la/Q04jXfpR0
Tsai Li Ming retweeted
The biggest @vllm_project meetup yet is starting soon in Beijing! Looking forward to meeting the many talented speakers and developer community here 🚀
Tsai Li Ming retweeted
Happening by popular demand! Whether you’re new to vLLM or a long-time contributor, join us next Thursday, October 23, for a vLLM Office Hours a session focused on helping you build, contribute, and collaborate with the community. Get a calendar invite: red.ht/office-hours
Tsai Li Ming retweeted
How do you make LLMs route queries to the right models or tools, efficiently and with low latency?
Join vLLM Office Hours on Oct. 9 to explore the new Semantic Router (github.com/vllm-project/sema…), a lightweight, high-performance router for @vllm_project.
red.ht/office-hours
Tsai Li Ming retweeted
What's next for Red Hat AI?
Join our Red Hat AI leaders @joefern1, @KatarkiTushar, Steven Huels, & Jeff DeMoss for a Q4 sneak peek into the latest releases and key new features that help enterprises address their biggest challenges.
Sign up: redhat.com/en/events/webinar…
Tsai Li Ming retweeted
Big congrats to Red Hat’s newest @vllm_project committer, Nicolo Lucchesi (@nick_lucche)! 🎉
Nicolo already made major contributions:
⚡️ Optimized disaggregated serving with KV transfer + Nixl for faster heterogeneous tensor-parallel setups.
🗣️ Designed & implemented vLLM’s speech-to-text interface, powering v1/audio/transcriptions and v1/audio/translations.
🤖 Advanced TPU integration with optimized sampling, efficient compilation, and improved preprocessing.
Now he’s leading the charge on "Disaggregated Everything" - building a tokens-in, tokens-out interface to unlock the next wave of vLLM performance. 🚀
Congrats, Nicolo!
Tsai Li Ming retweeted
v0.10.2 marks the first release with official aarch64 support for vLLM! You can now install vLLM directly onto @nvidia 's GB200. Along with the PyPI release, our docker image is also multi-platform so pulling the right image just works.
More perf enhancements on the way!
Tsai Li Ming retweeted
🚀 Thrilled to announce GuideLLM v0.3.0!
This release is highlighted by a brand new Web UI, containerized benchmarking, and powerful dataset preprocessing.
GuideLLM GitHub: github.com/vllm-project/guid…
(Thread 👇)
ALT GuideLLM metrics details: Dig into percentile breakdowns of latency & throughput vs. RPS
Tsai Li Ming retweeted
Qwen3-Next dropped yesterday and you can run it with Red Hat AI today.
✅ Day-zero support in vLLM
✅ Day-one deployment with Red Hat AI
Step-by-step guide: developers.redhat.com/articl…
The future of AI is open.
Tsai Li Ming retweeted
On August 23, we are co-hosting the Shanghai vLLM meetup with our friends at MetaX, so if you are in town and want to learn more about how the community is leveraging vLLM, then sign up and join in! @vllm_project lu.ma/3bex3de0
Tsai Li Ming retweeted
Happy 4th July!
Speed is the Moat & @AnushElangovan & his team Keeps Running Faster & Faster. Still lots of areas where ROCm has gaps but many are already closing
Tsai Li Ming retweeted
Join us: IMDA x Red Hat Technical Sharing Session!
🗓️ Thursday, 19 June 2025, 2.30pm @ Red Hat SG Office
⚡High-performance inference with vLLM
⚙️ Deploying LLMs in resource-limited environments
🌐 Scalable GenAI across hybrid cloud with llm-d
Register: red.ht/4jOhVFR
Tsai Li Ming retweeted
Cool to see @vllm_project used as part of @WhatsApp Trusted Execution Environment (TEE) Private Processing!
Paper here: ai.meta.com/static-resource/…
Tsai Li Ming retweeted
Thank you @AMD @LisaSu @AnushElangovan for Advancing AI together with @vllm_project! We look forward to the continued partnership and pushing the boundary of inference.
Tsai Li Ming retweeted
🇯🇵 Join us for an in-person @vllm_project meetup on Monday, June 16 in Tokyo. Or tune in via live stream!
Agenda:
-Intro to vLLM
-Japanese LLM adoption
-Model optimization w/ LLM Compressor
-Distributed inference w/ llm-d
-Q&A and lightning talks
RSVP: ossbyredhat.connpass.com/eve…
ALT vLLM meetup in Tokyo on June 16, 2025