CS @ CMU | Prev. Modal, YC X25
Joined May 2025
- Tweets96
- Following306
- Followers497
- Likes1K
The simulator first runs AccelSim for the target NVIDIA architecture on the kernel trace. Using the nsys-profiled trace, the relative kernel timings are replicated for the new estimates. On a small 125M param transformer, we found a relative error of <4% for end-to-end latency.
1
77
What’s next? Our ambition for this hackathon was to simulate the inference engine’s scheduling and batching dynamics on a group of simulators working as a cluster. Perhaps we would model this behavior “functionally” with modular components from Resin.
cascadesim.vercel.app
1
66
I wrote a blog post about what I did on my gap year. It felt like a crazy time working across startups. My hope is to provide a perspective for those taking time off from school or those interested in a sample from the experiences of young people in tech.
atoniolo76.github.io/writing…
Today was my last day at Modal.
I had an amazing 6 months on the training team, working on sandbox infra (RDMA) and on-policy distillation methods.
Currently feeling quite ambivalent about it: will be missing the people here but also excited to start college at CMU this fall!
This week I proxied InfiniBand uverb devices for gVisor’s runsc runtime. Most of the challenge was configuring a dev environment for running ibperf/all reduce tests on two H200 nodes. Not bad for a systrap platform!
1
7
534
Gotoni added SpiceDB's ReRBAC for humans & agents.
Enforcing permissions via a centralized store is powerful for two reasons:
1. Enforces consistent semantics across UX
2. Allows interoperability between applications i.e. re-using an SSH key
To your agents: Go rip on some GPUs!
2
8
575
Also recommend this read if you're interested in how Google designed an auth database that can scale to trillions of ACL requests per second: usenix.org/system/files/atc1…
1
235
How do you connect to Modal containers over SSH?
I spoke with a couple ML researchers over the past week and took a look at their awkward Modal setups.
My python package, gotoni, lets you open Modal containers in Cursor.
Credits to @marmikch for the original SSH setup!
How low can TTFT go?
We evaluated existing LLM load balancers and found that networking performance is a critical bottleneck.
Our routing policy, GORGO, improves TTFT by 5% through maximizing KV-cache reuse and minimizing delay.
Work done w/ @abinayaaaa and @foundinrome @ ART.
We aimed to make GORGO as reproducible as possible. In our repo, you will find support for running load balancing policies across @modal , @LambdaAPI , and @orgo .
Just wrapped up the first week of my internship @modal!
I was thoroughly impressed by the breadth of engineers on the team and have already learned so much from cloud resource solving to developer ergonomics.
Stay tuned for big updates from Modal’s Training team! RDMA go brr.
3
27
4,197
![[Kernel] Fallback to global memory in moe_lora_align_sum_kernel when … · atoniolo76/vllm@c94263e](https://pbs.twimg.com/card_img/2120359819575988224/fIUUYUZV?format=jpg&name=800x419)