Toronto, Ontario
Joined September 2020
- Tweets973
- Following385
- Followers3.2K
- Likes2.8K
I will say, I鈥檓 in love with pre-training optimizers. The fact that you can only use first order derivative info, that your implementation must be distributed-friendly, that it must maintain sample efficiency under scaling
It produces so many beautiful ideas
More likely to have learnt it from OS Kashmiri benchmarks, there鈥檚 many collections of Kashmiri text corpora on hugging face which I鈥檓 p sure their crawlers scrape to include as training data
you obv can, just go out there and make your name, improve on some OS baselines for niche tasks, implement something in a more efficient way, etc etc and put it out for ppl to see and you'll get reachouts yourself
If you order a waymo from Scarbs into downtown Toronto when it鈥檚 snowing and there鈥檚 salt on the road you get banned by anthropic for AI abuse btw
We鈥檙e bringing the magic of Waymo to Toronto! 馃嚚馃嚘
In the coming days, you鈥檒l begin seeing Waymo鈥檚 all-electric vehicles driving through Toronto鈥檚 streets as we lay the groundwork for our ride-hailing service.
Our goal is to begin serving riders in 2028.
Sign up for future updates: waymo.com/waymo-in-canada
but studying math for the sake of it was the most fun thing ever (but i do wish i picked up more pytorch internals, i had to learn it on the job instead which was hard)
the effect of batch size, LR schedules over variable # of steps, and the right LR multipliers for scaling are still very open problems
