Technical Fellow, Graphcore. Love beautiful code, and beautiful hardware to run it on.
Cambridge
Joined July 2009
- Tweets1.5K
- Following849
- Followers3.9K
- Likes1.6K
Pinned Tweet
Hey, I wrote a blog - I think it's fun, hope you do too!
A tiny implementation detail in low-precision arithmetic could be biasing your AI training 😲
This interactive deep dive from Graphcore Research's @Awfidius uncovers a subtle failure mode in stochastic rounding that only appears when randomness is limited, and how it can be addressed with one simple fix 🎲
Check it out in the link below! 👇
Andrew Fitzgibbon retweeted
A tiny implementation detail in low-precision arithmetic could be biasing your AI training 😲
This interactive deep dive from Graphcore Research's @Awfidius uncovers a subtle failure mode in stochastic rounding that only appears when randomness is limited, and how it can be addressed with one simple fix 🎲
Check it out in the link below! 👇
Andrew Fitzgibbon retweeted
🚨 Graphcore is hiring AI Research Interns! 🚨
Join us to work at the intersection of hardware and AI and help shape the future of AI systems. Whether you're excited about efficient inference, large-scale training, or advancing frontier-model capabilities, we’ve got cutting-edge projects for you to dive into.
Interested? Apply below 👇
Andrew Fitzgibbon retweeted
Our Papers of the Month for September is now live! We cover:
- LLM self-correction via RL
- Trillion-token FP8 training
- SOAP (Shampoo + Adam)
- Generative models for crystals
All framed in terms of "proper conditioning" (🧵)
graphcore-research.github.io…
Andrew Fitzgibbon retweeted
Graphcore Research internships are now open 🎉
We're looking for PhD students for next summer
We're interested in algorithms & tools for hardware-efficient ML, in areas like LLM training/inference, GNNs, knowledge graphs and frameworks
Spread the word! graphcore.ai/jobs
Andrew Fitzgibbon retweeted
Introducing `tandv` - a library for tracking and visualising the internal stats of your model. We hope this will help with low-precision, debugging and more.
(link in 🧵)
Andrew Fitzgibbon retweeted
Sure matplotlib is cool, but what if I want to load my loss curves into the 2006 hit Flash game LineRider?
Andrew Fitzgibbon retweeted
Our u-µP paper hit arXiv this morning! I'm so proud of this one — and grateful for a wonderful team who put so much into it 🥰
We add lots of good things to µP. Better sweeping, transfer, simple FP8. Already @cloneofsimo has a great thread on it, which I highly recommend
babe wake up, new muP paper dropped
arxiv.org/abs/2407.17465
And holy smokes does this look promising!
Andrew Fitzgibbon retweeted
Excited to present our work at @icmlconf WANT workshop: ⚖️ Scalify: scale propagation for efficient low-precision LLM training 🎉
github.com/graphcore-researc…
Andrew Fitzgibbon retweeted
babe wake up, new muP paper dropped
arxiv.org/abs/2407.17465
And holy smokes does this look promising!
As further feedback to the team that thought this would be a great innovation that would attract customers: I bought this tap because of Grohe's reputation for quality. The term "quickfix" had zero impact on my choice.
However, that term now means "low quality gimmick".
Same problem, and based on online searches, many others have it too.
Terrible wasteful design, saves 10 seconds when it works, costs hours when it doesn't, and a load of extra useless plastic is left in place having saved those 10 seconds.
This Post is from an account that no longer exists. Learn more
Andrew Fitzgibbon retweeted
Our team's summaries & analysis of our favourite papers from the last month.
We give our take on: Mamba-2, sparse-µP, contextual position encoding & matmul-free models 🧵
graphcore-research.github.io…
Andrew Fitzgibbon retweeted
Our latest edition of *Papers of the Month* is now available 📚
These are summaries of our team's favourite papers from March, including a new low-rank training procedure GaLore, and the supposed "Era of 1-bit LLMs" (really 1.58 bits)
Mini-version in 🧵
graphcore-research.github.io…
I'm moving from extremely rarely posting interesting content on twitter to doing the same on, wow I'm old, linkedin...
Andrew Fitzgibbon retweeted
So pleased to share that our OGB-LSC winning model GPS++ has just been published in TMLR.
GPS++: Reviving the Art of Message Passing for Molecular Property Prediction
Dominic Masters, Josef Dean, Kerstin Klaeser et al..
Action editor: Ole Winther.
openreview.net/forum?id=moVE…
#molecular #pcqm4mv2 #graph
I love that I'm being offered these - I guess the ad targeting knows I'm going to have a sudden urge to take up jewellery making long before I do...
I’m biased obvs, but having seen a preview of this talk, I can recommend it highly!
I'll start with "Julia meets the Intelligence Processing Unit" (pretalx.com/juliacon2023/tal…), about running #JuliaLang on the @graphcoreai IPU, spectacular example of how @llvmorg enables adopting new cool hardware in Julia, and how Julia opens up new doors in scientific computing