Research Scientist at Meta Superintelligence Labs | ex-@BellLabs
London, England
Joined August 2017
- Tweets178
- Following144
- Followers501
- Likes407
Akhil Mathur retweeted
📣 Our recent work at Meta: Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks
Rubric-based judging and reward modeling are powerful for open-ended tasks, but scaling requires controlled generation, not just free-form lists from an LLM.
Proud of this work from @LisaAlazraki's internship at Meta Superintelligence Labs. We take a systems-level view of agentic AI and show that capabilities emerge from coordination between agents, not just parametric scaling. #agents #msl #meta
It’s here! Meet Llama 3, our latest generation of models that is setting a new standard for state-of-the art performance and efficiency for openly available LLMs.
Key highlights
• 8B and 70B parameter openly available pre-trained and fine-tuned models.
• Trained on more than 15T tokens, 7x+ larger than Llama 2's dataset!
• Improved tokenizer with vocabulary of 128K tokens for better performance.
• State-of-the-art performance across industry benchmarks.
• New capabilities, including enhanced reasoning and coding.
• 3x more efficient training than Llama 2.
• New trust and safety tools with Llama Guard 2, Code Shield, and CyberSec Eval 2.
• Integrated into Meta AI, and available in more countries across our apps.
• And, just the beginning with more models and new capabilities coming soon!
Visit the Llama 3 website to read more and download the models. llama.meta.com/llama3
Akhil Mathur retweeted
1/5 Tired of self-supervised models that can't adapt to new data?
In our #WACV2024 paper, we propose Kaizen - an end-to-end approach for continual learning that performs knowledge distillation across both the pre-training and fine-tuning steps.
👉 arxiv.org/abs/2303.17235
Akhil Mathur retweeted
🤿 What is latent masking and why is it relevant to multimodal learning?
In our paper in ML4MHD at #ICML2023 we presented CroSSL, a new model that masks intermediate embeddings to improve multimodal learning.
📖 Paper: arxiv.org/abs/2307.16847
🎥 Video: drive.google.com/file/d/1lF5…
Akhil Mathur retweeted
This is huge: Llama-v2 is open source, with a license that authorizes commercial use!
This is going to change the landscape of the LLM market.
Llama-v2 is available on Microsoft Azure and will be available on AWS, Hugging Face and other providers
Pretrained and fine-tuned models are available with 7B, 13B and 70B parameters.
Llama-2 website: ai.meta.com/llama/
Llama-2 paper: ai.meta.com/research/publica…
A number of personalities from industry and academia have endorsed our open source approach: about.fb.com/news/2023/07/ll…
So happy to see this important work published. Model fairness should be a key consideration when we optimize machine learning pipelines for edge devices. Our paper offers a framework to approach on-device ML fairness, along with extensive experimental findings for speech models.
You’re unable to view this Post because this account owner limits who can view their Posts. Learn more
Great progress towards efficient and adaptive FL on ultra-constrained devices. Work led by @VincentMo6 during his internship at @BellLabs.
Federated Learning for constrained edge devices need to become more efficient and adaptive. In this preprint, we offer Centaur: a framework that adaptively partitions FL training across multiple clients owned by each user.
PDF: arxiv.org/abs/2211.04175
1/4
Akhil Mathur retweeted
🥁🥁 Join us for ACM SIGCHI Symposium for 𝐇𝐂𝐈 𝐚𝐧𝐝 𝐅𝐫𝐢𝐞𝐧𝐝𝐬 at IIT Bombay, India, 9-11 December 2022!🧵
Speakers: namastehci.in/#speakers
November 9th: Applications due
namastehci.in/#registration
🙏
Anupriya, @akhilmathurs, Anirudha Joshi, @Pushpendra_S__, @nehakumar
All train services between London and Cambridge are delayed because there is a swan 🦢 sitting on the track 🤷🏻♂️🤦♂️
Charlie Dean was outside the crease for more than 85% of all balls she started at the non-striker's end.
"Spirit of cricket!" 🙄
Shohreh Deldari (@ShohrehDeldari) from RMIT is now presenting her super cool work on cross modal self supervised learning. #UbiComp2022
Hyunsung Cho (@hciresearcher) is presenting her work FLAME 🔥 which shows how to make federated learning work in multi-device environments #UbiComp2022
Brilliant talk today by @IanTangC on our collaborative self-supervised work aimed at making HAR training data-efficient. #UbiComp2022 @ubicomp
In our upcoming paper at #IMWUT @ubicomp, we present ColloSSL: a technique for collaborative self-supervised learning among a group of devices. 🧵 (1/n) arxiv.org/abs/2202.00758
"At the stroke of the midnight hour, when the world sleeps, India will awake to life and freedom.
Happy 75th Independence Day! The journey has just begun #IndiaAt75
Looking forward to presenting our work and learning more about ML efficiency @ Deep Learning Indaba. Kudos to the incredible organizing team for putting this event together.
The exciting part is the incredible in-person talks and lineups we have planned in Tunis. Including:
- @KateKallot, Head of emerging areas at NVIDIA
- @akhilmathurs, Principal Research Scientist at Bell Labs
- @zngu, Associate Professor, University of Edinburgh
If you are attending #ICML2022, please join our spotlight talk on self-supervised FL on July 19 and poster presentation on July 21. Please feel free to DM if you are up for a chat on FL, self-supervision, and embedded ML.
icml.cc/virtual/2022/spotlig…
Glad to share that our work FLAME has been accepted to IMWUT '22. We explore algorithmic & system challenges for federated learning in the presence of multiple data-generating devices on a user. Proud of our intern @hciresearcher who led this work. arxiv.org/abs/2202.08922 @raswak
An excellent summary of our upcoming ICML paper on unsupervised FL. The paper is out on arXiv arxiv.org/abs/2205.11506. Also, a big thank you to the Flower framework flower.dev/ which helped us scale our FL experiments.
Hello, world! We present to you 🎶Orchestra🎶, an unsupervised framework for federated learning! The paper was recently accepted at #ICML.
Abs: arxiv.org/abs/2205.11506
