Yanping Huang retweeted
✨RL magic is in the air! Introducing DeepScaleR-1.5B-Preview—a fully open-source, 1.5B-parameter model trained with RL to surpass o1-preview for general math reasoning.
📜Blog: pretty-radio-b75.notion.site…
💻Github: github.com/agentica-project/…
Yanping Huang retweeted
Minimax new txt2video tool is great. I got quality video results. I went into their website (hailuoai.com/video - you can use your country phone to log in- not as Kling in the beginning that only allowed chinese numbers) and i translated their document with usage recommendations and prompts, and made some videos with the provided prompts. Have fun!
Yanping Huang retweeted
[1/5] Introducing Stylus 🖌️ - an #AI tool that automatically finds and adds the best adapters (LoRAs, Textual Inversions, Hypernetworks) to #StableDiffusion based on your prompt.
🗞️ Paper: arxiv.org/abs/2404.18928
🌎 Project Page: stylus-diffusion.github.io
Powered by GPipe on Lingvo!
How many languages can we support with Machine Translation? We train a translation model on 1000+ languages, using it to launch 24 new languages on Google Translate without any parallel data for these languages.arxiv.org/abs/2205.03983 Technical 🧵below: 1/18
Yanping Huang retweeted
Alpa is a framework that uses just one line of code to easily automate the complex model parallelism process for large #DeepLearning models. Learn more and check out the code. goo.gle/3y7xZ1f
Yanping Huang retweeted
Proud to release our last year of work on sparse expert models!
This started over a year ago when we found Switch Transformers pre-trained well, but some variants were unstable or fine-tuned poorly. The new SOTA ST-MoE-32B addresses this.
arxiv.org/abs/2202.08906
Yanping Huang retweeted
Open-domain dialog—where a model converses about any topic—is a key challenge for language models. Learn about LaMDA, a project to build dialog models that are more safe, grounded, high quality, and in line with our Responsible AI Principles ↓ goo.gle/3KJ2oXJ
Yanping Huang retweeted
MoE models can be a future for large language models, but having to distill into small models can be painful. Our work, task-level MoE for extracting subnetworks, is a simple idea to bypass distillation. And we're still at the very beginning :)
Blog: ai.googleblog.com/2022/01/le…
Read all about Task-level Mixture-of-Experts (TaskMoE), a promising step towards efficiently training and deploying large models, with no loss in quality and with significantly reduced inference latency ↓ goo.gle/3I5ulXj
Yanping Huang retweeted
We wrote a blogpost about our work on Task-level Mixture-of-Experts (TaskMoE), and why they're a great way to efficiently serve large models (vs more common approaches like training-> compression via distillation).
Read all about Task-level Mixture-of-Experts (TaskMoE), a promising step towards efficiently training and deploying large models, with no loss in quality and with significantly reduced inference latency ↓ goo.gle/3I5ulXj
Yanping Huang retweeted
Adding my two cents as a grad student — I genuinely think people here are very supportive & diversity is an active topic here, & Pedro D’s behaviors have not gone unnoticed. Before he retired, most faculty members used to openly disagree and challenge him on multiple issues, 🧵
Not gonna amplify that other tweet but I’ll instead point to this one. Speaking as a relatively newish member, I’ll say that I’ve found @uwcse to be a supportive place for women, w/ a strong commitment to diversity and inclusion from our leadership.
Yanping Huang retweeted
This is not how @uwcse — or any of us — imagined kicking off winter quarter, but we want to amplify this statement calling out the misogyny of an emeritus (retired) faculty member.
Today is the start of UW’s Winter Quarter, and #UWAllen is excited to welcome our students back. Without the benefit of a long break given our quarter system, instructors spent the holiday weekend preparing for the challenge of conducting the 1st week remotely due to omicron. 1/5
We share the recipes for training the 1.2T parameters GLaM MoE model on Arxiv today!
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
1.2T weight model that has better average zero and one-shot results than GPT-3 while using only ⅓ of compute for training.
Arxiv: arxiv.org/abs/2112.06905
Blog: goo.gle/3dBNLWQ
The latest GLaM model is also powered by GSPMD! ai.googleblog.com/2021/12/mo…
Our work shows that even a basic version of MoE can work extremely well for generative tasks at scale, and hence should be used as default for future scaling!
Today we introduce the Generalist Language Model (GLaM), a sparsely activated model that achieves better overall performance on 29 few-shot #NaturalLanguageProcessing benchmark tasks with just a third of the training energy cost. Learn more on the blog ↓ goo.gle/3dBNLWQ
We wrote a blogpost to unveil Google's model parallelism engine that allows the efficient training of large scale models like GShard-M4, LaMDA, and BigSSL.
Today we present an open-source system to scale neural networks — often critical for improving model performance — by automatically parallelizing the model across devices, which enables researchers to more efficiently build and train large-scale models. goo.gle/3EEwloj
Google demonstrates 63% computational efficiency, cutting edge in the industry, when training 480B parameter models on 2048 Cloud TPU-V4.
cloud.google.com/blog/topics… Link to model code: github.com/tensorflow/lingvo…
Yanping Huang retweeted
#EMNLP2021 Findings paper “Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference” w/ @bignamehyp, @ankurbpn, Maxim Krikun, @lepikhin, @lmthang, @orf_bnw about TaskMoE, an inference friendly alternative to token-based MoEs. Link: arxiv.org/abs/2110.03742 1/n
Yanping Huang retweeted
I am pleased to announce that the camera ready version of my new textbook, "Probabilistic Machine Learning: An Introduction", is finally available from probml.ai. Hardcopies will be available from MIT Press in Feb 2022.
Yanping Huang retweeted
This is unacceptable.
We now need to know more about @RobinhoodApp’s decision to block retail investors from purchasing stock while hedge funds are freely able to trade the stock as they see fit.
As a member of the Financial Services Cmte, I’d support a hearing if necessary.
More than half of all Robinhood users own at least some GameStop stock.
They are now unable to freely trade it; the app is only allowing users to close out their positions. vice.com/en/article/m7ak7y/r…
