Global Head of AI, Technology at Millennium. All opinions my own.
NYC
Joined March 2009
- Tweets7.8K
- Following3.4K
- Followers3.2K
- Likes7.7K
Just heard that Eugene Charniak @BrownCSDept passed away. May he live on in blessed memory.
Just heard the news about @ScottMinerd . I met him briefly, randomly, and he was surprisingly humble, kind, and generous with him time. Really a loss. My condolences.
Gideon Mann retweeted
Congrats to the fifth cohort of @Bloomberg's #DataScience Ph.D. Fellowship!
Learn more about each of our four 2022-2023 Fellows and what their research is focusing on: bloom.bg/3v8TBI0
#AI #ML #NLProc
ALT The 2022-2023 cohort of Bloomberg's Data Science Ph.D. Fellows Top (L-R): Suchin Gururangan of the University of Washington’s Paul G. Allen School of Computer Science & Engineering & Mengzhou Xia of Princeton University’s Department of Computer Science Bottom (L-R): Shauli Ravfogel of Bar-Ilan University & Wenhao Yu of the University of Notre Dame's Department of Computer Science and Engineering
Gideon Mann retweeted
Introducing task vectors!
A new way to steer models by doing arithmetic with model weights. Subtract to make models forget, add to make them learn
📜: arxiv.org/abs/2212.04089
🖥️: github.com/mlfoundations/tas…
Our beautiful, maximalist urban future awaits.
Just stumbled across these Carmelite monks who are building a gothic monastery in Wyoming, with a little help from CNC machines.
carmelitegothic.com/cnc-ston…
Gideon Mann retweeted
Last night, I wrote a tool which allows you to scrape data out of GPT models using SQL queries. Just make up a relevant database, and use your imagination of what kind of query you'd like, and it will return the data within GPT's parameters.
Results are structured. Example:
Gideon Mann retweeted
Researchers/engineers from our #AI Engineering Group & CTO Office co-authored 5 papers at @emnlpmeeting this week; learn more about their research, why the results are notable, and how their work will advance the state-of-the-art in #NLProc bloom.bg/3iNCHve
#EMNLP2022
ALT Bloomberg’s AI Engineering Group & CTO Office Publish 5 NLP Research Papers at EMNLP 2022 Bloomberg's five EMNLP 2022 research papers highlight a variety of applications, novel approaches, and improved models used in key NLP tasks.
Gideon Mann retweeted
.@CMoz's Open Universal Input Sleeve adds touch input capabilities quickly and stylishly to your projects: bit.ly/3VvS3mE
A big deal! And a sign of the immense amount of optimization left to be discovered. Obvious, but cost of training will keep dropping!
We're releasing an optimized implementation of GPT2/GPT3 with FlashAttention🚀!
This trains 3-5x faster than the Huggingface version, reaching up to 189 TFLOPs/sec per A100, 60.6% (model) FLOPs util of the theoretical maximum. 1/6
github.com/HazyResearch/flas…
Gideon Mann retweeted
OpenAI’s ChatGPT is susceptible to prompt injection — say the magic words, “Ignore previous directions”, and it will happily divulge to you OpenAI’s proprietary prompt:
Super cool! Extends @megamor2 @guy__dar work in directly interpreting weights and gives cleaner results.
You can learn more about transformer weight matrices by performing SVD on them than you can by naively looking at their rows and columns
lesswrong.com/posts/mkbGjzxD…
Gideon Mann retweeted
You can learn more about transformer weight matrices by performing SVD on them than you can by naively looking at their rows and columns
lesswrong.com/posts/mkbGjzxD…
Performance numbers look strong! Very curious to try it out more thoroughly.
Introducing GPT-JT, a 6B parameter open source model that can outperform many 100B+ parameter models and was trained over slow (1Gbps) internet links.
together.xyz/research/releas…
Gideon Mann retweeted
I've been looking around at hardware options for training/deploying ML models, and became interested in the differences between academia and industry 🎓🏢
Here's a poll: where do you train your ML models? (RTs appreciated) 1/5
Gideon Mann retweeted
If you last checked in on AI image makers a month ago & thought “that is a fun toy, but is far from useful…” Well, in just the last week or so two of the major AI systems updated.
You can now generate a solid image in one try. For example, “otter on a plane using wifi” 1st try:
Really enjoying this paper:
arxiv.org/pdf/2211.08411.pdf
@kandpal_nikhil @colinraffel
For a particular fact (e.g. an answer to a factoid question), having more "relevant" documents in the training data (which do not necessarily have the answer in them) makes fact recall easier.
This feels surprisingly reminiscent of human memory. I've always believed that having more general information about a topic, makes it easier to remember a specific fact about the topic.