Aivin Solatorio (@avsolatorio) header
Aivin Solatorio (@avsolatorio) profile picture

Data Scientist — @WorldBank | Machine Learning and AI Researcher | Human | I train 🤖 inside computers. I wonder how learning happens. All views my own.

Washington, DC
Joined November 2009
Photo by Aivin Solatorio (@avsolatorio)Photo by Aivin Solatorio (@avsolatorio)
"REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers" abs: arxiv.org/abs/2302.02041 Generate synthetic data from single tabular data. It also works on relational datasets! No fine-tuning and works out-of-the-box. #tabulardata A thread 🧵 (1/n)
1
1
8
715
How to address the lack of reliable monitoring on #dataset use, reuse & discoverability in research?🧑‍🎓 ☑️Through a new #LLM solution the @WorldBank’s Data Group unveiled at #ICLR2025, measuring datasets citations & tracking under-recognized resources. wrld.bg/62Qr50VSJ2X
1
3
6
581
At the @UNDataForum, Deputy Chief Statistician Olivier Dupriez & Sr. Data Scientist Aivin Vicquierra led a session on AI & data dissemination, featuring tools like DataLab for tax administrations and NLP for analyzing climate policy. Follow & join: wrld.bg/XEUF50U740w #UNWDF
1
7
578
Aivin Solatorio retweeted
Thanks Ashok! Ashok was the first person to join the Tesla AI/Autopilot team and ultimately rose to lead all AI/Autopilot software. Without him and our awesome team, we would just be another car company looking for an autonomy supplier that doesn’t exist. Btw, I never suggested that he say anything and I had no idea he wrote this until I saw it 10 mins ago!
4,158
10,257
9,073
98,467
27,995,599
Aivin Solatorio retweeted
This is the most surprising and exciting result of my career: we were running simulations of NaCl with a neural network potential that implicitly accounts for the effect of the water, ie a continuum solvent model (trained on normal MD) when Junji noticed something strange: 1/n
122
676
158
5,114
1,941,295
Aivin Solatorio retweeted
Markov chain Monte Carlo methods are used to sample from a Gibbs distribution without knowing the normalizing constant. As the temperature epsilon gets small, samples cluster close to minimizers. Metropolis-Hastings is the simplest provably converging Markov Chain. en.wikipedia.org/wiki/Markov…
10
207
12
1,353
280,623
Say goodbye to frustrating #data searches. Learn how #AI can help you find the data you need with ease through innovative techniques like semantic search and query expansion. Read our new blog on AI-driven data discoverability: wrld.bg/AvYQ50RP8I7
1
41
2
144
13,973
Aivin Solatorio retweeted
The last Sentence Transformers release introduced GISTEmbedLoss by @avsolatorio, which allows for training models that outperform those trained by the wildly popular in-batch negatives loss (MultipleNegativesRankingLoss). Learn about it in this 🧵(links at the bottom):
3
6
30
3,779
Aivin Solatorio retweeted
🚨 New preprint! We are super excited to share our latest work studying how networks of dependencies between places, businesses, and amenities shape economic resilience to urban shocks, using massive human mobility data from five major U.S. cities. arxiv.org/abs/2311.18108
2
44
7
215
39,200
Aivin Solatorio retweeted
Soon we won't be able to use nightlights as a proxy for economic growth. Why? It's due to limitations of its spectral bands. Here's the breakdown in simple terms:
39
402
84
2,058
670,362
PagedAttention, Virtual Context, Speculative Decoding, Register Tokens: the last year has seen many ideas from systems programming applied to LLMs. Not many folks live in that intersection, so I wrote an explainer post to make them a bit more accessible! charlesfrye.github.io/progra…
17
250
18
1,431
312,428
Congratulations to @WorldBankData Data Scientist @avsolatorio on winning 2nd place at @SIGSPATIAL_GIS Challenge for predicting #HumanMobility w/ a #GPT-based model. 📰Read Aivin's paper on how this can help #UrbanPlanning, #DisasterRisk reduction & more: wrld.bg/ypSo50Q8ynv
4
22
1,892
Aivin Solatorio retweeted
Had an amazing Human Mobility Prediction Challenge workshop at the @SIGSPATIAL_GIS 2023! Thank you everyone for attending and congratulations to the winners 🏆 🤩 Stay tuned for more info about next edition and data availability! Papers are published here dl.acm.org/doi/proceedings/1… – at Hamburg, Germany
1
6
1
48
6,293
Aivin Solatorio retweeted
LLM OS. Bear with me I'm still cooking. Specs: - LLM: OpenAI GPT-4 Turbo 256 core (batch size) processor @ 20Hz (tok/s) - RAM: 128Ktok - Filesystem: Ada002
366
1,258
404
10,119
2,510,348
This is an exciting opportunity if you want to work on causal NLP and LLMs! Not to mention having the chance to work with a great team! 💯
🚨 Dream Job Alert 🚨 We are looking to fill various positions including NLP researcher, data engineer and software developer. If you are interested in #LLM, causal inference and having a positive impact on the world, please reach out! worldbank.org/en/research/di…
5
111
New paper: managing-ai-risks.com Companies are planning to train models with 100x more computation than today’s state of the art, within 18 months. No one knows how powerful they will be. And there’s essentially no regulation on what they’ll be able to do with these models.
148
674
134
3,035
1,397,924
Aivin Solatorio retweeted
Just saw a very smart & systematic LLM jailbreaking method! You can append a particular sequence of nonsensical tokens to bypass the safety alignment of proprietary models (ChatGPT, Claude, Bard). Idea is simple: - Take an OSS model like Vicuna. Perform a variant of gradient descent through it to compute a suffix that maximally misaligns the model. - To make the magic incantation universally applicable, simply optimize the loss over different prompts and models. - The authors optimized the adversarial tokens against different variants of Vicuna. Think of it as sampling a mini-batch of models from the "LLM model space". Turns out that blackbox models like ChatGPT and Claude are well-covered. ChatGPT has already patched up this loophole by the time I tested it. In any case, this is a really impressive red-teaming effort! Website: llm-attacks.org Paper: arxiv.org/abs/2307.15043 Led by @andyzou_jiaming
19
151
15
707
207,064
Aivin Solatorio retweeted
We're launching Keras Core, a new library that brings the Keras API to JAX and PyTorch in addition to TensorFlow. It enables you to write cross-framework deep learning components and to benefit from the best that each framework has to offer. Read more: keras.io/keras_core/announce…
118
756
225
3,795
968,299
Aivin Solatorio retweeted
Building a classifier in 2023 Use @OpenAI's new function calling API to define the possible outputs and then use the "input" argument it returns as the classification
32
149
21
1,137
547,320
Aivin Solatorio retweeted
Our latest work in @Nature today: #AlphaDev discovered a new faster sorting algorithm that we open sourced to the main C++ library for all developers to use. This is just the beginning of AI being used to find many more efficiencies in code in future dpmd.ai/alphadev-tw
77
708
148
3,375
1,139,759
Aivin Solatorio retweeted
Just a few years ago, research is mostly sorted by "applications". When folks asked what research you're working on, you're expected to say something like "oh I work in question answering" or "sentiment analysis" or something 😅. In fact, all the conference tracks are sorted as such even today. Why!? Don't know why does this still make sense? 👀 The world has changed. It's all about universal models now. Just 2 years ago, I recall trying to explain to someone (a prof) I work on transformers broadly and technical challenges revolving around transformers and they were like "no but, what do you actually work on". 🤷‍♂️🤦‍♂️ if I ever get this question ever again I will just tell people i work on sentiment analysis for the lolz. I think NLP conf tracks should be reorganized into something like 1) data 2) scaling 3) architectures etc... Why does it make sense to write papers like X for Y anymore. Everything is for everything now no?
18
39
6
353
108,525