PhD student at @BoulderNLP @lecslab, intern at @AmazonScience Studying NLP for rare+endangered languages, multilinguality, automata, synthetic data
Boulder, CO
Joined November 2018
- Tweets1.9K
- Following357
- Followers349
- Likes108K
Pinned Tweet
Beyond honored that PolyGloss has won an Outstanding Paper award at #ACL2026
We’re really excited about NLP for endangered language documentation, and it’s inspiring to see that the community is too!!
Excited to announce that the PolyGloss paper has been accepted to @aclmeeting!
Previously, we trained models to help in endangered language documentation workflows by automatically predicting interlinear glosses. But real-world user studies revealed crucial issues...
This can actually be a negative, especially in educational settings!
One underrated aspect of coding with AI is that I never get stuck anymore.
In the olden days I'd sometimes hit a bug or other difficulty that could leave me spinning my wheels for days. I'd google fifty variations of the same query, read poorly-written docs I barely understood, post on StackOverflow and get no answer, pore through obscure technical blogs and bang my head against the screen but end up no closer to victory than when I started. Sometimes I'd give up completely.
This basically never happens anymore. AI doesn't magically solve everything but it cuts through all the crap so I quickly that I never feel like I'm making no progress.
Kids these days don't know how good they've got it!
“Coding is solved” is a weird phrase to me
“Coding” is not a math conjecture!
It’s the broad process of translating informal, underspecified requirements into a formal representation. That’s not really something that has a definable “solved” state
michael retweeted
ai coding skeptic: man this thing seems like a footmachinegun
ai coding fanboy: rookie mistake, you just gotta point another footmachinegun so it shoots the bullets out of the air before they hit your foot
There’s like a billion tweets about how AI coding is just “writing code in English”, but that’s just…obviously not true
Even without watermarking, there’s no guarantee that the quote won’t subtly change, and if you think this never happens you should probably look closer!
The answer is the internet will once again get substantially worse for everyone except advertisers
Yesterday me and my friends talked about the Dead Internet Theory
If nobody asks questions anymore on Stack Overflow, and sites like Reddit are now taken over by AI reply bots to promote brands, as well as AI reply bots on here, there is no real content anymore on the internet
And then there is no fresh training data anymore
I thought about something like, how would you find out the best outdoor action camera? Before I would search :
site:reddit.com best outdoor action camera
But now I just ask my AI which is the best
The problem is if nobody creates new content anymore and just asks their AI every question, the AI has nothing to train on anymore
And then I thought so what will happen? Will you get companies with giant warehouses that just buy stuff and review it with manually or with humanoid robots? Will you get humanoid robots backpacking in South East Asia to get life experience as training data? Or will we all train it by giving it access to our AI chats and user data?
I don't know but I think it's likely the web will dead in the future and AI companies will have to find training data elsewhere
A lot of the debate about whether LLMs are neurosymbolic is simply because “neurosymbolic” is an incredibly nebulous and broad term that you can use for virtually anything
While I understand (but don’t agree with) allowing AI writing for papers, I strongly believe it should be banned for author rebuttals. Otherwise what’s the point??
michael retweeted
Did you know Hermann Grassmann invented most of modern linear algebra but his thesis was largely considered too abstract and useless? I often wonder if an AI with extreme computational power will simply not need, hence not define, what an exterior algebra or a manifold is.
Is there any reason why not to do hidden-state distillation for SDPO (rather than logit distillation)?
Seems like that incorporate the privileged info in a much richer, faithful way versus blindly matching outputs
michael retweeted
the fable / sol summaries are getting more and more absurdly information-dense
1 in every 5 words is a novel compound word (ex: “rate-adjacent”) which itself is the model’s invented stand-in for a metaconcept that it developed that i need to interrogate it about
how to fix this?
michael retweeted
This is a reminder for me, as an NLP researcher: how long has it been since I last walked up to a CL poster and talked with the authors?
I appreciate that ACL still hosts many tracks dedicated to traditional CL methods — that’s what distinguishes us from ML conferences.
And I agree that, now that we’re equipped with such advanced tools, we should aim at questions that are profound to linguists, such as language acquisition and, my personal favorite, language evolution. (Highly recommend The Unfolding of Language; I’ve read it three times and it fascinates me every time.)
Here are a few more remarks, which I think are mostly what I heard at @psresnik’s talk, but might be partly what I wanted to hear. 😉
Computational Linguistics is part of AI. Indeed, in the age of LLMs, it has become the tentpole part of AI. (Exciting!) But computational linguistics is also part of linguistics. ACL has to maintain its identity as people who are interested in human language and languages. At the moment, you see that interest most clearly in the many and varied workshops at ACL. You don’t always see it clearly wandering the main conference poster hall, where ACL can look a bit like yet another ML conference.
ACL (and, really, most academic conferences) has always been a balance between multiple interests and groups: cognitive science vs. natural language engineering, theoretical models vs. empiricism, research vs. practical systems. This diversity is important. It is always the case that the pendulum of interests has swung markedly over time, including around 1990, when Philip (and I) got into the field, with the rise of empirical approaches.
But, still, there are plenty of worries at the moment: You can worry that things have swung so far in one direction that the pendulum is broken. You can worry that the opportunities for research in industry are rapidly disappearing as industry research labs retreat from open science (Chris: much more a thing in the US than China!). You can definitely worry that the growing size of recent conferences might lead to a success catastrophe.
On the one hand, we should be preparing interested students for the rich opportunities now available in industry for natural language engineering. But on the other hand, we should also be reinvigorating interest in computational linguistic science. This has to be less about some of the problems that centrally animated ACL for most of the past 75 years (machine translation, syntactic parsing, …) and more about still little understood problems (e.g., computational approaches to language acquisition, pragmatics, discourse, sociolinguistics, and linguistic style).
Philip’s talk was more about questions than answers, but he encouraged everyone to spend a bit more time thinking about possible futures. ∎
I’m incredibly grateful for my amazing collaborators @lltjuatja @mariavale505 @lexicutioner
When Lindia and I started this work three years ago, I would never have guessed it’d end up here