
Assoc Prof at UC San Diego @ucsd_cse, AI researcher
San Diego, CA
Joined December 2017
- Tweets75
- Following342
- Followers786
- Likes45
Taylor Berg-Kirkpatrick retweeted
Literary works are often translated into the same language multiple times, and each translation interprets the original in its own way.
How can we use these different translations to train better literary MT systems? Can synthetic references be as good as human ones?
Taylor Berg-Kirkpatrick retweeted
Today we release our first collaborator model @humansand! Ampersand is the first model with privacy built in (unlike most other models where privacy is an afterthought), co-designed in tandem with our product (more soon!!).
We designed our environment and simulations such that the post-training’s optimization pressure incentivizes decisions that abide by contextual privacy norms, through long horizon outs.
Replying to @humansand
Unlike assistant models trained with verifiable or preference reward, Ampersand was trained to understand the long term consequences of its actions through simulation feedback
ampersand.humansand.ai/blog
Taylor Berg-Kirkpatrick retweeted
🚨Did you know that your coding agent is writing your code in an insecure way, leaving exploitable vulnerabilities unpatched?
@dqwang122 trained agents to be better at *secure* vibe coding
Check the preprint and thread 👇
Taylor Berg-Kirkpatrick retweeted
Excited to share that our Strokes of Sonic Tradition project has been awarded a Humanities and AI Virtual Institute (HAVI) grant from Schmidt Sciences!
We will use AI to decode ancient guqin music by bridging the surviving notations and living practices.
smtd.umich.edu/u-m-researche…
Schmidt Sciences is excited to announce 19 new research teams using AI to expand the study of art, dance, history, music, literature and more across thousands of years of global culture through our Humanities and AI Virtual Institute program.
schmidtsciences.org/havi-202…
Taylor Berg-Kirkpatrick retweeted
Optimizers are usually benchmarked at one batch size, and scaling rules extend them to different batch sizes. We show: (1) frameworks to derive these rules are not as general as they claim to be, and (2) the best optimizer depends on the batch size. batch-size-scaling.com
Taylor Berg-Kirkpatrick retweeted
How to Play Jazz Piano - srush.github.io/jazz-piano/
A guide to jazz chords and harmonization with visualizations. (humanslop).
Taylor Berg-Kirkpatrick retweeted
pretty interesting document.
Anthropic released their 2021 research roadmap earlier this year (because of copyright litigation); I think some people might find it interesting: i.sm4.ca/concise-research-ro…
Taylor Berg-Kirkpatrick retweeted
Replying to @BachFrancis
(1/7) I'd like to provide some additional context for the remarks that can be found in this interview.
(2/7) My discomfort with the attention being paid to existential risk---and with the ongoing dominance of the public discussion of such risks by a few so-called thought leaders---is that there's an elephant in the room that is being missed. That elephant is the effect of all of this discussion on young people, and indeed the effect---beginning with social media a decade ago---of information technology on young lives broadly speaking.
(3/7) I recently read an interview by Jill Lepore in which she reports on discussions that she had recently with a group of college students: "They had just three things to say. One, social media completely destroyed their childhoods. Two, AI is destroying their college education and their likely prospects for employment. And three, what the hell are we all doing?" To add a fourth bullet that I've heard in similar discussions: "I'm not going to have children, because the existential risk is just too high."
(4/7) Please contemplate such a statement, those of you who think that it's OK that the public, widely diffused, discussion coming from seemingly authoritative figures is so adamant about existential risk.
(5/7) Let me add that I have little interest in a debate about what "superintelligence" might mean, or whether we're currently seeing the emergence of superintelligence. For what it's worth, I would certainly agree that the solution of extremely challenging math problems (for example) exhibits a very impressive form of intelligence, although I would want to argue that such intelligence is impossible to dissociate from the collective of humans that provided data to the machine learning system (which itself has become a kind of collective).
(6/7) Such philosophical debates are entertaining and even important, but when they then head in the direction of "when such superintelligence emerges, there's a good chance that it will want to kill all humans", I say "science fiction". Again, I don't think that it's bad to consider such thoughts, but I strongly believe that it's bad for society, and especially young people, for such thoughts, unaccompanied by any serious real-world analysis, to become a dominant form of discourse of the era.
(7/7) We are damaging human lives, and the consequences of that damage will no doubt be a major theme of the next several decades.
Taylor Berg-Kirkpatrick retweeted
Michael I. Jordan, machine learning demigod, UC Berkeley
... the young people who work at these companies operate in a world [Silicon Valley, Berkeley, etc.] where these kinds of discussions are commonplace. Many of them aren’t really engineers; they come from the humanities, philosophy, or other disciplines. They’re 25 years old, they’re naive, they don’t do engineering work, they’ve never worked at large companies where real products are developed for real people, with discipline. And they get together, a bit like in Parisian cafés in the 1920s, to talk about these grand science-fiction ideas. When these young people resign, they know they’re going to attract attention. It allows them to feel virtuous and to be seen as very important figures. They claim to be giving up money, but in reality, they already have their stock options. So all of this is a grand moralizing performance that doesn’t serve the real objective it claims to pursue. There’s a great deal of naiveté here, which goes hand in hand with a certain cynicism…
di.ens.fr/~fbach/MJordan_ITW…
Taylor Berg-Kirkpatrick retweeted
We’re Marin, the details are pretty much always available!
github.com/marin-community/m…
This time we’re deploying Triton router top-k and short conv, leaner ragged a2a transport, fused expert backward, and XLA scheduling fixes
Taylor Berg-Kirkpatrick retweeted
Put your agents to navigate all over the Minecraft world including Notre Dame, Hong Kong, Versailles, the White House, and many more!
Introducing an interesting project- Mine Odyssey, a benchmark for measuring spatial agentic intelligence in the wild!
TL;DR: We leverage a wide variety of Minecraft maps created by the amazing community (yes, games are great lol) and challenge models to navigate through them by reading maps, finding paths, opening doors, going up and down stairs, climbing ladders, and more.
Thanks to the incredible Minecraft modding community, we can also introduce a wide range of variations for ablation studies, including weather conditions, different HUDs, first-person vs. third-person views, and reference route lines, to investigate how these factors affect model performance.
When we first launched this project, we honestly didn't think any model would be able to solve these challenges smoothly. But now, GPT-6 Astra and GPT-6.1 Sol can make it all the way to the destination remarkably quickly, with significantly fewer turns! 😨 Meanwhile, other models, even when they manage to reach the destination, often take unnecessary detours or get lost along the way.
Our benchmark is fully open-source and Harbor-compatible!
Kudos to @SArt_Maxine @llleo0612
@junxian_he
More details: mine-odyssey.github.io/
Taylor Berg-Kirkpatrick retweeted
Can you make more efficient embeddings through souping?
Go check out @yasjafarii poster in Franciscan C to find out!! #colm2026
Taylor Berg-Kirkpatrick retweeted
More perf tweaks landed thanks to @mcmcescher! We're now up to ~40% more tokens-per-second than when we launched.
Taylor Berg-Kirkpatrick retweeted
Can you fingerprint agents from a few turns of non-adversarial conversation without access to system prompts or model internals?
Can we trace scammers and undisclosed API changes?
📍10:10 - 10:35 and 3:25-4:15 at HAIPS at #COLM2026 Square 19 & 20
Taylor Berg-Kirkpatrick retweeted
🚢Marin 535B-A23B training has crossed the halfway mark! Some observations and learnings so far:
Taylor Berg-Kirkpatrick retweeted
In SF this week. Feel free to reach out if you want to chat.
Will present Parcae at Poster Session 1 #44
Taylor Berg-Kirkpatrick retweeted
Open-weight language models match Jev's overall performance on system one tasks, without additional training.
Taylor Berg-Kirkpatrick retweeted
Full zeroth order training method "previewed" a few days ago:
- FineWeb val loss is super high (~5) so I don't consider this a meaningful amount of pre-training
- x-axis on plots is # tokens, but I think it should be something like # steps, GPU-hours, etc
Backprop has been the only credit assignment algorithm capable of training large neural nets.
Introducing Dust, the first zeroth-order method to pretrain transformers to approach and sometimes even *exceed* backprop with large amounts of computation (bitter lesson!)
- Dust is on the order of 1,000 to 10,000x more compute efficient than EGGROLL, the state-of-the-art ES method, for training transformers.
- We chose pretraining transformers because it's arguably the hardest task possible for zeroth-order optimization.
- Search in high dimensions is really misunderstood: zeroth-order usually gets better with overparameterization, not worse, i.e. at a fixed population size, larger models (up to 120x) reach a lower loss than smaller ones.
- Backprop-like gradients emerge from a large population of activation perturbations, and the alignment holds up at every scale we tested, up to 1B tokens, which is critical for scaling.
The core idea is a *virtual population*, which helps scale to large populations in transformers by perturbing activations in parallel.
w/ @bishmdl76, @cs_serdar, @akshayvegesna
Taylor Berg-Kirkpatrick retweeted
Catch Max and me presenting this @COLM_conf! Following our observation that token-level watermarking algs don't adequately use the available sequence-level entropy, we propose a semantic watermarking approach for low-entropy output scenarios. arxiv.org/abs/2601.11629
How do you watermark AI text when there's barely any randomness? 🤔
It is unclear how to effectively watermark low-entropy text— until now. Meet SeqMark at #COLM2026: a watermarking algorithm for constrained generation
Paper: arxiv.org/abs/2601.11629





































