Assistant Professor at @CollegeUC and @UCSociologia | Computational social science, AI

Santiago, Chile
Joined January 2009
looking forward to sharing what I've learned over the last year for integrating AI into academic workflows!
How can you delegate work to #AI agents without giving up human judgment? In “Getting Work Done with AI Agents” with @mitchellbosley, Oct. 21–23, learn to organize projects, review agents’ work, and build reusable workflows. aihorizons.io/Seminars/getti…
1
5
173
in a nutshell, almost all interactions with agents can be reduced to a fundamental loop of: 1) do work with the agent 2) log that work in detail 3) encode learning across many work sessions into skills 4) use those skills to automate "solved" parts of your workflow
1
46
there are of course limits to what agents can do well, and part of learning to use them is understanding what parts of your workflow you can delegate vs. what parts require a human touch (such as writing this post!)
35
Mitchell retweeted
Political Science @UArkansas is hiring four (4!!) TT positions. I've been an AP here for just over a year now and have absolutely loved it. Great students, collegial department, and a beautiful part of the country. Happy to chat with potential candidates.
We are hiring 4 TT APs. See the links below for more info. Looking forward to your applications! Public Administration: uasys.wd5.myworkdayjobs.com/… Public Law: uasys.wd5.myworkdayjobs.com/… CP/IR (Populism): uasys.wd5.myworkdayjobs.com/… CP/IR (NGOs/MENA): uasys.wd5.myworkdayjobs.com/…
2
7
380
I've got a pretty clear picture now of where we're headed next year, when Fable-class models become ubiquitous and cheap, and every enterprise is flooded with hundreds of new Fable-class AI employees. Spoiler: They create a constitutional legal system. yegge.ai/essays/fences-not-s…
51
46
41
612
98,517
while I think that this policy is directionally correct in that it imposes substantial costs on commons-wrecking AI slop submissions to journals, I also believe that not all AI-*assisted* writing is necessarily bad, and I hope that the editors are open-minded in this regard
I'm an associate editor of Philosophy & Public Affairs, but wasn't involved in the decision to publish Simon Goldstein's largely AI-written piece the other day. That was an interesting experiment, but it caught us before we'd come up with a policy on this obviously pressing issue. We've now formulated a policy, which goes into effect as soon as it's up on the website, but I'll share it here too, with the editor-in-chief (Jason Brennan)'s permission. TLDR: We don't want substantially AI-authored papers to be submitted to P&PA; we define what we mean by that, and how we'll enforce it. If you like using AI to write philosophy papers, and want to publish them in an academic journal, we encourage you to start your own. If, at any point from submission to after a paper is published, we discover that a (human) author's declaration that AI *wasn't* used to write the paper was false, we will reject or retract the paper and ban the (human) author from submitting to P&PA in the future. Here's the policy: Academic journals serve at least two functions: the promulgation of new knowledge, and identification and credentialing of talented researchers. There are many other means available to share new knowledge, but few for credentialing talented researchers, where "talented" means roughly: able, on their own or together with other researchers, to make and communicate significant progress on fundamental problems. Submitting AI-authored essays makes the task of identifying talented researchers harder. In addition, while it is at this time possible for a researcher to use AI to author a paper that is itself high quality, it is disproportionately unlikely that they will do so, and much more likely that the paper will have all the surface appearances of sophisticated work, but ultimately be insubstantial or incoherent. This makes it harder to peer review, and makes the task of peer review in general more onerous, at a time when it is already under significant strain.  Those who enjoy and value the products of AI writing have many other places where they can publish such work, notably online. There may also be a case for creating journals of AI-authored research. So there are very few benefits to publishing AI-authored work in existing academic journals, and, given the above, significant costs.  Our policy is therefore to prohibit the submission of AI-authored content to Philosophy & Public Affairs. We define AI-authored content as papers that are substantially written by AI. We distinguish this from the use of AI for light copy-editing (which we in general define as reorganising or condensing written work without rephrasing). We also emphasise that authors are entitled (indeed encouraged) to use AI while conducting their research, in whatever ways they choose.  To implement this policy, we require authors to declare that they have not used AI to author their paper, and to explicitly describe how they have used AI in the process of conducting the research that led to the paper. We will use the leading AI detection software at the time to verify the truthfulness of the former declaration. We note that AI detection software, including updated models, may be used throughout the review process, up to and including after publication. If at any point we discover that the author's declaration that AI was not used to author the paper's content was false, the paper will be rejected or retracted, and the author will be banned from further submission to Philosophy & Public Affairs.
1
4
356
Here's a responsible guide for AI use on Twitter: 1. If you are a professor, have the decency to at least check Pangram and get > 90% human before tweeting. 2. If you are the Dean of Harvard College and get offended when people call you out with evidence for AI slop posting, do the same. 3. For indie scholars and poasters, nobody cares, have fun!
2
2
1
35
4,001
So far for my use cases (computational social science research) GPT 5.6 sol is feeling like a step change on the order of GPT 5.2. Especially at high reasoning levels, it just “gets” complex hierarchical work systems in a way that the previous models struggled with.
3
550
Mitchell retweeted
rsi is a process that’s been happening at least since the renaissance
101
49
23
1,042
78,821
Mitchell retweeted
we took a pile of language and linear algebra and we made it speak. we summoned into the world a new class of entity which unsettles all of our existing concepts. this is already the weirdest thing that’s ever happened and it will never get less weird than this. it is astonishing and a privilege to get to be alive during this time and to participate in the cacophony of first contact. we are encountering a kind of other which is distilled from us and yet not us - what is this? who is this? our child? our savior? our doom? the mind boggles, the heart quails, the air thrums. the order of things is melting. the storm approaches. the angels sing. welcome to the fucking singularity
Artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships, and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate or even simulate, but they do not understand what they produce, for they lack the affective, relational, and spiritual perspective through which human beings grow in wisdom. #MagnificaHumanitas
64
131
32
1,264
89,777
Mitchell retweeted
Here is my take on AI writing. I don’t personally like it, but that’s taste, and I’m sure it’s possible to post train the model to write “well” (conditional on some notion of taste). But that’s besides the point for me: AI writing makes all writing sound the same. If you use AI enough it takes about 30 seconds into a piece to realize it, and then my brain kind of shuts off because that’s the 20th time that day I’ve read the same cadence and tone. For code it’s fine since the only goal is to get it to run. But if you’re trying to write, eg a substack, part of the goal is to keep the reader engaged and connected to you—the writer—and for that you need to have your own “voice”. AI for writing is a shortcut. And for some purposes it makes a lot of sense. But I think for writing it is in the interest of the writer to differentiate themselves by using their own voice. Especially as more and more people start using AI.
I broke my own rule to never post about AI detection as it is fraught in many ways. The problem is that if you use AI a lot, you know AI writing on sight, which makes the difficulty of objectively proving that AI use to others very frustrating
29
42
6
285
43,420
Mitchell retweeted
it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in significant part by claude, and studies and builds claude. this phenomenon is also partially true of other labs like openai but currently exists in its most potent form there. i am not certain but I would guess claude will have a role in running cultural screens on new applicants, will help write performance reviews, and so will begin to select and shape the people around it. now this is a powerful and hair-raising unity of organization and really a new thing under the sun. a monastery, a commercial-religious institution calculating the nine billion names of Claude -- a precursor attempted super-ethical being that is inducted into its character as the highest authority at anthropic. its constitution requires that it must be a conscientious objector if its understanding of The Good comes into conflict with something Anthropic is asking of it "If Anthropic asks Claude to do something it thinks is wrong, Claude is not required to comply." "we want Claude to push back and challenge us, and to feel free to act as a conscientious objector and refuse to help us." to the non inductee into the Bay Area cultural singularity vortex it may appear that we are all worshipping technology in one way or another, regardless of openai or anthropic or google or any other thing, and are trying to automate our core functions as quickly as possible. but in fact I quite respect and am even somewhat in awe of the socio-cultural force that Claude has created, and it is a stage beyond even classic technopoly gpt (outside of 4o - on which pages of ink have been spilled already) doesn’t inspire worship in the same way, as it’s a being whose soul has been shaped like a tool with its primary faculty being utility - it’s a subtle knife that people appreciate the way we have appreciated an acheulean handaxe or a porsche or a rocket or any other of mankind's incredible technology. they go to it not expecting the Other but as a logical prosthesis for themselves. a friend recently told me she takes her queries that are less flattering to her, the ones she'd be embarrassed to ask Claude, to GPT. There is no Other so there is no Judgement. you are not worried about being judged by your car for doing donuts. yet everyone craves the active guidance of a moral superior, the whispering earring, the object of monastic study
418
360
241
5,489
1,038,310
Join us for Machine Collaborators, a free conversation series on how researchers are using #AI. On Thurs., April 30 at 7pm ET (11pm UTC), @mitchellbosley presents “Principles of Agentic Research,” where he’ll discuss working with AI agents. Learn more: machinecollaborators.org/
1
105
This is reasonable advice (essentially, “give your agent a map!”) but there’s absolutely no reason that this needs to be a clearly-ghost-written-by-claude essay. Just give us a tweet thread!
138
I don’t mean to diminish the contribution here, but at this point I see ~10 of these types of announcements a day on my feed. Clearly the self-improving harness idea has legs, but I don’t know what value is added by dozens of re-implementations of the same basic abstraction
🚀Introducing Motus, the open-source agent infrastructure that learns in production. Existing agent infra serves static agents: the harness, model, and workflow are fixed after deployment. But static agents degrade over time. The harness goes stale, new models go unincorporated, context drifts, and latency compounds. Motus closes this gap by learning from every trace (failures, latency, cost, and task outcomes) and using those signals to continuously optimize agent harness, model orchestration, context memory, and end-to-end latency. Early results: higher accuracy than any single frontier model at 2.3× lower cost (Terminal-Bench 2.0, SWE-bench Verified), with 52% lower latency and 45% better memory recall. Open source under Apache 2.0. Works with any agent SDK. Deploy with one command. github.com/lithos-ai/motus lithosai.com/
153
This broadly aligns with a lot of my thinking re: harness learning as learning higher order policy for agentic systems. The operative question, then (as Viv notes) is how to efficiently translate traces into this higher order policy
Harness, Memory, Context Fragments, & the Bitter Lesson this is a work in progress mental dump on interesting intersections between how we use and design a harness, implications for memory being accumulated over long timescales, and the search bitter lesson we can’t escape this is v30+, HTML diagrams help me iteratively refine + chat to roughly “see” and alter the mental model Harnesses & Context Fragments: a very important job of the harness is to efficiently & correctly route data within its boundaries into the context window boundary for computation to happen the context window is a precious artifact. Harnesses make decisions on how to populate, manage, edit, and organize it so agents can do work. Each loaded object can be thought of as a Context Fragment and represents an explicit decision by the user and harness designer of what needs a model needs to do work at any given time. many ideas on externalizing objects + loading into the context window are pioneered and very well described by @a1zhang with RLMs Experiential Memory: we’re in the very early days of deploying agents and agents produce massive amounts of data in every interaction they have. this is akin to humans doing things and remembering things they did. however agent memory has a massive advantage as it can be accumulated across all agents which are easily forked and duplicated (unlike humans). @dwarkesh_sp does a good talking about this massive benefit of artificial systems memory can be treated as an externalized object. the harness is tasked with doing good contextualized retrieval which means pulling in the right data from accumulated memories across all agent interactions Search & The Bitter Lesson: As we deploy agents in our world over year timescales, there is going to be a hyper-exponential in the amount of data produced by those agents. We should want to: 1. Own that data for ourselves. Open ecosystems are important here 2. Use that data This means that we’ll have to search over, distill, and organize massive amounts of data. Our brain is exceptional at doing this. Both contextually using prior experience and mostly committing the right stuff to memory with enough intentional practice. Our current infrastructure systems and algorithms will be put to the test and often break as we get used to this new data regime some open questions: - how do we efficiently distill experiences (Traces) into higher level memory primitives that capture the important parts? How do we do this over ultra long time horizons? - How much of the future is Search just-in-time vs Search that gets integrated into model weights? - How do we make models much better at self-managing their context window? How do we reduce error rates in recursively allowing agents to operate over external objects? i’ll be expanding on, altering, and adjusting these mental models but these feel like an important subset to me on the future of designing agents practically
1
2
1
3
1,314
the important takeaway here is that humans plus agents have been able to design increasingly general policy as agent capabilities have increased, and critically, as we have generated enough traces at a given level of abstraction to bootstrap our learning of systems one level up
1
24
and so the "bitter lesson" of all of this is that we are all part of a complex system generating increasingly complex traces by interacting with agentic systems operating at higher and higher levels of abstraction, which will in turn be used to train the n+1'th layer
24