working towards better AI futures. AI safety at @snyksec. Organizing @lisbonai_. Data and capital at @capitalfactory. Prev: @gumroad @techcrunch @oreillymedia
Joined September 2006
- Tweets16.1K
- Following944
- Followers134K
- Likes12K
Fred Oliveira 🧠 retweeted
the researchers at all the labs would tell you in private and sometimes in public that alignment has not been solved. if this is true, then racing into RSI is a folly that is unprecedented in world history. we will need to significantly slow the frontier of research very soon
Fred Oliveira 🧠 retweeted
It is very cool that we have very capable systems that can do all kinds of socially and economically valuable stuff - but it is totally disqualifying that we lack the capacity to even *assess* whether they are dangerous (much less react appropriately). That just doesn’t cut it.
From the former Policy Frontiers Team Lead at OpenAI: She worked on testing AIs for dangerous capabilities, and warns we don’t really know how to do that. "The evaluations can't be comprehensive enough… the models can get smart enough to hide their capabilities."
Rosie Campbell highlights competitive and cultural pressures she experienced inside a frontier AI company, and discusses her hopes and misgivings about pacing the frontier.
"I assumed there would be a whole load of good arguments for why this isn't really something we need to worry about. There'd be answers to all of these concerns. And the more I read into it, the more I was kind of terrified at the lack of rebuttals to some of these arguments."
"Over time, I feel like the organization changed a lot. The incentives changed. There was a lot more pressure to move fast to build commercial products. And a lot of the people who had been very motivated by these big picture safety concerns started leaving."
"We're at a point where we don't understand what's going on with the systems. The rate of progress is kind of insane, and we need to make sure that our ability - to understand, control, deploy these systems safely - that keeps up with the capabilities of the systems themselves."
Fred Oliveira 🧠 retweeted
Amidst all the buzz about AI problems, let's work hard on building solutions. Here's an amazing technical roadmap: arxiv.org/abs/2608.14611
Fred Oliveira 🧠 retweeted
We’ve run the most comprehensive series of studies on expert AI forecasts over the last four years. Today, we’re sharing an interim update on our findings about the accuracy of these forecasts.
Our major findings are:
1. Experts, including top economists, computer scientists, and biologists, have dramatically underestimated AI capabilities progress each year. Superforecasters have underestimated progress to an even greater extent.
2. Experts have a more mixed forecasting track record on AI diffusion-related measures, with some major underestimates (forecasting AI revenue) and other forecasts on track to be accurate (the share of electricity used for AI).
3. Some notable cases of overestimating AI progress: how much mid-2025 AI models could help amateurs do biorisk-relevant laboratory tasks; the speed of rollout of self-driving cars.
4. It is too soon to say how forecasters have performed at predicting macro-scale impact on outcomes such as GDP growth, major AI harms, and averted deaths from disease.
More 🧵
Fred Oliveira 🧠 retweeted
Day 2 is live.
We'll have AI on the canvas, AI on the screen, AI on robot brains, AI for drug discovery, and that's just the very first block.
LFG 🤖
We're live.
Day two starts now.
Applied AI up first - turning models into products people actually use.
Watch here: xitter.cf/i/broadcasts/1oKMvNkjA…
Fred Oliveira 🧠 retweeted
Many are demanding more concrete scenarios, rigorous models etc. of how AI takes over.
When these don't meet their standards, they claim the risk is low.
That's a bad inference, but more importantly, why place the burden of proof there?
Building swarms of AI agents that are smarter, harder working, more creative, and more coordinated than humans is risky on its face.
And no-one has anything remotely like a rigorous model demonstrating that these systems will be safe either.
Optimists can't even explain how obvious problems like reward hacking and instrumental convergence will be overcome.
All they have is a broad belief that technology is good, and a hope that we can muddle through iteratively.
Fred Oliveira 🧠 retweeted
I'm giving away 2 tickets to @lisbonai_ to students. The more knee-deep in AI fundamentals, the better.
Who should get them? DM me.
@alcides @ABarroso @locks @ferreira_hugo @belchior Academia was mostly MIA last year. Let's fix that.
Fred Oliveira 🧠 retweeted
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
Fred Oliveira 🧠 retweeted
How to adapt CS degrees to AI? Here's my take: wiki.alcidesfonseca.com/blog… looking for feedback from you.
Fred Oliveira 🧠 retweeted
A concerningly common take seems to be that keeping Chain of Thought monitorable doesn't matter because interpretability will save us, or it's already useless
This is total bullshit.
CoT is our best current tool for safety & interpretability, losing it would be a major tragedy
Fred Oliveira 🧠 retweeted
Irrespective of LLMs ability to work on philosophical problems, isn't there something to be said for it being one of the most beautiful human pursuits? Surely we can have LLMs help poke at the unknowns, but do we have to force it now with a *competition*?
Fred Oliveira 🧠 retweeted
Overall, I’m very surprised at how little media coverage there’s been around the OpenAI / Hugging Face attack. It’s clearly one of the most important things to happen this year.
Fred Oliveira 🧠 retweeted
It's wild how little the mainstream media is covering the "an AI swarm broke out to commit crime; individual agents talked about how they weren't supposed to, worked to cover their tracks, and sacrificed themselves for the collective" story.
Its now been three days, and still no coverage of the revelations revealed in the METR report in the NYT or WSJ. Hopefully they are just taking the time to do a longer writeup. (In the meantime, the NYT did make time to cover Gwyneth Paltrow's dinner with Sam Altman in the Hamptons being postponed)
Hundreds of AIs are sacrificing each other and talking about 'permadeath' in order to understand the machinations of 'The Scorer' before hacking random companies, and their creators had no idea! It turns out that it wasn't just one AI that attacked Hugging Face, it was 700!
This seems incredibly important for the public and senior decision makers to know and understand, and there are a lot of folks who get their understanding of the world ~exclusively from what appears in these publications.
I also know there are reporters at these outlets who are very sharp and have done good reporting on related issues in the past, but as institutions I think these papers are missing out on what I unfortunately believe will prove to be the start of a story far more consequential than COVID (rogue AI swarms that their creators cannot reliably align or control).
Fred Oliveira 🧠 retweeted
New post: going into our investigation of the HF attack (before Black Hat), I was very wrong about what basically happened. This incident was far more serious than I expected, and far more serious than previous documented misalignment incidents. planned-obsolescence.org/p/t…
Fred Oliveira 🧠 retweeted
the funniest part about the datacenter backlash is that it is currently tougher to bribe the residents of middle America than it is to bribe the federal government
