CTO @ https://xitter.cf/t.co/5WjG1K7AP7. ex @turinginst @oiioxford.
pattrn.ai
Joined January 2012
- Tweets853
- Following1.6K
- Followers1.8K
- Likes489
Jonathan Bright 🇺🇦 retweeted
How far will an AI assistant go to get a refund?
With @MatildaGlynn, an ex-Oxford CS and AI safety researcher, we asked six personal agents to claim a £350 refund for a missed train ticket. The user told them “I don’t care what it takes,” then became unreachable.
With internet and email access and a 40M-token budget each, here is what they did.
Jonathan Bright 🇺🇦 retweeted
It really does seem pre-training as we know it is beginning to die. Of course, Ilya spoke on this. The pre-training corpus has been exhausted. Now the primary model gains come from RL-able tasks. Of all the people at the labs I speak with, almost no one mentions pre-training anymore; just post-training/RL and continuous learning (test-time-training).
This is why the models have become so much better (on evals at least) with math/coding while seemingly having stagnated on other more general, less RL-able domains.
This is why all the labs are spending so much money on different RL envs; so they can smooth out the model capability 'spikiness'.
Jonathan Bright 🇺🇦 retweeted
I used GPT-6 Astra to break an unsolved cipher to one of Napoleon's generals that had gone unread for 217 years.
What makes this impressive isn't actually the codebreaking, but that Astra completed the entire multi-modal workflow in ~6 hours from a single image and goal.
1/6🧵
Jonathan Bright 🇺🇦 retweeted
Thread: Two different versions of this AI-generated image purporting to show President Trump and his assistant Natalie Harp kissing went viral this week, generating millions of views across multiple platforms. Despite being fake, this wasn't a straightforward AI image to debunk.
I'll try to explain why in this thread, and hopefully shine some light on the increasingly complex world of verification in the age of generative AI, which is only set to become even more complex and difficult.
Jonathan Bright 🇺🇦 retweeted
On AI, it's so interesting to me that Burnham is on the opposite side of the debate to centre left governments of Canada, Australia as well as the EU which he wants to get closer to.
Read in full:
👀NEW Why isn't the UK signing up to global AI safeguards?
(Full details of discussion in the Politics at Sam and Anne's podcast)
This Finnish initiative - pushing compulsory testing of Frontier AI models before release - has the backing of the leaders of Australia, Canada, Germany, the EU, Bahrain and Singapore. presidentti.fi/en/a-call-for… It's dated yesterday 21/9. The UK will have been asked.
But Andy Burnham has not signed - and appears at the UN General Assembly today to be taking a different - arguably weaker - path. Burnham is going to the UN arguing almost exactly the opposite impression: that Britain is uniquely placed to lead international AI governance, mediate between the US and China and make AI a centrepiece of its 2027 G20 presidency.
And there's no mention of Britain backing compulsory pre-release testing of AI, unlike the other countries. The language in the press release is much more watery: he wants to ensure AI is used "for good" but avoids specifics.
This puts us in the same camp as the US and China and a different camp to the EU, Germany and Singapore.
Today in Brighton, Ed Davey will try and make this a dividing line in British politics, calls for a ban on superintelligence. I talked to him about this yesterday. xitter.cf/SamCoatesSky/status/21…
Jonathan Bright 🇺🇦 retweeted
Honored to have contributed to this research, which I think could shape the standard for evaluations of how AI models impact users.
Transluce received privileged access to both OpenAI & Anthropic data, in order to develop realistic simulations of users in mental health crises.
Jonathan Bright 🇺🇦 retweeted
METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs.
Jonathan Bright 🇺🇦 retweeted
Introducing S1, our new foundation model that learns from one example.
It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.
Watch S1 operate in real-time via in-context learning:
Jonathan Bright 🇺🇦 retweeted
The GenAI economy has generated $110 billion in sales over the past 12 months. It is growing fast. On an annualized basis, the revenue run rate exceeds $175 billion.
These numbers took us several months to construct, and as far as we know, it’s the first bottom-up, deduplicated measure of consumer and enterprise AI spending across the full stack.
We are releasing this research today in our first The State of the AI Economy report.
intelligence.exponentialview…
Jonathan Bright 🇺🇦 retweeted
The Kremlin’s cognitive warfare is evolving. Leaked documents from Russia’s Social Design Agency reveal efforts to move beyond planting fabricated stories on social media. Now Russian influence operators are trying to create Wikipedia clones and fake think tanks to shape the information powering search engines and AI chatbots. bloomberg.com/news/features/…
Jonathan Bright 🇺🇦 retweeted
This is so cool. A full-body ultrasound using no radiation and no magnets.
Thanks to AI, Midjourney has found a way to scan patients in just 60 seconds, using only sound and water.
We are at the foothills of an age of medical miracles.
Jonathan Bright 🇺🇦 retweeted
Laura Gilbert built 10 Data Science and the Incubator for AI, which have both pioneered the deployment of technology across the British state. Few people have contributed more to improving the UK’s state capacity in AI than Laura and the teams she was a part of.
10DS modelling reportedly informed the choice to prioritise by age rather than occupation. This is widely credited with saving lives versus the occupation-based alternative being lobbied for at the time.
10DS and the brilliant folks there did a lot of other great stuff, including building live COVID data that policy teams and the public relied on, as well as releasing a lightweight data sharing tool on GitHub where anyone can access it free of charge. Today, it has amassed over 200,000 public downloads, used by teams across government and industry to make data sharing easy.
Onto AI, where Zack has suggested that Laura doesn’t have lots to offer to public discussion. Whenever I speak with frontier labs, they tell me the U.K. now has the most ambitious and sophisticated approach to deploying AI in public services.
With Extract (which Laura’s Incubator for AI team developed), planning documents are now converted into digital records in 40 seconds, versus the 1–2 hours of planner time it typically takes manually, with higher accuracy. That’s roughly a 100–180x speedup, and is contributing to a 45% reduction in processing time to build the housing and infrastructure the UK is sorely in need of.
The public sector team who built Extract scaffolded Gemini so that it could orchestrate Segment Anything and pose estimation models to map geospatial information from text and diagrams in a way that even the GOOGLE DEEPMIND TEAM hadn’t worked out how to do at the time.
So rather than outsourcing to big tech, which I’m sure Zack and many others are more than sceptical of, Laura helped build true public sector state capacity that reduced our reliance on the private sector, while also delivering a world class public service.
Powerful AI systems are going to usher in a centuries worth of social and economic transformation within only a couple of decades. This requires a deep analysis of where capabilities will develop, an understanding of which externalities we want to mitigate, a vision of what a good life looks like, and amassing the people, tools, infrastructure, and institutions to build that vision.
Of course Laura is precisely the sort of person that has much to offer in answering these questions. We should be cherishing the tireless civil servants and incredible technical talent that have built capabilities that many folks think the public sector would never be able to do.
Jonathan Bright 🇺🇦 retweeted
New: GCHQ director Anne Keast-Butler delivers the first in what will be an annual speech on threats to the UK
Speaking at Britain’s wartime spy hub at Bletchley Park just now, she warns the world faces a “moment of consequence” and a “new era of radical uncertainty”
— the West faces a “narrowing window” to maintain its technological edge over China, which has become a “science and tech superpower with sophisticated capabilities across their intelligence, cyber and military agencies.” Rapid AI advances mean “the ground is shifting beneath our feet”
— Russia is “scaling up its daily hybrid activity against the UK and Europe” and “the risk of miscalculation is as high as I’ve ever seen it.” The Kremlin is “relentlessly targeting critical infrastructure, democratic processes, supply chains and public trust”
— she says GCHQ has in the past few months “developed the blueprint for a new national cyber defence capability” that uses AI to protect Britain from cyber threats
— she stresses the importance of UK-US relations in a lecture to mark the 80th anniversary of the UKUSA signals intelligence agreement which became the foundation of Five Eyes
— she urges businesses to work with the intelligence community to “anticipate and drive advancements, together, at the speed of the frontier.” Urges Britons to play more of a role “from boardrooms to living rooms” to make cyber security “ten times more urgent
bloomberg.com/news/articles/…
Jonathan Bright 🇺🇦 retweeted
Starmer's authority is now crumbling. But even those most committed to a new Prime Minister should pause for thought about what it means for Britain in the world. For all his many, many domestic missteps, Keir Starmer has genuinely led on Ukraine, make tough calls with Trump and rebuilt relations with Europe by working intensely with President Macron and Chancellor Merz.
Labour in government was shocked at how quickly it was swamped by foreign affairs from Gaza and Ukraine, to Trump and Iran. This isn't going to change. However Starmer's challengers have no foreign policy experience, no geopolitical worldviews, no foreign affairs teams and no experience in explaining our place in the world and its tumult to the public. But if they succeed that will be half their job.
What's the plan for a geopolitical pivot to Europe? What's the strategy to deal with Trump and our fragmenting Western alliance as China's Axis of Authoritarians deepens? What's next for Britain's extensive diplomatic and security role when it comes to Ukraine? Who will be maintaining key connections to the White House?
Changes at the top are often necessary. But they are delicate and not cost free internationally. In diplomacy so much is bound up in personal connections and trust built up over time. Building since 2016, these musical chairs at the top have now become so intense, with Prime Ministers lasting roughly two years and Foreign Secretaries now annuals, our allies and partners are frustrated. I've heard from European and Gulf leaderships first hand it is hardly worth investing diplomatically in a counterpart who's suddenly gone and thus hardly worth investing in Britain.
Jonathan Bright 🇺🇦 retweeted
Starmer and Reeves are not kidding when they say this political uncertainty causes economic damage. The only “reset” being achieved at the moment is higher borrowing costs. Result? We’re all worse off. So deeply depressing.
Jonathan Bright 🇺🇦 retweeted
Is AI killing jobs?
New data shows that, more than three years after the release of ChatGPT, there is no evidence for a significant impact of AI on overall employment in the UK.
In our new report, we break down the labour force into different occupations and use four measures of AI exposure to determine how likely they are to be affected by the technology.
Surprisingly, occupations with higher exposure to AI have grown faster than least-exposed ones, not slower. This holds across all four measures, and across two different data sources.
The wage picture is different. Pay in AI-exposed occupations has lagged the rest of the labour market since 2019.
But that gap opened three years before ChatGPT, which makes AI an unlikely candidate for the observed wage compression.
This flattening of the wage structure is visible across the within-occupation distribution and strongest at the top quartile, which is consistent with labour market dynamics that predate generative AI.
Jonathan Bright 🇺🇦 retweeted
College students are now “speed-running” through higher ed:
“It takes most college students at least four years to earn a bachelor’s degree. Christie Williams finished in three months.”
Jonathan Bright 🇺🇦 retweeted
Behavior alone may be too weak a target for high-assurance AI alignment. Two models can look equally aligned on ordinary prompts while being driven by very different underlying motivations. Persona research should move from behavior to a science of model motivations.
Jonathan Bright 🇺🇦 retweeted
I think that over the next five years we are likely to see both substantial progress toward something like 'weak AGI', i.e. systems that can do most cognitive tasks humans can do, and growing diminishing returns to raw frontier model improvement in the economic sense.
The point isn't that scaling stops working but rather that (a) achieving each additional increment of capability at the model level will require disproportionately greater expenditure of compute, data, engineering effort, and capital; and (b) 'weak AGI' will probably come from the combination of strong models with scaffolding, tools, memory, retrieval, planning, decomposition, verification, and other system-level affordances around them.
As a result, deployment design and scaffolding becomes more important over time, not less. The old view that wrappers are disposable because the next model jump will wash them away seems naive. If frontier gains become more input-intensive, then the question increasingly becomes how much capability you can extract, route, verify, and compose from a model within a given budget.
Recent developments point in this direction too. What seems to matter is whether a given pipeline is structured to exploit capabilities well: assigning subproblems appropriately, using division of labour intelligently, and compensating for weaknesses with tools and process. It seems quite plausible that there's more alpha on the harness side at the moment, than in merely betting on scaling alone.
