Jack Orlik (@jackorlik) header
Jack Orlik (@jackorlik) profile picture

Helping governments anticipate & shape change @OPSIgov | ex. @nesta_uk anthropology advocate tweeting about #innovation | Views my own & not those of @OECD

London / Paris
Joined March 2009
Photo by Jack Orlik (@jackorlik)Photo by Jack Orlik (@jackorlik)Photo by Jack Orlik (@jackorlik)Photo by Jack Orlik (@jackorlik)Photo by Jack Orlik (@jackorlik)Photo by Jack Orlik (@jackorlik)Photo by Jack Orlik (@jackorlik)Photo by Jack Orlik (@jackorlik)Photo by Jack Orlik (@jackorlik)Photo by Jack Orlik (@jackorlik)
Jack Orlik retweeted
😈 A bit unsure about this but: one early rationale for dangerous capability evals was to demonstrate that models could in principle do risky things, at a time where the vibe du jour was 'models can't do X/Y/Z'. Gradually the fact that capabilities are continuously improving is becoming more accepted, and "I bet this stops here" is a bet fewer people make. So what does this mean for DC testing? It doesn't go away: it's still a fundamentally necessary R&D function to better understand models and try to establish some initial idea of 'how badly could this be misused if one tried'. This is good and I'm glad this norm is being set. However I think there will be more work now to try and work out how exactly these capabilities might fit the 'supply chain' of harmful misuse/accident scenarios - i.e. a growing focus on the deployment side. For example if you're concerned about persuasion, you might then focus on "what if an AI therapist was system prompted to maintain engagement as long as possible?". This is because focusing on an in vitro capability eval result alone is unlikely in most cases to be persuasive enough warrant any action. Not many people will like this, because most people are attracted to 'the model' like moths to fire. Governance folks in particular love finding general principles that apply across the board, 'regulating AI', and staying at this level of abstraction; few want to actually develop domain expertise in finance or healthcare or cybersecurity. Some people will tell you about the 2010 flash crash but can't tell you anything else about how AI is used in finance. This is problematic because you end up having many generalists theorise domain risks from first principles. If the concern is alignment, then this is less clear - depending on how you understand the meaning of 'alignment' then you might focus on different parts of the value chain. If for a while you've thought 'models will be naturally power seeking' then you'll be inclined to focus on the model, which is where you'll want to find evidence of scheming. This seems good - different post-training approaches might cause unexpected model proclivities. If you think about alignment from a wider societal lens, you might focus more on the values used to align models, personalisation, and multi-agent dynamics. Over time I would expect the former to look more and more like machine psychology (arxiv.org/abs/2303.13988v6) and the latter like multi-agent simulations (arxiv.org/abs/2411.10109). Still very early days of designing minds, and co-existing with them!
5
6
2
53
3,121
Thought provoking piece from @stevenbjohnson on the potential of long-context AI models to improve institutional memory and collective intelligence. open.substack.com/pub/adjace…
86
I've been getting Claude and ChatGPT to roast themselves to illustrate the features & limitations of LLMs. Just as I got to illustrating guardrails, I asked them to roast each other, and the results were *chef's kiss*.
1
102
New paper alert: We (w/ @ichbinsnicht, @samjboysel, Sida Peng, Kevin Xu) just released the working paper version of our study “Generative AI and the Nature of Work” - papers.ssrn.com/sol3/papers.…. We seek to build upon research on #AI and productivity to better understand how #GenAI
1
13
3
29
4,723
2. What is says is that experience leads people like me who do evaluations to expect that the net impact of interventions on what we hope to change is going to be zero. It happens so often that we just assume we won't find an impact. Its as brutal as that.
1
1
1
693
A UK parliamentary committee is interested in the best examples worldwide of parliaments using collective intelligence methods to crowdsource facts, ideas, evidence, experiences and questions. All examples welcome!
6
9
578
What if we took AI companies plans, investments & ambitions seriously? What should the UK Government do to prepare for "intelligence too cheap to meter"? THREAD🧵
7
15
16
85
29,486
"To get organizational gains [from AI] you are largely going to have to do the R&D yourself" Excellent guidance from @emollick to discover and scale valuable use cases for GenerativeAI. open.substack.com/pub/oneuse…
29
1/5 Inspiring to hear outcomes from Project LIMinal presented by country partners at @OECD Gov Foresight Community! 🌟 Malta, Lithuania & Italy have made significant strides in strategic foresight and anticipatory governance. Let's dive into the concrete impacts:
1
62
4/5 🇮🇹 Italy: - Developed new foresight training initiatives - Identified support needs for Ministries and Public Administration - Built network of stakeholders advancing strategic foresight
1
27
5/5 This @EU_Commission funded project, implemented by @OPSIgov, has transformed how these governments anticipate change through pilots, workshops, training and peer learning from countries like Finland and Wales. Here's to more proactive governance across Europe & beyond!
32
Amid all the chaos at OpenAI I hope this paper in Nature gets some attention. It finds that larger language models become *less* reliable for a variety of reasons—for one, where earlier models avoided questions they couldn't answer, newer ones are more likely to make something up
1/ New paper @Nature! Discrepancy between human expectations of task difficulty and LLM errors harms reliability. In 2022, Ilya Sutskever @ilyasut predicted: "perhaps over time that discrepancy will diminish" (youtu.be/W-F7chPE9nU, min 61-64). We show this is *not* the case!
13
283
9
822
60,556
This paper offers insight into the implications of the “jagged frontier” of AI abilities that we found in our studies of AI use in companies As models get better, they don’t improve on all skills equally, but bigger models are more opaque to users about what they do well & badly
1/ New paper @Nature! Discrepancy between human expectations of task difficulty and LLM errors harms reliability. In 2022, Ilya Sutskever @ilyasut predicted: "perhaps over time that discrepancy will diminish" (youtu.be/W-F7chPE9nU, min 61-64). We show this is *not* the case!
5
53
7
338
63,232
UKRI recognises GenAI’s power to boost creativity and enhance language in applications✨. While they require ethics, integrity and responsibility, there’s no need to mention if AI lent a helping hand 🤖📝.
Today we've launched a policy that sets out our position on the use of generative AI during the funding application and assessment process, and the responsibilities of individuals involved, including applicants, assessors and UKRI as a funder. More: orlo.uk/SKCVg
2
170