challenge: invent a benchmark where
- the score range is between 10% and 70%
- the vibes based hierarchy is "correct", eg astra and fable beat sonnet and luna
- the benchmark is pointless/doesn't measure intelligence or capabilities
ryan retweeted
Some info on how we’ve been working towards safety cases at OpenAI
Note that until there is a safety “brief” for it, any type of frontier workload cannot start – this has real teeth, and has already been a good motivating force for a lot of safety work
obviously it's not going to be paperclips or stamps but I hope it's something more recognizable than computronium
openai admin console is so unintuitive to use it presents a genuine alignment/control risk. why is codex computer use necessary to figure out how to enable dots
I wonder how many honeypot companies with actual resources (money, reserved cloud compute, etc) the labs have set up
modified prisoner's dilemma where if everyone cooperates they get life sentences for antitrust violations
all solutions to this problem (even with infinite IQ classifiers) cannot work unless they re-evaluate continuously. sometimes you don't know how hard the task is going to be until you're 20m in!
not sure why I didn't think about this before but think we finally solved automatically picking an effort for subagents, sounds easy but:
- ask the main agent to pick it => gets handwaved
- pass the task to a classifier => small model struggles w/ picking the right level
add a field asking for a terse justification to pass to the classifier instead:
solved @ $0.03 per 1k subagents ✅
jev binary search for sparse data. "is there a revenue forecast in pages 1-16? 1-8? 5-8? ... etc" you could save cents, who's building this
1 year ago you would get better performance out of models by explicitly telling them they would be compacted (assume not part of RL training). How much things have changed since
ryan retweeted
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
ryan retweeted
I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.
Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…