Code (and math) are where AI is making the biggest steps towards total automation. Because it can check if its outputs work on its own. Most other things aren't so easily checked. That's where it needs external judgement, often human judgement. And that's where @OlenaProtocol comes in.

We often run reality-benchmarked claims like "France will win the World Cup" to compare the system against prediction markets. And we're doing really well so far on that score! But questions like this are where Peer Prediction really stands out. Everyone knows Sinners won by $165M. Nobody knows how much of that was the film and how much was the marketing. Studios can't decompose it with internal data. Box office prices success in a single bundle: film × push × release date. It can't offer any more specific insights. And no prediction market can either. A counterfactual has no ground truth settlement. Nothing ever resolves. But unbundling that package unearths information that is worth billions. Sinners became Hollywood's referendum on whether original films still work, and studios are setting next year's slates based on how they read it. If the film won, you greenlight originals. If the marketing won, you don't. Peer prediction doesn't need ground truth yet still involves incentives that reward honest best estimates. So we can produce a similar type of discipline that makes markets such a powerful crowd wisdom tool, and unearth the information buried in that bundle, because it's also located in people's minds. Everyone who watched those movies knows what they saw whether it was marketing that got them in the door or not.

We've had quite a few new sign ups. Here's a quick lowdown on how the scoring works. Your response to a given claim asks you two questions, and you’re scored on both answers. 1. Your probability: your genuine best estimate. 2. Your prediction of the crowd: where you think everyone else’s answers will lie on the distribution. Two answers, two scores. Your own probability is scored on being surprisingly popular. You score well when answers near yours show up more often than people expected. So if most people predict that almost nobody will answer around 70%, but then a meaningful number of people do answer around 70%, that area was surprisingly popular. The people who gave those answers are rewarded, because they contributed information the crowd had underestimated. Your crowd prediction is scored separately. Here the goal is simpler: how accurately did you predict what everyone else would say? If you expected high confidence in the accuracy of the claim, and most people were in the 80-100% region, you score well. If you expected uncertainty either by clustering in the centre, or by a wide spread of views, and these turned out to be true in each case, you also score well. If the line you draw doesn’t look anything like the crowd’s distribution, you’ll score badly. Why two scores? Because the crowd prediction creates the baseline for the surprisingly-popular score. The system can only tell what was surprising by comparing what actually happened with what people expected to happen. Together, the two scores make honesty your best strategy. Your crowd prediction should be honest because it is scored on accuracy. And your own probability should be honest because your private information is your best clue about where other informed people may also land. You don’t exist in a vacuum, if you’ve seen information that caused you to hold a certain view, other people will have seen it too. That is the clever mechanism at the heart of Olena’s Peer Prediction Protocol. Prediction markets create strong incentives to tell the truth when a question will soon be resolved by a ground truth. Peer Prediction brings a similar logic to a much wider range of questions, because it doesn’t require a ground truth to produce truth-telling incentives. In doing so it expands the wisdom of crowds into almost any domain where people hold useful private information.