ALT Claude 2's performance on the GRE, USMLE, and Multistate Bar Exam. Claude 2 on GRE: Verbal: 165; Analytical Writing: 5.0; Quant Reasoning: 154. Claude 2 on USMLE: Step 1 (5-shot) 68.9; Step 2: 63.3; Step 3: 67.2. Claude 2 on Multistate Bar Exam (5-shot): 76.5

ALT A 52B language model can evaluate the validity of its own proposed answers - separating the correct and incorrect responses - on questions from TriviaQA, Lambada, Arithmetic, GSM8k, and Codex HumanEval. We have weighted the overall contribution from each of these five datasets equally.

ALT Graph showing the fraction of neurons of which quickly suggest an interpretation in regular transformer language models vs SoLU across a range of scales. Baseline models are ~30% of neurons with an interpretation, while SoLU is ~60%.

ALT Experimental Setup. From a large original text dataset (left), we draw 90% of our desired training dataset in a non-repeated fashion, and 10% as repeats of a tiny portion of the original dataset (right). We hold constant that 10% of total training tokens will come from repeats, but we vary the repeated fraction in our runs. In other words, the sample to be repeated might be very small, like 0.01% of the total training tokens repeated 1000x, or relatively large, like 1% of the total training tokens repeated 10x. A small, held-back portion of the original dataset (yellow in left figure), not including any repeated data, is used as a test set and is the test loss reported in all subsequent figures.

ALT A graph showing the difference in performance between context distilled, static HH RLHF, Online HH RLHF, and Online Helpful RLHF models. Online Helpful RLHF models do best - close to the distribution of scores for professional writers.