CTO at Theia Insights. PhD from @cambridge_cl. Co-organiser of fever workshop (https://xitter.cf/t.co/9FTkHRwec2)

Cambridge, England
Joined October 2013
Hi #FactChecking NLPers! As we are preparing to wrap up our image-text claims shared task, a reminder that @FEVERworkshop is welcoming submissions on all related topics with deadline on Dec19. openreview.net/group?id=eacl… fever.ai/
Excited to announce the 9th @FEVERworkshop collocated with #EACL2026 in Morocco: fever.ai/index.html with a new shared task on #factchecking image-text claims with evidence from the web using the AVerImaTeC dataset: arxiv.org/abs/2505.17978 Deadline 2nd of December! And...
5
1
9
4,535
James Thorne retweeted
⁉️Why do reward models suffer from over-optimization in RLHF? We revisit how representations are learned during reward modeling, revealing “hidden state dispersion” as the key, with a simple fix! 🧵 Meet us at @icmlconf! 📅July 16th (Wed) 11AM–1:30PM 📍East Hall A-B E-2608
1
3
21
940
Pleased to announce the next @FEVERworkshop at ACL2025! Regular workshop papers (ARR and direct submissions) due 15th of April! And new shared task focusing on reproducible and efficient verification of real world claims! Check fever.ai/ and get keen!
1
13
3
38
5,491
Looking forward to attending #NeurIPS2024 this week. Come meet @itsnamgyu who will present on block-transformer, which speeds up decoding by up to 20x.
Do you know your LLM uses less than 1% of your GPU at inference? Too much time is wasted on KV cache memory access ➡️ We tackle this with the 🎁 Block Transformer: a global-to-local architecture that speeds up decoding up to 20x 🚀 @kaist_ai @LG_AI_Research w/ @GoogleDeepMind 🧵
2
4
26
2,247
NVIDIA's earnings call wasn’t about semiconductors — it was a vision for AI factories, 24/7 inference, and industry transformation. Using TIIC, we reveal $NVDA's diversification shows the future is broader than chips. What's your take? #AI #NVIDIA #LLMs linkedin.com/pulse/nvidias-e…
1
2
713
📢 Excited to share our new paper with @fabreetseo: "Beyond Position: How Rotary Embeddings Shape Representations and Memory in Autoregressive Transformers"! arxiv.org/abs/2410.18067 Keep reading to find out how RoPE affects Transformer models beyond just positional encoding 🧵
7
25
5
170
24,092
James Thorne retweeted
We are hosting wonderful NLP colleagues at KAIST on their way to ACL Bangkok! 🤩 On-site registration is closed, but the talks will be broadcast on Zoom. Please join us! Date/Time: Aug 10, 2024, 10:05-12:30 KST (UCT+9) Parallel Session 1: Advanced Language Models and AI Capabilities, Moderator @seo_minjoon kaist.zoom.us/my/aliceatkais… 10:05 – 10:30 @mohitban47 (UNC Chapel Hill): Multimodal Generative LLMs: Unification, Planning Agents, Evaluation 10:35 – 11:00 @IAugenstein (U of Copenhagen): Revealing the Parametric Knowledge of Language Models 11:05 – 11:30 Yoon Kim (MIT): Efficient Alternatives to Attention for Sequence Modeling 11:35 – 12:00 @kaiwei_chang (UCLA): From Learning through Labels to Learning through Language 12:05 – 12:30 @PangWeiKoh (UW): Scaling Retrieval-Based Language Models with a Trillion-Token Datastore Parallel Session 2: Specialized Applications and Societal Impact of NLP, Moderator @NoSyu z.nosyu.kr 10:05 – 10:30 @VioletNPeng (UCLA): Controllable and Creative Natural Language Generation 10:35 – 11:00 @computermacgyve (Oxford University): Information Integrity and Generative AI: Challenges and Opportunities 11:05 – 11:30 @sunipa17 (Google): Geo Cultural Representation in NLP 11:35 – 12:00 @CamachoCollados (Cardiff University): Practical and Specialised NLP Solutions: The Case of Social Media 12:05 – 12:30 @TristanNaumann (Microsoft Research): Biomedical NLP / Real World Evidence in the Era of Generative AI
2
40
3
114
23,410
James Thorne retweeted
Aligning a diffusion model on preference data WITHOUT a reference model could be nice no? So, @krasul and I are ideating the use of ORPO to align SDXL 1.0 on PickAPic. Diffusion ORPO with LoRA 💫 Code and model ⬇️ huggingface.co/sayakpaul/dif…
3
13
1
60
18,172
James Thorne retweeted
🔥Mixtral-8x22B-base + ORPO🔥 7k data & 1.3 hours to build a strong human-aligned 140B chat model🦾 👉IFEval: 65% 👉BBH: 59% 👉MT-Bench: 8.17 More models will be added to the Zephyr-ORPO collection with @argilla_io and @huggingface , stay-tuned😃
The new Mixtral-8x22B base model is a total beast for fine-tuning and has produced some of the highest scores I've ever seen on IFEval and BBH 🤯 We teamed up with @argilla_io and @kaist_ai to cook up a brand new recipe for Zephyr models 🪁 huggingface.co/HuggingFaceH4… 🧑‍🍳 Align the base model with Odds Ratio Preference Optimisation (ORPO). This novel algorithm does not require an SFT step to achieve high performance and is thus much more computationally efficient than methods like DPO and PPO 🦫 Use a brand new dataset of 7k high-quality, multi-turn preferences that has been developed by our friends at @argilla_io: huggingface.co/datasets/argi… As usual, we are open sourcing the training code in the Alignment Handbook for the community to build on: github.com/huggingface/align… This has been a epic speed run with @jiwoohong98 @nlee288 @alvarobartt - now I can finally sleep 😂
2
17
70
13,992
James Thorne retweeted
Great work! I’m glad our benchmarks KoBBQ, KOLD, and CLIcK are used to evaluate HyperCLOVA_X. Making progress in Korean LLM! 🇰🇷🇰🇷🇰🇷 arxiv.org/abs/2307.16778 arxiv.org/abs/2205.11315 arxiv.org/abs/2403.06412
Happy to share that our #HyperCLOVA_X tech report was released!
1
5
1
43
7,316
This is actually huge: - No SFT stage (e.g., Zephyr used 200k examples) - Preference tuning with 7K examples only (other models trained with at least 60k samples) I've put a lot of care & love building the DPO version of the amazing Capybara dataset from @ldjconfirmed so I'm really pleased to see these results. Let's double down on useful open data for OSS AI developers and researchers
📢New model, Mistral-ORPO-Capybara-7k in ORPO collection!🧵 With 💡ORPO💡 + 7k Capybara preference pair by @argilla_io🔥 + Mistral (7B), you can get the human-aligned chat model within 2.5 hours of fine-tuning👀 👉AlpacaEval 2.0 (LC): 15.9% 👉MT-Bench: 7.44 👉IFEval: 61.27%
2
17
72
6,519
🚀Calling all astronaut economists! Introducing Theia Theme Connect, where you can travel the cosmos by connecting companies and themes. Can you navigate through the stars and matrix of data before your oxygen runs out? 🔗Click here to launch into action: connect.theiainsights.com/?u…
1
1
4
313
New preprint: Align LLMs without reference models using ORPO ⬇️ Code and checkpoints available on @huggingface
Align LLMs with the preference dataset ONLY with 💡ORPO💡 We introduce ORPO, alignment without reference model & SFT! With awesome dataset from @argilla_io + Mistral(7B) + ORPO, we present 🌟Mistral-ORPO-β🌟 🧵 👉 AlpacaEval 2.0: 12.20% 👉 IFEval: 66.19% 👉 MT-Bench: 7.32
6
712
James Thorne retweeted
I can’t keep up with all these new papers and datasets from my students 😊 @j6mes and I wanted to give #kaist undergrad students research experience with LLMs in their own languages, and this is one of the outcomes of that project!
Excited to share our new paper ✨CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean✨ presented at LREC-COLING 2024 📌TL; DR We release CLIcK benchmark, Cultural and Linguistic Intelligence in Korean, and test 13 LMs using CLIcK (1/🧵)
2
25
2,671
Pleased to announce that Theia Insights has raised $6.5M to develop foundational AI technologies for the investment industry. Well done team!
🚀Theia Insights is pleased to share news of our recent seed financing. We have secured $6.5M from some incredible investors to continue building #AI-based foundational technologies for the global #investment community - read more here: theiainsights.com/blog/seed-…
1
25
1,063
Congratulations Seonghyeon!
FLASK🧪 is accepted to ICLR 2024 as Spotlight! Check out our work and see you in Vienna!
3
686
James Thorne retweeted
A group photo from the poster presentation of »AmbiFC: Fact-Checking Ambiguous Claims with Evidence«, co-authored by @Max_Glockner (@UKPLab), Ieva Staliūnaitė (@Cambridge_CL), @j6mes (@kaist_ai) , Gisela Vallejo (@unimelb), @vlachos_nlp (@Cambridge_CL) and @IGurevych. #EMNLP2023
1
6
1
40
2,967
Come and enjoy on Fri, Oct 27 at 6 pm, as @RMBandService and @ktigers_ent do a joint performance at 📍Gwanghwamun Plaza in front of the Sejong Center to celebrate 🇬🇧🇰🇷140yrs.
3
8
1,676