
CTO at Theia Insights. PhD from @cambridge_cl. Co-organiser of fever workshop (https://xitter.cf/t.co/9FTkHRwec2)
Cambridge, England
Joined October 2013
- Tweets764
- Following581
- Followers1.2K
- Likes1K
James Thorne retweeted
Hi #FactChecking NLPers! As we are preparing to wrap up our image-text claims shared task, a reminder that @FEVERworkshop is welcoming submissions on all related topics with deadline on Dec19. openreview.net/group?id=eacl… fever.ai/
Excited to announce the 9th @FEVERworkshop collocated with #EACL2026 in Morocco: fever.ai/index.html with a new shared task on #factchecking image-text claims with evidence from the web using the AVerImaTeC dataset: arxiv.org/abs/2505.17978 Deadline 2nd of December! And...
James Thorne retweeted
⁉️Why do reward models suffer from over-optimization in RLHF?
We revisit how representations are learned during reward modeling, revealing “hidden state dispersion” as the key, with a simple fix!
🧵
Meet us at @icmlconf!
📅July 16th (Wed) 11AM–1:30PM
📍East Hall A-B E-2608
James Thorne retweeted
Pleased to announce the next @FEVERworkshop at ACL2025! Regular workshop papers (ARR and direct submissions) due 15th of April! And new shared task focusing on reproducible and efficient verification of real world claims! Check fever.ai/ and get keen!
Looking forward to attending #NeurIPS2024 this week. Come meet @itsnamgyu who will present on block-transformer, which speeds up decoding by up to 20x.
Do you know your LLM uses less than 1% of your GPU at inference? Too much time is wasted on KV cache memory access ➡️ We tackle this with the 🎁 Block Transformer: a global-to-local architecture that speeds up decoding up to 20x 🚀
@kaist_ai @LG_AI_Research w/ @GoogleDeepMind 🧵
NVIDIA's earnings call wasn’t about semiconductors — it was a vision for AI factories, 24/7 inference, and industry transformation. Using TIIC, we reveal $NVDA's diversification shows the future is broader than chips. What's your take? #AI #NVIDIA #LLMs linkedin.com/pulse/nvidias-e…
James Thorne retweeted
📢 Excited to share our new paper with @fabreetseo: "Beyond Position: How Rotary Embeddings Shape Representations and Memory in Autoregressive Transformers"!
arxiv.org/abs/2410.18067
Keep reading to find out how RoPE affects Transformer models beyond just positional encoding 🧵
James Thorne retweeted
We are hosting wonderful NLP colleagues at KAIST on their way to ACL Bangkok! 🤩 On-site registration is closed, but the talks will be broadcast on Zoom. Please join us!
Date/Time: Aug 10, 2024, 10:05-12:30 KST (UCT+9)
Parallel Session 1: Advanced Language Models and AI Capabilities, Moderator @seo_minjoon
kaist.zoom.us/my/aliceatkais…
10:05 – 10:30 @mohitban47 (UNC Chapel Hill): Multimodal Generative LLMs: Unification, Planning Agents, Evaluation
10:35 – 11:00 @IAugenstein (U of Copenhagen): Revealing the Parametric Knowledge of Language Models
11:05 – 11:30 Yoon Kim (MIT): Efficient Alternatives to Attention for Sequence Modeling
11:35 – 12:00 @kaiwei_chang (UCLA): From Learning through Labels to Learning through Language
12:05 – 12:30 @PangWeiKoh (UW): Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
Parallel Session 2: Specialized Applications and Societal Impact of NLP, Moderator @NoSyu
z.nosyu.kr
10:05 – 10:30 @VioletNPeng (UCLA): Controllable and Creative Natural Language Generation
10:35 – 11:00 @computermacgyve (Oxford University): Information Integrity and Generative AI: Challenges and Opportunities
11:05 – 11:30 @sunipa17 (Google): Geo Cultural Representation in NLP
11:35 – 12:00 @CamachoCollados (Cardiff University): Practical and Specialised NLP Solutions: The Case of Social Media
12:05 – 12:30 @TristanNaumann (Microsoft Research): Biomedical NLP / Real World Evidence in the Era of Generative AI
James Thorne retweeted
Aligning a diffusion model on preference data WITHOUT a reference model could be nice no?
So, @krasul and I are ideating the use of ORPO to align SDXL 1.0 on PickAPic.
Diffusion ORPO with LoRA 💫
Code and model ⬇️ huggingface.co/sayakpaul/dif…
James Thorne retweeted
🔥Mixtral-8x22B-base + ORPO🔥
7k data & 1.3 hours to build a strong human-aligned 140B chat model🦾
👉IFEval: 65%
👉BBH: 59%
👉MT-Bench: 8.17
More models will be added to the Zephyr-ORPO collection with @argilla_io and @huggingface , stay-tuned😃
The new Mixtral-8x22B base model is a total beast for fine-tuning and has produced some of the highest scores I've ever seen on IFEval and BBH 🤯
We teamed up with @argilla_io and @kaist_ai to cook up a brand new recipe for Zephyr models 🪁
huggingface.co/HuggingFaceH4…
🧑🍳 Align the base model with Odds Ratio Preference Optimisation (ORPO). This novel algorithm does not require an SFT step to achieve high performance and is thus much more computationally efficient than methods like DPO and PPO
🦫 Use a brand new dataset of 7k high-quality, multi-turn preferences that has been developed by our friends at @argilla_io: huggingface.co/datasets/argi…
As usual, we are open sourcing the training code in the Alignment Handbook for the community to build on: github.com/huggingface/align…
This has been a epic speed run with @jiwoohong98 @nlee288 @alvarobartt - now I can finally sleep 😂
James Thorne retweeted
Great work! I’m glad our benchmarks KoBBQ, KOLD, and CLIcK are used to evaluate HyperCLOVA_X. Making progress in Korean LLM! 🇰🇷🇰🇷🇰🇷
arxiv.org/abs/2307.16778
arxiv.org/abs/2205.11315
arxiv.org/abs/2403.06412
Happy to share that our #HyperCLOVA_X tech report was released!
James Thorne retweeted
This is actually huge:
- No SFT stage (e.g., Zephyr used 200k examples)
- Preference tuning with 7K examples only (other models trained with at least 60k samples)
I've put a lot of care & love building the DPO version of the amazing Capybara dataset from @ldjconfirmed so I'm really pleased to see these results.
Let's double down on useful open data for OSS AI developers and researchers
📢New model, Mistral-ORPO-Capybara-7k in ORPO collection!🧵
With 💡ORPO💡 + 7k Capybara preference pair by @argilla_io🔥 + Mistral (7B), you can get the human-aligned chat model within 2.5 hours of fine-tuning👀
👉AlpacaEval 2.0 (LC): 15.9%
👉MT-Bench: 7.44
👉IFEval: 61.27%
James Thorne retweeted
🚀Calling all astronaut economists! Introducing Theia Theme Connect, where you can travel the cosmos by connecting companies and themes. Can you navigate through the stars and matrix of data before your oxygen runs out? 🔗Click here to launch into action: connect.theiainsights.com/?u…
New preprint:
Align LLMs without reference models using ORPO ⬇️
Code and checkpoints available on @huggingface
Align LLMs with the preference dataset ONLY with 💡ORPO💡
We introduce ORPO, alignment without reference model & SFT! With awesome dataset from @argilla_io + Mistral(7B) + ORPO, we present 🌟Mistral-ORPO-β🌟
🧵
👉 AlpacaEval 2.0: 12.20%
👉 IFEval: 66.19%
👉 MT-Bench: 7.32
James Thorne retweeted
I can’t keep up with all these new papers and datasets from my students 😊 @j6mes and I wanted to give #kaist undergrad students research experience with LLMs in their own languages, and this is one of the outcomes of that project!
Pleased to announce that Theia Insights has raised $6.5M to develop foundational AI technologies for the investment industry. Well done team!
🚀Theia Insights is pleased to share news of our recent seed financing. We have secured $6.5M from some incredible investors to continue building #AI-based foundational technologies for the global #investment community - read more here: theiainsights.com/blog/seed-…
James Thorne retweeted
A group photo from the poster presentation of »AmbiFC: Fact-Checking Ambiguous Claims with Evidence«, co-authored by @Max_Glockner (@UKPLab), Ieva Staliūnaitė (@Cambridge_CL), @j6mes (@kaist_ai) , Gisela Vallejo (@unimelb), @vlachos_nlp (@Cambridge_CL) and @IGurevych. #EMNLP2023
James Thorne retweeted
⭐️Congratulations to everyone at KAIST AI who have a total of 20 papers accepted to #NeurIPS2023! Check them here 🧵docs.google.com/document/d/1…
James Thorne retweeted
⭐️Congratulations to everyone at KAIST AI who has a total of 11 papers accepted to #EMNLP2023! Check them here 📷🧵
docs.google.com/document/d/1…
James Thorne retweeted
Come and enjoy on Fri, Oct 27 at 6 pm, as @RMBandService and @ktigers_ent do a joint performance at 📍Gwanghwamun Plaza in front of the Sejong Center to celebrate 🇬🇧🇰🇷140yrs.





























