Joined April 2010
Beyza Ermiş retweeted
turns out you can get frontier performance by having a swarm of tiny models working on the problem together. great work from the research team on this one :) cohere.com/blog/lessons-from…
Announcing new research on scaling multi-agent teams for long horizon software engineering tasks. 🤖🧩📈 Work led by @keenansamway, @mrdanieldsouza, Omar Darwiche Domingues, @mziizm, @yichern_tan, and @nouhadziri Read the paper: cohere.link/tStRYJS Check out the blog post: cohere.link/0TzgKoy
18
41
4
420
37,584
Beyza Ermiş retweeted
This is how we trained Soft-SVeRL! Highly recommended over just yoloing a scalar score. It has additional mathematical guarantees: arxiv.org/abs/2605.28561
Replying to @adithya_s_k
2/ If you can verify, verify. If you can't, break it down. Half the environments are plain pass/fail. The 925 workplace tasks become 5,125 yes/no checks, each graded on its own, most by a model given the expected answer. Half right counts as wrong.
2
1
376
Beyza Ermiş retweeted
He who controls the verifier controls the intelligence. Great to see a consensus emerge about scaling verification using criteria decomposition and repeated-verification. We found this to be especially important for co-training the generator and the verifier for recursive self-improvement. arxiv.org/pdf/2605.28561
Scaling self-verification with DeepSeek V4 Flash beats Claude Fable 5 on Terminal-Bench 2.1, while being 11x cheaper 💰 As open-source models become more capable, they can now generate large numbers of high-quality candidate solutions and verify their own outputs at very low cost. For example, we find that sampling just 5 solutions with DeepSeek V4 Flash and ranking them using the same model with LLM-as-a-Verifier can lead to a significant boost in accuracy (79% → 88%), outperforming closed frontier models on Terminal-Bench. Try it out today: github.com/llm-as-a-verifier… More on verification scaling in my previous post.
1
2
25
3,640
Beyza Ermiş retweeted
My talk is now available on YouTube. Thanks @Cohere_Labs for hosting me and making this public.
How can LLM agents exhibit reliability in dynamic environments? 🤖⚡🔄 @SharonYixuanLi discusses key strategies and challenges for reliable agent behavior. Watch the full session: youtu.be/VfuCUgFmroI
9
1
51
7,303
Beyza Ermiş retweeted
Reasoning models are getting better at producing answers, but capability without self-knowledge hits a ceiling fast. In our recent paper we start from a simple observation 💡: Confidence isn't one thing, before reasoning confidence is a forecast, after, it becomes verification.
Announcing new research on calibrating confidence in reasoning models 🎯⚖️, led by @ConorFinlay3, @joshuakurien_, @TheyCallMeMr_, @mziizm, and @beyzaermis. 📜Read the paper: arxiv.org/abs/2606.24281
1
5
1
34
4,918
Beyza Ermiş retweeted
1/ Reasoning models can verbalize how confident they are that their answers are correct. The problem: these confidence estimates are often poorly calibrated. Models can be highly confident even when their answers are wrong.
1
3
6
833
Beyza Ermiş retweeted
Can an LLM know when not to trust its own answer? 🤔 Reasoning models are powerful but overconfident, and their verbalised confidence is generally unreliable. We built CALIBER: an RL method that teaches a model to estimate its confidence twice, before and after reasoning.
1
3
11
1,180
Beyza Ermiş retweeted
Excited to share North Mini Code: Our first open-source coding model. It is small but very strong in agentic coding for its size🔥 I’m incredibly proud of the team — Shipping a model like this takes outstanding research, engineering, infrastructure, and collaboration.
Introducing Cohere's first open-source coding model: North Mini Code Small & efficient, designed for agentic performance and built for community input.
2
11
1
53
3,791
Really glad I got to be part of this one. Huge credit to @TheyCallMeMr_ for leading the work, and grateful to the whole team!
1/ RLVR has driven big gains in math and code because many outputs admit reliable automatic checks: an answer matches the expected result, or a program passes tests. But many real tasks are not like that. Code can be functionally correct but qualitatively terrible or a response may satisfy 4 syntactic constraints but fail 1 semantic constraint.
2
13
1,647
Beyza Ermiş retweeted
Our latest agentic reasoning model — fast, multilingual, multimodal and fully open source ♥️ Built to run efficiently, available for all.
Introducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
4
1
40
2,336
Beyza Ermiş retweeted
Command A+ from @cohere is out now :) its our best model yet and its open source apache 2.0
55
129
19
1,327
205,208
Beyza Ermiş retweeted
Great to join forces with Reliant AI and to grow our Berlin team. Healthcare and biopharma need trustworthy AI that can run on-prem, and that is able to ground it statements on the latest biomedical research. Our state-of-the-art context engine together with tech&datasets from Reliant, will make AI for pharma even better.
A major step forward for sovereign enterprise AI in healthcare and biopharma. Cohere has acquired @reliant_ai 🇨🇦🇩🇪
2
6
28
2,071
Beyza Ermiş retweeted
Exciting news for the future of AI! @cohere and Aleph Alpha are partnering to build a global sovereign AI platform. As demand for AI grows, we are accelerating the development of next-generation frontier models while upholding strong standards for data security and ethical AI.
🚀 Sovereign AI for the world. Cohere & Aleph Alpha form transatlantic AI powerhouse anchored in Canada & Germany! Combining our global scale with European R&D excellence to build sovereign, enterprise-grade AI. Security, privacy & trust for businesses & governments worldwide. #SovereignAI #AIPartnership Learn more: businesswire.com/news/home/2… Image from left to right: Rolf Schumann, Schwarz Digits, Samuel Weinbach, Aleph Alpha, Aidan Gomez, Cohere, Minister Solomon, Canada, Minister Wildberger, Germany
10
10
83
9,090
Extremely proud to present ✨Tiny Aya ✨ Tiny 🤏 but mighty 💪3.35B parameter models, massively multilingual from the ground up 🌎🌍🌏, built with immense care w.r.t language representation🤗 We had a blast building this! 💗 Have at it! 🎆
Introducing ✨Tiny Aya✨, a family of massively multilingual small language models built to run where people actually are. Tiny Aya delivers strong multilingual performance in 70+ global languages in a 3.35B parameter model, efficient enough to run locally, even on a phone.
11
43
5,555
Beyza Ermiş retweeted
Very proud to share today we’re releasing Tiny Aya✨🤏: small enough to run on your phone, strong enough to support 70+ languages. Proof that multilingual progress comes from intentional design --- and evaluation that measures balance, not just peaks 🕯️
Introducing ✨Tiny Aya✨, a family of massively multilingual small language models built to run where people actually are. Tiny Aya delivers strong multilingual performance in 70+ global languages in a 3.35B parameter model, efficient enough to run locally, even on a phone.
5
10
70
7,120
So happy this is out in EACL Findings! 🥳 Loved being able to contribute a bit, big congrats to @_joestacey_ and all the authors! Really nice practical takeaways on robustness for closed-source LLM fine-tuning.
Super excited to have my last PhD paper about NLI robustness published at EACL Findings😍 We investigate how to make closed-source LLMs more robust after fine-tuning. Here are the paper highlights 🧵
2
4
19
2,468
Proud to share this work led by @oliverjbolton 🎉 We introduce SimMerge, a practical way to make model merging more reliable at scale. Paper and results in the thread 👇
🧩In modern LLM development, we often end up with many specialized checkpoints. Merging into one model is attractive, but the results depend a lot on the selected merge method & order.  Our paper introduces SimMerge: a simple way to choose the merge configuration automatically.
10
28
5,396
Sadly, this echoes what many authors and ACs have been feeling. The peer-review system is under serious strain. We can't keep pretending it's fine.
I am an AC for ICLR 2026. One of the papers in my batch was just withdrawn. The authors wrote a brief response, explaining why the reviewers failed at their job. I agree with most of their comments. The authors gave up. They are fed up. Just like many of us. I understand. We pretend the emperor has clothes, but he is naked. Here is the final part of their withdrawal notice. I took the liberty to make it public, to highlight that what we are doing with AI conference reviews these last few years is, basically, madness. --- Comment: We thank the reviewers for their time. However, upon reading the reviews for our paper, it became immediately apparent that the four "reject" ratings are not based on good-faith academic disagreement, but on a critical failure to read the submitted paper. The reviews are rife with demonstrably false claims that are directly contradicted by the text. The core justifications for rejection rely on asserting that key components are "missing" when they are explicitly detailed in the manuscript. Some specific examples are (and many are even fake claims). Claim: Harder tasks like GSM8K are missing. Fact: GSM8K results are in many tables, like Table 2 (Section 4.2) and Appendix G. Claim: The method does not use per-layer ranks. Fact: This is the entire point of our method. The reviewer clearly mistook our method for the baselines. (Section 2, Table 1). Claim: The GP kernel is not specified. Fact: It is specified in Appendix E (Table 6). Claim: There is no ablation of the method's three stages. Fact: Section 4.4 ("Ablation Study") and Appendix J are dedicated to this. Reviewers have a fundamental responsibility to read and evaluate the work they are assigned. The nature of these errors is so fundamental, so systemic in overlooking explicit content, that it goes far beyond what "limited time" or "oversight" can explain. This work has gone through several rounds of revision over the last year. In earlier submissions, the paper usually received borderline or weak-accept scores. Numerous signs strongly suggest that some reviewers are relying entirely on AI tools to automatically generate peer reviews, rather than fulfilling their fundamental responsibility of personally reading and evaluating manuscripts. We strongly protest this. This is a gross disrespect to the authors. It is a flagrant desecration of the reviewer's sacred duty. It fundamentally undermines the integrity of the entire peer-review process. Given that the reviews are not based on the actual content of our paper, we have decided to withdraw the submission. We leave this comment so that future readers of the OpenReview page are aware that the items described as "missing" are already present in the submitted manuscript. These negative reviews for this submission are factually unsound and do not reflect the content of the paper. We cannot and will not accept an assessment that is not based on the work we actually submitted.
13
1,555
First acceptance at @NeurIPSConf :) Super thrilled to share that our work, ‘The Leaderboard Illusion’ got accepted at @NeurIPSConf 2025! 🎉 Huge congrats to all my coauthors! And special shout out to @mziizm @sarahookr @beyzaermis @YiyangNan — learnt a lot from each of you 💙
It is critical for scientific integrity that we trust our measure of progress. The @lmarena_ai has become the go-to evaluation for AI progress. Our release today demonstrates the difficulty in maintaining fair evaluations on @lmarena_ai, despite best intentions.
9
16
227
30,217
Beyza Ermiş retweeted
Our scholar program is our big bet on a delicate balance: driving impactful research in ML and guiding the next generation of thinkers and researchers. Looking forward to reviewing all the applications! 🎶
As we start to review applications, we're seeing so much expertise, energy, and thought from applicants around the world. Thank you for engaging with this process so generously, and congratulations on your applications. The potential you all bring to ML research is inspiring. ✨
5
13
65
9,442