PhD student @ The University of Sheffield UKRI CDT Speech and Language Tech.
Joined January 2015
- Tweets1.1K
- Following1.4K
- Followers460
- Likes6.7K
Constantinos Karouzos retweeted
What is TMLR doing under a 300% increase in submissions?
- Suspensions of >150 authors
- Publicly naming major violations
- Submission quotas distinguishing slop patterns
- Desk rejections 6% → 53%; focus on clarity
- AI reviews evaluating soundness
blog.tmlr.org/2026/adapting-…
Constantinos Karouzos retweeted
We’ve been locked out of this account for a while😅 but we’re back, with some good news! 🎉
We have a bunch of new Sheffield NLP papers at @emnlpmeeting @aaclmeeting CIKM and INLG 📚
Congrats to everyone involved! 👏
More details below 👇
Constantinos Karouzos retweeted
Emails sent to [email protected] are often quite rude. Before pressing send, please keep in mind that ARR is run mostly by volunteers. PLEASE TREAT US WITH RESPECT!
#ARR @ReviewAcl
We have just implemented a self-assessment checker for this and now deployed at aclrollingreview.org/qualifi….
Constantinos Karouzos retweeted
We discovered a sophisticated recording device that was hidden inside one of our offices. We used our models to understand the device and where it had come from, identifying that a specific employee had planted it a few days earlier. We referred the matter to the FBI, and we’re grateful to the agents and prosecutors who moved quickly to make an arrest. The FBI’s investigation is ongoing and we will continue to support their work.
At this time, we have no indications of this individual inappropriately accessing or exfiltrating model weights, source code, or customer data. The employee was terminated.
We take protecting our technology, data, and people seriously, and as a matter of national importance. Our security program is designed to protect against a wide range of threat actors, from insider risks to cybercriminals to nation states, and this case is an example of how AI can help address these sophisticated security threats.
Constantinos Karouzos retweeted
Ben Affleck reveals he writes Python, understands convolutional neural networks, worked extensively with GPUs, and used his celebrity status to get private looks at Google and OpenAI’s video models
“I’ve always been kind of into computers since I was young. Then, when film started to move from analog film to digital, I became more interested in that aspect of it. The visual-effects workflow for many years has included machine learning, so I can write pretty shitty Python scripts and stuff like that.
“With convolutional neural networks, which were the precursors to what the transformer can do, which is much more computation simultaneously, you would do things like look at what’s called a tensor. That’s the numerical translation of a visual image in numbers, like the batch number, the frame number and the red, green and blue values of each pixel in each frame. It’s just that simple. That numeric is called a tensor.
“You’d use a convolutional neural network to identify patterns that reveal what’s called edge detection or feature extraction, which is identifying patterns well enough to know, this is where the window ledge is, so we can more easily take the green-screen image out and replace it with something.
“That was familiar to me early on because, prior to Artists Equity, I had a small visual-effects company. I’ve worked with GPUs a lot too. The visual-effects guys said, ‘Hey, you should see. There are a couple: Google and this other company, OpenAI, are doing really interesting stuff with transformers in video.’
“I’ve learned that I can actually just call up and go, ‘Hey, it’s Ben Affleck. Can I come see what you’re doing?’ Sometimes people say yes, to my astonishment.”
$ANET Andy Bechtolsheim says AI has made optics demand about 10 times bigger in five years and the industry is only at the "very beginning"
"So I guess I don't need to tell you that AI has been driving this incredible increase in demand for high-speed optics, which is probably now 10 times bigger than it used to be five years ago..."
"And what I want to talk to you today is that we're not at the end of this journey, but rather the very beginning. There's easily another order of magnitude increase in bits needed for the next generation kind of data centers."
"So what I want to talk about first is what's happening at the data center level, and how we as an industry have to get together to solve these problems of this very rapid and high demand growth as fast as possible, since AI can't wait."
______
For full set of takeaways to Andy's AI Datacenter and Optics lecture: firesidealpha.substack.com/p…
Constantinos Karouzos retweeted
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
Constantinos Karouzos retweeted
We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:
Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:
Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:
Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:
Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!
In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.
Constantinos Karouzos retweeted
We're expanding our approach to content provenance to include text in response to EU regulatory requirements, while recognizing the significant limitations of current text watermarking technology.
Our tools already help verify whether an image or audio file was created with our models. This work builds on those efforts to help people better understand when content may have been generated or edited with an OpenAI model.
In the EU, we’ll start watermarking eligible text from ChatGPT and Codex over the coming weeks to comply with the EU AI Act.
Customers using our API can turn on text watermarking for select models worldwide today.
Constantinos Karouzos retweeted
Super excited to share that FinePhrase has been selected as an Oral Spotlight at #COLM2026!
I will present on Wednesday afternoon next week. First in Oral Session 4 at 3:30PM PT and then in Poster Session 4 between 4:30 and 6:30PM (Poster F-4-126).
If you're in SF, I am looking forward to seeing you at the talk or in the poster session!
What is FinePhrase all about?
Synthetic data is a standard component in training large language models, yet systematic comparisons across design dimensions, including rephrasing strategy, generator model, and source data, remain absent.
We conduct extensive controlled experiments, generating over one trillion tokens, to identify critical factors in rephrasing web text into synthetic pretraining data.
Our results reveal that structured output formats, such as tables, math problems, FAQs, and tutorials, consistently outperform both curated web baselines and prior synthetic methods.
Notably, increasing the size of the generator model beyond 1B parameters provides no additional benefit.
Our analysis also demonstrates that the selection of the original data used for mixing substantially influences performance.
By applying our findings, we develop FinePhrase, a 486-billion-token open dataset of rephrased web text.
We show that FinePhrase outperforms all existing synthetic data baselines while reducing generation costs by up to 30 times.
We provide the dataset, all prompts, and the generation framework to the research community.
Listen to the paper to learn more: alphaxiv.org/abs/2604.13977v…
Thanks to all my collaborators without whom this work would not have been possible! @_gucciiiii, Michal Štefánik, @gui_penedo, @HKydlicek, @eliebakouch, @_lewtun, @edwardbeeching, @thibaudfrere, Colin Raffel, @lvwerra, @Thom_Wolf
Constantinos Karouzos retweeted
A last piece of my PhD work is out!
arxiv.org/abs/2609.39827
We ask whether "pre-pretraining" (PPT) (i.e., training on synthetic non-natural language data briefly prior to pre-training on natural language data) helps at scale.
This is the largest ever PPT study! (1/5)
ALT Prior work sits at or below 1B parameters and short training horizons. We span four scales, four PT mixtures, five PPT tasks, and up to 100B PT tokens to test the effectiveness of PPT in a more realistic setting.
ALT PPT is claimed to induce structural priors that transfer to natural language grammar.
Constantinos Karouzos retweeted
We have just implemented a self-assessment checker for this and now deployed at aclrollingreview.org/qualifi….
Constantinos Karouzos retweeted
the number of people I talk to who think they should leave the field because end2end research is about to be AuToMAted is ridiculous. In terms of tools, open problems, and our ability to attack them, this is BY FAR the best time in recent history to do AI research.
Constantinos Karouzos retweeted
📢 ARR October 2026 authors
Please review the new ACL Sustainable Reviewing Policy and FAQ to make sure your submissions follow the new requirements.
Policy: aclweb.org/portal/content/ac…
FAQ: aclrollingreview.org/sustain…
#ARR #NAACL2027 #COLING2027 #NLProc
Constantinos Karouzos retweeted
Now that Astra proved millenium prize problems, do we believe it may be getting close to be a reasonable reviewer for neurips submissions? what, no? oh ok. maybe let's wait after p v np is settled.
Constantinos Karouzos retweeted
How can you simulate a city using LLM-powered agents?
@prb977 builds on his previous award-winning work to expand to 10k agents with memory, social networks, and family dynamics that efficiently simulate human movement in a real city (here, San Francisco) for a week!
Can thousands of LLM agents live in a city and move like real people?
We study this in UrbanLLMind—accepted at #COLM2026—a city-grounded simulation of persistent LLM agents validated on real travel data.
Joint work with @anas_ant @dpfoser @smreia
📄 openreview.net/forum?id=jaK0… 🧵👇
Constantinos Karouzos retweeted
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
Constantinos Karouzos retweeted
Checking that a major mathematical proof is correct can take years. Formalization—converting the mathematical reasoning into a form computer proof assistants like Lean can verify—can help.
Last month, Claude completed the first formalized proof of Fermat’s Last Theorem, one of the most famous theorems of all time. This was a project experts thought would take many years. It is the largest Lean proof ever written.
Fermat’s Last Theorem was first proven in 1995 by Sir Andrew Wiles, more than 350 years after it was conjectured. Our proof, which totals over 13 million lines of code, provides machine verification. More importantly, it proves over 29,000 other theorems that the proof requires, across many areas of math which had never before been formalized.
We see this as a major step in the long process of firming up the core of mathematical knowledge, building on work from three centuries of mathematicians and hundreds of contributors to Lean and Mathlib. We are optimistic that AI-assisted verification of mathematical proofs will help reduce the burden of refereeing mathematics in an era where more proofs are being produced than ever before.
You can read about the process on our Science Blog: anthropic.com/research/forma…
And see the complete proof on GitHub: github.com/anthropics/fermat…
Constantinos Karouzos retweeted
EMNLP 2026 is shaping up to be the largest NLP conference in Europe this year! Put your brand in front of leading AI & NLP talent to build visibility, recruiting, and partnerships with the people shaping the field.
⏰ Deadline: September 15
🔗 2026.emnlp.org/sponsors/
#EMNLP2026