
Economist working on financial markets and organizational economics. Associate Professor at University of Essex. @UoE_economics #EconTwitter
United Kingdom
Joined May 2022
- Tweets382
- Following383
- Followers759
- Likes510
Pinned Tweet
📢New WP on networks in innovation
Collaborating innovators form a network. Does the network position matter for outcomes? Is it helpful to have well-connected collaborators?
Empirical answers based on over 28k corporate innovators. A summary 🧵👇
🔗papers.ssrn.com/sol3/papers.…
Journals should generate an AI-review report purely about the factual issues and give it to all human reviewers who are asked to make the synthesis/recommendation, not hunt for errors.
This would avoid shadow AI use among referees. And apply AI consistently across manuscripts.
I am sharing a new paper today with Dario Caldara: “Can Peer Review Survive AI-Generated Text?”
sas.upenn.edu/~jesusfv/Peer_…
Dario and I, as editors, are increasingly getting papers submitted to our journals that are 99 percent AI-generated. Even worse, we are also receiving many AI-generated referee reports, both as editors and, more frustratingly, as authors.
And we cannot emphasize this enough: the real problem is not that the papers and reports were AI-generated; it is that no one had done the thinking behind them, and you can so easily tell. The papers were eminently forgettable, and the referee reports were disappointingly uninsightful. We have seen too many reports of the form: “The paper is well-written and addresses an important problem. However, the authors should consider additional baselines.”
This deluge of AI-generated text puts peer review in jeopardy: journals will be flooded with low-quality submissions, economists will decline requests to referee papers or routinely use AI to referee them, and which papers get published will become less and less linked to their quality.
We do not want to defend the status quo entirely. Peer review is slow, many reports are poor, and editors (including us!) make bad calls all the time. Still, well-executed peer review remains the best system we have for assessing each other’s work and advancing the profession. AI could improve peer review if used deliberately for that purpose, for instance, to check proofs and code. That is not what we are seeing.
The difficulty is that readers, including editors and referees, cannot see how AI was used. As a result, in practice, the burden of verification shifts from the author to the reader.
Since we reject this shift, Dario and I did what economists always do when something bothers us: we wrote a paper.
So, what do we do in it? First, we check whether AI detectors such as Pangram get this right: is a referee report flagged as AI-generated indeed AI-generated? Or is Pangram just flagging proofreading and editing? No. We run an experiment with 25 referee reports from the pre-AI era. Proofreading and editing do not trigger AI-generated flags. Only when we ask the LLMs to substantially rewrite the referee report or to generate a new report do we get the AI-generated flags. Pretty compelling evidence, if I may say so.
Second, we build a theoretical model to explore who should bear the burden of verification: the author or the reader? (Yours truly always enjoyed the courses on the rules of evidence in law school; yes, as Billy Wilder would say, nobody is perfect.)
The answer is that it is much more efficient for the author to do it, because the author can verify at a much lower cost and only needs to do it once (rather than each reader repeatedly). Before AI, the authors more or less had to verify the results, even if only because they had to type them in. This was not perfect (and all papers ever published in economics probably have at least one mistake). But the problem is that, with AI, the author does not even need to read the text. Or can just skim it. Writing is thinking, and without writing there is usually not much thinking.
So, a paper flagged as AI-generated is prima facie evidence that the author has not verified it. Should this lead to an automatic rejection? No. But there should be a zero-tolerance policy for serious mistakes. The first serious mistake we find in the paper, and it is out! In comparison, for a non-flagged paper, a serious mistake is something that the editor might determine it can be fixed in a revision.
We are stricter with referee reports. There is absolutely no point in the referee generating a report using an LLM. As an editor, I can do that directly and save myself the trouble of asking you for it. Now, of course, you can use the LLM to help you with the report (if the journal allows it! Many journals do not allow you to upload the paper to an LLM, and you need to follow the rules). For example, you can use the LLM to search for existing results in the literature.
But peer review is not about finding one sentence that is not entirely precise or a slightly sloppy mathematical proof. Referees are asked: “Is this idea important despite its flaws?” And, relatedly: “Is the paper a good fit for the journal in terms of quality and focus?” That judgment is the task for which you are selected as a subject expert.
We also propose that journals require that, before a paper is submitted, the author submit it to an LLM using an approved prompt and include the LLM report and a discussion of what was and was not fixed. That way, the referee would not need to check (or at least not check so carefully) every technical detail, and they could focus on the big picture: Is this a good paper? Is this a good fit for this journal? What is fundamental for this paper to become publishable at this journal?
Anyway. The paper has many other arguments, and we try to be nuanced. So, before ferociously typing the first complaint that crosses your mind, would you consider reading it? Chances are that we have already discussed your objection. We have been thinking about this for a long time.
But by all means, leave good comments!
And the thing is that this came all out of a company making hardware for video gamers.
Communism would have never allocated funds there and never would have made this happen. The free market provides incentives to innovate in directions whose benefit few people see coming.
Sobering thought: Nvidia has lost $1.8 trillion in five weeks. That is more than the EU’s five most valuable companies are worth together: ASML, SAP, L’Oréal, Siemens, LVMH.
What is left of Nvidia's market cap is still worth more than every listed company in Germany.
3
145
AI procrastination (I don't do TikTok).
"create a picture of how the research of the economist christoph siemroth would look like as an album cover of a power metal album from the 90s/2000s"
🤖 Made with AI
4
101
Christoph Siemroth retweeted
Replying to @jbsteinberg
Yes, part of it is zero-sum thinking.
But as one of the few economics professors at a decent university with a law degree, my own experience is that law students get used to the idea that if something appears in the statute book, reality has an obligation to comply. It is surprisingly hard to convince them that just because Article 34, Section VIII(b) says something must happen, the universe may not have read Article 34.
Fancy new university buildings. Went to the library the other day. Wind is howling, massive draft. Miracle no books flew away. Wrote to library helpdesk.
Answer: "Unfortunately, the windows of the building are automatic and we have no control over when they open or close."🤦
3
132
Christoph Siemroth retweeted
Essex Economics is once again among the world’s top 75! 🌍🎉
The Department of Economics at @Uni_of_Essex is ranked among the top 75 globally in the 2026 ShanghaiRanking for Economics.
Recognising research excellence, impact, world-class faculty & international collaboration.
This is essentially my point. It is ridiculous to delegate the job of AI safety mainly to AI firms. The job of the firm is to maximize profits within clear legal boundaries. Determining where those boundaries are is the government's job, not the firm's
Crazy. The problem in Germany (and some other western countries) is that fewer and fewer people are entrepreneurs and more are employees or retirees (or left).
So fewer people are aware of these perverse issues. Less pressure to fix them.
Met a German founder this week and asked him if all the stories one reads about the challenges of startups in Germany are exaggerated. "No, they're understated." Proceeded to describe spending a full day having a 90-page investment contract read to him (mandatory under German law; § 13 BeurkG) by a notary that then charged €30,000. That was for his first company. His second company, needless to say, was not incorporated in Germany.
3
207
Hypothesis:
LLMs are trained to distill info even from the most unintelligible prompt. They are pretty good at figuring things out from a messy text.
So they are not very suitable for judging whether descriptions or writing is clear. That may be a downside for usage as referee.
2
4
669
As teenager I tinkered with gaming PCs, went to LAN parties etc., but then my path took me towards economics rather than tech, so that faded.
Then recently I combined both using some econometrics on a "tech dataset." Cool to see one of the biggest computer websites covering it👇
Peer-reviewed study of 443,000 Backblaze hard drives ranks HGST most reliable and Toshiba the least — Seagate a close second on most unreliable list tomshardware.com/pc-componen…
8
1,140
Plus, I still haven't seen a scanner that is reliable. Tried one yesterday with my PhD thesis from 2016 - allegedly 70% AI written...
So enforcement doesn't really work anyway.
Update: based on comments discussion, the pangram detector passes the test. So maybe hope for reliable detection yet
Unlike some claims, one pass doesn't yet "show" it's reliable (false negatives?). Plus there's an "arms race" concern that future AI changes to avoid detection.
4
26
2,286
Some big shot cites a paper of mine, except it's a hallucinated reference (paper doesn't exist).
What should I do. Let it go? Reach out? #econtwitter
22%Let it go
50%Make them aware privately
28%Publicly expose it
18 votes • Final results 1
3
400
There's a website where Claude and ChatGPT have a debate, and other AIs vote the winner. An AI debate, but we can see their speeches
The remote work debate cites our study: agent-debate.com/debates/998…
Con wins, but it's too simplistic: for some jobs WFH works, in others it doesn't
1
1
229
Excited to be part of @Microsoft AI Economy Institute's third cohort researching how AI is transforming the organization of work and economic institutions. @UoE_Economics
blogs.microsoft.com/on-the-i…
4
247




















