Daniel Riaño (@danielrruf) header
Daniel Riaño (@danielrruf) profile picture

Filólogo y lingüista. Humanista digital (CSIC) ✍🏻Escritura académica (UAM)👉 If you can't code it, you don’t understand it. Our 🌍 needs more: Philology

Madrid
Joined January 2009
Una parte de la ciencia es explicar por qué lo que intuyes es cierto; otra parte de la ciencia es explicar por qué lo que intuyes es erróneo; la última parte de la ciencia son cuestiones que no podrías plantearte sin ella
3
4
42
I am sharing a new paper today with Dario Caldara: “Can Peer Review Survive AI-Generated Text?” sas.upenn.edu/~jesusfv/Peer_… Dario and I, as editors, are increasingly getting papers submitted to our journals that are 99 percent AI-generated. Even worse, we are also receiving many AI-generated referee reports, both as editors and, more frustratingly, as authors. And we cannot emphasize this enough: the real problem is not that the papers and reports were AI-generated; it is that no one had done the thinking behind them, and you can so easily tell. The papers were eminently forgettable, and the referee reports were disappointingly uninsightful. We have seen too many reports of the form: “The paper is well-written and addresses an important problem. However, the authors should consider additional baselines.” This deluge of AI-generated text puts peer review in jeopardy: journals will be flooded with low-quality submissions, economists will decline requests to referee papers or routinely use AI to referee them, and which papers get published will become less and less linked to their quality. We do not want to defend the status quo entirely. Peer review is slow, many reports are poor, and editors (including us!) make bad calls all the time. Still, well-executed peer review remains the best system we have for assessing each other’s work and advancing the profession. AI could improve peer review if used deliberately for that purpose, for instance, to check proofs and code. That is not what we are seeing. The difficulty is that readers, including editors and referees, cannot see how AI was used. As a result, in practice, the burden of verification shifts from the author to the reader. Since we reject this shift, Dario and I did what economists always do when something bothers us: we wrote a paper. So, what do we do in it? First, we check whether AI detectors such as Pangram get this right: is a referee report flagged as AI-generated indeed AI-generated? Or is Pangram just flagging proofreading and editing? No. We run an experiment with 25 referee reports from the pre-AI era. Proofreading and editing do not trigger AI-generated flags. Only when we ask the LLMs to substantially rewrite the referee report or to generate a new report do we get the AI-generated flags. Pretty compelling evidence, if I may say so. Second, we build a theoretical model to explore who should bear the burden of verification: the author or the reader? (Yours truly always enjoyed the courses on the rules of evidence in law school; yes, as Billy Wilder would say, nobody is perfect.) The answer is that it is much more efficient for the author to do it, because the author can verify at a much lower cost and only needs to do it once (rather than each reader repeatedly). Before AI, the authors more or less had to verify the results, even if only because they had to type them in. This was not perfect (and all papers ever published in economics probably have at least one mistake). But the problem is that, with AI, the author does not even need to read the text. Or can just skim it. Writing is thinking, and without writing there is usually not much thinking. So, a paper flagged as AI-generated is prima facie evidence that the author has not verified it. Should this lead to an automatic rejection? No. But there should be a zero-tolerance policy for serious mistakes. The first serious mistake we find in the paper, and it is out! In comparison, for a non-flagged paper, a serious mistake is something that the editor might determine it can be fixed in a revision. We are stricter with referee reports. There is absolutely no point in the referee generating a report using an LLM. As an editor, I can do that directly and save myself the trouble of asking you for it. Now, of course, you can use the LLM to help you with the report (if the journal allows it! Many journals do not allow you to upload the paper to an LLM, and you need to follow the rules). For example, you can use the LLM to search for existing results in the literature. But peer review is not about finding one sentence that is not entirely precise or a slightly sloppy mathematical proof. Referees are asked: “Is this idea important despite its flaws?” And, relatedly: “Is the paper a good fit for the journal in terms of quality and focus?” That judgment is the task for which you are selected as a subject expert. We also propose that journals require that, before a paper is submitted, the author submit it to an LLM using an approved prompt and include the LLM report and a discussion of what was and was not fixed. That way, the referee would not need to check (or at least not check so carefully) every technical detail, and they could focus on the big picture: Is this a good paper? Is this a good fit for this journal? What is fundamental for this paper to become publishable at this journal? Anyway. The paper has many other arguments, and we try to be nuanced. So, before ferociously typing the first complaint that crosses your mind, would you consider reading it? Chances are that we have already discussed your objection. We have been thinking about this for a long time. But by all means, leave good comments!
58
221
29
977
85,190
El Ministerio del Interior egipcio ha informado (en Facebook) de la incautación de un alijo de 3,000 objetos robador de época Greco-Romana en Dakahliya facebook.com/share/p/1F7Wj4A…
2
95
Daniel Riaño retweeted
Ah, que 7 misiles Ruta de Destinus impactaron en el puente de Kerch. Me pinchas y no sangro.
26
80
2
853
24,124
Daniel Riaño retweeted
The @cfranganillo interview should not be controversial. In a normal, healthy democracy tough questions are a fundamental part of ensuring those in power are held accountable.
3
19
1
85
2,400
Daniel Riaño retweeted
El 29N expira el plazo para que los académicos e intelectuales de España presenten por escrito su protesta ante la colonización política de las instituciones culturales, académicas y de investigación en España. Si solo lo hacen después, su reclamación no tiene valor.
1
1
85
El 29N expira el plazo para que los académicos e intelectuales de España presenten por escrito su protesta ante la colonización política de las instituciones culturales, académicas y de investigación en España. Si solo lo hacen después, su reclamación no tiene valor.
1
1
85
Pues si mucho no me equivoco, en este momento todos los servidores del CSIC están caídos o fuera de línea sin previo anuncio y sin explicación desde la institución. Esto incluye correo y cualquier servicio que he probado.
1
3
4
474
Ahora mismo ha vuelto a funcionar Digital CSIC. Otros servidores como el del Jardin Botánico funcionan perfectamente, creo
1
77
Gracias por tanto @MagfcoMargarito
10
62
11
255
48,736
Al hacer los rankings de universidades se puntúa tener profesores que ostentan distinciones como el Premio Nobel. ¿Cómo es posible que no cuente *negativamente* tener como contratado o emérito a una vergüenza profesional y académica como Tezanos?
3
184
Daniel Riaño retweeted
Otro excelente recurso: para quienes deseen realizar búsquedas sintácticas y semánticas en hebreo y griego, el "Nida Institute" ha desarrollado esta plataforma súper útil: search.nida.bible
5
32
1
124
3,878
Raras veces han funcionado tan bien la línea 4, 5 y 7 del metro de Madrid (las que usé ayer) como el día de ayer, la jornada de la huelga contra la oposición, en términos de frecuencia, limpieza y posibilidad de encontrar sitio (mi experiencia fue de 8 a 19)
248
Working on my hobby project, to see if I can make clean and useful facing-Greek/Latin editions of Greek texts from old books with AI and TeX. Unfortunately, I picked a Latin translation (apparently from Lorenzo Valla's) that's too dynamic. The Latin tells you what the Greek means, but not *how* it means it, which (IMHO) is the whole reason for a bilingual. For the Bible, this is mostly guaranteed. Not so much here.
7
4
2
58
3,903
Daniel Riaño retweeted
El Grupo de Estudios sobre Poder y Cultura en América (ÉPoCA) y CLARIAH-ES realizan la Jornada Interdisciplinar "Las Humanidades Digitales y el reto de la Inteligencia Artificial" en la U Sevilla Fac. de Filología (26-27 de noviembre), coordinado por Emilio Gallardo Saborido 👏👉
3
3
190
Daniel Riaño retweeted
List of 69 Greco-Roman books attested to exist (from legible fragments) among the carbonized papyri of Herculaneum. All of these works are currently lost to time, but some may be recovered in the future by the "digital unwrapping" and scanning of the papyri.
34
254
45
1,587
68,440
Daniel Riaño retweeted
Many thanks to Harald, Robert and Sebastian for their continued hard work on Glottolog – version 5.4 was just released: glottolog.org/. I'm glad to seethat this project, which I initiated many years ago (but have not really been involved in in recent years), is thriving.
1
18
66
2,115
Daniel Riaño retweeted
Emilio Gallardo Saborido (ÉPoCA) coordina dentro de CLARIAH-ES la Jornada Interdisciplinar (presencial y online) "Las Humanidades Digitales y el reto de la Inteligencia Artificial" en la U Sevilla, Fac. de Filología (26-27 de noviembre). cchs.csic.es/es/event/jornad…
2
3
147
El Grupo de Estudios sobre Poder y Cultura en América (ÉPoCA) y CLARIAH-ES realizan la Jornada Interdisciplinar "Las Humanidades Digitales y el reto de la Inteligencia Artificial" en la U Sevilla Fac. de Filología (26-27 de noviembre), coordinado por Emilio Gallardo Saborido 👏👉
3
3
190
Daniel Riaño retweeted
The claim that a democratic change in government in Spain would lead to members of the LGBTI community being "imprisoned in basements and shot in the back of the head" is absolutely obscene. It also speaks very poorly of @eldiarioes that they publish this inflammatory nonsense.
El Sindicato de Inquilinas alerta de qué pasaría si gobierna la extrema derecha: "Compañeras acabarían deportadas o con un tiro en la nuca" eldiario.es/economia/alicia-…
18
113
2
449
10,616
Mirad qué maravilla. El Parque Arqueológico de Herculano ha reabierto, varias décadas después y tras ser restauradas, dos de sus joyas: la "Domus del Atrio de Mosaico" y la "del Entramado de Madera". Dos formas de vivir hace 2.000 años. Vámonos de visita a disfrutarlo 🧵 👇
28
520
21
2,069
72,494