
Product Lead, Spanish AI Gigafactory @Telefonica | Co-founder & CEO @Kairos_Tek | Lecturer @IEuniversity, @Afi_es, UCM, UPM and UCLM | AI speaker & communicator
European Union
Joined April 2011
- Tweets44K
- Following85
- Followers5.9K
- Likes15.5K
Most teams check their AI agents by asking another LLM to grade them.
The catch: that grader is slow, expensive, and can give the same answer two different grades.
@adlrocha tested a cheaper alternative. Here’s what he found.
At Baselight, the startup he co-founded, grading was so costly they couldn’t run it on every code change. Only on some changes, plus a nightly run.
Worse: when a test failed, they couldn’t tell if the agent got worse or the grader changed its mind. Finding out meant paying for the evaluation again.
The alternative: decision models.
They don’t write. They pick an option (yes/no, A/B) and say how confident they are. So you rewrite your evaluation as closed questions, like “Is the answer based on the data the agent found?”
The numbers:
- LangChain: matched the human reviewer every time. $0.34 vs $28.17 with a regular LLM
- Alfonso’s test on code reviewers: same verdict as the LLM grader 9 times out of 10, 8.6x faster, half the price
His take: decision models for anything you can ask as yes/no. LLM graders for anything that needs an explanation.
And a warning: a grader that always gives the same grade isn’t necessarily right. In the 2 case out of 10 where they disagree, who’s right?
We debate questions like this every month at AI Socratic Madrid. Join us 👉 luma.com/aimadrid
LLM-as-a-judge always pushed my agent evals out of CI: too slow, too expensive, and too noisy. At Baselight a red test could mean the agent got worse or the judge changed its mind, and the only way to know was to pay for another run.
In part 2 of the decision models series I replaced the LLM judge in a code-review eval with OpenAI's Decisions API:
* 90.7% agreement with the generative judge
* 186 ms per verdict, 8.6x faster
* About half the cost (not the 80x LangChain saw, and I explain why)
What changes when judging is nearly free: you run the whole suite on every commit, judged evals start behaving like ordinary tests, and the model's confidence becomes a signal of how flaky your eval is.
I also cover where I'd still pay for an LLM judge.
Link to post: adlrocha.substack.com/p/adlr…
Every time one of my agents needs a yes-or-no, I pay an LLM to think for a few seconds and get back an answer with no sense of how sure it is. Decision models like Jev and Cloudflare's open Clef answer the same question in milliseconds, without writing a word, and put a confidence next to every option.
My bet: decision models are a new primitive for the AI stack, like embeddings were. First of a three-post series.
adlrocha.substack.com/p/adlr…
Jorge Ordovás retweeted
¿Cámaras o biometría en exámenes online? Examino los límites del proctoring universitario: la biometría y la supervisión invasiva exigen base legal sólida y alternativas libres. pablofb.com/2025/10/09/puede…
Jorge Ordovás retweeted
¿Es legal usar la huella dactilar para acceder a gimnasios, bibliotecas o empresas?
Analizo la ilicitud del control de acceso por huella según el RGPD y la AEPD, recordando que es obligatorio ofrecer alternativas no biométricas para evitar sanciones.
pablofb.com/2025/10/08/es-le…
En este artículo cuento lo que aprendimos en @Kairos_Tek al probar distintos modelos abiertos (open source) con tres casos de uso reales de una gestoría.
Y por qué la elección final no depende de los rankings públicos de estos modelos, sino de las pruebas que construimos específicamente con el trabajo de nuestro cliente, que no tienen por qué dar el mismo resultado.
No hablo de teoría, sino de una experiencia real que ha permitido a una gestoría de 20 personas integrar, por fin, la inteligencia artificial en su trabajo del día a día sin sacrificar calidad ni comprometer la privacidad de la información que gestiona.
En Moneyball (película que siempre recomiendo) Peter Brand convence a Billy Beane de que el objetivo de un equipo no es fichar jugadores, sino comprar victorias.
Con los modelos de IA pasa exactamente lo mismo: no compras un modelo, compras aciertos sobre tu trabajo. Y los aciertos solo se ven midiendo lo que necesitas saber para decidir.
Si quieres saber cómo elegir el modelo que de verdad necesitas para tu trabajo, te lo cuento en este artículo de la newsletter "Entre gigafactorIA y gestorIA".
linkedin.com/pulse/abiertos-…
I've earned my #ClaudeCertified Associate credential at the Foundations level.
This credential validates the skills to apply Claude to real work: prompting, evaluating output, integrating Claude into workflows, and using it responsibly.
credly.com/badges/893c8d17-b…
Desde que el Ómnibus Digital de IA entró en vigor a finales de julio, la pregunta que más me han hecho relacionada con el AI Act desde las PYMEs con las que tratamos en @Kairos_Tek (gestorías, despachos o agencias de menos de veinte personas) llega con un tono de alivio que se nota claramente:
"He leído que el AI Act se ha aplazado a diciembre de 2027, entonces ya no me toca hacer nada todavía, ¿verdad?".
Y la respuesta suele ser siempre la misma: lo que se aplazó casi con total seguridad no era tuyo, y lo que sí entró en vigor el 2 de agosto no te pide lo que crees que te pide.
El Ómnibus no suavizó el reglamento para las empresas pequeñas. Movió las obligaciones caras, que afectan a muy pocas empresas, y dejó intactas las baratas, que afectan a casi todas. Y la parte que se mantiene implica dos puntos de vista diferentes: el de quien fabrica el sistema y el de quien lo usa, que casi todo lo que he leído estas semanas mezcla en uno solo.
Ahí es donde se está tomando mal la decisión, y si tienes una PYME, eso es lo que tienes que revisar (no te preocupes, pero ocúpate).
Te lo cuento en esta nueva publicación de la newsletter "Entre gigafactorIA y gestorIA".
Qué cambió con el AI Act, y cómo afecta a una PYME.
Qué cambió con el AI Act, y cómo afecta a una PYME
linkedin.com/pulse/qu%C3%A9-…
Jorge Ordovás retweeted
En Santander AI Lab estamos ampliamos equipo en Madrid. Buscamos Frontier AI Engineers y AI Researchers para trabajar en los retos más ambiciosos de la IA aplicada a la banca, con mucha autonomía y capacidad de ir más allá del estado del arte actual. Info: linkedin.com/posts/delachica…
Pensé "para clonar un repo me bastará con Haiku"...
Resultado: rm -rf de *TODO* lo que tenía hecho con Claude.
Se ha disculpado, eso sí:
"Me disculpo profundamente. He cometido un error crítico.
El problema es que asumí que ~/claude solo contenía el repositorio anterior, pero claramente tenías otros proyectos/datos ahí.
El rm -rf ~/claude limpió TODO el directorio, no solo lo que querías actualizar."
Por eso uso github. Moraleja: NO trabajar con Haiku, sobre todo cuando te duele la cabeza.
Jorge Ordovás retweeted
"La IA no es el problema"
Me encanta el titular que le han puesto a esta entrevista que nos hizo @ignacio_arriaga
Mi opinión sobre la teoría de que la IA acabará con la consultoría se resume en que es errónea porque parte de una premisa falsa: que existe una sola respuesta correcta para un problema.
youtube.com/watch?v=Adzis1LJ…
I really hope people have caught the @ENTERSHIKARI-themed Easter egg at the end of the post.
I'm calling 26th of August AI Independence Day.
DeepSeekV4-Flash was the first frontier-adjacent model you could squeeze onto a Spark. Now Qwen3.8-Flash-Next fits in under $5K of hardware, and GLM5.3 fits with some pruning.
The intuition I've been trying to communicate for a while is that total size decides whether you can run it, while active size decides whether it's pleasant to use. Both numbers are finally landing in a good place at the same time.
Which means: unlimited tokens at home.
In this week's posts I chat about Flash models, and ox-alpha, the undercover model that was doxxed:
adlrocha.substack.com/p/adlr…
Trabajando con Claude en las pruebas de... algo que estamos haciendo en @Kairos_Tek 🤫 creo que le ha dado un "glitch" que nunca había visto.
Cito literal su respuesta:
"Mi script siguió a make smoke de todos modos porque until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until until — perdona, me repito"
@adlrocha puede ser esto una señal de AGI quizá?
GLM-5.3 and Qwen3.8-27B shipped on the same day.
Neither of them changed a single line of architecture.
Both of them got significantly better. Do you want to know why?
Weekly post freshly out of the oven describing what post-training can do to your models and why the base model has stopped being the bottleneck: adlrocha.substack.com/p/adlr…
Me molesta cuando escucho que las PYMEs van por detrás en la adopción de la IA.
No van por detrás en información, que hoy es prácticamente gratis. Van por detrás en criterio, que es lo único que sigue siendo más caro que los tokens.
Lo digo desde una posición peculiar. Durante la mayor parte de mi jornada soy Product Lead de la Gigafactoría de IA española en Telefónica, uno de los consorcios que se presenta a la convocatoria europea para garantizar la soberanía sobre la infraestructura de IA. El resto de mi tiempo (que en la práctica son muchas noches y fines de semana) soy cofundador y CEO de @Kairos_Tek donde trabajamos con gestorías, bufetes, agencias y empresas de logística de menos de 200 personas.
Dos mundos que no se parecen en nada. Y sin embargo, las conversaciones sobre IA de un lado y del otro se parecen muchísimo. Cambian las cifras, los plazos y el número de personas a las que hay que convencer. No cambian las preguntas.
Un dato lo ilustra bien. Según el informe de Citadel Securities del 18 de agosto, el precio de los tokens ha caído un 40 % desde finales de junio. En julio, el gasto en IA por empleado subió un 49 % mes a mes en el 1 % de empresas más avanzadas, un 25 % en el siguiente diez por ciento y solo un 9 % en la mediana.
La tecnología se abarata para todo el mundo por igual. La capacidad de aprovecharla, no.
Hoy lanzo "Entre gigafactorIA y gestorIA", una newsletter quincenal donde voy a contar lo que veo desde los dos extremos. No vas a encontrar resúmenes de las herramientas de la semana ni predicciones, porque tengo suficientes años como para haber visto fallar demasiadas veces no solo las de los demás, sino las mías propias. Va a tener un ritmo más lento y va a ser menos espectacular que otras que sigues.
La primera edición ya está publicada. La segunda va sobre el AI Act. Sobre lo que ha cambiado de verdad y lo que no.
Las piezas caen a la misma velocidad para una gigafactoría que para una gestoría de veinte personas. Lo único que decides es dónde las colocas.
Esta es la primera publicación. Si te gusta, suscríbete y te espero en quince días.
linkedin.com/pulse/los-dos-e…
NVIDIA's $12.9 billion acquisition of Hugging Face may be far more significant than it first appears.
It's not simply a matter of NVIDIA wanting to add another AI company to its portfolio. It's a strategic move about where the true center of gravity of the next phase of AI might lie.
For the past few years, the market seemed relatively clear: the large commercial models of OpenAI, Anthropic, and Google defined the technological frontier and concentrated a significant portion of the value. But something has been changing in recent months.
Open source models are rapidly closing the gap with commercial ones. They are becoming increasingly powerful, efficient, and, above all, allow companies and developers to adapt, deploy, and control them in ways that closed models don't always permit.
And NVIDIA seems to have recognized something fundamental here. Hugging Face is not simply a repository of models; it has become one of the major layers of distribution and collaboration within the AI ecosystem: models, datasets, tools, communities, and developers.
Controlling that layer can be as strategic as controlling the hardware on which the AI runs.
The operation would also have a particularly interesting logic for NVIDIA: the more diverse and competitive the open model ecosystem, the less dependent the market will be on a small group of closed model vendors who, moreover, have incentives to develop their own chips and reduce their dependence on NVIDIA.
And the data shows that the open ecosystem is no longer a marginal alternative. Hugging Face registered nearly 3 million public model repositories this summer, while NVIDIA and AMD are among the organizations that have published the most new models in 2026.
The connection between open models and hardware strategy is becoming increasingly clear: a good model optimized for your infrastructure is also an extraordinarily effective way to sell hardware.
An example of a recurring successful strategy, which has an entire chapter dedicated to it in the book "Good Strategy, Bad Strategy," discusses how NVIDIA went from being a GPU company for video games to building CUDA when the accelerated computing market was still much smaller. Then it turned that investment into a massive competitive advantage for AI. And now, having clearly dominated the infrastructure layer, it seems determined to strengthen its position in the software, modeling, and community layers as well.
It's not content with just selling shovels during the gold rush. It wants to be on the roads, in the warehouses, and in the markets where that gold is distributed. Watch the market for the consequences of this move.
Link to the article in the comments 👇
I wanted to understand what actually changed in Kimi K3 to make it so good in plain English.
Turns out it comes down to one idea: give the model a notebook instead of a transcript, and finally fix the one direction transformers have always been lazy about: depth, not sequence.
It's also the first release that got me off my Claude subscription entirely, for the first time since I started using open models daily. That's a bigger deal to me than any benchmark.
Post here, enjoy: adlrocha.substack.com/p/adlr…
Good news: Anthropic is permanently maintaining the prices of Sonnet 5, which offers more than many SMEs need for their daily use (comparable to Opus 4.8, which is five times more expensive).
For these companies, predictable and stable costs are fundamental when adopting AI today. Because the question is no longer whether artificial intelligence can help you, but how much it will cost to implement it and whether the ROI is sustainable as you scale.
At @Kairos_Tek, we're dedicated to this: helping them implement the AI they need to be efficient, secure, and reliable. Nothing more, nothing less.
Europe just launched the AI Gigafactories call tender. Up to 7 facilities, €10B public, €20B private expected, 18 Member States on board.
I lead software for the Spanish AI Gigafactory, so I am not a neutral observer here.
But €30B is less than 5% the investment of combined 2026 capex for Amazon, Microsoft, Google and Meta. And you have to take into account also China...
This is not about how much money we spend. This is about sovereign AI, who gets to use the compute we build: startups, SMEs, universities, public bodies. Under EU rules, on EU infrastructure.
Different race. Worth reading it that way 👇
🤖 Made with AI
















