What's the best path here ?
Just closed a six-figure deal for our AI-native ERP. €150k. First enterprise contract.
Here's the part nobody prepares you for: two thirds of the contract is services. Implementation, training, stitching with legacy systems. €100k of work that has nothing to do with building the product.
The founder dilemma:
Do it in-house — cash in, reduce burn, delay the next raise. But your engineering team is now doing consulting instead of shipping product.
Subcontract it out — stay focused on core product, ship faster, keep the roadmap clean. But you give away most of the margin and lose control of the customer experience.
Genuinely torn on this one. What would you do?
The methodological choice in this paper from the university of Munich released yesterday, should rewire how you evaluate LLMs in production: the surface fidelity gap.
It measures the distance between how on-task a model sounds and whether it actually respected the brief. Llama-70B has the largest gap. +0.80. The output reads aligned. The constraints are gone.
The knows-but-violates rate measures non-compliance despite the model correctly recalling the rule. It ranges from 8% to 99% across seven models. Structured checkpointing reduces it. It does not close the gap. For anyone designing business processes on top of LLMs, the lesson is methodological. Reading the output is not evaluation.
Asking the model 'what were the rules?' is not evaluation. Evaluation is checking the output against a structured brief with binary-decidable constraints. That was the only mechanism the paper found that reliably catches violations. If your QA layer is a human skim or a model self-check, you are measuring how aligned the output sounds. Not whether it complied.
Replying to @levelsio
The real question is who's watching what they shipped while you slept
Replying to @dan__rosenthal
Layer 5 is the one that matters most and the one everyone will skip. We built an entire quality pipeline for the same reason — agents produce at 10x speed but nobody catches the complexity debt they leave behind until it breaks in production
Replying to @PawelHuryn
The 10-20x is real. What nobody mentions is that at 10x speed you also accumulate debt 10x faster. The system that compounds isn't the agent — it's the one watching the agent
Replying to @antirez
11k system prompt is prompt obesity. Same disease as the 2000-line God class, just a different layer. We'll end up building linters for prompts the way we built them for code
Replying to @businessbarista
The moat isn't the tech anymore — it's the boring stuff nobody wants to build. Tax compliance, e-invoicing mandates, labor law across jurisdictions. AI made building easy. Regulation made it defensible
Replying to @NateMatherson
Managing agents is still managing. We had to build an entire quality pipeline just to catch what our AI sessions were shipping too fast. Agents don't need 1:1s — they need instrumentation
Replying to @Yoroomie
Can confirm. Building a modular business OS with a partner marketplace — every day is a different fire in a different module in a different country
Replying to @james406
Still not enough context to fit a single enterprise client's "small list of changes"
Replying to @dunkhippo33
The hardest version of this is when the shiny object is revenue. A big services contract lands and suddenly you're choosing between cash now and product later. Staying the course means saying no to money that's already on the table
Replying to @itsalexvacca
The services vs software tension is real from the other side too. We just closed a six-figure ERP deal — two thirds of it is implementation services. The temptation is to do it all in-house for the cash. But every hour your engineers spend on services is an hour they're not shipping product. The hard part isn't choosing — it's admitting which business you're actually building
Replying to @gillianxobrien
The best companies don't fit a category — they create one. VCs who need you to fit a box are telling you more about their limits than yours


