@ipapapa

Engineering @Google | ex-industry: @SnowflakeDB @Netflix @IBM | ex-academia: @Purdue @NCState

 California, USA
ipapapa.github.io/
Joined March 2010
  • Tweets1.5K
  • Following667
  • Followers611
80Media
Ioannis Papapanagiotou retweeted
Ruslan Meshenberg@rusmeshenbergMay 27
An overview of how we're integrating AI in the reliability tech and practices - sre.google/resources/practic… credit: @ipapapa
3
1
11
922
Benchmarked: H100 vs Gemini Flash. Result: H100 choked on prefill. Gemini flew. Without complex disaggregation, a single H100 is just an expensive heater. ioannisp.medium.com/the-real…

The Real Cost of Self-Hosted RAG: Benchmarking CPU vs. H100 vs. Gemini 3.0 Flash

In the world of Enterprise AI, the “Self-Hosted vs. Managed” debate is often driven by ideology rather than engineering data. Teams assume…

ioannisp.medium.com
2
215
Choosing a text splitter for RAG? 📏 Character: Splits by count (e.g. 2000 chars) → Simple, predictable → Cuts mid-sentence, mid-table 🔢 Token (TikToken): Splits by LLM tokens → Better for context planning → Still arbitrary cuts
1
127
📑 Markdown/Header: Splits by structure → Respects semantic boundaries → Uneven chunk sizes - which affects retrieval consistency and costs. All splitters have flaws - you're just choosing which flaw to accept.
84
I've been recommending RAG to save costs. Time to reconsider. Testing Gemini Flash 3 with 22 technical manuals + implicit caching (75% off repeated prefixes): Full documents as system prompt became: → Cost-competitive with RAG → Higher quality responses
2
1
141
more replies
The trade-off isn't cost vs quality anymore. It's: • Cold start cost (Full Docs higher) • Query volume (Full Docs wins at scale) • Context stability (Full Docs needs consistent prefix) RAG retrieves fragments. Full Docs preserves relationships.
1
76
For technical support chatbots with stable system prompts? Full Documents + caching might be the better architecture. Caching changes the ROI calculation completely.
70
Running a high-performance multilingual reranker on a 1 vCPU instance is a recipe for latency. The local ms-marco model was "good enough," but I needed "perfect" handling of foreign technical terminology without crashing the server.
1
83
more replies
The new reranker is so effective it retrieved too many similar manuals. I countered this with a System Prompt "Phase Filter"—explicitly instructing the LLM to verify requirements against the retrieved specs.
1
62
Retrieval: If you are dealing with non-English technical RAG, don't struggle with generic local models. The specialized multilingual APIs are worth every millisecond of latency saved.⚡️ #AI #RAG
63
Just spent the weekend debugging a high-precision RAG pipeline for technical manuals. 🛠️
1
72
more replies
The LLM kept "lazy-loading" generic definitions from general category files instead of reading the specific manuals. I injected a "Critical Instruction" into the System Prompt. Result: It now ignores the easy answer and digs for the specific brands.
1
64
Model: Gemini 3.0 Pro (Preview) Embeddings: gemini-embedding-001 Reranker: ms-marco-MiniLM (Local) The accuracy difference is night and day. If you aren't reranking, you aren't really searching. 🚀 #AI #RAG #Engineering #Gemini #LLMs
83
This work on Program of Thoughts (PoT) presents a compelling method for enhancing LLM performance on numerical reasoning tasks. By having the model generate a program to be executed, it effectively disentangles the reasoning and computational steps. arxiv.org/abs/2211.12588

Program of Thoughts Prompting: Disentangling Computation from...

Recently, there has been significant progress in teaching language models to perform step-by-step reasoning to solve complex numerical reasoning tasks. Chain-of-thoughts prompting (CoT) is by far...

arxiv.org
53
Load more