so Trieve was really a bad idea because it's mostly true that xgboost is all you need big enterprises who spend a lot on search are primarily spending money on staff and data pipelines to tune xgboost for ranking a top-k set of results that comes back from their search engine db sparse/dense/multi vector(s) scores are ultimately just data points which go into the xgboost model that decides on ranking (this is called "learning to rank" if you want to google) it is important to have as many data points as you can get. most companies will add things like # of sales, likes, etc. tooling for this isn't great, the elastic/solr plugins have outdated documentation and data engineering with LLMs is still way too intensive startups don't use xgboost for their "optimized retrieval pipelines" and that's a travesty someone out there needs to work on making it better/easier

Cheating at Search with LLMs We've been doing this thing for a while at Trieve that we've been calling "cheating at Search with LLMs" and I thought it'd be cool to talk about it. The Problem: Smart Product Comparisons For our Gen AI sales associate Shopify app, we wanted to make it possible to do cool things like generate a comparison table for any two products. Take this example from the brand LifeStraw, which sells filterable straws. If a customer asks to "compare the Sip against the LifeStraw" (two different products in their portfolio), we need to quickly look inside their catalog to determine which two products to fetch. The challenge? No traditional keyword, semantic, or hybrid search would ever be intelligent enough without an LLM to understand the exact two products being discussed. Our Solution: Let the LLM Do the Hard Work So we cheat. Here's how it works: 1. First, we do a standard search with the user's query and get the top 20 results. Each group represents a product, and each chunk within that group is a variant of that product (like different colors or pack sizes). 2. Then we use a tool called "determine relevance" that asks the LLM to rank each product as high, medium, or low relevance to the query. We pass each product's JSON, HTML, description text, and title to the LLM. 3. The LLM examines each product and makes the call. For example, it might mark the LifeStraw Sip Cotton Candy variant as "high" relevance, and the regular Life Straw as "high" relevance, while everything else gets "medium" or "low." 4. We then use these relevance rankings to display only the most relevant products to the user. Making It Fast Despite making 20+ LLM calls in the background, the experience feels instantaneous to the user thanks to semantic caching on all the tool calls. If I run the same comparison again, it's blazing fast. Going Even Further We extend this approach to other aspects of search: - Price Filters: We have a tool call that extracts min and max price parameter - Category Determination: For stores with predefined categories, we use LLMs to determine the right category - Format Selection: We use tool calls to decide whether to generate text or images - Context Retention: If a user follows up with "tell me more about the Life Straw's filtration," we don't need to search again - we just use the same products from before Why This Matters It literally feels like cheating, which is incredible. In the early days, we spent a ton of time building super high-relevance search pipelines. But with modern LLMs, that's unnecessary. You can just fetch 20 things, give the LLM the query and each fetched item, and ask it which ones are relevant. Absolute madness. Intelligence as a commodity.