RAG From Scratch: Multi-Representation Indexing Our RAG From Scratch video series walks through impt RAG concepts in short / focused videos w/ code. This is the 12th video in our series and focuses on some useful tricks for indexing full documents. 🔧 Problem: Many RAG approaches focus on splitting documents into chunks and retrieving some number based on similarity to an input question for the LLM. But chunk size and chunk number can be difficult to set and affect results if they do not provide full context for the LLM to answer a question. 💡Idea: Proposition indexing (@tomchen0 et al) is a nice paper that uses an LLM to produce document summaries ("propositions") that are optimized for retrieval. We've built on this with two retrievers. (1) multi-vector retriever embeds summaries, but returns full documents to the LLM. (2) parent-doc retriever embeds chunks but also returns full documents to the LLM. The idea is to get best of both worlds: use concise representations (summaries or chunks) that are well-suited to retrieval, but link them to documents, which have full context for generation. The approach is general, and can also be applied to tables or images: create / index a summary, but return the raw table or raw image for LLM generation. This gets around challenges w/ directly embedding tables (they can be split by poorly tuned chunk size) or images (need multi-modal embeddings), using a summary as a representation for text-based similarity search. 📽️ Video: youtu.be/gTCU9I6QqCE 💻 Code: github.com/langchain-ai/rag-… 🧠 References: 1/ Proposition indexing: arxiv.org/pdf/2312.06648.pdf 2/ Multi-vector: python.langchain.com/docs/mo… 3/ Parent-document: python.langchain.com/docs/mo… 4/ Blog applying this to tables: blog.langchain.dev/semi-stru… 5/ Blog applying this to images w/ eval: blog.langchain.dev/multi-mod…