This blog post was inspired by Cole Medin's excellent Youtbube video titled: very RAG Strategy Explained in 13 Minutes (No Fluff) which can be found here.
In the world of AI, one of the most exciting techniques today is Retrieval-Augmented Generation, or RAG for short. Simply put, RAG means combining a large language model (LLM) with a knowledge base (like your documents, files, articles) so that the AI doesn’t only rely on what it “knows” from training but can look something up and use that information in its answers. In his video, Cole Medin walks through 11 different strategies you can apply to build or improve a RAG system. These are ways of tweaking how you store data, how you ask questions, and how you pull relevant context, all in order to give the best possible answer. Let’s break them down at a high level.
The Basics: What RAG Workflows Look Like
Before diving into the strategies, here’s the typical workflow of a RAG system:
- Data Preparation – You take your documents (reports, transcripts, knowledge bases), break them into smaller chunks, create embeddings (vectors) and store them in a searchable database (often a vector database).
- Retrieval + Generation – When a user asks a question, you embed that query, search the vector database to find the most relevant chunks, then pass those chunks (as extra context) into your LLM. The LLM uses that retrieved info plus its general knowledge to generate a response.
These two phases are common to all RAG systems. What varies is how you do each step. That’s where the 11 strategies come in.
The 11 RAG Strategies
Here are the strategies, described in simple terms so you can get a feel for what each one is and when you might use it.
1. Re-ranking
In this strategy you do a two-step retrieval: first you pull many candidate chunks from the database, then you use a second, more precise model (like a cross-encoder) to re-rank them and pick the best few. This helps make sure the LLM gets only the most relevant context, instead of dozens of semi-relevant ones. Use this when you have lots of document chunks and need to filter for quality.
2. Agentic RAG (Hybrid Search)
Here, the AI agent has the flexibility to choose how it searches. It might do a classic semantic search, but it could also read the full text of a document if needed. It combines multiple search types. This is useful when your documents vary in format or you expect the agent to decide “should I search broadly or dive deep?” depending on the question.
3. Knowledge Graphs
Instead of only using vector‐similarity search, you build a knowledge graph: a structure of entities (people, concepts, items) and their relationships. Then your retrieval can ask relational questions (“Which products are made by which supplier and sold in which market?”) rather than just “which text is similar to my query?” Great when your domain is richly connected (legal, biology, supply chain) and relationships matter.
4. Contextual Retrieval
In this approach, you don’t just store chunks passively. You enrich each chunk by adding information that tells how it fits into its larger document (for example: “this section is the conclusion of a report about workflow optimization”). Then during search you use that extra context so the retrieval is smarter. Use this if your documents have structure and you want smarter chunk/context alignment.
5. Query Expansion
Here you take the user’s query and—before you search—use an LLM to expand or reformulate it (add more detail, clarify intent) so you can catch more relevant chunks. Then you search using that improved query. Useful when users ask in vague ways and you want to increase precision.
6. Multi-Query RAG
This is a variant of query expansion: instead of one reformulated query, you generate multiple different queries (via an LLM) and run them in parallel searches. You then gather results from all of them. Good when you want more coverage and are okay with extra compute/cost.
7. Context-Aware Chunking
This strategy goes back to data preparation. Instead of just chopping documents every X characters or pages, you try to split them intelligently, keeping document boundaries or logical sections intact. This ensures chunks make sense and embeddings are more accurate. Use this when your document structure is important and you want the chunks to retain cohesive meaning.
8. Late Chunking
A more advanced preparation method: here you apply embedding to the full text first, and only then you chunk (so each chunk retains broader embedding context). The idea is to preserve more of the document’s full context in each chunk. It’s more complex and expensive, but can help in domains where context really matters (e.g., long reports, narratives).
9. Hierarchical RAG (Using Metadata)
In this method you build layers: chunks reference parent/child relationships (for example: paragraph → section → document). You might search at the small chunk level for precision, then pull the entire parent document for more context, depending on what you find. Good when you want both precise retrieval and the option to get larger context if needed.
10. Self-Reflective RAG
This one introduces a “feedback loop.” After you do your initial retrieval and generation, you have the LLM evaluate the result (e.g., rate whether the answer is good). If it’s not high enough quality, you go back, refine retrieval or search again and try again. Useful when you want to raise the bar on answer quality automatically, though it increases complexity and cost.
11. Fine-Tuned Embeddings
Finally: instead of using a generic embedding model for all your documents and queries, you fine-tune or adapt an embedding model to your specific domain (say, legal contracts, medical texts, sentiment‐rich user feedback). This often yields better similarity matching for your particular use case. This is ideal when you have a lot of domain‐specific text and care a lot about high relevance.
Which Strategies Should You Use?
If you’re just getting started, you don’t need all 11 at once. In the video, Cole suggests that a solid practical starting point is to pick three strategies and combine them. For example:
- Re-ranking (to ensure relevant chunks)
- Agentic RAG (to give flexible search options)
- Context-Aware Chunking (to store your data smartly in the first place)
From there, you can layer in others as needed (for example, if your domain is highly relational you might add knowledge graphs; if quality matters a lot you might add self-reflective RAG; if you have domain-specific data you might fine-tune embeddings).
Final Thoughts
RAG unlocks big potential: by combining your knowledge base with a powerful LLM, you can build smarter AI agents that know your data and use it. The key is not just to pick RAG but to pick how you’re going to implement it. The 11 strategies above give you a menu of options — pick the ones that match your data, your domain, and your performance vs cost trade-offs. If you’re curious and want to go deeper, I highly recommend watching Cole Medin’s video (linked above) which walks through each strategy with clarity and examples.