Prompt Engineering (L1): Just ask – no extra data, no training
RAG (L1-L2): Retrieve relevant documents, add to prompt context
Fine-Tuning (L3): Train the model on your specific data
Agents (L3-L4): Let AI use tools (search, APIs, code execution)
Pattern
When to use
Knowledge update
Cost
Prompt Engineering
Simple tasks, general knowledge
Static (model cutoff)
Lowest
RAG
Need current data, proprietary docs
Real-time (as docs update)
Low-Medium
Fine-Tuning
Need consistent style/behavior, large dataset
Static (until re-trained)
Medium-High
Agents
Need actions (search, API calls, computation)
Dynamic (via tools)
Variable
📚 RAG (Retrieval-Augmented Generation)
📌 What is RAG?
RAG combines a retrieval system (search for relevant documents) with an LLM (generate answer using those documents). This allows the LLM to answer questions about proprietary or up-to-date information not in its training data.
🔄 RAG Pipeline
User Query: "What is our company's vacation policy?"
↓
1. CONVERT query to embedding (vector)
↓
2. SEARCH vector database for similar documents
↓
3. RETRIEVE top-k relevant chunks
↓
4. AUGMENT prompt with retrieved context
↓
5. LLM generates answer based on context
↓
Response: "According to the employee handbook (page 12), employees get 20 days of PTO..."
📌 Key Components of RAG
Embeddings: Numerical representations of text (vectors)
Vector Database: Stores embeddings for fast similarity search (Pinecone, Weaviate, Chroma, Qdrant)
Chunking: Splitting documents into manageable pieces
Retrieval Strategy: How many chunks? Similarity threshold? Reranking?
Python RAG Example (using Chroma + OpenAI):
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI
# Load documents
documents = load_your_documents()
# Create vector store
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(documents, embeddings)
# Create retrieval QA chain
qa_chain = RetrievalQA.from_chain_type(
llm=ChatOpenAI(model="gpt-4"),
retriever=vectorstore.as_retriever(search_kwargs={"k": 4})
)
# Query
answer = qa_chain.run("What is our vacation policy?")
🎓 Fine-Tuning
📌 What is Fine-Tuning?
Fine-tuning takes a pre-trained model and continues training it on your specific dataset. This changes the model's weights to specialize in your task, domain, or style.
📌 When to Fine-Tune (vs. RAG)
Fine-Tuning wins when:
You need consistent style/tone (e.g., brand voice, legal writing)
You have thousands of labeled examples
You want to reduce prompt length/cost
The knowledge is static and core to the domain
RAG wins when:
Information changes frequently
You have a large corpus of documents
You need citations/sources
You have few training examples
Fine-Tuning Example (OpenAI API):
# Prepare training data (JSONL format)
{
"messages": [
{"role": "system", "content": "You are a customer support agent..."},
{"role": "user", "content": "My order hasn't arrived."},
{"role": "assistant", "content": "I'm sorry to hear that. Can you provide your order number?"}
]
}
# Upload file and create fine-tuning job
openai.File.create(file=open("training.jsonl"), purpose="fine-tune")
openai.FineTuningJob.create(training_file=file.id, model="gpt-3.5-turbo")
🔢 Embeddings & Vector Databases
📌 Embeddings 101
Embeddings convert text into a list of numbers (vector) that captures semantic meaning. Similar texts have similar vectors.
Concept
Explanation
Dimension
Number of features in the vector (e.g., 1536 for OpenAI embeddings)
Cosine Similarity
Measure of similarity between two vectors (-1 to 1, higher = more similar)
Vector Database
Optimized for storing embeddings and fast similarity search
Creating Embeddings (OpenAI):
from openai import OpenAI
client = OpenAI()
response = client.embeddings.create(
model="text-embedding-3-small",
input="Your text string goes here"
)
embedding = response.data[0].embedding # vector of length 1536
📌 Popular Vector Databases
Pinecone – Managed cloud service, fast, easy to start
Chroma – Lightweight, embedded, great for prototyping
pgvector – Vector search inside PostgreSQL
🤖 Agents & Tool Use
📌 What are AI Agents?
Agents combine LLMs with tools (APIs, search, calculators, code execution). The LLM decides which tool to use, in what order, and how to combine results.
Simple Agent Flow:
User: "What's the weather in Paris and should I pack an umbrella?"
↓
LLM decides: Need weather data → calls weather API
↓
Tool: Weather API returns: "Rainy, 15°C"
↓
LLM generates: "It's rainy and 15°C in Paris. Yes, you should pack an umbrella!"
📌 Agent Frameworks
LangChain – Most popular, extensive tool integrations
AutoGPT – Autonomous agents for multi-step tasks
BabyAGI – Simple task-driven autonomous agent
Semantic Kernel – Microsoft's framework for AI orchestration
🔀 Choosing the Right Pattern
💡 Decision Flowchart (Mental Model)
Do you need up-to-date or proprietary info? → RAG
Do you need to take actions (search, API calls)? → Agents
Do you have thousands of examples and need consistent style? → Fine-Tuning
Otherwise → Prompt Engineering (try this first!)
📌 Hybrid Approaches (Real World)
RAG + Agents: Agent retrieves documents, then executes actions based on them
Fine-Tuned + RAG: Fine-tune for style/format, RAG for factual retrieval
Multi-Agent Systems: Different agents with different roles (planner, researcher, writer)
✍️ Exercises
Exercise 1.04.1 – Pattern selection
For each scenario, choose the best pattern (Prompt Engineering, RAG, Fine-Tuning, or Agents):
A customer support chatbot that needs to answer questions from a constantly updated FAQ document
An AI that writes email replies in your company's consistent, friendly brand voice using 5000 past emails
An assistant that needs to book flights, check calendars, and send emails based on user requests
A general-purpose explanation tool for programming concepts (no external data needed)
Answers:
1. RAG (constantly updated FAQ → needs retrieval)
2. Fine-Tuning (consistent style, large dataset of examples)
3. Agents (needs to take actions: book flights, check calendars)
4. Prompt Engineering (general knowledge, no external data needed)
Exercise 1.04.2 – RAG terminology
Match each RAG term to its definition:
a) Embedding
b) Vector Database
c) Chunking
d) Similarity Search
Definitions: 1) Splitting documents into smaller pieces; 2) Numerical representation of text; 3) Finding vectors close to a query vector; 4) Storage optimized for embedding retrieval
Answers:
a-2, b-4, c-1, d-3
Exercise 1.04.3 – Design an integration
You're building an internal AI tool for employees to ask questions about company policy documents (HR manual, IT security, expense policy). Documents are updated monthly. Design the integration approach and explain why.
Sample Design:
- Use RAG (documents updated monthly → retrieval needed)
- Process: Convert policies to embeddings on update → Store in vector database → Query: embed question, retrieve relevant chunks → Augment prompt with chunks → LLM generates answer with citation
- Add metadata filtering (e.g., only retrieve IT security when question is about security)
- Optionally add re-ranking for improved relevance