Ch6. Machine Learning — Transformers, LLMs, and the AI Landscape

Why Transformers Were Necessary

Early deep learning for sequential data (text, audio) relied on Recurrent Neural Networks (RNNs). RNNs pass state from one time step to the next, but they have fundamental limitations.

RNN limitations:
  1. Long-term dependency problem
     "The trophy didn't fit in the suitcase because [15 words] it was too large."
     → Hard to connect "trophy" and "it" across many steps
     → Information from distant positions fades during training

  2. Sequential processing → no parallelism
     → Poor GPU utilization, slow training

  3. Vanishing / exploding gradients
     → LSTM and GRU helped but did not fully solve the problem

Google’s 2017 paper “Attention Is All You Need” introduced the Transformer, which solved all of these problems simultaneously.

---

The Attention Mechanism — Intuition

The key idea of attention: “Pay more attention to the words most relevant to what you’re currently processing.”

Intuitive Example

Translation: "I went to school today"

When generating "went":
→ High attention weight on "went" in the source (direct mapping)
→ Moderate attention on "today" (slightly relevant)
→ Low attention on "I", "school" (mostly unrelated)

Instead of processing left-to-right blindly,
attention actively "looks up" relevant information

Scaled Dot-Product Attention

Each word is transformed into three vectors:
  Query (Q):  "What am I looking for?"
  Key   (K):  "What information do I hold?"
  Value (V):  "The actual content to pass along"

Attention score:
  score  = Q × Kᵀ / √dₖ   (scaling stabilizes gradients)

Attention weights:
  weight = softmax(score)   (converts to probability distribution)

Output:
  output = weight × V
  → Values from relevant positions are weighted more heavily

Multi-Head Attention

Single head: captures one type of relationship

Multi-head: captures multiple relationship types simultaneously

Head 1: syntactic relationships ("subject-verb agreement")
Head 2: semantic relationships ("cat is related to animal")
Head 3: coreference ("it" refers to which entity?)
...

→ Concatenate all heads + linear projection
→ Richer contextual representation

---

Transformer Architecture (Encoder-Decoder)

Full Transformer structure:

Input Sequence              Output Sequence
     ↓                            ↑
[Positional Encoding]    [Positional Encoding]
     ↓                            ↑
┌─────────────────┐      ┌───────────────────────┐
│   Encoder × N   │      │      Decoder × N       │
│                 │      │                       │
│ Multi-Head      │  →   │ Masked Multi-Head     │
│ Attention       │      │ Attention             │
│      ↓          │      │        ↓              │
│ Add & Norm      │      │ Cross-Attention       │
│      ↓          │      │ (uses encoder output) │
│ Feed Forward    │      │        ↓              │
│      ↓          │      │ Feed Forward          │
│ Add & Norm      │      │        ↓              │
└─────────────────┘      └───────────────────────┘
                                  ↓
                         Linear + Softmax
                         (predict next token)

Positional Encoding

Attention has no inherent notion of order
→ "The dog bit the man" and "The man bit the dog" would be identical

Solution: add positional encodings to input embeddings
→ Each position gets a unique sinusoidal wave vector
→ Model can distinguish position

PE(pos, 2i)   = sin(pos / 10000^(2i/d_model))
PE(pos, 2i+1) = cos(pos / 10000^(2i/d_model))

---

BERT vs. GPT

Two landmark pre-trained Transformer models, built on different halves of the architecture.

BERT (Bidirectional Encoder Representations from Transformers)

Developer: Google (2018)
Architecture: Transformer encoder only

Pre-training tasks:
1. Masked Language Model (MLM):
   "The [MASK] sat on the mat" → predict "cat"
   → Learns bidirectional context (looks both left and right)

2. Next Sentence Prediction (NSP):
   Predict whether two sentences are consecutive

Strengths:
→ Rich understanding of sentence meaning
→ Excels at: text classification, NER, question answering
→ Not suited for text generation

GPT (Generative Pre-trained Transformer)

Developer: OpenAI (2018 → GPT-1, 2, 3, 4, o1, o3, ...)
Architecture: Transformer decoder only

Pre-training task:
Language modeling: predict the next token given all previous tokens
"The cat sat on" → "the" → "mat" (left-to-right only)

Strengths:
→ Excellent at fluent text generation
→ Text completion, dialogue, creative writing, code generation
→ Powerful few-shot and zero-shot generalization (from GPT-3 onward)

	BERT	GPT
Architecture	Encoder	Decoder
Attention direction	Bidirectional	Unidirectional (left → right)
Strength	Understanding (classification)	Generation (text completion)
Representative use	Sentiment analysis, NER	ChatGPT, GitHub Copilot

---

Using LLMs Effectively

Prompt Engineering

Optimizing how you instruct an LLM to maximize output quality

Zero-shot:
  "Classify the sentiment of this review:
   'The shipping took two weeks.'"
  → No examples provided

Few-shot:
  "Examples:
   'Great food!' → Positive
   'Service was rude.' → Negative
   Review: 'The shipping took two weeks.' → ?"
  → 2–10 examples included

Chain-of-Thought (CoT):
  "Think step by step:
   1. Identify key sentiment words
   2. Consider negations and intensifiers
   3. Assign final sentiment"
  → Effective for complex reasoning tasks

RAG — Retrieval-Augmented Generation

LLM limitations:
  - Knowledge cutoff: unaware of events after training
  - Domain gaps: lacks knowledge of internal documents
  - Hallucination: confidently generates false facts

RAG solution:
  1. Build an external knowledge base (store documents in a vector DB)
  2. Retrieve relevant documents for each user query (similarity search)
  3. Provide the query + retrieved documents to the LLM
  4. LLM generates an answer grounded in the retrieved evidence

Benefits:
  → Incorporates up-to-date information
  → Reduces hallucinations
  → Can cite sources

Fine-Tuning

Pre-training:
  → Hundreds of billions of parameters; trained on massive general text
  → Acquires broad language understanding and generation capability

Fine-tuning:
  → Additional training on domain-specific or task-specific data
  → Starts from pre-trained weights; adjusts with hundreds to thousands of samples

Full Fine-tuning:
  → Updates all parameters
  → Very high compute cost

PEFT (Parameter-Efficient Fine-Tuning):
  → Updates only a small subset of parameters

LoRA (Low-Rank Adaptation):
  → Freezes original weights; learns two small low-rank matrices
  → Trains ~0.1% of parameters with near-equivalent performance
  → Enables fine-tuning large models on a single consumer GPU

# Hugging Face BERT fine-tuning (conceptual)
from transformers import BertForSequenceClassification, Trainer, TrainingArguments

model = BertForSequenceClassification.from_pretrained(
    'bert-base-uncased',
    num_labels=2
)

training_args = TrainingArguments(
    output_dir='./results',
    num_train_epochs=3,
    per_device_train_batch_size=16,
    learning_rate=2e-5,          # small LR for fine-tuning
    evaluation_strategy='epoch',
    save_strategy='epoch',
    load_best_model_at_end=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
)
trainer.train()

---

AI Ethics and Hallucination

Hallucination

LLMs sometimes generate false information with high confidence

Example:
Q: "Who won the Nobel Prize in Physics in 2024?"
A (bad LLM): "Dr. James Smith won for his work on quantum entanglement."
→ A completely invented person and fact

Root cause:
→ LLMs predict the statistically most likely next token
→ Plausibility ≠ factual accuracy
→ Information absent from training data is guessed, not known

Mitigation:
→ RAG: ground answers in retrieved trusted documents
→ Lower temperature: reduces diversity → more conservative outputs
→ Require citations: "cite your sources"
→ Verification layers: add a fact-checking model

Core AI Ethics Principles

Fairness:
  → Biased training data → biased model outputs
  → Example: hiring model trained on historically male-dominated data
             systematically disadvantages female applicants
  → Mitigation: diverse data, fairness metric monitoring

Transparency:
  → Black-box models cannot explain their decisions
  → XAI (Explainable AI): SHAP, LIME
  → EU AI Act: high-risk AI systems must be explainable

Privacy:
  → LLMs may memorize and reproduce personal data from training
  → Differential Privacy techniques can limit this

Safety:
  → Prevent harmful content generation: RLHF, safety filters
  → Defend against adversarial attacks (jailbreaking)

---

ML Career Roadmap

Role Overview

Data Scientist:
  → Business problem → ML problem formulation
  → EDA, modeling, insight generation
  → Skills: Python, SQL, statistics, scikit-learn

ML Engineer:
  → Productionize models (MLOps)
  → Scaling, pipeline automation
  → Skills: Python, Docker, Kubernetes, MLflow

Deep Learning Researcher:
  → Novel architectures and algorithms
  → Paper implementation and experiments
  → Skills: linear algebra, calculus, PyTorch, paper reading

AI Product Manager:
  → AI product strategy and roadmap
  → Bridge between technology and business
  → Skills: ML literacy, data interpretation, communication

Learning Path

[Months 0–6: Foundation]
  Python → NumPy/pandas → statistics
  → scikit-learn → enter a beginner Kaggle competition

[Months 6–12: Core ML]
  ML theory (Ch1–5 of this series) → 1–2 projects
  → TensorFlow or PyTorch basics → CNN hands-on

[Year 2: Intermediate]
  NLP (Ch4) → Transformer internals → BERT fine-tuning
  → MLOps basics (Docker, FastAPI, MLflow)
  → Kaggle Silver → Gold medal target

[Advanced]
  LLM applications (RAG, fine-tuning) → paper implementation
  → Open-source contributions → portfolio
  → Job search

Essential Resources

Online courses:
  - Andrew Ng — Machine Learning Specialization (Coursera)
  - fast.ai Practical Deep Learning (free)
  - Hugging Face NLP Course (free)

Practice platforms:
  - Kaggle (competitions, datasets, notebooks)
  - Google Colab (free GPU)
  - Hugging Face Hub (pre-trained model repository)

Foundational papers:
  - "Attention Is All You Need" (Transformer)
  - "BERT: Pre-training of Deep Bidirectional Transformers"
  - "Language Models are Few-Shot Learners" (GPT-3)

---

Key Concept Cards

Attention mechanism ★★★★★
Q·K computes relevance scores → softmax → weighted sum of V. Dynamically determines “which words to focus on.” Q = query, K = keys, V = values.

Transformer encoder vs. decoder ★★★★★
Encoder = bidirectional attention → strong understanding (BERT). Decoder = unidirectional attention → strong generation (GPT).

RAG ★★★★☆
Addresses LLM knowledge cutoff and hallucination by retrieving relevant external documents and grounding the answer. = search + generation.

Hallucination ★★★★☆
LLMs generate statistically plausible — not necessarily factual — tokens. Training data gaps are filled by guess, not knowledge.

Fine-tuning vs. Prompt Engineering ★★★☆☆
Fine-tuning directly modifies model weights with domain data (high cost, high performance). Prompt engineering optimizes instructions without changing weights (fast and cheap). Try prompts first; fine-tune for narrow specialized domains.

---

Practice Quiz

Q1. How did Transformers solve the limitations of RNNs?

RNNs suffer from sequential processing (no parallelism) and long-term dependency loss. Transformers compute relationships between all positions in a sequence simultaneously via attention, eliminating sequential bottlenecks. GPU-friendly matrix operations make training dramatically faster, and attention directly connects any two positions regardless of distance.

Q2. What is the key architectural and functional difference between BERT and GPT?

BERT uses the Transformer encoder with bidirectional masked language modeling — it sees context from both sides of each token. Best for tasks requiring text understanding: classification, NER, QA. GPT uses the Transformer decoder with unidirectional left-to-right language modeling — each token only sees tokens before it. Best for text generation: dialogue, completion, code generation.

Q3. Explain what RAG is and why it is needed.

LLMs have a knowledge cutoff, lack proprietary domain knowledge, and can hallucinate. RAG retrieves relevant documents from an external vector database based on the user’s query, then provides both the query and the retrieved text to the LLM as context. The LLM generates an answer grounded in the evidence, reducing hallucinations and enabling up-to-date responses.

Q4. Why is LoRA more practical than full fine-tuning for most practitioners?

Full fine-tuning updates billions of parameters, requiring multiple high-end GPUs and enormous cost. LoRA freezes the original weight matrices and learns two small low-rank decomposition matrices instead — updating as little as 0.1% of parameters while achieving near-equivalent performance. This makes fine-tuning large models feasible on a single consumer GPU.

Q5. Give two concrete reasons why ML practitioners must consider AI ethics.

First, bias: a hiring model trained on historical data (predominantly male hires) will systematically rank female candidates lower — an actual incident at Amazon. This creates legal liability and perpetuates societal inequity. Second, privacy: using patient records without sufficient anonymization in a medical AI model risks exposing personal health information, violating HIPAA (US) and GDPR (EU) regulations, with severe legal consequences.