🗺️ Roadmap: The Mathematical Imperative — Why AI Demands Specialized Hardware¶

Below is the syllabus timeline of lessons in this module. Track your study progress here.

📈 Progress¶

---

📁 7 Deconstructing Artificial Intelligence Workloads¶

• 7.1a Artificial Intelligence (AI): the broadest category — any machine mimicking cognition
• 7.1b Machine Learning (ML): learning patterns from data without explicit programming
• 7.1c Deep Learning (DL): ML using multi-layer artificial neural networks
• 7.1d Generative AI (GenAI): models that produce new content (text, images, code, audio)
• 7.1e Supervised, Unsupervised, and Reinforcement Learning — the three paradigms
• 7.2a Data Ingestion: collecting, labeling, and versioning training datasets
• 7.2b Data Preparation and Feature Engineering: preprocessing, tokenization, normalization
• 7.2c Pre-Training: learning from a massive corpus from scratch (compute-intensive)
• 7.2d Fine-Tuning: adapting a pre-trained model to a specific domain or task
• 7.2e Inference: running the trained model to generate predictions in production
• 7.2f RLHF (Reinforcement Learning from Human Feedback): aligning model behavior
• 7.2g Model evaluation, benchmarking, and regression testing
• 7.3a The Transformer architecture: attention mechanisms explained intuitively
• 7.3b Parameters: what billions of weights actually are and why they need VRAM
• 7.3c Tokens: how LLMs process text — tokenization, vocabulary size, and BPE
• 7.3d Context windows: why longer context requires quadratically more memory
• 7.3e Embedding models, rerankers, and the retrieval-augmented generation (RAG) pipeline

📁 8 The CPU-GPU Compute Divide — Why CPUs Cannot Scale for AI¶

• 8.1a CPUs optimized for latency: branch prediction, out-of-order execution, large caches
• 8.1b GPUs optimized for throughput: thousands of simple cores, SIMD execution
• 8.1c The 'smart professors vs. assembly line workers' analogy — and why AI needs the factory
• 8.1d Amdahl's Law: the diminishing returns of parallelism and its AI implications
• 8.2a Arithmetic intensity: FLOPs per byte of memory accessed
• 8.2b Identifying whether a neural network layer is compute-bound or memory-bound
• 8.2c Implications for choosing precision, batch size, and hardware

📁 9 The Mathematics of Deep Learning — Tensors, Operations & Numerical Precision¶

• 9.1a Scalars: single values — a learning rate, a loss value
• 9.1b Vectors: 1D arrays — a word embedding, a bias term
• 9.1c Matrices: 2D arrays — a fully connected layer's weight matrix
• 9.1d Tensors: N-dimensional arrays — a batch of images, sequences, or activations
• 9.1e Tensor shapes, strides, and contiguity in memory
• 9.2a Why General Matrix Multiply (GEMM) accounts for 90%+ of DNN compute
• 9.2b Dot products, matrix-vector, and batched matrix multiplication
• 9.2c Why GEMM maps perfectly to GPU SIMD hardware
• 9.3a FP64 (double precision): 64-bit floating point — scientific computing gold standard
• 9.3b FP32 (single precision): 32-bit — the historical default for deep learning
• 9.3c FP16 (half precision): 16-bit — 2x storage savings, 2x bandwidth, with stability risks
• 9.3d BF16 (Brain Float 16): Google's format with FP32 exponent range — preferred for training
• 9.3e INT8 and INT4: quantized formats for inference acceleration — how quantization works
• 9.3f FP8 (E4M3/E5M2): NVIDIA Hopper's new training format
• 9.3g Mixed-precision training: using FP16/BF16 for forward pass, FP32 for master weights
• 9.3h Loss scaling: preventing underflow in half-precision gradients