Albert BF

The AI knowledge tree

Every branch of learning-based AI except computer vision and robotics, as structured courses only: no articles, no blog posts, no "read this repo". Ten tiers ordered by dependency, each one stating what it builds on. Within a tier the domains are siblings and can be taken in any order.

Courses last checked

MathematicsClassical MLDeep learningNLP & LLMsRLModalitiesMLSysMLOpsSafetyResearch

Every entry names the course, the institution, and its code and edition where they matter, so it resolves to exactly one thing, plus its expected workload where the provider states one. Genuine substitutes form a "pick one" group; dominated courses are left out. Computer vision and robotics are deliberately out of scope.

Tier 0. Mathematical foundations

Everything downstream assumes this material. Probability and statistics carry the most weight later.

No prerequisites

Linear algebra & linear systems

  • Linear Algebra (18.06SC) MIT OCW (Strang)18.06SC ~150 hours total freeuniv
  • Matrix Methods in Data Analysis, Signal Processing, and Machine Learning (18.065) MIT OCW (Strang)18.065 12 units (3-0-9) · ~12 h/week freeuniv builds on Linear Algebra (18.06SC)
  • Introduction to Linear Dynamical Systems (EE263) Stanford Engineering Everywhere (Boyd)EE263 3 units · ~9 h/week freeuniv builds on Linear Algebra (18.06SC)

Calculus & matrix calculus

  • Multivariable Calculus (18.02SC) MIT OCW (Auroux)18.02SC ~150 hours total freeuniv
  • Matrix Calculus for Machine Learning and Beyond (18.S096, IAP 2023; now 18.063) MIT OCW (Edelman & Johnson)18.S096 3 units · ~42 h total freeuniv builds on Multivariable Calculus (18.02SC)

Probability & statistics

  • Pick one

    • Statistics 110: Probability HarvardStat 110 10 weeks, 5–10 h/week freeuniv
    • Probability - The Science of Uncertainty and Data (6.431x) MITx (MIT Open Learning, via MITx Online)6.431x 16 weeks, 10–14 h/week freepaidunivcert
  • Fundamentals of Statistics (18.6501x) MITx (MIT Open Learning, via MITx Online)18.6501x 17 weeks, 10–14 h/week freepaidunivcert builds on Statistics 110: Probability or Probability - The Science of Uncertainty and Data (6.431x)

Information theory

  • Information Theory, Pattern Recognition and Neural Networks Cambridge (MacKay) 16 lectures freeuniv

Optimization

  • Convex Optimization I (EE364A) Stanford Engineering Everywhere (Boyd)EE364A 3 units · ~9 h/week freeuniv
  • Optimization for Machine Learning (CS-439) EPFL (Jaggi & Flammarion)CS-439 8 ECTS (≈200–240 h total) freeuniv builds on Convex Optimization I (EE364A)

Tier 1. Classical machine learning

The pre-deep-learning core: estimators, generalisation, and the theory the rest leans on.

Builds on Mathematical foundations

Core ML

The ML Specialization is an optional, calculus-free on-ramp before CS229.

  • Machine Learning Specialization (3 courses) DeepLearning.AI + Stanford Online (Ng) 2 months at 10 h/week courserapaid

  • Pick one

    • Machine Learning (CS229) StanfordCS229 3–4 units · ~9–12 h/week freeuniv
    • Introduction to Machine Learning (10-301/10-601, Spring 2026) CMU (Gormley & Virtue)10-601 12 units · ~12 h/week freeuniv

Learning theory

Probabilistic modelling & inference

  • Statistical Rethinking Max Planck (McElreath) 20 lectures (~26 h video) freeseries

  • Pick one

    • Probabilistic Graphical Models (10-708, Spring 2020) CMU (Xing)10-708 12 units · ~12 h/week freeuniv
    • Probabilistic Machine Learning University of Tübingen (Hennig) 9 ECTS (≈225–270 h total) freeuniv
    • Probabilistic Graphical Models Specialization (3 courses) Stanford (Koller) 4 months at 10 h/week courserapaid

Causal inference

  • Introduction to Causal Inference Brady Neal 13 lectures over 15 weeks freeseries

Tier 2. Deep learning core

Backprop, optimisation, architectures, generative models.

Builds on Classical machine learning

Deep learning fundamentals

  • Pick one

    • Introduction to Deep Learning (11-785) CMU11-785 12 units · ~12 h/week freeuniv
    • Deep Learning (6.7960, Fall 2024) MIT OCW (Isola, Beery & Bernstein)6.7960 12 units (3-0-9) · ~12 h/week freeuniv

Generative models

  • Deep Unsupervised Learning (CS294-158, Spring 2024) UC Berkeley (Abbeel)CS294-158 4 units · ~12 h/week freeuniv
  • Introduction to Flow Matching and Diffusion Models (6.S184, IAP 2026) MIT (Holderrieth & Erives)6.S184 6 lectures + 3 labs freeuniv

Graph & geometric learning

  • Machine Learning with Graphs (CS224W) StanfordCS224W 60 videos (~22 h video) freeuniv
  • Geometric Deep Learning (GDL100, AMMI 2022) AMMI (Bronstein, Bruna, Cohen & Veličković)GDL100 12 lectures + 5 seminars (~18 h video) freeuniv

Tier 3. NLP, transformers & LLMs

From word vectors to building a language model end to end. CS336 is the densest item in the tree.

Builds on Deep learning core

NLP foundations

Encoders, embeddings & retrieval

No course yet covers ModernBERT-era encoders or large-scale embedding training.

Transformers & building LLMs

Post-training is taught in CS336 (lectures 15-16, assignment 5) and tier 4's deep RL courses.

  • Language Modeling from Scratch (CS336, Spring 2026) Stanford (Hashimoto & Liang)CS336 5 units · ~15 h/week freeuniv
  • Transformers United V6 (CS25, Spring 2026) StanfordCS25 1 unit · ~3 h/week freeseries

Beyond transformers: SSMs, linear attention & diffusion LMs

No dedicated course exists yet. This lecture derives S4 and Mamba; the rest is in CS336 lecture 4, CS25 V6 and 6.S184 lecture 5.

  • Scaling Sequence Length (Lecture 22 of 11-711 Advanced NLP, Spring 2026) CMU (Welleck)11-711 one 80-min lecture freeuniv

Agents & tool use

  • AI Agents Course Hugging Face 8 units × ~3–4 h (≈24–32 h total) freecert
  • Agentic AI (CS294/194-196, Fall 2025) UC Berkeley RDI (Dawn Song)CS294/194-196 13 lectures (2 h each) freeuniv

Tier 4. Reinforcement learning

A standalone branch. Feeds post-training (RLHF, RLVR/GRPO) back into tier 3.

Builds on Deep learning core

Classical RL

  • Pick one

    • Reinforcement Learning Specialization (4 courses) University of Alberta 2 months at 10 h/week courserapaid
    • Introduction to Reinforcement Learning (COMPM050/COMPGI13, 2015) UCL (Silver), recordings by Google DeepMindCOMPM050/COMPGI13 10 lectures (~16 h video) freeuniv

Deep RL

  • Pick one

    • Deep Reinforcement Learning (CS224R, Spring 2025) Stanford (Finn)CS224R 3 units · ~9 h/week freeuniv
    • Deep Reinforcement Learning (CS185/285, Spring 2026) UC Berkeley (Levine)CS185/285 3 units · ~9 h/week freeuniv

Tier 5. Other modalities & applications

Speech, time series, recommenders, and multimodal models. Multimodal work necessarily touches vision.

Builds on Deep learning core

Speech & audio

  • Spoken Language Processing (CS224S, Spring 2025) Stanford (Maas)CS224S 2–4 units · ~6–12 h/week freeuniv

Time series & forecasting

  • Practical Time Series Analysis SUNY 3 weeks at 10 h/week courserapaid

Recommenders & large-scale mining

  • Recommender Systems Specialization (5 courses) University of Minnesota 2 months at 10 h/week courserapaid
  • Mining Massive Datasets (CS246) StanfordCS246 3–4 units · ~9–12 h/week freeuniv

Multimodal learning

  • How to AI (Almost) Anything (MAS.S60, Spring 2025) MIT Media Lab (Liang)MAS.S60 9 units (2-0-7) · ~9 h/week freeuniv

Tier 6. ML systems (MLSys)

Kernels, frameworks, distributed training, serving, and compression: how models run fast at scale.

Builds on NLP, transformers & LLMs

GPU & parallel programming

  • Parallel Computing (CS149, Fall 2023) Stanford (Fatahalian & Olukotun)CS149 3–4 units · ~9–12 h/week freeuniv
  • Accelerated Computing (6.S894, Fall 2025) MIT (Ragan-Kelley)6.S894 12 units (1-2-9) · ~12 h/week freeuniv builds on Parallel Computing (CS149, Fall 2023)

DL frameworks & compilers

  • Deep Learning Systems: Algorithms and Implementation (10-414/714) CMU (Chen & Dettmers)10-414 12 units · ~12 h/week freeuniv

Distributed training & LLM serving

  • Pick one

    • Data Systems for Machine Learning (CSE 234, Winter 2025) UC San Diego (Hao Zhang)CSE 234 4 units · ~12 h/week freeuniv
    • Large Language Model Systems (11-868, Spring 2026) CMU (Lei Li)11-868 12 units · ~12 h/week freeuniv
  • Stanford MLSys Seminar Series Stanford ongoing, ~1 h/week freeseries

Efficient ML & compression

  • TinyML and Efficient Deep Learning Computing (6.5940, Fall 2024) MIT (Song Han)6.5940 12 units (3-0-9) · ~12 h/week freeuniv

Tier 7. MLOps & production engineering

Shipping, operating, and monitoring models and the data pipelines behind them.

Builds on Deep learning core

MLOps

  • Machine Learning in Production (17-445/17-645, Spring 2026) CMU (Kästner & Le Goues)17-445 12 units · ~12 h/week freeuniv
  • MLOps Zoomcamp DataTalks.Club 9 weeks, self-paced freeseries

LLMOps

  • LLM Zoomcamp DataTalks.Club 10 weeks, self-paced freecert

Data engineering

  • Data Engineering Zoomcamp DataTalks.Club 9 weeks, self-paced freecert

Cloud ML certifications

  • Pick one

    • Exam Prep Plan: AWS Certified Machine Learning Engineer - Associate (MLA-C02) AWS Skill BuilderMLA-C02 ~1 month at 4 h/week paidcert
    • Professional Machine Learning Engineer Certification (learning path) Google Cloud (Google Skills) ~58 h (17 activities) freepaidcert

Tier 8. Safety, security, privacy & governance

Alignment, interpretability, red teaming, privacy, and policy.

Builds on NLP, transformers & LLMs

AI safety & alignment

  • Technical AI Safety BlueDot Impact 6 weeks at ~5 h/week freecert
  • ARENA - Alignment Research Engineer Accelerator ARENA 4–5 weeks (in-person) freeseries

LLM security & red teaming

  • AI Red Teaming 101 Microsoft (AI Red Team) 10 episodes freeseries

Privacy-preserving ML

  • Algorithms for Private Data Analysis (CS 860, Fall 2020) University of Waterloo (Kamath)CS 860 19 lectures freeuniv

Ethics, governance & policy

  • Ethics of AI (TKT21031) University of Helsinki (MOOC.fi)TKT21031 2 ECTS (≈50–60 h total) freeunivcert
  • Frontier AI Governance BlueDot Impact ~40 h total freecert
  • AIGP: Artificial Intelligence Governance Professional training IAPPAIGP ~13 h (7 modules) paidcert

Tier 9. Research practice

Craft rather than knowledge: reading a field, reproducing results, and writing them up.

Builds on NLP, transformers & LLMs and ML systems (MLSys)

Doing and writing research

  • AI Research Experiences (CS197, Fall 2022) Harvard (Rajpurkar)CS197 21 lectures freeuniv
  • Writing in the Sciences (SOM-Y0010) Stanford Online (Sainani)SOM-Y0010 3 weeks at 10 h/week (≈30 h total) courserapaidcert