02 Modern Regression and Cross Validation

In Block 02, we cover:

• Modern Regression:
  • Matrix formulation of multivariate regression.
  • Elements of multivariate calculus
• Cross Validation:
  • k-fold Cross validation
  • Relationship to model choice
• Implementations in R:
  • Regression for time series
  • Regression for feature matrices

Lectures:

Regression:

• 02.1 Modern Regression
  • Reference R code (02.1-Regression.R)
  • Matrix Multiplication Cheat Sheet
• 2.1.P Portfolio on Information Criteria

Cross Validation:

• 02.2 Cross Validation
• 2.2.P Portfolio on leave-one-out Cross Validation

Workshop:

• 02.3 Model Selection (.Rmd)
• 02.3 Model Selection (.html)

Assessments:

• The main long form portfolio Formative Portfolio.
• Block02 on Noteable via Blackboard (2026-27 link)

Reference material:

For Regression:

• Cosma Shalizi’s Modern Regression Lectures (Lectures 4-9 for basic material; Lectures 13-14 for Linear Algebra approach)
• Matrix Multiplication Cheat Sheet
  • A complete reference is The Matrix Cookbook
• Sam Roweis’ Matrix Identities
• Cosma Shalizi’s Modern Regression Lectures
• Further reading in chapters 2.3 and 3.2 of The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Friedman, Hastie and Tibshirani)

  For Model Comparison:

• Cosma Shalizi’s Modern Regression Lectures (Lectures 26,28)
• Further reading in Chapters 2.3 and 7.10 of The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Friedman, Hastie and Tibshirani).

Navigation

Previous: Block 01. Next: Block 03.