03 Latent Structures, PCA, and Clustering

In Block 3 we cover:

• Motivation for latent structures
• Principal Components Analysis
  • How to calculate PCA
  • What PCA is good for
  • Relationship to SVD and other Spectral Embeddings
• Clustering
  • Algorithmic Clustering
  • Hierarchical Clustering
  • Model based clustering
• Implementations in R:
  • Spectral Clustering as a pipeline element for classification

Lectures:

• 03.1-Latent Structures and PCA
  • Reference R code
• 03.2-Clustering
  • Reference R code

Worksheets:

• Worksheet 3.1 Latent Spaces and PCA
• Worksheet 3.2 Clustering
• Portfolio 03

Workshop:

• 03.3 PCA and Clustering (.Rmd)
• 03.3 PCA and Clustering (.html)

Assessments:

• Assessment 1 will be set in this week; see Assessments. This is a summative assessment (i.e. does contribute to your grade) and will be due in Week 7.

Reference material:

For PCA:

• Cosma Shalizi’s Advanced Data Analysis, Lecture 18
• Boyd and Vandenberghe: Convex Optimization is an excellent and thorough resource.
• I showed Kalman: Leveling with Lagrange: An Alternate View of Constrained Optimization

For Clustering:

• Tibsherani’s Data Mining lecture notes (Lecture 2 and Lecture 5)
• 5 clustering algorithms you need to know
• The fastcluster packages for R and python implements “fastest” \(O(N^2)\) versions of hierarchical clustering.
• Python resources comparing hdbscan
• Scikit Learn Diagram

Navigation

Previous: Block 02. Next: Block 04.