6.867 Machine Learning (Fall 2003)

Home

Syllabus

Lectures

Projects

Problem sets

Exams

References

Matlab

Fall 2002
Fall 2001

 6.867 Lectures

Lectures

Mon/Wed 2:30-4pm in 35-225

Lecture material:

  • Lecture 1 (Wed 9/3): Introduction
    pdf slides, 6 per page postscript

  • Lecture 2 (Mon 9/8): Linear regression, estimation/estimator, generalization
    (Jordan & Bishop: chapters 5-5.3)
    pdf slides, 6 per page postscript

  • Lecture 3 (Wed 9/10): Additive models, statistical regression models, maximum likelihood estimation
    (Jordan & Bishop: chapter 4 (up to eq 4.20); chapters 5-5.3 & 5.6)
    pdf slides, 6 per page postscript

  • Lecture 4 (Mon 9/15): Bias/variance of estimators, active regression, examples
    (no relevant reading in the book)
    pdf slides, 6 per page postscript

  • Lecture 5 (Wed 9/17): classification, discriminant, mixture models
    (Jordan & Bishop: chapter 4 mixture models; chapter 6-6.3.1)
    pdf slides, 6 per page postscript

  • Lecture 6 (Wed 9/24): logistic regression, gradient ascent, back propagation, regularization
    (additional notes on regularization)
    pdf slides, 6 per page postscript

  • Lecture 7 (Mon 9/29): regularization, regularized logistic regression, support vector machine
    pdf slides, 6 per page postscript

  • Lecture 8 (Wed 10/1): support vector machines, solution, kernels
    (Additional notes on largrange multipliers, optional paper: postscript)
    pdf slides, 6 per page postscript

  • Lecture 9 (Mon 10/6): text classification, generative models, feature selection
    pdf slides, 6 per page postscript

  • Lecture 10 (Wed 10/8): feature selection, combination of methods, boosting + midterm discussion
    pdf slides, 6 per page postscript

  • MIDTERM (Wed 10/15)

  • Lecture 11 (Mon 10/20): boosting, algorithm, performance
    (optional papers: brief tutorial postscript, statistical view postscript)
    pdf slides, 6 per page postscript

  • Lecture 12 (Wed 10/22): complexity, VC-dimension, model selection
    pdf slides, 6 per page postscript

  • Lecture 13 (Mon 10/27): compression and model selection, probabilistic models
    (Jordan & Bishop: chapter 4 statistical concepts, mixture models; chapter 9 up to 9.2)
    pdf slides, 6 per page postscript

  • Lecture 14 (Wed 10/29): Mixture models, EM, regularization
    (Jordan & Bishop: chapter 4 statistical concepts, mixture models; chapter 9 up to 9.2)
    pdf slides, 6 per page postscript

  • Lecture 15 (Mon 11/3):
    Mixtures and conditional mixtures
    (Jordan & Bishop: chapter 9)
    pdf slides, 6 per page postscript

  • No lecture (Wed 11/5)

  • Lecture 16 (Med 11/12):
    Clustering, k-means, spectral clustering
    pdf slides, 6 per page postscript

  • Lecture 17 (Mon 11/17):
    Semi-supervised clustering, clustering by dynamics, Markov and hidden Markov models.
    pdf slides, 6 per page postscript

  • Lecture 18 (Wed 11/19):
    Hidden Markov models, forward-backward, EM
    pdf slides, 6 per page postscript

  • Lecture 19 (Mon 11/24):
    Viterbi, alignment, graphical models
    pdf slides, 6 per page postscript

  • Lecture 20 (Wed 11/26):
    Bayesian networks
    pdf slides, 6 per page postscript

  • Lecture 21 (Mon 12/1):
    Bayesian networks, quantitative probabilistic inference
    pdf slides, 6 per page postscript

  • Lecture 22 (Wed 12/3):
    Exact inference, junction trees
    pdf slides, 6 per page postscript

  • Lecture 23 (Mon 12/8):
    Exact inference (briefly), approximate inference, and review for the final
    pdf slides, 6 per page postscript

  • FINAL EXAM (Wed 12/10)