6.891 Machine Learning

This course satisfies the AI graduate requirement for Area II graduate students in EECS.

Course Description

This introductory course on machine learning will give an overview of many techniques and algorithms in machine learning, beginning with topics such as simple perceptrons and ending up with more recent topics such as boosting, support vector machines, hidden Markov models, and Bayesian networks. The course will give the student the basic ideas and intuition behind modern machine learning methods as well as a bit more formal understanding of how and why they work. The underlying theme in the course is statistical inference as this provides the foundation for most of the methods covered.

Topics to be covered:

  • Classification, regression
  • Neural networks, back-propagation
  • Generalization, regularization
  • Feature selection, feature induction
  • Combining multiple models, additive models, boosting
  • Support vector machines, kernels
  • Complexity, VC-dimension
  • Model selection, structural risk minimization
  • Density estimation, parametric, non-parametric
  • Clustering, agglomerative, mixture models
  • Hidden markov models, forward-backward algorithm, EM
  • Bayesian networks, graphs, estimation, model selection
  • Markov random fields, dynamic Bayesian networks
  • Time series prediction, independent component analysis

    • Prerequisites

    We assume basic understanding of probability/statistics and linear algebra, in addition to familiarity with programming in Matlab. Having taken these courses (or their equivalents) should be sufficient background:

  • 6.001 Structure and Interpretation of Computer Programming
  • 6.041 or 18.05 Intro. to Probability
  • 18.06 Linear Algebra
    •

    Courses related to 6.891 Machine Learning include:

  • 6.231 Dynamic Programming and Stochstic Control
  • 6.252 Nonlinear Programming
  • 6.262 Discrete Stochastic Processes
  • 6.432 Stochastic Processes, Detection and Estimation
  • 6.441 Transmission of Information
  • 9.520 Networks for Learning: Regression and Classification
  • 9.641 Introduction to Neural Networks
  • 15.062 Data Mining: Algorithms and Applications
  • 15.065 Decision Analysis
  • 18.441/3 Statistical Inference
    •  Many of the above courses cover in detail topics that are only briefly covered in 6.891.

    Last year's course page.

    Next year's course page.