Copyright 2017 - CSIM - Asian Institute of Technology

Selected Topic: Machine Learning

Course code: AT70.9022
Credits: 3(3–0)
This course is elective

Course objectives

Intelligent systems, such as speech recognition systems, document classification systems, and character recognition systems, are concerned with the transformation of input data (e.g. speech, documents, or bitmaps) into desired output data (words, document classes, or characters, respectively). To obtain an efficient system, this transformation function must be carefully constructed and its parameters must be properly adjusted. Machine learning is concerned with the automatic learning of these parameters from training examples. It draws heavily on computer science, algorithms and data structures, probability, statistics, and optimization. This course covers fundamental concepts as well as state of the art algorithms in machine learning. The grading system relies on homework, student presentations of research from the primary academic literature, and a project.

Learning outcome

Introduction. Probability. Linear classifiers. Support vector machines. Clustering and mixture models. Subspace projection. Sequential data. Graphical models. Student presentations of primary research papers.


Programming Experience, Mathematical Sophistication.

Course outline

I.            Introduction
1.      Example of intelligent systems
2.      Probability theory
3.      K-nearest neighbors
4.      Kernel function
II.         Probability
1.      Bayesian classifier
2.      Gaussian model
3.      Maximum likelihood estimation
4.      Maximum a posteriori
5.      Non-parametric methods
III.       Linear classifiers
1.      Exponential family
2.      Minimum squared error training
3.      Ho-Kashyap algorithm
4.      Perceptron algorithm
5.      Kernel perceptron
6.      Multiclass classification
IV.      Support vector machines
1.      Introduction to statistical learning theory
2.      Margin of a linear classifier
3.      Optimal separating hyperplane
4.      Dual problem formulation
5.      Sequential minimal optimization (SMO) algorithm
6.      Comparison to neural networks
V.         Clustering and mixture models
1.      Self-organizing maps
2.      Hierarchical clustering
3.      K-means clustering
4.      Kernel K-means
5.      Mixture of Gaussians
6.      Expectation-Maximization (EM) algorithm
7.      Other clustering techniques
VI.      Subspace projection
1.      Principal component analysis (PCA)
2.      Hebbian learning
3.      Kernel PCA
4.      Multidimensional scaling (MDS)
5.      Linear discriminant analysis (LDA)
6.      Nullspace LDA
7.      Independent component analysis (ICA)
VII.    Sequential data
1.      Edit distance
2.      Markov models
3.      Hidden Markov models
4.      Viterbi algorithm
VIII.Graphical models
1.      Bayesian networks
2.      Conditional independence
3.      Markov random fields
4.      Inference in graphical models
IX.      Student presentations of research papers

Learning resources

Reference books

  • Christopher Bishop. Pattern Recognition and Machine Learning. Springer. 2006.
  • Richard O. Duda, Peter E. Hart, and David G. Stork. Pattern Classification. 2nd Edition. Wiley-Interscience. 2000.


Assignments                                        -10%
Project/Student’s Presentations       - 30%
Midterm and                                          - 20%
Final                                                        - 40%

Back to the list


Login Form


School of Engineering and technologies     Asian Institute of Technology