Copyright 2017 - CSIM - Asian Institute of Technology

Selected Topic: Machine Learning

Course code: AT70.9022
Credits: 3(3–0)
This course is elective

Course objectives

Intelligent systems, such as speech recognition systems, document classification systems, and character recognition systems, are concerned with the transformation of input data (e.g. speech, documents, or bitmaps) into desired output data (words, document classes, or characters, respectively). To obtain an efficient system, this transformation function must be carefully constructed and its parameters must be properly adjusted. Machine learning is concerned with the automatic learning of these parameters from training examples. It draws heavily on computer science, algorithms and data structures, probability, statistics, and optimization. This course covers fundamental concepts as well as state of the art algorithms in machine learning. The grading system relies on homework, student presentations of research from the primary academic literature, and a project.

Learning outcome

Introduction. Probability. Linear classifiers. Support vector machines. Clustering and mixture models. Subspace projection. Sequential data. Graphical models. Student presentations of primary research papers.

Prerequisite(s)

Programming Experience, Mathematical Sophistication.

Course outline

I.            Introduction
1.      Example of intelligent systems
2.      Probability theory
3.      K-nearest neighbors
4.      Kernel function
 
II.         Probability
1.      Bayesian classifier
2.      Gaussian model
3.      Maximum likelihood estimation
4.      Maximum a posteriori
5.      Non-parametric methods
 
III.       Linear classifiers
1.      Exponential family
2.      Minimum squared error training
3.      Ho-Kashyap algorithm
4.      Perceptron algorithm
5.      Kernel perceptron
6.      Multiclass classification
 
IV.      Support vector machines
1.      Introduction to statistical learning theory
2.      Margin of a linear classifier
3.      Optimal separating hyperplane
4.      Dual problem formulation
5.      Sequential minimal optimization (SMO) algorithm
6.      Comparison to neural networks
                                                                                                                                                        
V.         Clustering and mixture models
1.      Self-organizing maps
2.      Hierarchical clustering
3.      K-means clustering
4.      Kernel K-means
5.      Mixture of Gaussians
6.      Expectation-Maximization (EM) algorithm
7.      Other clustering techniques
 
VI.      Subspace projection
1.      Principal component analysis (PCA)
2.      Hebbian learning
3.      Kernel PCA
4.      Multidimensional scaling (MDS)
5.      Linear discriminant analysis (LDA)
6.      Nullspace LDA
7.      Independent component analysis (ICA)
 
VII.    Sequential data
1.      Edit distance
2.      Markov models
3.      Hidden Markov models
4.      Viterbi algorithm
 
VIII.Graphical models
1.      Bayesian networks
2.      Conditional independence
3.      Markov random fields
4.      Inference in graphical models
 
IX.      Student presentations of research papers

Learning resources

Reference books

  • Christopher Bishop. Pattern Recognition and Machine Learning. Springer. 2006.
  • Richard O. Duda, Peter E. Hart, and David G. Stork. Pattern Classification. 2nd Edition. Wiley-Interscience. 2000.

Grading

Assignments                                        -10%
Project/Student’s Presentations       - 30%
Midterm and                                          - 20%
Final                                                        - 40%

Back to the list

 

Login Form

Search

School of Engineering and technologies     Asian Institute of Technology