198:535 Pattern Recognition

Spring 2006

Instructor: Casimir Kulikowski
Cuorse meeting Time: 6:30-9pm
Venue: Hill 254
Office : CoRE 323
Office Hours: By Appointment
Email : (construct from) kulikows cs rutgers edu

TA: Akshay Vashist
Office Hours: Wednesdays 11am-1pm, CoRE 346
Email : (construct from) vashisht cs rutgers edu

Course Description

The course will cover the fundamental in the statistical theory of classifier design for analysis of complex experimental data. Starting with classical models and methods of statistical decision-making and prediction, clustering, and pattern recognition, it will relate them to current approaches in machine learning, data mining, multi-criteria decision making, optimization, and their applications.
There will be weekly homeworks for the first 2/3 of the class. The homeworks will constitute 20-30% of the grade. There will be a closed-book midterm exam which will contribute 30-40% of the grade. Students will also carry out projects developing software for the analysis of synthetic and real-world data. This will make up rest of the grade. The course will draw on material from several texts as well as papers from the recent literature.

References

Primary
[DHS]   R.O. Duda, P.E. Hart and D.G. Stork: Pattern Classification , 2nd edition, John Wiley & Sons, 2001.

Others
[HTF]   T. Hastie, R. Tibshirani and J. Friedman: The elements of Statistical Learning: Data Mining, Inference and Prediction, Springer-Verlag, 2001.
[WK]   S.M. Weiss and C.A. Kulikowski: Computer Systems that learn, Morgan Kaufmann, San Mateo, California, 1991.
[KF]   K. Fukunaga: Introduction to Statistical Pattern Recognition, 2nd edition Academic Press, New York 1990.
[SW]   S. Watanabe: Pattern Recognition: Human and Mechanical , John Wiley & Sons, New York 1985.

Preliminary Schedule

Date/Week

Topics [References/Reading]

 HW
Week 1 (Jan. 17)

Organization of the class
Intro to PR, clustering & related fields.
Intro to Bayesian decision theory.
[DHS 2.1 & 2.2]

HW1 Solutions
Prob. 2.1 - 2.4 DHS
Due: Jan. 24
Week 2 (Jan. 24)

Bayesian Decision Theory - relation to minimax
and hypothesis testing (Neyman-Pearson).
Multivariate Normal densities and their discriminants.
Error probabilities and bounds [DHS 2.3-2.8]
Estimation of performance [WK]

HW2 Solutions
Prob. 5,6 DHS
Due: Jan. 31
Week 3 (Jan. 31)

Binary and other discrete features.
Bayesian Belief Networks.
Introduction to Maximum Likelihood
[DHS 2.9-2.11 & 31.-3.2]

 HW3 Solutions
Prob. 2.13, 2.14, 2.32 from DHS
(optional problem 2.24) Due: Feb. 7
Week 4 (Feb. 7)

Bayesian parameter estimation,
sufficient statistics and complexity
[DHS 3.3-3.8]

 HW4 Solutions
Prob. 3.3, 3.15 from DHS
(optional problems 3.1 & 3.7) Due: Feb. 14
Week 5 (Feb. 14)

PCA, MDA, IDA, EM, HMMs
[DHS 3.9-3.10 & selections from Ch. 10]
Week 6 (Feb. 21)

Performance analysis and algorithm independent ML [DHS Ch. 9]

 HW5 Solutions
Prob. 3.17, 3.38 from DHS
(Optional problem 3.39) Due Feb. 28
Week 7 (Feb. 28)

Non-parametric techniques for density and
posterior probabilities, series expansion approximations
[DHS Ch. 4] Slides1 Slides2

 HW6 Solutions
Prob. 4.3, 4.13 DHS
Due Mar. 7
Week 8 (Mar. 7)

Linear discriminant functions and perceptrons.
[DHS 5.1-5.5]

HW7 Solutions
Prob. 5.4, 5.5, 5.10 DHS
Mar. 14

SPRING BREAK
Week 9 (Mar. 21)

Other descnet procedures.
[DHS Ch. 5.6-5.10] Slides3 Slides4 Slides5
Week 10 (March 28)

Midterm

SCOP HW Due Apr. 18
Week 11 (Apr. 4)

Support Vector Machines
[DHS Ch. 5.11-5.12]

 HW9 Prob. 5.32, 5.34, DHS (Extra Credit 5.33)
Due Apr. 11
Week 12 (Apr. 11)

Multilayer Perceptron
Slides6
Week 13 (April 18)

Unsupervised learning and clustering. [DHS 10.1-10.9]

 Term Project Due ??
Week 14 (Apr. 25)

Validity, dimensionaltiy reduction. [DHS 10.10-10.14]
Week 15 (May 2)

Project Presentations