Probabilistic Graphical Models
provide a way of modeling high dimensional random structures and have found wide applications.
The popular Hidden Markov Models, Markov Random fields. LDA, fall within this framework.
A graphical model is a graph whose nodes are random variables. The graphical model
formalism uses the structure of the graph to code independence relations. The goal of
this course is to provide a systematic introduction to the underlying probability and
Some of the topics that will be covered are :
- Basic probability and statistics: Independence, Conditional independence, Multivariate Normal distribution.
Estimation of parameters, Maximum Likelihood, Bayesian methods. Exponential families
- Directed graphical models (Bayesian networks). D-separation and conditional
Independence. Markov equivalence. I-equivalence. Undirected Graphical Models (Markov Networks). Markov Networks and Independence. Gibbs distribution and Markov networks.
- Gaussian networks, Gaussian Bayesian Networks. Gaussian markov random fields.
Hidden Markov models, Kalman filters, Markov random fields, Generative modeling of data, LDA.
- Exact inference in Bayesian networks: Junction tree algorithm, Belief Propagation, Forward - Backward algorithm in HMM
- Approximation inference: Variational techniques, MCMC techniques, Gibbs sampling
- Parameter learning Learning in fully observed models, multinomial and multivariate learning, EM algorithm
- Structure learning. Search over DAGs, Search over DAG patterns, Model averaging, AIC, BIC
- Daphne Koller and Nir Friedman., Probabilistic Graphical Models.
- Richard Neapolitan., Learning Bayesian.
- Parts of these two texts will form the core of the course. As additional topics are discussed
relevant references will be provided.