Probabilistic Graphical Models

Graphical models provide a way of modeling high dimensional random structures and have found wide applications. The popular Hidden Markov Models, Markov Random fields. LDA, fall within this framework. A graphical model is a graph whose nodes are random variables. The graphical model formalism uses the structure of the graph to code independence relations. The goal of this course is to provide a systematic introduction to the underlying probability and statistical issues

Some of the topics that will be covered are :

• Basic probability and statistics: Independence, Conditional independence, Multivariate Normal distribution. Estimation of parameters, Maximum Likelihood, Bayesian methods. Exponential families
• Directed graphical models (Bayesian networks). D-separation and conditional Independence. Markov equivalence. I-equivalence. Undirected Graphical Models (Markov Networks). Markov Networks and Independence. Gibbs distribution and Markov networks.
• Gaussian networks, Gaussian Bayesian Networks. Gaussian markov random fields. Hidden Markov models, Kalman filters, Markov random fields, Generative modeling of data, LDA.
• Exact inference in Bayesian networks: Junction tree algorithm, Belief Propagation, Forward - Backward algorithm in HMM
• Approximation inference: Variational techniques, MCMC techniques, Gibbs sampling
• Parameter learning Learning in fully observed models, multinomial and multivariate learning, EM algorithm
• Structure learning. Search over DAGs, Search over DAG patterns, Model averaging, AIC, BIC

References

• Daphne Koller and Nir Friedman., Probabilistic Graphical Models.
• Richard Neapolitan., Learning Bayesian.
• Parts of these two texts will form the core of the course. As additional topics are discussed relevant references will be provided.