Logistics | Course Info | Syllabus | Other Resources

Logistics

Please use Piazza for all questions related to lectures and coursework. For SCPD students, please email scpdsupport@stanford.edu or call 650-741-1542.

Instructor

Stefano Ermon

Stefano Ermon
ermon [at] cs.stanford.edu

Website

Course Assistants

Shengjia Zhao

Shengjia Zhao
zhaosj12 [at] stanford.edu

Stephanie Wang

Stephanie Wang
steph17 [at] stanford.edu

Nate Gruver

Nate Gruver
ngruver [at] stanford.edu

Owen Wang

Owen Wang
ojwang [at] stanford.edu

Chris Yeh

Chris Yeh
chrisyeh [at] stanford.edu

Zhangyuan Wang

Zhangyuan Wang
zywang17 [at] stanford.edu

Course Information

Course Description

Probabilistic graphical models are a powerful framework for representing complex domains using probability distributions, with numerous applications in machine learning, computer vision, natural language processing and computational biology. Graphical models bring together graph theory and probability theory, and provide a flexible framework for modeling large collections of random variables with complex interactions. This course will provide a comprehensive survey of the topic, introducing the key formalisms and main techniques used to construct them, make predictions, and support decision-making under uncertainty.

The aim of this course is to develop the knowledge and skills necessary to design, implement and apply these models to solve real problems. The course will cover: (1) Bayesian networks, undirected graphical models and their temporal extensions; (2) exact and approximate inference methods; (3) estimation of the parameters and the structure of graphical models.

Prerequisites

Students are expected to have background in basic probability theory, statistics, programming, algorithm design and analysis.

Readings

Required Textbook: (“PGM”) Probabilistic Graphical Models: Principles and Techniques by Daphne Koller and Nir Friedman. MIT Press.

Course Notes: Available here. Student contributions welcome!

Lecture Videos: Instructions on finding them.

Further Readings:

Grading Policy

Homeworks (70%): There will be five homeworks with both written and programming parts. Each homework is centered around an application and will also deepen your understanding of the theoretical concepts. Homeworks will be posted on Piazza.

Final Exam (30%): Wednesday, March 20, 2019, 8:30-11:30am.

Extra Credit (+3%): You will be awarded with up to 3% extra credit if you answer other students’ questions on Piazza in a substantial and helpful way, or contribute to the course notes on GitHub with pull requests.

Assignments

Written Assignments: Homeworks should be written up clearly and succinctly; you may lose points if your answers are unclear or unnecessarily complicated. You are encouraged to use LaTeX to writeup your homeworks (here is a template), but this is not a requirement.

Homework Submission: All students (non-SCPD and SCPD) should submit their assignments electronically via Gradescope.

Late Homework: You have 6 late days to use at any time during the term without penalty. For a particular homework, you can use only two late days. Once you run out of late days, you will incur a 25% penalty for each extra late day you use. Each late homework should be clearly marked as “Late” on the first page.

Regrade Policy: If you believe that the course staff made an error in grading, you may submit a regrade request through Gradescope within one week of receiving your grade. Please be as specific as possible with your regrade request.

Collaboration Policy and Honor Code: You are free to form study groups and discuss homeworks and projects. However, you must write up homeworks and code from scratch independently without referring to any notes from the joint session. You should not copy, refer to, or look at the solutions from previous years’ homeworks in preparing your answers. It is an honor code violation to intentionally refer to a previous year’s solutions, either official or written up by another student. Anybody violating the honor code will be referred to the Office of Community Standards.

Syllabus

Week Dates Topics Required Readings Assignments
1 Jan. 8 & 10 Introduction, Probability Theory, Bayesian Networks PGM Ch. 1-3 HW 1 released
2 Jan. 15 & 17 Undirected models PGM Ch. 4
3 Jan. 22 & 24 Learning Bayes Nets PGM Ch. 16-17 HW 2 released
4 Jan. 29 & 31 Exact Inference; Message Passing PGM Ch. 9-10 HW 3 released
5 Feb. 5 & 7 Sampling PGM Ch. 12
6 Feb. 12 & 14 MAP Inference; Structured prediction PGM Ch. 13 HW 4 released
7 Feb. 19 & 21 Parameter Learning PGM Ch. 19-20
8 Feb. 26 & 28 Bayesian Learning; Structure Learning PGM Ch. 17-18 HW 5 released
9 Mar. 5 & 7 Exponential families; variational inference PGM Ch. 8 & 11; GEV Section 3
10 Mar. 12-14 Advanced topics and conclusions

Many thanks to David Sontag, Adnan Darwiche, Vibhav Gogate, and Tamir Hazan for sharing material used in slides and homeworks.

Other Resources

There are many software packages available that can greatly simplify the use of graphical models. Here are a few examples: