Logistics | Course Info | Syllabus | Other Resources

Logistics

Please use Piazza for all questions related to lectures and coursework. For SCPD students, please email scpdsupport@stanford.edu or call 650-741-1542.

Instructor

Stefano Ermon

Stefano Ermon
ermon [at] cs.stanford.edu

Website

Course Assistants

Jiaming Song

Jiaming Song
tsong [at] stanford.edu

Akshat Jindal

Akshat Jindal
akshatj [at] stanford.edu

Albert Gu

Albert Gu
albertgu [at] stanford.edu

Daniel Cai

Daniel Cai
danieltc [at] stanford.edu

Huizi Mao

Huizi Mao
huizi [at] stanford.edu

Tri Dao

Tri Dao
trid [at] stanford.edu

Course Information

Course Description

Probabilistic graphical models are a powerful framework for representing complex domains using probability distributions, with numerous applications in machine learning, computer vision, natural language processing and computational biology. Graphical models bring together graph theory and probability theory, and provide a flexible framework for modeling large collections of random variables with complex interactions. This course will provide a comprehensive survey of the topic, introducing the key formalisms and main techniques used to construct them, make predictions, and support decision-making under uncertainty.

The aim of this course is to develop the knowledge and skills necessary to design, implement and apply these models to solve real problems. The course will cover: (1) Bayesian networks, undirected graphical models and their temporal extensions; (2) exact and approximate inference methods; (3) estimation of the parameters and the structure of graphical models.

Prerequisites

Students are expected to have background in basic probability theory, statistics, programming, algorithm design and analysis.

Readings

Required Textbook: (“PGM”) Probabilistic Graphical Models: Principles and Techniques by Daphne Koller and Nir Friedman. MIT Press.

Course Notes: Available here. Student contributions welcome!

Lecture Videos: Instructions on finding them.

Further Readings:

Grading Policy

Homeworks (70%): There will be five homeworks with both written and programming parts. Each homework is centered around an application and will also deepen your understanding of the theoretical concepts. Homeworks will be posted on Piazza.

Final Exam (30%): Wednesday, March 18, 2020, 8:30 - 11:30 a.m.

Extra Credit (+3%): You will be awarded with up to 3% extra credit if you answer other students’ questions on Piazza in a substantial and helpful way, or contribute to the course notes on GitHub with pull requests.

Assignments

Written Assignments: Homeworks should be written up clearly and succinctly; you may lose points if your answers are unclear or unnecessarily complicated. You are encouraged to use LaTeX to writeup your homeworks (here is a template), but this is not a requirement.

Homework Submission: All students (non-SCPD and SCPD) should submit their assignments electronically via Gradescope.

Late Homework: You have 6 late days to use at any time during the term without penalty. For a particular homework, you can use only two late days. Once you run out of late days, you will incur a 25% penalty for each extra late day you use. Each late homework should be clearly marked as “Late” on the first page.

Regrade Policy: If you believe that the course staff made an error in grading, you may submit a regrade request through Gradescope within one week of receiving your grade. Please be as specific as possible with your regrade request.

Collaboration Policy and Honor Code: You are free to form study groups and discuss homeworks and projects. However, you must write up homeworks and code from scratch independently without referring to any notes from the joint session. You should not copy, refer to, or look at the solutions from previous years’ homeworks in preparing your answers. It is an honor code violation to intentionally refer to a previous year’s solutions, either official or written up by another student. Anybody violating the honor code will be referred to the Office of Community Standards.

Syllabus

Week Dates Topics Required Readings Assignments
1 Jan. 7 & 9 Introduction, Probability Theory, Bayesian Networks PGM Ch. 1-3 HW 1 (Jan 7 - 21)
2 Jan. 14 & 16 Undirected models PGM Ch. 4
3 Jan. 21 & 23 Learning Bayes Nets PGM Ch. 16-17 HW 2 (Jan 22 - 31)
4 Jan. 28 & 30 Exact Inference; Message Passing PGM Ch. 9-10 HW 3 (Jan 31 - Feb 11)
5 Feb. 4 & 6 Sampling PGM Ch. 12
6 Feb. 11 & 13 MAP Inference; Structured prediction PGM Ch. 13 HW 4 (Feb 11 - 25)
7 Feb. 18 & 20 Parameter Learning PGM Ch. 19-20
8 Feb. 25 & 27 Bayesian Learning; Structure Learning PGM Ch. 17-18 HW 5 (Feb 25 - Mar 10)
9 Mar. 3 & 5 Exponential families; variational inference PGM Ch. 8 & 11; GEV Section 3
10 Mar. 10 & 12 Advanced topics and conclusions

Many thanks to David Sontag, Adnan Darwiche, Vibhav Gogate, and Tamir Hazan for sharing material used in slides and homeworks.

Additional TA Sessions

Friday 3:30p-4:20p at Gates B01. Attendence is optional but encouraged.

Other Resources

There are many software packages available that can greatly simplify the use of graphical models. Here are a few examples:


The Stanford University Honor Code

The Honor Code (https://communitystandards.stanford.edu/policies-and-guidance/honor-code) is a part of this course. The Honor Code is the university's statement on academic integrity written by students in 1921. It articulates university expectations of students and faculty in establishing and maintaining the highest standards in academic work. The Honor Code is an undertaking of the students, individually and collectively: that they will not give or receive aid in examinations; that they will not give or receive unpermitted aid in class work, in the preparation of reports, or in any other work that is to be used by the instructor as the basis of grading; that they will do their share and take an active part in seeing to it that others as well as themselves uphold the spirit and letter of the Honor Code. The faculty on its part manifests its confidence in the honor of its students by refraining from proctoring examinations and from taking unusual and unreasonable precautions to prevent the forms of dishonesty mentioned above. The faculty will also avoid, as far as practicable, academic procedures that create temptations to violate the Honor Code. While the faculty alone has the right and obligation to set academic requirements, the students and faculty will work together to establish optimal conditions for honorable academic work.

Students with Documented Disabilities

Students who may need an academic accommodation based on the impact of a disability must initiate the request with the Office of Accessible Education (OAE). Professional staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty dated in the current quarter in which the request is being made. Students should contact the OAE as soon as possible since timely notice is needed to coordinate accommodations. The OAE is located at 563 Salvatierra Walk (phone: 723-1066, URL: http://oae.stanford.edu).

Names and Pronouns

Use the names and pronouns (e.g., they/them, she/her, he/him, just a name, or something else) indicated by your classmates for themselves. If you don’t want to share a set of pronouns for yourself, that is perfectly acceptable, too. If your name or pronouns change during the course, we invite you to share this with us and/or other students, so we may talk with you and refer to your ideas in discussion as you would wish.