MVE187, Computational methods for Bayesian statistics, 2018/19

Latest news

Welcome to the course! The schedule for the course can be found in TimeEdit.

Teachers

Course coordinator, teacher, and lab supervisor: Petter Mostad

Course literature

Our required reading will be selected from three different textbooks, all available online, and some occasional extra material described in lecture notes. Specifically, we will use:

(A) Albert: Bayesian Computation with R. Available as e-book; see also here. We cover (parts of) the chapters listed under "Program".
(RC) Robert and Casella: Introducing Monte Carlo Methods with R. Available as e-book. Also available: Solutions to odd-numbered exercises, and errata / additional errata. We cover parts of the chapters listed under "Program".
(B) Bishop: Pattern Recognition And Machine Learning, available here. We cover parts of the chapters listed under "Program".
Lecture notes for some of the course lectures will be made available below. Occasionally they will contain additional material not covered in the texts above.

Some additional reading, for the interested student:

Gelman et al: Bayesian Data Analysis. Good book for learning about Bayesian theory and data analysis.
Liu: Monte Carlo Strategies in Scientific Computing. Available as e-book.
Johansen and Evers: Monte Carlo Methods. Available here.
Robert and Casella: Monte Carlo Statistical Methods. Available as e-book.
Gentle, Härdle and Mori: Handbook of Computational Statistics. Available as e-book.

English-Swedish mathematical dictionary

Program

Lectures

Rooms: Tuesday lectures are in Pascal, while Thursday lectures are in Euler.

Day	Literature (e.g., A2 means chapter 2 in Albert (see above))	Contents
Tuesday 4/9, 13:15 - 15:00,	Lecture Notes.	Lecture 1: Introduction. Motivation: Problems with classical frequentist inference. Basic ideas of Bayesian statistics.
Thursday 6/9 13:15 - 15:00	Lecture Notes. A2 (except 2.5), A3. B2 (2.1, 2.4.1, 2.4.2).	Lecture 2: Basics ideas of conjugacy. Simple computations. Mixtures. The exponential family of distributions.
Tuesday 11/9, 13:15 - 15:00	Lecture Notes. A4.1 - A4.4. B2.3.1 - B2.3.6 Rcode examples	Lecture 3: Some multivariate conjugacies. Bayesian inference by discretization. Low dimensional inference.
Thursday 13/9, 13:15 - 15:00	Lecture Notes. A5.1-5.8. RC2, RC3.1-2. B11.1.1-2. Rcode examples	Lecture 4: Inference by simulation. Monte Carlo integration. Basic simulation methods.
Tuesday 18/9, 13:15 - 15:00	Lecture Notes. A6-7. RC6,7,8. B11.	Lecture 5: Introduction to Markov chain Monte Carlo (MCMC) methods.
Thursday 20/9, 13:15 - 15:00	R code. A6-7,10. RC6,7,8. B11.	Lecture 6: More on MCMC.
Tuesday 25/9, 13:15 - 15:00	R code. Lecture Notes. A6-7. RC6,7,8. B11.	Lecture 7: Hierarchical models. Gibbs sampling.
Thursday 2/10, 13:15 - 15:00	Lecture Notes. R code. A6-7. RC6,7,8. B11.	Lecture 8: Checking convergence. More on MCMC simulations.
Tuesday 4/10, 13:15 - 15:00	Lecture Notes. R code. A6-7. RC6,7,8. B11.	Lecture 9: Extensions of Metropolis-Hastings. More basic simulation methods.
Thursday 28/9 13:15 - 15:00	Lecture Notes. R code. B9. RC5.	Lecture 10: Some information theory. The EM algorithm.
Tuesday 9/10, 13:15 - 15:00	Lecture Notes. B8.	Lecture 11: Introduction to Graphical Models.
Thursday 11/10, 13:15 - 15:00	Lecture Notes. R code. B8.	Lecture 12: Inference for Graphical Models. The Forward-Backward algorithm.
Tuesday 16/10, 13:15 - 15:00	Lecture Notes. Rcode, Rcode, Rcode. B13.	Lecture 13: Inference for Graphical Models. The Viterbi algorithm. The Baum-Welch algorithm.
Thursday 18/10, 13:15 - 15:00	Lecture Notes. A8. A study on medical age assessment. with Presentation	Lecture 14: Model choice. Bayesian modelling and Bayesian inference in practice.
Tuesday 23/10, 13:15 - 15:00	Lecture Notes.	Lecture 15: Some ways forward. Variational Bayes. Approximate Bayesian Computation (ABC).
Thursday 25/10, 13:15 - 15:00		Lecture 16: Review.

The final written exam is held Saturday 27 October 14:00 - 18:00.

In addition to the lectures, you will work individually and in groups on theoretical exercises and computer exercises. During the course, Petter Mostad will be available every Thursday 15:15 - 17:00 in the computer room MVF22 (except 25 October) and Tuesdays 10:00 - 11:45 (except 25 September, when the time is 15:15 - 17:00) in his office MVH3017 to answer questions.

Recommended exercises:
Learning R: A (i.e., Albert) 1.6: Exercise 4, RC (i.e., Robert and Casella) Exercise 1.19
One-parameter models: A2.9 Exercises 1,4,5; A3.9 Exercises 1,3,4.
Several parameters, discretization: A4.8 Exercises 1, 4, 7.
Simple simulation: RC Exercises 2.11, 2.12, 2.18, 2.22.

Bayesian computation: A5.13: Exercises 1, 4.
MCMC: A6.13: Exercises 2, 4. A7.12: Exercises 1, 2. A10.7: Exercises 1, 3. RC Exercises 6.7, 6.8, 7.11, 8.2, 8.8
Simulation methods: RC Exercise 3.13
EM algorithm: RC Exercises 5.8, 5.9, 5.10.
Model comparison: A8.11: Exercises 1, 3
Inference for Markov chains: Extra exercises, with solutions.

Computer labs

To understand and learn the methods of this course, it is essential to work with examples on a computer. Our textbooks contain a large number of exercises, and recommended exercises will be listed above.

As an obligatory part of the course, each student must do 3 assignments. The deadlines for these are 20 September, 4 October, and 18 October. Details about the assignments will be available via PingPong for Chalmers students and GUL for GU students. Answers must also be handed in via PingPong/GUL. Although students are welcome to cooperate in their work, each student must be prepared to explain orally all details of their own written answers.

The weekly computer labs will function as support for students, and an opportunity to get individual help with either exercises from the textbooks or with the assignments. Students choose and prioritize themselves what to work with, and how to work. The computer labs are held in MVF22 15:15 - 17:00 every Thursday starting 4/10 and ending 18/10.

As all the course material uses the R language for examples and illustrations, students should also use this language. Students who are not familiar with this language need to study it individually during the first weeks of the course. See, for example, the introductory chapters of our textbooks.

Course requirements

The learning goals of the course can be found in the course plan. To paraphrase, the goal is to give students a firm understanding of the principles of Bayesian inference and how they differ from frequentist inference principles, as well as a good technical capability for making such computational inference in a range of models of medium complexity.

Assignments

See under Computer labs above.

Examination

To pass the course you need to

have approved answers to the assignments. This is registered as a separate 2 hp project in Ladok, so assignments can be approved a different year from the final written exam.
Pass the final written exam. No aids are allowed during the exam. Your grade for the course is based on the grade from the written exam. The exam will contain questions asking you to describe/explain/prove theory, and questions asking you to apply theory to specific situations, to obtain specific equations or computational algorithms. Examples of questions from previous exams and practice exams that are relevant for you will be given under "Old exams" below.

The final written exam will give a maximal score of 30 points.
The Chalmers grading scale is 12-17.5: 3; 18-23.5: 4; 24-30: 5.
The GU grading scale is 12-21.5: G; 22-30: VG.

Examination procedures

In Chalmers Student Portal you can read about when exams are given and what rules apply on exams at Chalmers. In addition to that, there is a schedule when exams are given for courses at University of Gothenburg. Before the exam, it is important that you sign up for the examination. If you study at Chalmers, you can do this from the Chalmers Student Portal, and if you study at University of Gothenburg, you sign up via GU's Student Portal.

At the exam, you should be able to show valid identification.

After the exam has been graded, you can see your results in Ladok by logging on to your Student portal.

At the annual (regular) examination:
When it is practical, a separate review is arranged. The date of the review will be announced here on the course homepage. Anyone who can not participate in the review may thereafter retrieve and review their exam at the Mathematical Sciences Student office. Check that you have the right grades and score. Any complaints about the marking must be submitted in writing at the office, where there is a form to fill out.

At re-examination:
Exams are reviewed and retrieved at the Mathematical Sciences Student office. Check that you have the right grades and score. Any complaints about the marking must be submitted in writing at the office, where there is a form to fill out.

Course Evaluation

At the beginning of the course at least two students representatives from Chalmers and GU will be nominated. The teachers will meet the elected representatives 3 times over the period to keep track of how the course is going.

Student representatives Chalmers:
Aditya Bhadravathi Sridhara
Agathe Gourrat
Isak Hjortgren
Sandra Jansson
Sharif Zahiri

Student representatives GU:
Sebastian Ahlman
Margareta Carlerös
Sören Richard
Zeinab Tir Sahar
Chengjie Wang

The largest change from the 2017 course edition is that the Bishop textbook has been added to the textbooks we use. This means that more of the required reading is covered in actual textbooks. Also, weekly office hours have been added to the schedule.

Old exams

Exam 2019-08-29 with suggested solutions.

Exam 2019-01-08 with suggested solutions.

Exam 2018-10-27 with suggested solutions.

Exam 2018-01-02 with suggested solutions.

Exam 2017-10-21 with suggested solutions.

=============================================

Recent exams in MVE186/MSA100:

Exam 2017-06-05 (extra, irregularly scheduled exam) with suggested solutions.

Exam 2017-01-02, with suggested solutions. You may skip question 8.

Exam 2016-10-22, with suggested solutions.

Some older exams in MVE186/MSA100:

Exam 2015-10-24, with suggested solutions: You may skip questions 2, 4, and possibly 6.

Exam 2015-01-05. You may skip questions 1, 4, and 6, and possibly 5.

Exam 2014-10-27, with suggested solutions.

A mock exam from 2014. You may skip question 2.

Some even older exams in MVE185/MSA100:

Exam 2009-10-24, with suggested solutions: You may skip question 7.

Exam 2008-10-25, with suggested solutions: You may skip question 2.