SYST/STAT 664 Bayesian Inference

and Decision Theory

Spring Semester, 2011
The objective of this course is to introduce students to the theory of Bayesian inference and decision making and to introduce applications in information technology and engineering. Students will learn the fundamentals of the Bayesian theory of inference, including probability as a representation for degrees of belief, the likelihood principle, the use of Bayes Rule to revise beliefs based on evidence, conjugate prior distributions for common statistical models, and methods for approximating the posterior distribution. Graphical models are introduced for representing complex probability and decision models by specifying modular components. A brief overview is given of basic concepts in decision analysis, including influence diagrams, decision trees, and utility theory.

Instructor: Kathryn Blackmond Laskey

email: klaskey@gmu.edu
web: http://www.ite.gmu.edu/~klaskey
phone: 703-993-1644
Office hours: Wednesday 3:00-4:00, Thursday 5:00-6:00 or by appointment,
Office location: Room 2214 ENGR

Prerequisites: STAT 544 or STAT 554 or equivalent (a strong grounding in probability with calculus)

Textbook

Required text: This book, published in 2009, was not available the last time this course was taught. It provides about the right level of coverage and is a very modern treatment. An electronic edition of this book is available to George Mason University students and faculty from the university library.

Hoff, Peter D., A First Course in Bayesian Statistical Methods, Springer, 2009.

Recommended reference book: This is the most comprehensive text and reference book on Bayesian methods I have found. The hyperlink below contains reviews, exercises, data sets and software.

Gelman, A., Carlin, J., Stern, H. and Rubin, D., Bayesian Data Analysis (2nd edition), Chapman & Hall, 2004.

Alternate text: Although it is far less comprehensive, and not as useful as a reference, some students have found the text by Peter Lee accessible and helpful. Again, the hyperlink contains additional information, including exercises, solutions, errata and software.
Lee, Peter, Bayesian Statistics: An Introduction (3rd edition), Arnold, 1997.

Software

All students are expected to have access software package that can perform elementary data analysis functions such as scatterplots, histograms, hypothesis tests, confidence intervals, linear regression, etc. Many of the homework and exam exercises can be managed with a full-featured spreadsheet package such as Microsoft Excel. Others will require more power (see below).
We will make use of R, a powerful statistical graphics and computing language, and MCMCpack, an R package for Bayesian computation. R code for many of the examples in the text is available from the author's web site.

Requirements

Grades will be based on the following:

Homework assignments 20%

Midterm exam (take-home) 30%

Final exam (take-home) 30%

Project 20%

Homework problems will be due one week from the day they are assigned. Eight to ten assignments will be given through the semester. Students are encouraged to work together on homework exercises, but solutions must be written up individually. Exams will be take-home and will be similar to the homework problems. Students are expected to work by themselves on the exams.

Policies and Resources

Schedule

The topics for each unit are listed below, along with readings from the text. The midterm exam will be posted on March 8 and will be due on March 23 at the start of class. It will include all material covered up through the date the exam is distributed. The final exam will be posted on May 3 and will be due on Wednesday, May 11 at 10:00PM. The final exam will be cumulative. All students are required to do a data analysis project. The project is due on Monday, May 16 at 11:59 PM.

Unit 1 Course Overview
Week 1 Hoff, Chapter 1

Unit 2 Random variables, Parametric Models and Inference from Observation
Weeks 2-3 Hoff, Chapter 2

Unit 3 Statistical Models with a Single Parameter Weeks 3-4 Hoff, Chapter 3

Unit 4 Monte Carlo Approximation
Weeks 5-6 Hoff, Chapter 4

Unit 5 The Normal Model
Week 6-7 Hoff, Chapter 5

Unit 6 Gibbs Sampling
Week 8-9
Hoff, Chapter 6

Unit 7 Hierarchical Bayesian Models
Week 10-11 Hoff, Chapter 8

Unit 8 Hypothesis Tests and Bayes Factors
Weeks 11-12 Hoff, Chapter 9

Unit 9
Linear Regression Weeks 12-13 Hoff, Chapter 9, readings

Unit 10
Metropolis-Hastings Sampling
Week 14 Hoff, Chapter 10

Unit 1	Course Overview	Week 1	Hoff, Chapter 1
Unit 2	Random variables, Parametric Models and Inference from Observation	Weeks 2-3	Hoff, Chapter 2
Unit 3	Statistical Models with a Single Parameter	Weeks 3-4	Hoff, Chapter 3
Unit 4	Monte Carlo Approximation	Weeks 5-6	Hoff, Chapter 4
Unit 5	The Normal Model	Week 6-7	Hoff, Chapter 5
Unit 6	Gibbs Sampling	Week 8-9	Hoff, Chapter 6
Unit 7	Hierarchical Bayesian Models	Week 10-11	Hoff, Chapter 8
Unit 8	Hypothesis Tests and Bayes Factors	Weeks 11-12	Hoff, Chapter 9
Unit 9	Linear Regression	Weeks 12-13	Hoff, Chapter 9, readings
Unit 10	Metropolis-Hastings Sampling	Week 14	Hoff, Chapter 10