SYST/STAT 664

Bayesian Inference and Decision Theory

Kathryn Blackmond Laskey
Department of Systems Engineering
George Mason University
Course Description

Spring, 2007
ST1 Room 120


Many students find statistics to be a confusing collection of rules and procedures, the applicability of which seems to have little rhyme or reason.  The Bayesian approach provides a unified and intuitively appealing approach to the problem of drawing inferences from observations.   Bayesians view statistical inference as a problem in belief dynamics, or use of evidence about a phenomenon to revise and update our knowledge about it. Bayesian statistics is a scientifically justifiable way to integrate informed expert judgment with empirical data.  For a Bayesian, statistical inference cannot be treated entirely independently of the context of the decisions that will be made on the basis of the inferences.  This course introduces students to the Bayesian theory and modern computational methods for Bayesian inference.  Students will learn the commonalities and differences between the Bayesian and frequentist approaches to statistical inference, how to approach a statistics problem from the Bayesian perspective, and how to combine data with informed expert judgment in a sound way to derive useful and policy relevant conclusions.  Students will learn the necessary theory to develop a firm understanding of when and how to apply Bayesian and frequentist methods, and will also learn practical procedures for inference, hypothesis testing, and developing statistical models for phenomena.  Specifically, students will learn the fundamentals of the Bayesian theory of inference, including probability as a representation for degrees of belief, the likelihood principle, the use of Bayes Rule to revise beliefs based on evidence, conjugate prior distributions for common statistical models, methods for approximating the posterior distribution.  Bayesian networks, influence diagrams, and general graphical models are introduced for representing complex probability and decision models by specifying modular components.  Assignments make use of modern computational techniques and focus on appling the methods to practical problems.

Course Syllabus

Internet Lectures

This course is being offered as both a regular lecture course and an Internet course using the Network EducationWare (NEW) software developed at GMU's Network and Simulation Lab.  Students may attend lectures in person or may participate from home over the Internet. Students logged in electronically can hear lectures, see vugraphs, see what the instructor writes on the electronic whiteboard, and ask questions. All lectures will be recorded and can be played back at students' convenience.  Instructions for using the student client software can be found here.

Class Email List

Everyone in the class will be subscribed to the class email list. The email list will be used to communicate with students outside of class. If you wish to subscribe using an email other than your GMU email, you may do so by visiting the GMU listserv web site.  All email correspondence with the instructor should say STAT664 or SYST664 in the subject line.

Lecture Notes

Lecture notes for each chapter will be made available before the first class covering that chapter. You will need to download Adobe Acrobat Reader to read these lecture notes.  Links to the previous year's notes are provided for those who like to read ahead.  Be advised -- I make changes every year.

Homework Assignments

Email submission of homework assignments is accepted.  Please make sure your name and the assignment number are on every page, and that your name, the assignment number and the number "664" appear in the filename.  (Example: Smith_STAT664_HW1.doc).  Homework is due in class on the assigned due date. If it is submitted before 23:59 the day after the due date, you will receive 75% credit.  If it is submitted up to 1 week late, you will receive 50% credit.  If you have extenuating circumstances, please contact me in advance, and I will consider giving you additional time to complete the assignment for partial credit. There is a WebCT dropbox for electronic assignments.

Exams

Study Aids

Sites of Possible Interest

A recent web search on "Bayesian" yielded over 400,000 hits (in 0.21 seconds).  When you have some time, it would be a good idea to throw in a modifier or two to reflect your individual interests and browse.  There is lots of very interesting stuff out there.  I've culled a few tidbits to get you started.  Many of these sites also contain useful links to other Bayesian sites.
Professional Societies:
International Society for Bayesian Analysis
American Statistical Association Section on Bayesian Statistical Science
A Non-Comprehensive Sampling of Research Groups and Stat Departments with Bayesian Orientation:
Microsoft Machine Learning and Applied Statistics Group
Duke University Institute for Statistics and Decision Sciences
University of Washington Statistics Department
Carnegie Mellon University Department of Statistics
Bayesian Statistical Software:
Bayesian Analysis Using Gibbs Sampling
FirstBayes
Companion software to Peter Lee text in R and C++
Kevin Murphy's Bayesian Network Toolbox for MATLAB
Miscellaneous
The Bayesian Songbook (includes Frequentist Frenzy by world renowned songwriter Kathryn Blackmond Laskey)
Bayesians Worldwide