Bayesian Decision Theory and Machine Learning

Kathryn Blackmond Laskey
Department of Systems Engineering
George Mason University

This talk presents the Bayesian approach to machine learning. The talk begins with an overview the basic philosophy and approach of Bayesian decision theory. Next, application of the decision theoretic approach to machine learning is discussed. In decision theory, learning is viewed as a problem of inference, in which a prior distribution and data are used to infer a posterior distribution for parameters of interest. Problems in machine learning may be contrasted with more traditional statistical inference problems. Machine learning problems are characterized by very high dimensional parameter spaces and by models that are not "identifiable" -- that is, it may not be possible to distinguish on the basis of available training data which of several candidate representations is the "correct" one. This suggests an alternative characterization of the machine learning problem in decision theoretic terms. The learning task is viewed as acquiring a problem representation that has high utility, where utility depends both on accuracy (i.e., projected performance on problems not in the training set) and computational complexity. Theoretical and pragmatic arguments for Bayesian methods are presented. A summary of recent research in knowledge representations and learning methods is presented. A few applications of Bayesian learning are discussed.

View Presentation

Download Presentation