Advanced Object-Oriented Programming in R by Thomas Mailund

Author:Thomas Mailund
Language: eng
Format: epub
Publisher: Apress, Berkeley, CA

Bayesian Linear Regression

The simplest form of linear regression fits a line to data points. Imagine we have vectors x and y, and we wish to produce coefficients w[1] and w[2] such that y[i] = w[1] + w[2] x[i] + e[i] where the e is a vector of errors that we want to make as small as possible. We typically assume that the errors are identically normally distributed when we consider it a statistical problem, and so we want to have the minimal variance of the errors. When fitting linear models with the lm function, you get the maximum likelihood values for the weights w[1] and w[2], but if you wish to do Bayesian statistics you should instead consider this weight vector w as a random variable, and fit it to the data in x, while y means updating it from its prior distribution to its posterior distribution.

A typical distribution for linear regression weights is the normal distribution. If we consider the weights’ multivariate normal distributed as their prior distribution, then their posterior distribution given the data will also be normally distributed, which makes the mathematics very convenient.

We will assume that the prior distribution of w is a normal distribution with mean zero and independent components, so a diagonal covariance matrix. This means that, on average, we believe the line we are fitting to be flat and going through the plane origin, but how strongly we believe this depend on values in the covariance matrix. This we will parameterize with a so-called hyperparameter, a, that is the precision—one over the variance—of the weight components. The covariance matrix will have 1/a on its diagonal and zeros off-diagonal.

We can represent a distribution over weights as the mean and covariance matrix of a multinomial normal distribution and construct the prior distribution from the precision like this:

weight_distribution <- function(mu, S) {

structure(list(mu = mu, S = S), class = "wdist")

}

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.