Next: Decision Theory [2+2* P] Up: NNA_Exercises_2012 Previous: Linear models for regression

# Linear models for regression II[4 P]

Apply curve fitting by finding the MAP solution for a linear model with Gaussian basis functions and a target function with Gaussian noise. You are required to use MATLAB for this assignment. A template for the MATLAB file as presented in the lecture is available for download on the course homepage1. Complete the lines marked with in the file biasvariance.m (search for the tag HOMEWORK) as required for the following points.

a)
Generate datasets, each containing data points, where is drawn uniformly and independently from the interval and is given by the deterministic function and a zero mean Gaussian random variable with precision (inverse variance) .

b)
Implement Gaussian basis functions defined by

with for the linear model.

c)
Calculate the MAP solutions for the linear model for each of the datasets. The prior distribution for the weights of the linear model are given by

with and where denotes the identity matrix. Determine the average prediction

where denotes the output of the model trained on datatset , the integrated squared bias and the integrated variance

where and , and the average test error (mean squared error) for 1000 test samples drawn from the same distribution as the training samples. Determine each of this quantities for . Plot and interpret their dependence on and discuss the results in the context of the bias-variance trade-off. Create a plot that looks like Figure 1. What regularization parameter should be chosen.

Next: Decision Theory [2+2* P] Up: NNA_Exercises_2012 Previous: Linear models for regression
Haeusler Stefan 2013-01-16