next up previous
Next: Logistic regression [3 P] Up: NNA_Exercises_2012 Previous: Perceptron learning rule [2

Iterative Reweighted Least Squares [3* P]

Show that the Hessian matrix $ {\bf H}$ for the logistic regression model, given by $ {\bf H} = {\bf\Phi}^T {\bf R} {\bf\Phi}$ , is positive definite. Here $ {\bf R}$ is a diagonal matrix with elements $ {\bf R}_{nn} = y_n (1-y_n)$ , and $ y_n$ is the output of the logistic regression model for input vector $ x_n$ . Hence show that the error function is a convex function of $ {\bf w}$ and that it has a unique minimum.

Haeusler Stefan 2013-01-16