Derive an expression for the outer product (Quasi-Newton) approximation to the Hessian matrix for a network having outputs with a softmax output unit activation function

and output unit activations , where , and a cross-entropy error function , corresponding to the result

with for the sum-of-squares error function

and a linear output unit activation function, i.e. .

Haeusler Stefan 2013-01-16