91Ó°ÊÓ

Show that the 'tanh' function and the logistic sigmoid function (3.6) are related by $$ \tanh (a)=2 \sigma(2 a)-1 . $$ Hence show that a general linear combination of logistic sigmoid functions of the form $$ y(x, \mathbf{w})=w_{0}+\sum_{j=1}^{M} w_{j} \sigma\left(\frac{x-\mu_{j}}{s}\right) $$ is equivalent to a linear combination of 'tanh' functions of the form $$ y(x, \mathbf{u})=u_{0}+\sum_{j=1}^{M} u_{j} \tanh \left(\frac{x-\mu_{j}}{s}\right) $$ and find expressions to relate the new parameters \(\left\\{u_{1}, \ldots, u_{M}\right\\}\) to the original parameters \(\left\\{w_{1}, \ldots, w_{M}\right\\}\).

( \(\star\) ) Consider a data set in which each data point \(t_{n}\) is associated with a weighting factor \(r_{n}>0\), so that the sum-of-squares error function becomes $$ E_{D}(\mathbf{w})=\frac{1}{2} \sum_{n=1}^{N} r_{n}\left\\{t_{n}-\mathbf{w}^{\mathrm{T}} \phi\left(\mathbf{x}_{n}\right)\right\\}^{2} $$ Find an expression for the solution \(\mathrm{w}^{\star}\) that minimizes this error function. Give two alternative interpretations of the weighted sum-of- squares error function in terms of (i) data dependent noise variance and (ii) replicated data points.

Consider the linear basis function model in Section \(3.1\), and suppose that we have already observed \(N\) data points, so that the posterior distribution over \(\mathbf{w}\) is given by (3.49). This posterior can be regarded as the prior for the next observation. By considering an additional data point \(\left(\mathbf{x}_{N+1}, t_{N+1}\right)\), and by completing the square in the exponential, show that the resulting posterior distribution is again given by (3.49) but with \(\mathbf{S}_{N}\) replaced by \(\mathbf{S}_{N+1}\) and \(\mathbf{m}_{N}\) replaced by \(\mathbf{m}_{N+1}\).