Learning Materials
Features
Discover
Chapter 2: Problem 22
Show that the inverse of a symmetric matrix is itself symmetric.
Over 30 million students worldwide already upgrade their learning with 91影视!
These are the key concepts you need to understand to accurately answer the question.
All the tools & learning materials you need for study success - in one app.
Get started for free
In this exercise, we prove that the binomial distribution \((2.9)\) is normalized. First use the definition (2.10) of the number of combinations of \(m\) identical objects chosen from a total of \(N\) to show that $$ \left(\begin{array}{l} N \\ m \end{array}\right)+\left(\begin{array}{c} N \\ m-1 \end{array}\right)=\left(\begin{array}{c} N+1 \\ m \end{array}\right) $$ Use this result to prove by induction the following result $$ (1+x)^{N}=\sum_{m=0}^{N}\left(\begin{array}{l} N \\ m \end{array}\right) x^{m} $$ which is known as the binomial theorem, and which is valid for all real values of \(x\). Finally, show that the binomial distribution is normalized, so that $$ \sum_{m=0}^{N}\left(\begin{array}{l} N \\ m \end{array}\right) \mu^{m}(1-\mu)^{N-m}=1 $$ which can be done by first pulling out a factor \((1-\mu)^{N}\) out of the summation and then making use of the binomial theorem.
( \(\star)\) The uniform distribution for a continuous variable \(x\) is defined by $$ \mathrm{U}(x \mid a, b)=\frac{1}{b-a}, \quad a \leqslant x \leqslant b . $$ Verify that this distribution is normalized, and find expressions for its mean and variance.
\((\star \star)\) Show that the entropy of the multivariate Gaussian \(\mathcal{N}(\mathbf{x} \mid \boldsymbol{\mu}, \boldsymbol{\Sigma})\) is given by $$ \mathrm{H}[\mathbf{x}]=\frac{1}{2} \ln |\boldsymbol{\Sigma}|+\frac{D}{2}(1+\ln (2 \pi)) $$ where \(D\) is the dimensionality of \(\mathrm{x}\).
This exercise demonstrates that the multivariate distribution with maximum entropy, for a given covariance, is a Gaussian. The entropy of a distribution \(p(\mathbf{x})\) is given by $$ \mathrm{H}[\mathbf{x}]=-\int p(\mathbf{x}) \ln p(\mathbf{x}) \mathrm{d} \mathbf{x} $$ We wish to maximize \(\mathrm{H}[\mathbf{x}]\) over all distributions \(p(\mathbf{x})\) subject to the constraints that \(p(\mathbf{x})\) be normalized and that it have a specific mean and covariance, so that $$ \begin{aligned} &\int p(\mathbf{x}) \mathrm{d} \mathbf{x}=1 \\ &\int p(\mathbf{x}) \mathbf{x} \mathrm{d} \mathbf{x}=\boldsymbol{\mu} \\ &\int p(\mathbf{x})(\mathbf{x}-\boldsymbol{\mu})(\mathbf{x}-\boldsymbol{\mu})^{\mathrm{T}} \mathrm{d} \mathbf{x}=\mathbf{\Sigma} . \end{aligned} $$ By performing a variational maximization of (2.279) and using Lagrange multipliers to enforce the constraints \((2.280),(2.281)\), and \((2.282)\), show that the maximum likelihood distribution is given by the Gaussian (2.43).
Consider a \(D\)-dimensional Gaussian random variable \(\mathrm{x}\) with distribution \(\mathcal{N}(\mathbf{x} \mid \boldsymbol{\mu}, \boldsymbol{\Sigma})\) in which the covariance \(\boldsymbol{\Sigma}\) is known and for which we wish to infer the mean \(\boldsymbol{\mu}\) from a set of observations \(\mathbf{X}=\left\\{\mathbf{x}_{1}, \ldots, \mathbf{x}_{N}\right\\}\). Given a prior distribution \(p(\boldsymbol{\mu})=\mathcal{N}\left(\boldsymbol{\mu} \mid \boldsymbol{\mu}_{0}, \boldsymbol{\Sigma}_{0}\right)\), find the corresponding posterior distribution \(p(\boldsymbol{\mu} \mid \mathbf{X}) .\)
What do you think about this solution?
We value your feedback to improve our textbook solutions.
Explanations 
Textbooks 
Flashcards 
91影视 AI 
Notes 
Study Plans 
Study Sets 
Find a degree 
Magazine