91Ó°ÊÓ

A vector \(\mathbf{x} \in \mathbb{R}^{n}\) minimizes the Euclidean norm \(\|\mathbf{b}- A\mathbf{x}\|_{2}\), if this norm is minimized for this vector among all possible vectors in \(\mathbb{R}^{n}\). This means that for any other vector \(\mathbf{x}' \in \mathbb{R}^{n}\), we have \(\|\mathbf{b}- A\mathbf{x}\|_{2} \leq \|\mathbf{b}- A\mathbf{x}'\|_{2}\).

Now we rewrite the given expression in terms of the singular value decomposition: \[ A \mathbf{x} = U \Sigma V^T \mathbf{x} \] Let's substitute the expression of \(\mathbf{x}\) and rewrite the Euclidean norm: \[ \|\mathbf{b} - A\mathbf{x}\|_{2} = \|\mathbf{b} - U \Sigma V^T (A^{+} \mathbf{b} + \sum_{i=r+1}^{n} c_i \mathbf{v}_i)\|_{2} \]

Let's expand the expression inside the norm: \[ \mathbf{b} - U \Sigma V^T (A^{+} \mathbf{b} + \sum_{i=r+1}^{n} c_i \mathbf{v}_i) = \mathbf{b} - U \Sigma V^T A^{+} \mathbf{b} - U \Sigma V^T\left(\sum_{i=r+1}^{n} c_i \mathbf{v}_i\right) \] Since \( V^T A^{+} = (AV)^+ V^T = I\), \[ \mathbf{b} - U \Sigma V^T A^{+} \mathbf{b} - U \Sigma V^T\left(\sum_{i=r+1}^{n} c_i \mathbf{v}_i\right) = U U^T \mathbf{b} - U \Sigma \left(\sum_{i=r+1}^{n} c_i \mathbf{u}_{i}\right) \] Now we have: \[ \|U U^T \mathbf{b} - U \Sigma \left(\sum_{i=r+1}^{n} c_i \mathbf{u}_{i}\right)\|_{2} \] Since \(U\) is orthogonal, we can safely remove the \(U\) terms in the expression: \[ \|U^T \mathbf{b} - \Sigma \left(\sum_{i=r+1}^{n} c_i \mathbf{u}_{i}\right)\|_{2} \] Note that the remaining sum in the norm is equal to zero because, by definition, each \(\mathbf{v}_i\) corresponds to singular values with rank less than the given rank \(r < n\). Then, the original condition of minimizing the norm is satisfied, and our proof is complete. Hence, a vector \(\mathbf{x} \in \mathbb{R}^{n}\) minimizes the Euclidean norm \(\|\mathbf{b}- A\mathbf{x}\|_{2}\), if and only if \[ \mathbf{x}=A^{+} \mathbf{b}+c_{r+1} \mathbf{v}_{r+1}+\cdots+c_{n} \mathbf{v}_{n} \]