/*! This file is auto-generated */ .wp-block-button__link{color:#fff;background-color:#32373c;border-radius:9999px;box-shadow:none;text-decoration:none;padding:calc(.667em + 2px) calc(1.333em + 2px);font-size:1.125em}.wp-block-file__button{background:#32373c;color:#fff;text-decoration:none} Problem 11 Problems inoolving \(\ell_{1}\) ... [FREE SOLUTION] | 91影视

91影视

Log out

Learning Materials

Features

Discover

Chapter 4: Problem 11

Problems inoolving \(\ell_{1}\) - and \(f_{\infty}\)-norms. Formulate the following problems as L.P. Explain in detail the relation besween the optimal solution of each problem and the solution of its equivaleat L.P. (a) Minimixe \(\|A x-b\|_{\infty}\left(f_{\infty}\right.\)-norm approximation) (b) Minimire \(\| A x-4 / 1\left(\ell_{i}\right.\)-horm approximation). (c) Minimire \(\| A x-\) bll sabject to \(\|x\|_{\infty} \leq 1\). (d) Minimize \(\|x\|_{1}\) subjoct to \(\|A x-b\|_{2} \leq 1\). In each problem, \(A \in \mathbf{R}^{m-n}\) and \(b \in \mathbf{R}^{\prime \prime \prime}\) are given. (See \(\\{6.1\) for mare problems involving appraximation and constrained approximation.)

Short Answer

Expert verified
(a) To minimize \(\|A x - b\|_{\infty}\), the equivalent LP is: \[ \begin{array}{ll} \text{minimize} & t \\ \text{subject to} & t \geq (A x - b)_i \\ & t \geq -(A x - b)_i \\ & i = 1,\dots,m \end{array} \] The optimal value of this LP is the same as the optimal value of \(\|A x - b\|_{\infty}\). (b) To minimize \(\| A x - b\|_1\), the equivalent LP is: \[ \begin{array}{ll} \text{minimize} & \sum_{i=1}^m y_i \\ \text{subject to} & y_i \geq (A x - b)_i \\ & y_i \geq -(A x - b)_i \\ & i = 1,\dots,m \end{array} \] The optimal value of this LP is equal to the optimal value of \(\|A x - b\|_1\). (c) To minimize \(\|A x - b\|_\infty\) subject to \(\|x\|_{\infty} \leq 1\), the equivalent LP is: \[ \begin{array}{ll} \text{minimize} & t \\ \text{subject to} & t \geq (A x - b)_i \\ & t \geq -(A x - b)_i \\ & -1 \leq x_j \leq 1 \\ & i = 1,\dots,m \\ & j = 1,\dots,n \end{array} \] The optimal value of this LP is the same as the optimal value of \(\|A x - b\|_{\infty}\) subject to \(\|x\|_{\infty} \leq 1\). (d) For minimizing \(\|x\|_1\) subject to \(\|A x - b\|_{2} \leq 1\), we cannot directly formulate an equivalent LP problem, as the constraint is quadratic and not linear. This problem will require an alternate optimization approach.

Step by step solution

01

Set up the LP

We want to minimize the maximum value of \(|A x - b|\). We introduce a new variable \(t\), with the constraint that \(t \geq |(A x - b)_i|\) for each \(i = 1, \dots, m\). The equivalent LP is: \[ \begin{array}{ll} \text{minimize} & t \\ \text{subject to} & t \geq (A x - b)_i \\ & t \geq -(A x - b)_i \\ & i = 1,\dots,m \end{array} \]
02

Relate the solutions

The optimal solution of the above LP yields the minimum value of \(t\) that satisfies the constraints. This value of \(t\) is the same as the optimal value of the original problem \(\|A x - b\|_{\infty}\). (b) Minimize \(\| A x - b\|_1\):
03

Set up the LP

We can rewrite the \(l_1\)-norm as a sum of absolute values, i.e., \(\|A x - b\|_1 = \sum_{i=1}^m |(A x - b)_i|\). Introduce new variables \(y_i\) for each \(i=1,\dots, m\), with the constraints that \(y_i \geq (A x - b)_i\) and \(y_i \geq -(A x - b)_i\). The equivalent LP is: \[ \begin{array}{ll} \text{minimize} & \sum_{i=1}^m y_i \\ \text{subject to} & y_i \geq (A x - b)_i \\ & y_i \geq -(A x - b)_i \\ & i = 1,\dots,m \end{array} \]
04

Relate the solutions

The optimal solution of the above LP minimizes the sum of \(y_i\) values, which are equivalent to the absolute values of the residuals. Hence, the optimal value of this LP is equal to the optimal value of the original problem \(\|A x - b\|_1\). (c) Minimize \(\|A x - b\|_\infty\) subject to \(\|x\|_{\infty} \leq 1\):
05

Set up the LP

We can rewrite the constraints as follows: \(-t \leq (A x - b)_i \leq t\) for each \(i = 1, \dots, m\) and \(-1 \leq x_j \leq 1\) for each \(j = 1, \dots, n\). The equivalent LP is: \[ \begin{array}{ll} \text{minimize} & t \\ \text{subject to} & t \geq (A x - b)_i \\ & t \geq -(A x - b)_i \\ & -1 \leq x_j \leq 1 \\ & i = 1,\dots,m \\ & j = 1,\dots,n \end{array} \]
06

Relate the solutions

The optimal solution of the above LP yields the minimum value of \(t\) that satisfies all the constraints. This value of \(t\) is the same as the optimal value of the original problem \(\|A x - b\|_{\infty}\) subject to \(\|x\|_{\infty} \leq 1\). (d) Minimize \(\|x\|_1\) subject to \(\|A x - b\|_{2} \leq 1\):
07

Note on LP formulation

The problem can't be formulated as a linear programming problem because of the quadratic constraint \(\|A x - b\|_{2} \leq 1\). This constraint is convex but not linear. A different optimization approach is needed to solve this problem.
08

Conclusion

In this problem, we cannot directly formulate an equivalent LP problem, as the constraint is quadratic and not linear. This problem will require an alternate optimization approach.

Unlock Step-by-Step Solutions & Ace Your Exams!

  • Full Textbook Solutions

    Get detailed explanations and key concepts

  • Unlimited Al creation

    Al flashcards, explanations, exams and more...

  • Ads-free access

    To over 500 millions flashcards

  • Money-back guarantee

    We refund you if you fail your exam.

Over 30 million students worldwide already upgrade their learning with 91影视!

Key Concepts

These are the key concepts you need to understand to accurately answer the question.

\(\ell_{1}\)-norm
The \(\ell_{1}\)-norm is a mathematical concept used to calculate the sum of absolute values in a vector. It is often employed in optimization problems to measure deviations. The \(\ell_{1}\)-norm is extremely helpful in situations where you need to minimize the total deviation, considering all individual differences equally. For a vector \(v\), the \(\ell_{1}\)-norm, denoted as \(||v||_1\), is computed as:\[||v||_1 = |v_1| + |v_2| + \dots + |v_n|\]This concept is crucial in various fields, such as data science, where it is used in the context of sparse solutions. When used in optimization, especially linear programming (LP), the \(\ell_{1}\)-norm allows us to minimize the total error or residuals by converting the problem into a form that is linear, and hence easier to solve with LP techniques. The challenge often lies in formulating the problem so that it fits into an LP framework, requiring the introduction of extra variables to handle absolute values.
\(\ell_{\infty}\)-norm
The \(\ell_{\infty}\)-norm, also known as the max-norm or uniform norm, is used to measure the maximum absolute value within a vector. This norm is useful in optimization problems when the goal is to minimize the largest deviation or error. It offers a different perspective compared to the \(\ell_{1}\)-norm by focusing on the worst-case scenario.For a vector \(v\), the \(\ell_{\infty}\)-norm is expressed as:\[||v||_{\infty} = \max\{|v_1|, |v_2|, \dots, |v_n|\}\]In the context of optimization, especially LP, the \(\ell_{\infty}\)-norm problem is often about keeping all deviations below a certain threshold, or minimizing the largest residual. This requires carefully setting up LP constraints to ensure that all elements conform to the maximum deviation specified by the \(\ell_{\infty}\)-norm.
Optimization Problems
Optimization problems are mathematical challenges where the goal is to find the best solution from all possible solutions. They involve various types of mathematical formulations to either maximize or minimize a specific function鈥攁n objective function鈥攐ften subject to certain constraints. Points of interest in optimization problems include:
  • Objective Function: This is the main function being optimized, typically representing cost, energy, profit, etc.
  • Constraints: Restrictions or conditions that the solution must satisfy.
  • Variables: These are the elements we can control and adjust to find the optimal outcome.
Optimization problems can be linear, as in linear programming, or nonlinear, depending on the nature of the objective function and constraints. In linear programming, both the objective function and constraints are linear, which allows the use of standard LP methods to find solutions. Effective optimization techniques are key in fields such as economics, engineering, logistics, and operation research, where resources need to be allocated efficiently.
Convex Optimization
Convex optimization is a subset of optimization problems where the objective function is convex, meaning any line segment connecting two points on the graph of the function lies above or on the graph. Similarly, the feasible region defined by the constraints is also convex. This property simplifies the problem significantly, as any local minimum will also be a global minimum. Key characteristics of convex optimization include:
  • Convex Functions: Functions where their second derivative is always positive or zero.
  • Convex Sets: These set boundaries where for any two points within the set, the line segment joining them is entirely within the set.
  • Globally Optimal Solutions: Due to the shape of the problem, solutions found are optimal for the whole problem space, not just locally optimal.
Convex optimization is widely used in various fields because of its properties that allow finding solutions more efficiently and reliably. This makes it preferable in high-stakes applications where optimal solutions are critical.

One App. One Place for Learning.

All the tools & learning materials you need for study success - in one app.

Get started for free

Most popular questions from this chapter

Square \(L P\). Consider the LP $$ \begin{array}{ll} \text { minimize } & e^{T} x \\ \text { subject to } & A x \leq b \end{array} $$ with \(A\) squate and nonsingular. Show that the optimal value is given by $$ p^{*}=\left\\{\begin{array}{cc} c^{T} A^{-1} b & A^{-r} c \leq 0 \\ -\infty & \text { otherwise } \end{array}\right. $$

Gencralization of veighted-stm soalorizetion. In \(14.7 .4\) we thowed how to obtain Pareto optimal solutions of a vectoe optimization problers by replacing the vector objective \(f_{0}=\) \(\mathbf{R}^{n} \rightarrow \mathbf{R}^{v}\) with the scalar objective \(\lambda^{2} f_{0}\), where \(\lambda>k=0\). Let \(\psi=\mathbf{R}^{\circ} \rightarrow \mathbf{R}\) be a \(K\)-increasing funetion, \(1 . t_{-1}\) satiofying $$ u \leq k v,=\neq v \Longrightarrow v(u)

\(\mathrm{~ C e p e c i t y ~ o f ~ e ~ c o m m s n i c a t i o n ~ c h a n n e l ~ W e ~ c o n s i d e r ~ a ~ c o m i m u n i c a t i o n ~ c h a n s e l , ~ w i t h ~ i}\) \(X(t) \in\left\\{1_{1++,} n\right\\}\), and output \(Y(t) \in\\{1, \ldots, m\\}\), for \(t=1,2, \ldots\) (in scconds, say). The relation between the inpat and the output is given statistically: $$ p_{i j}=\operatorname{prob}(Y(t)=i \| X(t)=j), \quad i=1, \ldots, m, \quad j=1, \ldots, n $$ The matrix \(P\) ' \(\mathrm{R}^{\text {"we }} \mathrm{~ i s ~ c a l l e d ~ t h e ~ c h a n n e}\) a discrite memoryless charneL. A famoas result of Shannan states that information can be sent over the commanication. channel, with arbitrarily stoall pcobability of error, at any rate less than a bumber \(C_{0}\), called the channel capecity, in bits per second. Shannon abo showed that the capacity of a discrete memoryleso channel can be found by solving an optimizatioa problem. Asoume that \(X\) has a probability distribotion denoted \(x \in \mathbf{R}^{=}\), Le. $$ x_{i}=\operatorname{prob}(X=j), \quad j=1, \ldots, n $$ The mutual moformutoen between \(X\) and \(Y\) is given by Thou the chanael capacity \(C\) is givea by where the supremum is over all possible protubility distributions for the imput \(\Lambda\). Le.s \(\operatorname{atst} x \neq 0,1^{7} z=1\) Show how the charnel capucity can be computed using convex optinixation. Hint. Introduce the variahle \(y=P x\), which gives the probability distribation of the output \(Y^{\prime}\), and show that the mutual information can be expressed ns where \(c_{j}=\sum_{i=}, p_{0} \mathrm{~ l o g}\).

Spparating huperplanes and nplerrs. Suppone yon arv siven two wes of points in \(\mathbf{R}^{\text {", }}\) \(\left(v^{\prime}, v^{2}+\ldots, v^{k}\right)\) and \(\left\\{w^{\prime}, w^{2}+\ldots, w^{2}\right)\). Formulate the following two problems as LP frasiluility problems. (a) Determine a hyperplane that separates the two sets, i.e., find \(a \in \mathbf{R}^{n}\) and \(b \in \mathbf{R}\). with \(a \neq 0\) such that $$ a^{T} v^{i} \leq b, \quad i=1, \ldots, K_{i} \quad a^{T} w^{i} \geq b, \quad i=1, \ldots . L $$ Note that we require \(a \neq 0\), so you have to make sare that your formulation exclades the trivial solution \(a=0, b=0\). You can asesume that $$ \operatorname{rank}\left[\begin{array}{cccccccc} v^{1} & v^{2} & \ldots & v^{k} & w^{1} & w^{2} & \ldots & w^{L} \\ 1 & 1 & \cdots & 1 & 1 & 1 & \ldots & 1 \end{array}\right]=n+1 $$ (i.e., the affine hull of the \(K+L\) points hiss dimension \(n\) ). (b) Determine a spibere separating the two sets of points, \(\angle e_{4}\) find \(x_{y} \in \mathbf{R}^{\text {" and } R \geq 0}\) such that $$ \left\|r^{\prime}-x_{4}\right\|_{z} \leq R, \quad i=1, \ldots, K_{1} \quad\left\|w^{\prime}-x_{n}\right\|_{2} \geq R, \quad i=1, \ldots, L $$ (Here \(x_{e}\) is the center of the sphere, \(R\) is its radius.) (See chapter 8 for uore on separating hyperplanes, separating splueres, and related topies.)

Eyuinalent conver problems. Show that the following three couvex problems are equivaleat. Carefully explain bow the solution of each problem is obtained from the solution of the other problens The peoblean data are the matrix \(A \in R^{-\infty \times a}\) (with rows \(a_{i}^{T}\) ), the vectot \(b \in \mathbf{R}^{\prime \prime}\), and the constant \(M>0\). (a) The robust irast-sqoares problem $$ \text { misimive } \sum_{i=1}^{m} \phi\left(\theta_{i}^{T} x-b_{i}\right) $$ with variable \(x \in \mathbf{R}^{n}\), where \(\phi: \mathbf{R} \rightarrow \mathbf{R}\) is delined is $$ \phi(u)= \begin{cases}w^{2} & |v| \leq M \\ M(2|\mathrm{u}|-M r) & |u|>M\end{cases} $$ (This function is kuown as the Huler penalty functiod; see \(56.1 .2\) ) (b) The feast-secuares problem with nariabe weights $$ \begin{aligned} &\text { minimize } \\ &{\text { subject to }} \sum_{w<0}{\sum_{i+1}^{m}}\left(a_{i}^{T} x-b_{i}\right)^{2} /\left(w_{i}+1\right)+M^{2} 1^{T} w \end{aligned} $$ witb variables \(x \in \mathbf{R}^{\prime \prime}\) and \(w \in \mathbf{R}^{m}\), and domain \(D=\left\\{(\tau, w) \in \mathbf{R}^{-} \times \mathbf{R}^{-1} \mid w>-1\right\\}\) Hint. Optimire over we asuming \(x \mathrm{~ i s ~ f i x n l , ~ t o ~ e s t a b l i o h ~ a ~ r w a t i o}\) in part (a). (This problem can be interpreted as a weighted least-squares peoblem in which we are allowed to adjust the weight of the ith residual. The weiglat is one if \(w_{1}=0\), and decreases if we increase wt, The socond term in tbe objcetive penalizes large values of \(w\), i.e., lasge adjustanents of the weights) (c) The enadnafue progrum minimize \(\sum_{i=1}^{m}\left(v_{7}^{2}+2 . M v_{0}\right)\) nubect to \(-u-v \leq \Delta z-b \leq u+v\) \(0 \leq v \leq M 1\) \(v \geq 0 .\)
See all solutions

Recommended explanations on Math Textbooks

Statistics

Read Explanation

Applied Mathematics

Read Explanation

Logic and Functions

Read Explanation

Mechanics Maths

Read Explanation

Decision Maths

Read Explanation

Pure Maths

Read Explanation
View all explanations

What do you think about this solution?

We value your feedback to improve our textbook solutions.

Study anywhere. Anytime. Across all devices.