91Ó°ÊÓ

(Bertsekas (1976)) A farmer annually produces \(X_{k}\) units of a certain crop and stores \(\left(1-u_{k}\right) X_{k}\) units of his production, where \(0 \leq u_{k} \leq 1 .\) He invests the remaining \(u_{k} X_{k}\) units, thus increasing next year's accumulated output to a level \(X_{k+1}\) given by $$ X_{k+1}=X_{k}+w_{k} u_{k} X_{k}, \quad k=0,1, \ldots, N-1 $$ The scalars \(W_{k}\) are independent random variables with an identical probability distribution that depends neither on \(x_{k}\) nor \(u_{k} .\) Furthermore, \(E\left[W_{k}\right]=\bar{w}>0 .\) The problem is to find the optimal policy that maximizes the expected output accumulated over \(N\) years, $$ E\left[X_{N}+\sum_{k=0}^{N-1}\left(1-u_{k}\right) X_{k}\right] $$ Show that one optimal control is given by: (i) If \(\bar{w}>1\), then \(u_{0}^{*}\left(x_{0}\right)=\cdots=u_{N-1}^{*}\left(x_{N-1}\right)=1\). (ii) If \(0<\bar{w}<1 / N\), then \(u_{0}^{*}\left(x_{0}\right)=\cdots=u_{N-1}^{*}\left(x_{N-1}\right)=0\). (iii) If \(1 / N \leq \bar{w} \leq 1\), then $$ \begin{gathered} u_{0}^{*}\left(x_{0}\right)=\cdots=u_{N-\bar{k}-1}^{*}\left(x_{N-\bar{k}-1}\right)=1 \\\ \vdots \\ u_{N-\bar{k}}^{*}\left(x_{N-\bar{k}}\right)=\cdots=u_{N-1}^{*}\left(x_{N-1}\right)=0 \end{gathered} $$ where \(\bar{k}\) is such that \(1 /(\bar{k}+1) \leq \bar{w}<1 / \bar{k}\). Note that this control consists of constant functions.