91Ó°ÊÓ

Consider a process \(\left\\{X_{H}, n=0,1, \ldots\right\\}\) which takes on the values 0,1 , or 2 . Suppose $$ \begin{aligned} &P\left\\{X_{n+1}=j \mid X_{n}=i, X_{n-1}=i_{n-1}, \ldots, X_{0}=i_{0}\right\\} \\ &\quad=\left\\{\begin{array}{ll} P_{i j}^{\mathrm{I}}, & \text { when } n \text { is even } \\ P_{i j}^{\mathrm{II}}, & \text { when } n \text { is odd } \end{array}\right. \end{aligned} $$ where \(\sum_{j=0}^{2} P_{i j}^{\mathrm{I}}=\sum_{j=0}^{2} P_{i j}^{\mathrm{II}}=1, i=0,1,2 .\) Is \(\left\\{X_{n}, n \geqslant 0\right\\}\) a Markov chain? If not, then show how, by enlarging the state space, we may transform it into a Markov chain.

Prove that if the number of states in a Markov chain is \(M\), and if state \(j\) can be reached from state \(i\), then it can be reached in \(M\) steps or less.

A particle moves on a circle through points which have been markéd \(0,1,2,3,4\) (in a clockwise order). At each step it has a probability \(p\) of moving to the right (clockwise) and \(1-p\) to the left (counterclockwise). Let \(X_{n}\) denote its location on the circle after the \(n\) th step. The process \(\left[X_{n}, n \geqslant 0\right\\}\) is a Markov chain. (a) Find the transition probability matrix. (b) Calculate the limiting probabilities.

Consider three urns, one colored red, one white, and one blue. The red urn contains 1 red and 4 blue balls; the white urn contains 3 white balls, 2 red balls, and 2 blue balls; the blue urn contains 4 white balls, 3 red balls, and 2 blue balls. At the initial stage, a ball is randomly selected from the red urn and then returned to that urn. At every subsequent stage, a ball is randomly selected from the urn whose color is the same as that of the ball previously selected and is then returned to that urn. In the long run, what proportion of the selected balls are red? What proportion are white? What proportion are blue?

Each morning an individual leaves his house and goes for a run. He is equally likely to leave either from his front or back door. Upon leaving the house, he chooses a pair of running shoes (or goes running barefoot if there are no shoes at the door from which he departed). On his return he is equally likely to enter, and leave his running shoes, either by the front or back door. If he owns a total of \(k\) pairs of running shoes, what proportion of the time does he run barefooted?

Consider a Markov chain with states \(0,1,2,3,4\). Suppose \(P_{0,4}=1\); and suppose that when the chain is in state \(i, i>0\), the next state is equally likely to be any of the states \(0,1, \ldots, i-1 .\) Find the limiting probabilities of this Markov chain.

Let \(P^{(1)}\) and \(P^{(2)}\) denote transition probability matrices for ergodic Markov chains having the same state space. Let \(\pi^{1}\) and \(\pi^{2}\) denote the stationary (limiting) probability vectors for the two chains. Consider a process defined as follows: (i) \(X_{0}=1\). A coin is then flipped and if it comes up heads, then the remaining states \(X_{1}, \ldots\) are obtained from the transition probability matrix \(P^{(1)}\) and if tails from the matrix \(P^{(2)}\). Is \(\left\\{X_{n}, n \geqslant 0\right\\}\) a Markov chain? If \(p=P\\{\) coin comes up heads \(\\}\), what is \(\lim _{n \rightarrow \infty} P\left(X_{n}=i\right)\) ? (ii) \(X_{0}=1\). At each stage the coin is flipped and if it comes up heads, then the next state is chosen according to \(P^{(1)}\) and if tails comes up, then it is chosen according to \(P^{(2)}\). In this case do the successive states constitute a Markov chain? If so, determine the transition probabilities. Show by a counterexample that the limiting probabilities are not the same as in part (i).

Suppose that on each play of the game a gambler either wins 1 with probability \(p\) or loses 1 with probability \(1-p\). The gambler continues betting until she or he is either winning \(n\) or losing \(m\). What is the probability that the gambler quits a winner?

For the Markov chain with states \(1,2,3,4\) whose transition probability matrix \(\mathbf{P}\) is as specified below find \(f_{i 3}\) and \(s_{i 3}\) for \(i=1,2,3\). $$ \mathbf{P}=\left[\begin{array}{llll} 0.4 & 0.2 & 0.1 & 0.3 \\ 0.1 & 0.5 & 0.2 & 0.2 \\ 0.3 & 0.4 & 0.2 & 0.1 \\ 0 & 0 & 0 & 1 \end{array}\right] $$

For a branching process, calculate \(\pi_{0}\) when (a) \(P_{0}=\frac{1}{47} \cdot P_{2}=\frac{3}{4}\) (b) \(P_{0}=\frac{1}{4}, P_{1}=\frac{1}{2}, P_{2}=\frac{1}{4}\) (c) \(P_{0}=\frac{1}{6}, P_{1}=\frac{1}{2}, P_{3}=\frac{1}{3}\)