91Ó°ÊÓ

Let the transition probability matrix of a two-state Markov chain be given, as in Example 4.2, by $$ \mathbf{P}=\left\|\begin{array}{cc} p & 1-p \\ 1-p & p \end{array}\right\| $$ Show by mathematical induction that $$ \mathbf{P}^{(n)}=\left\|\begin{array}{|ll} \frac{1}{2}+\frac{1}{2}(2 p-1)^{n} & \frac{1}{2}-\frac{1}{2}(2 p-1)^{n} \\ \frac{1}{2}-\frac{1}{2}(2 p-1)^{n} & \frac{1}{2}+\frac{1}{2}(2 p-1)^{n} \end{array}\right\| $$

Suppose that coin 1 has probability \(0.7\) of coming up heads, and \(\operatorname{coin} 2\) has probability \(0.6\) of coming up heads. If the coin flipped today comes up heads, then we select coin 1 to flip tomorrow, and if it comes up tails, then we select \(\operatorname{coin} 2\) to flip tomorrow. If the coin initially flipped is equally likely to be \(\operatorname{coin} 1\) or \(\operatorname{coin} 2\), then what is the probability that the coin flipped on the third day after the initial flip is coin 1? Suppose that the coin flipped on Monday comes up heads. What is the probability that the coin flipped on Friday of the same week also comes up heads?

Let \(\mathrm{P}\) be the transition probability matrix of a Markov chain. Argue that if for some positive integer \(r, \mathrm{P}^{r}\) has all positive entries, then so does \(\mathrm{P}^{n}\), for all integers \(n \geqslant r\).

Specify the classes of the following Markov chains, and determine whether they are transient or recurrent: $$\mathbf{P}_{1}=\left\|\begin{array}{lll} 0 & \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & 0 & \frac{1}{2} \\ \frac{1}{2} & \frac{1}{2} & 0 \end{array} \mid, \quad \mathbf{P}_{2}=\right\| \begin{array}{cccc} 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 1 \\ \frac{1}{2} & \frac{1}{2} & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array} \|$$ $$\mathbf{P}_{3}=\left\|\begin{array}{|ccccc|} \frac{1}{2} & 0 & \frac{1}{2} & 0 & 0 \\ \frac{1}{4} & \frac{1}{2} & \frac{1}{4} & 0 & 0 \\ \frac{1}{2} & 0 & \frac{1}{2} & 0 & 0 \\ 0 & 0 & 0 & \frac{1}{2} & \frac{1}{2} \\ 0 & 0 & 0 & \frac{1}{2} & \frac{1}{2} \end{array} \mid, \quad \mathbf{P}_{4}=\right\| \begin{array}{ccccc} \frac{1}{4} & \frac{3}{4} & 0 & 0 & 0 \\ \frac{1}{2} & \frac{1}{2} & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & \frac{1}{3} & \frac{2}{3} & 0 \\ 1 & 0 & 0 & 0 & 0 \end{array} \|$$

Show that if state \(i\) is recurrent and state \(i\) does not communicate with state \(j\), then \(P_{i j}=0 .\) This implies that once a process enters a recurrent class of states it can never leave that class. For this reason, a recurrent class is often referred to as a closed class.

Coin 1 comes up heads with probability \(0.6\) and \(\operatorname{coin} 2\) with probability \(0.5 . \mathrm{A}\) coin is continually flipped until it comes up tails, at which time that coin is put aside and we start flipping the other one. (a) What proportion of flips use coin 1? (b) If we start the process with \(\operatorname{coin} 1\) what is the probability that \(\operatorname{coin} 2\) is used on the fifth flip?

A DNA nucleotide has any of four values. A standard model for a mutational change of the nucleotide at a specific location is a Markov chain model that supposes that in going from period to period the nucleotide does not change with probability \(1-3 \alpha\), and if it does change then it is equally likely to change to any of the other three values, for some \(0<\alpha<\frac{1}{3}\). (a) Show that \(P_{1,1}^{n}=\frac{1}{4}+\frac{3}{4}(1-4 \alpha)^{n}\). (b) What is the long-run proportion of time the chain is in each state?

Let \(Y_{n}\) be the sum of \(n\) independent rolls of a fair die. Find \(\lim _{n \rightarrow \infty} P\left\\{Y_{n}\right.\) is a multiple of 13\(\\}\) Hint: Define an appropriate Markov chain and apply the results of Exercise \(20 .\)

In a good weather year the number of storms is Poisson distributed with mean \(1 ;\) in a bad year it is Poisson distributed with mean 3. Suppose that any year's weather conditions depends on past years only through the previous year's condition. Suppose that a good year is equally likely to be followed by either a good or a bad year, and that a bad year is twice as likely to be followed by a bad year as by a good year. Suppose that last year-call it year 0 -was a good year. (a) Find the expected total number of storms in the next two years (that is, in years 1 and 2 ). (b) Find the probability there are no storms in year 3 . (c) Find the long-run average number of storms per year.

Consider the following approach to shuffling a deck of \(n\) cards. Starting with any initial ordering of the cards, one of the numbers \(1,2, \ldots, n\) is randomly chosen in such a manner that each one is equally likely to be selected. If number \(i\) is chosen, then we take the card that is in position \(i\) and put it on top of the deck-that is, we put that card in position 1 . We then repeatedly perform the same operation. Show that, in the limit, the deck is perfectly shuffled in the sense that the resultant ordering is equally likely to be any of the \(n !\) possible orderings.