91Ó°ÊÓ

Each individual in a population of size \(N\) is, in each period, either active or inactive. If an individual is active in a period then, independent of all else, that individual will be active in the next period with probability \(\alpha .\) Similarly, if an individual is inactive in a period then, independent of all else, that individual will be inactive in the next period with probability \(\beta .\) Let \(X_{n}\) denote the number of individuals that are active in period \(n\). (a) Argue that \(X_{n}, n \geqslant 0\) is a Markov chain. (b) Find \(E\left[X_{n} \mid X_{0}=i\right]\). (c) Derive an expression for its transition probabilities. (d) Find the long-run proportion of time that exactly \(j\) people are active. Hint for \((\mathrm{d}):\) Consider first the case where \(N=1\).

Let \(A\) be a set of states, and let \(A^{c}\) be the remaining states. (a) What is the interpretation of $$ \sum_{i \in A} \sum_{j \in A^{c}} \pi_{i} P_{i j} ? $$ (b) What is the interpretation of $$ \sum_{i \in A^{e}} \sum_{j \in A} \pi_{i} P_{i j} ? $$ (c) Explain the identity $$ \sum_{i \in A} \sum_{j \in A^{c}} \pi_{i} P_{i j}=\sum_{i \in A^{c}} \sum_{j \in A} \pi_{i} P_{i j} $$

An individual possesses \(r\) umbrellas that he employs in going from his home to office, and vice versa. If he is at home (the office) at the beginning (end) of a day and it is raining, then he will take an umbrella with him to the office (home), provided there is one to be taken. If it is not raining, then he never takes an umbrella. Assume that, independent of the past, it rains at the beginning (end) of a day with probability \(p\). (a) Define a Markov chain with \(r+1\) states, which will help us to determine the proportion of time that our man gets wet. (Note: He gets wet if it is raining, and all umbrellas are at his other location.) (b) Show that the limiting probabilities are given by $$ \pi_{i}=\left\\{\begin{array}{ll} \frac{q}{r+q}, & \text { if } i=0 \\ \frac{1}{r+q}, & \text { if } i=1, \ldots, r \end{array} \quad \text { where } q=1-p\right. $$ (c) What fraction of time does our man get wet? (d) When \(r=3\), what value of \(p\) maximizes the fraction of time he gets wet

Let \(\left\\{X_{n}, n \geqslant 0\right\\}\) denote an ergodic Markov chain with limiting probabilities \(\pi_{i} .\) Define the process \(\left\\{Y_{n}, n \geqslant 1\right\\}\) by \(Y_{n}=\left(X_{n-1}, X_{n}\right)\). That is, \(Y_{n}\) keeps track of the last two states of the original chain. Is \(\left\\{Y_{n}, n \geqslant 1\right\\}\) a Markov chain? If so, determine its transition probabilities and find $$ \lim _{n \rightarrow \infty} P\left\\{Y_{n}=(i, j)\right\\} $$

Consider the Ehrenfest urn model in which \(M\) molecules are distributed between two urns, and at each time point one of the molecules is chosen at random and is then removed from its urn and placed in the other one. Let \(X_{n}\) denote the number of molecules in urn 1 after the \(n\) th switch and let \(\mu_{n}=E\left[X_{n}\right]\). Show that (a) \(\mu_{n+1}=1+(1-2 / M) \mu_{n}\). (b) Use (a) to prove that $$ \mu_{n}=\frac{M}{2}+\left(\frac{M-2}{M}\right)^{n}\left(E\left[X_{0}\right]-\frac{M}{2}\right) $$

A particle moves among \(n+1\) vertices that are situated on a circle in the following manner. At each step it moves one step either in the clockwise direction with probability \(p\) or the counterclockwise direction with probability \(q=1-p\). Starting at a specified state, call it state 0 , let \(T\) be the time of the first return to state 0 . Find the probability that all states have been visited by time \(T\). Hint: Condition on the initial transition and then use results from the gambler's ruin problem.

In the gambler's ruin problem of Section 4.5.1, suppose the gambler's fortune is presently \(i\), and suppose that we know that the gambler's fortune will eventually reach \(N\) (before it goes to 0 ). Given this information, show that the probability he wins the next gamble is $$ \begin{array}{ll} \frac{p\left[1-(q / p)^{i+1}\right]}{1-(q / p)^{i}}, & \text { if } p \neq \frac{1}{2} \\ \frac{i+1}{2 i}, & \text { if } p=\frac{1}{2} \end{array} $$

Consider a branching process having \(\mu<1\). Show that if \(X_{0}=1\), then the expected number of individuals that ever exist in this population is given by \(1 /(1-\mu)\). What if \(X_{0}=n ?\)

For a branching process, calculate \(\pi_{0}\) when (a) \(P_{0}=\frac{1}{4}, P_{2}=\frac{3}{4}\). (b) \(P_{0}=\frac{1}{4}, P_{1}=\frac{1}{2}, P_{2}=\frac{1}{4}\). (c) \(P_{0}=\frac{1}{6}, P_{1}=\frac{1}{2}, P_{3}=\frac{1}{3}\).

At all times, an urn contains \(N\) balls?-some white balls and some black balls. At each stage, a coin having probability \(p, 0