91Ó°ÊÓ

The first passage probabilities \(f_{ij}\) are defined as the probability that the Markov chain will first visit state \(j\) from state \(i\). They can be calculated recursively using the following formula: $$ f_{ij}^{(n)} = \sum_{k \neq j} p_{ik} f_{kj}^{(n-1)}, \quad n \geq 1, $$ where \(f_{ij}^{(n)}\) denotes the probability that the process first enters state \(j\) at step \(n\) and \(p_{ik}\) is the transition probability from state \(i\) to state \(k\). For \(i=1,2,3\), we will compute \(f_{i3}\) using the formula above: 1. When \(i = 1\), \(f_{13}^{(n)} = 0.1\cdot f_{33}^{(n-1)}+0.2\cdot f_{23}^{(n-1)}+0.4\cdot f_{13}^{(n-1)}\). 2. When \(i = 2\), \(f_{23}^{(n)} = 0.2\cdot f_{33}^{(n-1)}+0.5\cdot f_{23}^{(n-1)}+0.1\cdot f_{13}^{(n-1)}\). 3. When \(i = 3\), \(f_{33}^{(n)} = 0.2\cdot f_{33}^{(n-1)}+0.4\cdot f_{23}^{(n-1)}+0.3\cdot f_{13}^{(n-1)}\). With these recursions, we can calculate the first passage probabilities until the desired convergence is obtained.

The mean first passage time, \(s_{ij}\), is the expected time it takes for the Markov chain to reach state \(j\) for the first time from state \(i\). It can be calculated using the first passage probabilities, \(f_{ij}^{(n)}\), as follows: $$ s_{ij} = \sum_{n=1}^{\infty} n \cdot f_{ij}^{(n)}. $$ Using the previously calculated first passage probabilities, \(f_{ij}^{(n)}\), for \(i=1,2,3\), we can now compute the mean first passage times, \(s_{i3}\), by summing the series for each \(i\). Remember that, in general, we must use numerical methods to compute both first passage probabilities and mean first passage times. By iterating over the recursions, we can obtain these values to a desired level of precision.