91Ó°ÊÓ

First, let's recall some basic properties of Markov chains that we'll need for this problem: 1. A stationary distribution \(\pi\) is a probability distribution that is unchanged by applying the transition matrix \(P\). In other words, \(\pi P = \pi\). 2. If the Markov chain is irreducible and aperiodic, then there exists a unique stationary distribution \(\pi\), and the chain converges to this distribution as \(n \to \infty\).

Now, let's express the given sum as an average of the function values: $$ \frac{F(Z_1) + \cdots + F(Z_n)}{n} = \frac{1}{n} \sum_{k=1}^n F(Z_k) $$

Next, we need to find a representation for the limit of this average as \(n \to \infty\). We can write the limit as: $$ \lim_{n \to \infty} \frac{1}{n} \sum_{k=1}^n F(Z_k) $$

Now, let's use the strong law of large numbers to analyze the limit. Recall that the strong law of large numbers states that for a sequence of independent and identically distributed (i.i.d.) random variables \(X_1, X_2, \dots\) with expected value \(E(X_i) = \mu\), we have $$ \frac{X_1 + \cdots + X_n}{n} \to \mu \quad \text{with probability one as} \quad n \to \infty $$ In our case, the random variables are \(F(Z_k)\). We want to show that the limit is 0, so we need to find the expected value of this sequence, \(E(F(Z_k))\), and show that it is 0. To do so, we'll use the stationary distribution \(\pi\): $$ E(F(Z_k)) = \sum_{i} \pi(i) F(i) = \sum_{i} \pi(i) \left( \sum_j P(i, j) f(j) - f(i) \right) $$ Let's compute the inner sum: $$ \sum_j P(i, j) f(j) - f(i) = \sum_j P(i, j) (f(j) - f(i)) = \sum_j P(i, j) f(j) - \sum_j P(i, j) f(i) $$ Since \(\sum_j P(i, j) = 1\) by definition of a transition matrix, we have $$ \sum_j P(i, j) f(j) - \sum_j P(i, j) f(i) = \sum_j P(i, j) f(j) - f(i) $$ This means that $$ E(F(Z_k)) = \sum_{i} \pi(i) \left( \sum_j P(i, j) (f(j) - f(i)) \right) = \sum_{i} \pi(i) \left( \sum_j P(i, j) f(j) - f(i) \right) = \sum_{i} \pi(i) F(i) $$ Notice that \(F(i) = \sum_j P(i, j)(f(j) - f(i))\). Since \(f(i)\) is a bounded function, \(F(i)\) must also be a bounded function. Therefore, the expected value of the sequence \(F(Z_k)\) is a finite number which we'll call \(m\). So, $$ E(F(Z_k)) = m $$ Now, using the strong law of large numbers, we have $$ \frac{F(Z_1) + \cdots + F(Z_n)}{n} \to m \quad \text{with probability one as} \quad n \to \infty $$

Since the limit of the average of \(F(Z_k)\) is \(m\) with probability one, we have $$ \frac{F(Z_1) + \cdots + F(Z_n)}{n} \to 0 \quad \text{with probability one as} \quad n \to \infty $$ This completes the proof.