91Ó°ÊÓ

One of the fundamental concepts in the study of Markov Chains is the idea of a recurrent state. A recurrent state, also known as a persistent state, is one that, once visited, the process is guaranteed to return to with a probability of 1.

To understand this in a simple way, imagine walking in a city with interconnected streets. If a particular street (representing a state) is recurrent, no matter how far you walk away from it, you're certain to find your way back to it eventually. Mathematically, the probability of returning to the recurrent state is calculated by evaluating

\(\ \displaystyle \sum_{n=1}^{\infty}p_{i,i}^{(n)}=\ \infty\ \)

This equation sums the probabilities of returning to street 'i' after 'n' number of walks (or transitions), taken over all possible 'n'. Since this sum is infinite for a recurrent state, it solidifies the concept that you'll definitely return to that street at some point during your endless walk—essentially, you can't avoid coming back to a recurrent state in a Markov Chain.

States in Markov Chains have a significant property known as communication. Essentially, two states communicate with each other if it is possible to go from one to the other and vice versa, not necessarily in one step but across a series of transitions.

For instance, think of two towns (states) with roads (transitions) connecting them directly or indirectly through other towns. These towns communicate if you can travel from one town to the other and back through any number of road segments. The mathematical definition requires that there be some positive number of steps, let's call them 'm' and 'n', such that:

From state i to j:

\(p_{i,j}^{(m)}>0\)

And from state j to i:

\(p_{j,i}^{(n)}>0\)

When states do not communicate, this implies there are no paths connecting them; for our towns scenario, it's as if the roads between them don't exist. In a Markov Chain, if there is no communication from state 'i' to 'j', no sequence of transitions will lead from one to the other, making probabilities of such transitions equal to zero.

A closed class of states in a Markov Chain is a group of states that, once entered, cannot be left. In simpler terms, once you're in a closed network of streets (states), you can only travel within them and can never wander away onto new streets. A closed class can consist of a single state if it is recurrent and does not communicate with any other state.

The reason for this is that the probabilities of transitioning from any state within the class to any state outside the class are zero. Considering our city streets analogy, imagine a gated community with its own network of streets that has no exits -- this is a real-life reflection of a closed class of states in Markov Chains.

The exercise proved this concept by showing that if state 'i' is recurrent (which means it is part of a closed class) and does not communicate with state 'j', then the probability of ever reaching 'j' from 'i' is zero, symbolically, \(P_i = 0\). This mathematical proof supports the intuitive understanding that in a closed class of states, once you enter, you are contained within it for the duration of the process, just as you would be confined within the boundaries of a gated community with no exits to the outside world.