91Ó°ÊÓ

Conditional probability is a fundamental concept in probability theory. It allows us to calculate the probability of an event happening, given that another event has already occurred. In the context of Markov Chains, this is crucial because it helps us to determine the likelihood of transitioning between states:

* Given certain conditions on the path taken.

For example, suppose we're looking at a Markov chain starting in a specific state and want to know the probability of being in another specific state after a number of steps, without entering an excluded state. We use conditional probability to exclude those unwanted paths and focus only on the acceptable ones. This conditional probability is expressed as:

\[ P\{X_{n}=m \mid X_{0}=i, X_{k} eq r, k=1, \ldots, n\} \]

This formula represents the probability that the Markov Chain is at state \(m\) at time \(n\) given it started at state \(i\) and hasn't visited state \(r\). It essentially filters the chain's paths to only include the desired ones.

Transition probabilities are the backbone of Markov Chain models. They dictate how the process moves from one state to another. In our exercise, there are specific transitions to observe:

* If it moves from state \(i\) to state \(j\) without involving state \(r\).

To modify a Markov Chain and exclude certain states, like state \(r\), we redefine the transition probabilities. These new probabilities are calculated to redistribute the likelihood of moving from \(i\) to \(j\) over the remaining states, excluding \(r\):

\[ \underline{Q}_{i, j}=\frac{P_{i, j}}{1-P_{i, r}}, \quad i, j eq r \]

This equation transforms the transition matrix by scaling probabilities. This adjustment keeps the total probability of transitioning from a state equal to one, even when certain states are removed.

The state space in a Markov Chain is the set of all possible states the system can be in. In our scenario, the state space includes every state the process might occupy along its journey over time.

* However, sometimes we remove certain states, like state \(r\) in our question.

When we exclude a state, the state space gets reduced. For our conditionally modified Markov Chain, we have a state space formulated by eliminating the restricted state \(r\). This simplification allows us to focus only on transitions that are permissible within the new boundaries. These changes impact how probabilities are distributed across remaining states, influencing the overall behavior and analysis of the chain.

Probability Theory underpins the understanding of random processes and Markov Chains. It gives us tools and methods to analyze the likelihood of events occurring and to make predictions about these events.

In the context of the exercise, several principles of probability theory are applied:

• The concept of conditional probability to account for restrictions in our problem scenario.
• Rescaling probabilities, so the total likelihood across transitions without forbidden states still sums up to 1.
• The use of long-term behavior predictions through stage transitions.

By organizing these components, probability theory allows us; to make meaningful assessments about how processes unfold over time, even when barred paths or missing states are considered, such as in Markov Chains.