91Ó°ÊÓ

A transition matrix, often denoted by the letter \( P \), is a square matrix used in stochastic processes to describe the transitions of a Markov chain. Each entry \( p_{ij} \) of the transition matrix \( P \) represents the probability of moving from state \( i \) to state \( j \) in a single time step. Transition matrices are a fundamental tool for modeling a system that undergoes random state changes.
In our example, the transition matrix is:

• \( P = \begin{bmatrix} 0 & 1 & 0 \ 1/3 & 1/3 & 1/3 \ 1 & 0 & 0 \end{bmatrix} \)

This matrix illustrates the probability of moving from one state to another in the system, with each row summing up to 1. This is a critical feature of any transition matrix because it guarantees that the probabilities account for all possible outcomes.

A linear system is a set of linear equations that models the relationships between different variables. To find the steady-state distribution in a Markov chain, you'll set up a linear system based on the transition matrix.
For a given transition matrix \( P \), you need to find a steady-state vector \( \pi \) such that \( \pi P = \pi \). This implies that the system is in balance, with no net change in distribution over time.

• Example of linear equations set from the matrix:
  • \( \pi_1 = \frac{1}{3}\pi_2 + \pi_3 \)
  • \( \pi_2 = \pi_1 + \frac{1}{3}\pi_2 \)
  • \( \pi_3 = \frac{1}{3}\pi_2 \)

These equations result from the rule \( \pi P = \pi \) and they must be simultaneously satisfied to solve for \( \pi \). We're essentially looking for a solution where the variables \( \pi_1, \pi_2, \pi_3 \) don't change over transitions.

The steady-state vector \( \pi \) is a probability vector that describes a stable distribution in which the Markov chain does not change anymore, regardless of further transitions. This vector is a crucial concept for understanding long-term behavior in stochastic processes.
For the vector \( \pi \) to be valid:

• Each element must be non-negative to represent a probability.
• The sum of all elements must equal 1.

Initially, the solution from step 3 of our process was not a valid steady-state vector because it included negative values. Multiplying by -2 normalized the elements to non-negative values, and ensured they summed up to 1. The corrected steady-state vector is:

• \( \pi = \begin{bmatrix} 1 & 3 & -1 \end{bmatrix} \)

This final form ensures that \( \pi \) satisfies the condition \( \pi P = \pi \), maintaining the stochastic nature of the process.

The Gauss-Jordan elimination method is a systematic process of performing row operations to transform a matrix into its Reduced Row Echelon Form (RREF). This method is used to solve systems of linear equations efficiently.
In simple steps:

• Convert the equations into an augmented matrix format.
• Apply row operations (swap, scale, and row add) to simplify the matrix.
• Continue until you get a diagonal form, which makes it easy to extract solutions.

For our linear system, the augmented matrix is given by:

• \( \begin{bmatrix} -1 & \frac{1}{3} & 1 & | & 0 \ 1 & -\frac{2}{3} & 0 & | & 0 \ 0 & \frac{1}{3} & -1 & | & 0 \end{bmatrix} \)

After applying the Gauss-Jordan method, we reach the RREF:

• \( \begin{bmatrix} 1 & 0 & 0 & | & -\frac{1}{2} \ 0 & 1 & 0 & | & -\frac{3}{2} \ 0 & 0 & 1 & | & \frac{1}{2} \end{bmatrix} \)

This form makes it simple to read off the solution for each variable \( \pi_1, \pi_2, \pi_3 \), although adjustments, such as normalizing, might be necessary to fit the context.