91Ó°ÊÓ

A transition matrix in the context of Markov Chains is a square matrix that describes the transitions of a system from one state to another. Each entry in the matrix represents the probability of moving from one state to another in a single time step. For example, consider a transition matrix \( P \):\[P = \begin{bmatrix} 0.5 & 0.3 \ 0.5 & 0.7 \end{bmatrix}\]In this matrix, the first row \([0.5, 0.3]\) represents the probabilities of moving from state 1 to state 1 (0.5) and state 1 to state 2 (0.3). Likewise, the second row \([0.5, 0.7]\) indicates the probabilities for movements from state 2 to states 1 (0.5) and 2 (0.7).

• Each row of the matrix sums to 1, which reflects that transition probabilities account for all possible states.
• Every element in the matrix is non-negative, representing valid probabilities.

Understanding transition matrices is crucial as they depict the entire dynamic behavior of a Markov process, capturing how states evolve over time.

A state vector in a Markov Chain is a column vector where each element represents the probability of the system being in a specific state at a given time. Given an initial state, this vector evolves over iterations driven by the transition matrix. For instance:The initial state vector \( \mathbf{x}_{0} \):\[\mathbf{x}_{0} = \begin{bmatrix} 0.5 \ 0.5 \end{bmatrix}\]The elements in \( \mathbf{x}_{0} \) indicate that there is an initial probability of 0.5 for the system being in state 1 and 0.5 for being in state 2.

• State vectors must have elements that sum to 1, as they represent a complete probability distribution across all states.
• Each element is non-negative, consistent with probability rules.

The state vector is multiplied by the transition matrix to predict the system's state in the next time step, iteratively updating the probabilities as the system progresses.

Matrix multiplication plays a central role in Markov Chains, specifically when calculating the evolution of state vectors. Each multiplication step updates the state probabilities according to the transition matrix. Let's see how this works.In the exercise, the initial state vector \( \mathbf{x}_{0} \) is multiplied by the transition matrix \( P \) to find the next state vector \( \mathbf{x}_{1} \):\[\mathbf{x}_{1} = P \cdot \mathbf{x}_{0} = \begin{bmatrix} 0.5 & 0.3 \ 0.5 & 0.7 \end{bmatrix} \cdot \begin{bmatrix} 0.5 \ 0.5 \end{bmatrix} = \begin{bmatrix} 0.4 \ 0.6 \end{bmatrix}\]To transition to the next state, \( \mathbf{x}_{1} \) is again multiplied by \( P \), producing \( \mathbf{x}_{2} \):\[\mathbf{x}_{2} = P \cdot \mathbf{x}_{1} = \begin{bmatrix} 0.5 & 0.3 \ 0.5 & 0.7 \end{bmatrix} \cdot \begin{bmatrix} 0.4 \ 0.6 \end{bmatrix} = \begin{bmatrix} 0.38 \ 0.62 \end{bmatrix}\]

• Matrix multiplication here involves taking each row of the transition matrix and performing a dot product with the state vector.
• The result is a new vector reflecting updated state probabilities for the next time step.

This iterative process through matrix multiplication allows us to track how the probabilities develop over time in a Markov Chain.