91Ó°ÊÓ

Stochastic process can be defined as the sequence of the random variable

\({X_1}...{X_i}\)noted against discrete time parameter.

Here\({X_i}\)is the i^th time period. In stochastic process the value \({X_i}\)depends on the preceding values observed at times \(t - 1,t - 2...,1\).

A ball throwing game between three boys A, B and C. A throws ball to B with probability 0.2 and to C with probability 0.8 throws ball to A with probability 0.6

and to C with probability 0.4.finally C throws the ball to A and to B with equal probability that is 0.5

Let’s consider that the game forms a markov chain. So, the transition probabilities of the game can be given as

Hence the transition matrix is given as:

\(p = \left[ {\begin{array}{*{20}{c}}0&{0.2}&{0.8}\\{0.6}&0&{0.4}\\{0.5}&{0.5}&0\end{array}} \right]\)

Each of the three boys is equally likely to have the ball at a certain time n.

Find out the boy who is most likely to have the ball at time \(n + 2\)

Lets find the seconds step transition matrix.

\({p^{\left( 2 \right)}} = p \times p\)

\(\begin{aligned}{}{p^{\left( 2 \right)}} &= \left[ {\begin{aligned}{}0&{0.2}&{0.8}\\{0.6}&0&{0.4}\\{0.5}&{0.5}&0\end{aligned}} \right] \times \left[ {\begin{aligned}{}0&{0.2}&{0.8}\\{0.6}&0&{0.4}\\{0.5}&{0.5}&0\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{}{0.52}&{0.4}&{0.08}\\{0.2}&{0.32}&{0.48}\\{0.3}&{0.1}&{0.6}\end{aligned}} \right]\end{aligned}\)

Defining a vector \(v = \left[ {\begin{aligned}{}{\frac{1}{3}}&{\frac{1}{3}}&{\frac{1}{3}}\end{aligned}} \right]\)

That represents the probability of holding the ball at time n by each of the boys. So, the probability of holding the ball at time \(n + 2\) can be obtained by multiplying v by \({p^{\left( 2 \right)}}\).

\(\begin{aligned}{}v{p^{\left( 2 \right)}} &= \left[ {\begin{aligned}{}{\frac{1}{3}}&{\frac{1}{3}}&{\frac{1}{3}}\end{aligned}} \right] \times \left[ {\begin{aligned}{}{0.52}&{0.4}&{0.08}\\{0.2}&{0.32}&{0.48}\\{0.3}&{0.1}&{0.6}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{}{0.34}&{0.273}&{0.386}\end{aligned}} \right]\end{aligned}\)

It can be observed that each of the three boys is equally likely to have the ball at a certain time n, boy C is most likely with probability 0.386 to have the ball at time\(\left( {n + 2} \right)\).