91Ó°ÊÓ

Recall that for a two-state continuous-time Markov chain, the expected occupation time in state 0 by time t is given by: \(m(t) = E[O(t)]\) where \(O(t)\) denotes the occupation time in state 0. Now, let's consider the occupation time by time \(t + h\), where \(h\) is a small positive number: \(m(t + h) = E[O(t + h)]\) Since the continuous-time Markov chain is memoryless, the expectation of the occupation time by time \(t + h\) can be computed as the sum of occupation time by time \(t\) and the probability of being in state 0 at time \(t\), multiplied by the small increment \(h\), neglecting the higher-order terms of \(h\): \(m(t + h) = m(t) + P_{00}(t)h + o(h)\) That concludes part (a).

To compute the derivative of the expected occupation time m(t), we use the limit definition of the derivative: \(m'(t) = \lim_{h \to 0} \frac{m(t + h) - m(t)}{h}\) Now, substitute the expression for m(t + h) that we derived in part (a): \(m'(t) = \lim_{h \to 0} \frac{m(t) + P_{00}(t)h + o(h) - m(t)}{h}\) Simplify the expression: \(m'(t) = \lim_{h \to 0} \frac{P_{00}(t)h + o(h)}{h}\) Divide by h: \(m'(t) = \lim_{h \to 0} P_{00}(t) + \frac{o(h)}{h}\) As h goes to 0, the term \(\frac{o(h)}{h}\) goes to 0. Therefore, we are left with: \(m'(t) = P_{00}(t)\) Now, recall that for a two-state continuous-time Markov chain, the transition probability from state 0 to state 0 is given by: \(P_{00}(t) = \frac{\mu}{\lambda+\mu}+\frac{\lambda}{\lambda+\mu}e^{-(\lambda+\mu) t}\) So, we can write the derivative of the expected occupation time as: \(m'(t) = \frac{\mu}{\lambda+\mu}+\frac{\lambda}{\lambda+\mu}e^{-(\lambda+\mu) t}\) That concludes part (b).

Now that we have derived an expression for the derivative of the expected occupation time m'(t), we can find m(t) by integrating m'(t) with respect to t: \(m(t) = \int m'(t) dt = \int (\frac{\mu}{\lambda+\mu}+\frac{\lambda}{\lambda+\mu}e^{-(\lambda+\mu) t}) dt\) Now, break the integral into two parts: \(m(t) = \int \frac{\mu}{\lambda+\mu} dt + \int \frac{\lambda}{\lambda+\mu} e^{-(\lambda+\mu) t} dt\) Integrate each part: \(m(t) = \frac{\mu}{\lambda+\mu}t - \frac{\lambda}{(\lambda+\mu)^2}(e^{-(\lambda+\mu) t}) + C\) Here, C is the integration constant. To find C, we can use the initial condition m(0) = E[O(0)] = 0, because the occupation time in state 0 at time 0 is zero: \(0 = \frac{\mu}{\lambda+\mu}(0) - \frac{\lambda}{(\lambda+\mu)^2}(e^{-(\lambda+\mu)(0)}) + C\) Solve for C: \(C = \frac{\lambda}{(\lambda+\mu)^2}\) Now, substitute the value of C back into the expression for m(t): \(m(t) = \frac{\mu}{\lambda+\mu}t - \frac{\lambda}{(\lambda+\mu)^2}(e^{-(\lambda+\mu) t}) + \frac{\lambda}{(\lambda+\mu)^2}\) This is the expected occupation time in state 0 by time t for the given two-state continuous-time Markov chain. That concludes part (c).