To model a stochastic process as a Markov chain, you define N states (N > 1) and determine a transition matrix P for the chain. You decide that the chain is irreducible and ergodic and calculate a steady-state vector pi. A colleague visits your work area and glances (with your permission) at your work. He notes that your transition matrix P contains the number 1 on the main diagonal of the matrix, and suggests you have made an error in your efforts. Explain why your colleague is correct. Solution First fact: In a Markov chain, a state is said to be ergodic if it is aperiodic and positive recurrent. If all states in a Markov chain are ergodic,then the chain is said to be ergodic. A Markov chain is ergodic if it is possible to pass from any state to any other state (Markov\'s theorem) Then, the transition matrix will have no zero values. Second basic fact: But since we are talking about probability of transition of a state to another state, sum of values of row elements (of any particular row) should be 1. Now if one element on diagonal is 1, rest in that row will have to be zero (from second fact ; remember that no negative vaules are allowed) which contradicts first fact. Hence, there will have to be an error. Hope this helps. Please Rate. Thanks & Cheers.

1. To model a stochastic process as a Markov chain, you define N states (N > 1) and determine a
transition matrix P for the chain. You decide that the chain is irreducible and ergodic and
calculate a steady-state vector pi. A colleague visits your work area and glances (with your
permission) at your work. He notes that your transition matrix P contains the number 1 on the
main diagonal of the matrix, and suggests you have made an error in your efforts. Explain why
your colleague is correct.
Solution
First fact: In a Markov chain, a state is said to be ergodic if it is aperiodic and positive recurrent.
If all states in a Markov chain are ergodic,then the chain is said to be ergodic. A Markov chain is
ergodic if it is possible to pass from any state to any other state (Markov's theorem) Then, the
transition matrix will have no zero values.
Second basic fact: But since we are talking about probability of transition of a state to another
state, sum of values of row elements (of any particular row) should be 1.
Now if one element on diagonal is 1, rest in that row will have to be zero (from second fact ;
remember that no negative vaules are allowed) which contradicts first fact. Hence, there will
have to be an error.
Hope this helps.
Please Rate.
Thanks & Cheers