Solutions by Mohammed Alasmar

1. Full name: MOHAMMED ALASMAR
Email: M.ALASMAR@SUSSEX.AC.UK
GENERAL INFORMATION
Problem & Approaches to Solutions
MATHS CHALLENGE1
http://users.sussex.ac.uk/~gg44/rep2rep/1
The problems that have been solved:
Problem 1 - Cartesian Plane
Problem 2 - Box of Chocolates
Problem 4 - Maze
1
30/09/2018

2. Q1
2

3. !" + !$
2
!"
!$
&" &$
&" + &$
2
Ø The midpoint between the points: &", !" and &$, !$ is given by the following
formula (see Fig.1):
Midpoint =
()*(+
$
,
,)*,+
$
Ø The coordinates of the midpoint are integers iff:
• &" + &$ is an even number AND
• !" + !$ is an even number
Ø The sample space contains the following possible outcomes:
&" + &$ -.-/ , &" + &$ 011 , !" + !$ -.-/ and !" + !$ 011
Ø The probability of each outcomes is:
2[ &" + &$ -.-/] = 2[ &" + &$ 011] = 2[ !" + !$ -.-/] = 2[ !" + !$ 011] = 0.5
i.e., the probability of the sum of two integers to be even = 0.5, and the probability of the sum
of two integers to be odd = 0.5 (we prove these probabilities in solution #2)
Ø Table.1 shows all the possible outcomes with their probabilities. The joint probabilities in Table. 1 are as follows:
2[ &" + &$ -.-/ ∩ !" + !$ -.-/] = 2[ &" + &$ -.-/ ∩ !" + !$ 011] =
2[ &" + &$ 011 ∩ !" + !$ -.-/] = 2[ &" + &$ 011 ∩ !" + !$ 011] = 0.25
Ø The probability that the middle point of the segment that joins the two integer points has integer coordinates =
2[ &1 + &2 ;<;= ∩ !1 + !2 ;<;=] = 0.25
Fig 1
Table 1
&" + &$ -.-/ &" + &$ >??
!" + !$ -.-/ 0.25 0.25 0.5
!" + !$ 011 0.25 0.25 0.5
0.5 0.5 1
3

4. !" #$#%
!" &''
!( #$#%
!( &''
!( #$#%
!( &''
0.5
0.5
0.5
0.5
0.5
0.5
Fig 2
Ø The midpoint between the points: !", -" and !(, -( is given by the following formula:
Ø The coordinates of the midpoint are integers iff:
• !" + !( is an even number AND
• -" + -( is an even number
Ø The sum of two integers is even iff both integers are either even or odd, i.e.
/0/1 + /0/1 = /0/1
/0/1 + 344 = 344
344 + /0/1 = 344
344 + 344 = /0/1
Ø The probability that the sum of two integers !" + !( is even =
5[ 78 + 79 :;:<] = > !" #$#%. !( #$#% + > !" &''. !( &'' = 0.25 + 0.25 = 0.5
, as depicted in the tree diagram in Fig.2.
Ø Similarly, the probability that the sum of two integers !1 + !2 is odd =
5[ 78 + 79 ABB] = > !1 /0/1. !2 344 + > !1 344. !2 /0/1 = 0.25 + 0.25 = 0.5
Ø This means that the probability of each outcomes is:
>[ !" + !( #$#%] = >[ !" + !( &''] = >[ -" + -( #$#%] = >[ -" + -( &''] = 0.5
Midpoint =
CDECF
(
,
GDEGF
(
> !" #$#%. !( #$#%
= 0.5×0.5 = 0.25
> !" &''. !( &''
= 0.5×0.5 = 0.25
> !" #$#%. !( &''
= 0.5×0.5 = 0.25
> !" &''. !( #$#%
= 0.5×0.5 = 0.25
Cont…
4

5. !" + !$ %&%'
!" + !$ ())
*" + *$ %&%'
*" + *$ ())
*" + *$ %&%'
*" + *$ ())
0.5
0.5
0.5
0.5
0.5
0.5
P !" + !$ %&%' ∩ *" + *$ %&%' = 0.5×0.5 = 0.25
Fig 3
Ø Fig.3 shows the tree diagram for all possible outcomes with their probabilities.
Ø The probability that the middle point of the segment that joins the two points has integer coordinates (see Fig.3) =
P !" + !$ %&%' ∩ *" + *$ %&%' = 0.5×0.5 = 0.25
5

6. !" #$#%
!" &''
!( #$#%
!( &''
0.5×0.5×0.5×0.5 = 0.0625
!( #$#%
!( &''
!" #$#%
!" &''
!( #$#%
!( &''
!( #$#%
!( &''
!" #$#%
!" &''
!( #$#%
!( &''
!( #$#%
!( &''
!" #$#%
!" &''
!( #$#%
!( &''
!( #$#%
!( &''
0( #$#%
0( &''
0( #$#%
0( &''
0" #$#%
0" &''
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5×0.5×0.5×0.5 = 0.0625
0.5×0.5×0.5×0.5 = 0.0625
0.5×0.5×0.5×0.5 = 0.0625
Ø The probability that the middle point of the segment that joins them has integer coordinates (see Fig.4)
= 0.0625 + 0.0625 + 0.0625 + 0.0625 = 0.25
Midpoint =
23425
(
,
73475
(
Ø The sum of two integers is even iff both
integers are either even or odd, i.e.
898: + 898: = 898:
;<;= + >?? = >??
>?? + ;<;= = >??
@AA + @AA = BCBD
Ø The coordinates of the midpoint are integers iff:
• 0" + 0( is an even number AND
• !" + !( is an even number
Fig 4
6

7. !" !# $" $# !" + !# $" + $# Is !" + !# AND
$" + $# even?
0 0 0 0 1 1 YES
0 0 0 1 1 0 NO
0 0 1 0 1 0 NO
0 0 1 1 1 1 YES
0 1 0 0 0 1 NO
0 1 0 1 0 0 NO
0 1 1 0 0 0 NO
0 1 1 1 0 1 NO
1 0 0 0 0 1 NO
1 0 0 1 0 0 NO
1 0 1 0 0 0 NO
1 0 1 1 0 1 NO
1 1 0 0 1 1 YES
1 1 0 1 1 0 NO
1 1 1 0 1 0 NO
1 1 1 1 1 1 YES
Midpoint =
()*(+
#
,
-)*-+
#
Ø The coordinates of the midpoint are integers iff:
• !" + !# is an even number AND
• $" + $# is an even number
Ø Let’s use the binary digit 1 to represent the ‘even’ state and the binary digit 0 to
represent the ‘odd’ state.
Ø Hence, the following facts can be represented as shown in Table 2.
./.0 + ./.0 = ./.0, 2. ., 4 + 4 = 4
5657 + 899 = 899, 2. ., 4 + : = :
899 + 5657 = 899, 2. ., : + 4 = :
;<< + ;<< = =>=?, 2. ., : + : = 4
Ø There are 4 inputs in this binary system (!", !#, $" and $#), so we get 2^4 = 16
different binary presentations (from 0 to 15), as shown in Table 3.
Ø There are 4 cases out of 16 where both !" + !# and $" + $# are even
numbers (as shown in Table 3). This means that the probability that the midpoint
has integers = 4/16 = 0.25.
!" !# !" + !#
4 4 4
1 0 0
0 1 0
: : 4
Table 2
$" $# $" + $#
4 4 4
1 0 0
0 1 0
: : 4
Table 3
Inputs
7

8. Q2 8

9. 1. 1 1 1 1 1 1
2. 1 1 1 1 1 2
3. 1 1 1 1 1 3
4. 1 1 1 1 2 2
5. 1 1 1 1 2 3
6. 1 1 1 1 3 3
7. 1 1 1 2 2 2
8. 1 1 1 2 2 3
9. 1 1 1 2 3 3
10. 1 1 1 3 3 3
11. 1 1 2 2 2 2
12. 1 1 2 2 2 3
13. 1 1 2 2 3 3
14. 1 1 2 3 3 3
15. 1 1 3 3 3 3
16. 1 2 2 2 2 2
17. 1 2 2 2 2 3
18. 1 2 2 2 3 3
19. 1 2 2 3 3 3
20. 1 2 3 3 3 3
21. 1 3 3 3 3 3
22. 2 2 2 2 2 2
23. 2 2 2 2 2 3
24. 2 2 2 2 3 3
25. 2 2 2 3 3 3
26. 2 2 3 3 3 3
27. 2 3 3 3 3 3
28. 3 3 3 3 3 3
9
Large Box (LB)
Small Box (SB)
Step 1: Finding the number of flavours groups in the Small Box (SB).
Each box is filled at random with some selection of flavours (f1, f2 and f3), i.e., the
small box will contain one of these flavours groups (10 groups):
1. 1 1 1
2. 1 1 2
3. 1 1 3
4. 1 2 2
5. 1 2 3
6. 1 3 3
7. 2 2 2
8. 2 2 3
9. 2 3 3
10. 3 3 3
(note that numbers here represent flavours i.e., 1 represents f1, 2 represents f2
and 3 represents f3)
It is obvious that we can form 10 different groups in the small box. These groups
contain unordered sampling (for example f1f2f3 is same as f3f2f1 as order is not
important) with replacement (i.e., we might get the same flavour more than once
in the box). This number of groups can be confirmed through this combination
calculation (we explain this formula in Solution #2):
6
3
#Flavours + BoxSize − 1
BoxSize
=
3 + 3 − 1
3
=
5
3
=
5!
3!2!
= 10
Step 2: Similarly, finding the number of flavours groups in the
Large Box (LB). The large box will contain one of these
flavours groups (28 groups):
#Flavours + BoxSize − 1
BoxSize
=
3 + 6 − 1
6
=
8
6
=
8
2
= 28
Cont…

10. 10
5/6
1/6
Step 3: The number of groups with all flavours (i.e., !1!2!3) are as follows:
in the small box: 1 out of 10 groups,
and in the large box: 10 out of 28 groups (see all the possible groups in the previous page)
Step 4: the large box will be chosen only when the dice comes 6 (i.e., with probability 1/6),
otherwise the small box will be chosen (i.e., with probability of 5/6).
Thus, %['(] = 5/6 ./0 %[1(] = 1/6
Step 4: the question asks for the probability that the dice came 6 (i.e., 1() given that the box has all 3 flavours (i.e., !1!2!3)
This is a conditional probability that can be written as follows (the conditional probability between two events):
%[ 1( | !1!2!3 ] =
% 1( ∩ !1!2!3
% !1!2!3
= =
1
6
×
10
28
1
6
×
10
28
+
5
6
×
1
10
=
0.0595
0.143
= 0.417

11. 1. 1 1 1 1 1 1
2. 1 1 1 1 1 2
3. 1 1 1 1 1 3
4. 1 1 1 1 2 2
5. 1 1 1 1 2 3
6. 1 1 1 1 3 3
7. 1 1 1 2 2 2
8. 1 1 1 2 2 3
9. 1 1 1 2 3 3
10. 1 1 1 3 3 3
11. 1 1 2 2 2 2
12. 1 1 2 2 2 3
13. 1 1 2 2 3 3
14. 1 1 2 3 3 3
15. 1 1 3 3 3 3
16. 1 2 2 2 2 2
17. 1 2 2 2 2 3
18. 1 2 2 2 3 3
19. 1 2 2 3 3 3
20. 1 2 3 3 3 3
21. 1 3 3 3 3 3
22. 2 2 2 2 2 2
23. 2 2 2 2 2 3
24. 2 2 2 2 3 3
25. 2 2 2 3 3 3
26. 2 2 3 3 3 3
27. 2 3 3 3 3 3
28. 3 3 3 3 3 3
11
Step 1: The number of flavours groups in the Large Box (LB). The
large box will contain one of these flavours groups (28 groups):
#Flavours + BoxSize − 1
BoxSize
=
3 + 6 − 1
6
=
8
6
= 28
Step 2: We would like to present the sample groups in Step 1 using some codes. For
example,
So on … for the all samples.
This coding scheme requires six symbols ‘×’ (corresponding to box size = 6) and two
separation symbols ‘|’ (one less than the number flavours)
Step 3: We can formulate the total number of groups based on this coding scheme. It is
clear that the total number of combinations (which we have already shown in step 1) =
(why 8 choose 6? Ans. In the code there are 6 ‘x’ symbols plus 2 ‘|’ symbols, so the total
options is 6+2=8 and we want to choose 6 (box size) out of the 8 options.
Thus, the general formula can be written as follows:
)*)*)*)*)*)* = xxxxxx||
)*)*)*)*)*)- = xxxxx||x
..
)*)*)*)*)-). = xxxx|x|x
..
)*)*)*)-).). = xxx x xx
..
)*)*)*).).). = xxx||xxx
Coding
8
6
=
8
2
=
/!
-! 1!
= 28
Cont…

12. 12
Step 4: similarly, the total number of available groups if box size is 3 =
Step 5: Now, the number of samples (groups) that contain all flavours =
If BoxSize = 6, then the number of samples (groups) that contain all flavours =
If BoxSize = 3, then the number of samples (groups) that contain all flavours =
#Flavours + BoxSize − 1
BoxSize
=
3 + 3 − 1
3
=
5
3
= 10
()()()()()() = xxxxxx||
()()()()()(, = xxxxx||x
..
()()()()(,(- = xxxx|x|x
..
()()()(,(-(- = xxx x xx
..
()()()(-(-(- = xxx||xxx
BoxSize − .
BoxSize − /
6 − 1
6 − 3
=
5
3
= 10
3 − 1
3 − 3
=
2
0
= 1
Step 6: the question asks for the probability that the dice came 6 (i.e., 23) given that the box has all 3 flavours (i.e., (1(2(3)
This is a conditional probability that can be written as follows (the conditional probability between two events):
4[ 23 | (1(2(3 ] =
4 23 ∩ (1(2(3
4 (1(2(3
= =
1
6
×
10
28
1
6
×
10
28
+
5
6
×
1
10
=
0.0595
0.143
= 0.417

13. Q4
13

14. G1 G2
R1
R2R3
R4
C1
C2
C3C4
Deadlock?
C1 C2 C3 C4 R1 R2 R3 R4 G1 G2
C1 0 1/3 0 0 0 1/3 1/3 0 0 0
C2 1/3 0 1/3 1/3 0 0 0 0 0 0
C3 0 1/3 0 0 1/3 0 0 0 0 1/3
C4 0 1/3 0 0 0 0 0 1/3 1/3 0
R1 0 0 0 0 1 0 0 0 0 0
R2 0 0 0 0 0 1 0 0 0 0
R3 0 0 0 0 0 0 1 0 0 0
R4 0 0 0 0 0 0 0 1 0 0
G1 0 0 0 0 0 0 0 0 1 0
G2 0 0 0 0 0 0 0 0 0 1
P =
Step 2: Generate the total transition matrix (the absorbing
Markov chain):
Step 1: Give name to each state in the maze, as shown in Fig 1.
There are 10 states in this maze: C1, C2, C3, C4, R1, R2, R3, R4,
G1 and G2.
Note that the states G1, G2, R1, R2, R3 and R4 of the
Markov chain are called absorbing, as the probabilities:
PG1G1 = PG2G2 = PR1R1 = PR2R2 = PR3R3 = PR4R4 = 1, i.e., if the
chain reaches any of these states it is stuck there forever.Fig 1
Cont…
14

15. Step 3: The probability that the dot will end in one of the green squares(C1 is the starting position X0) can be described
by the conditional probability as follows:
P(Xn = G1| X0 =C1) + P(Xn = G2| X0 =C1), n → ∞
= PC1G1
#+ PC1G2
#
% #
=Step 4: Use Matlab to find:
C1
C2
C3
C4
R1
R2
R3
R4
G1
G2
C1 C2 C3 C4 R1 R2 R3 R4 G1 G2
PC1G1
#+ PC1G2
#= 0.0556 + 0.0556 = 0.1112
15
The Markov chain converges to the values shown in this matrix (we used n=1000 steps, i.e., % &''' )
Step 5: Finally, from this matrix we can find the required provability:

16. G1 G2
R1
R2R3
R4
C1
C2
C3C4
C1 C2 C3 C4 R1 R2 R3 R4 G1 G2
C1 0 1/3 0 0 0 1/3 1/3 0 0 0
C2 1/3 0 1/3 1/3 0 0 0 0 0 0
C3 0 1/3 0 0 1/3 0 0 0 0 1/3
C4 0 1/3 0 0 0 0 0 1/3 1/3 0
R1 0 0 0 0 1 0 0 0 0 0
R2 0 0 0 0 0 1 0 0 0 0
R3 0 0 0 0 0 0 1 0 0 0
R4 0 0 0 0 0 0 0 1 0 0
G1 0 0 0 0 0 0 0 0 1 0
G2 0 0 0 0 0 0 0 0 0 1
P =
Step 2: Generate the total transition matrix (the absorbing
Markov chain):
Step 1: Give name to each state in the maze, as shown below.
There are 10 states in this maze: C1, C2, C3, C4, R1, R2, R3, R4,
G1 and G2.
Note that C1, C2, C3 and C4 are transient states, while
R1, R2, R3, R4 ,G1 and G2 are absorbing states.
Note that the states G1, G2, R1, R2, R3 and R4 of the
Markov chain are called absorbing, as the probabilities:
PG1G1 = PG2G2 = PR1R1 = PR2R2 = PR3R3 = PR4R4 = 1, i.e., if the
chain reaches any of these states it is stuck there forever.
16
Cont…

17. C1 C2 C3 C4
C1 0 1/3 0 0
C2 1/3 0 1/3 1/3
C3 0 1/3 0 0
C4 0 1/3 0 0
R1 R2 R3 R4 G1 G2
C1 0 1/3 1/3 0 0 0
C2 0 0 0 0 0 0
C3 1/3 0 0 0 0 1/3
C4 0 0 0 1/3 1/3 0
C1 C2 C3 C4
C1 1 -1/3 0 0
C2 -1/3 1 -1/3 -1/3
C3 0 -1/3 1 0
C4 0 -1/3 0 1
! = # − % &'
=
C1 C2 C3 C4
C1 1.1667 0.5000 0.1667 0.1667
C2 0.5000 1.5000 0.5000 0.5000
C3 0.1667 0.5000 1.1667 0.1667
C4 0.1667 0.5000 0.1667 1.1667
Step 3: The transition matrix has the following format:
Thus,
% = and ( =
Now, # − % =
RQ
I0
Transient states Absorbing states
The entries of W are ‘mean times’ spent in transient states.
For example, the entry WC1C2 is the mean time spent in state C2 given
that the initial state is C1.
TransientAbsorbing
17
The inverse of
the matrix # − %
Cont…

18. where
! is the initial transient state (which is C1 in this question),
" is the last transient state before absorption (which are C4 and C3 in this question) and
# is all relevant absorbing states (which are G1 and G2 in this question)
Hence,
$ %&'( = " | %+ = ! = ,-. /
0
$.0
$ %&'( = 13 | %+ = 11 = ,4(45. $4578 = 0.1667 ×
1
3
= =. =>>?
$ %&'( = 14 | %+ = 11 = ,4(4A. $4A7( = 0.1667 ×
1
3
= =. =>>?
Step 4: Now, the probability of reaching an absorbing state when starting
from an initial state can be obtained using:
18
, = B − D '( =
C1 C2 C3 C4
C1 1.1667 0.5000 0.1667 0.1667
C2 0.5000 1.5000 0.5000 0.5000
C3 0.1667 0.5000 1.1667 0.1667
C4 0.1667 0.5000 0.1667 1.1667
Step 5: Finally, The probability that the dot will end in
one of the green squares = 0.0556 + 0.0556 = 0.1112

19. C1
R2
R3
C2
C4
C3
C1
G2
R1
C2
R2
R3
C2
G1
R4
C2
C4
C3
C1
G2
R1
C2
R2
R3
C2
G1
R4
C2
C4
C3
C1
G2
R1
C2
R2
R3
C2
G1
R4
C2
C4
C3
C1
G2
R1
C2
R2
R3
C2
G1
R4
C2
P[G1]=
!
"
×
!
"
×
!
"
+ 3
!
"
×
!
"
×
!
"
×
!
"
×
!
"
+ 9
!
"
×
!
"
×
!
"
×
!
"
×
!
"
×
!
"
×
!
"
=
!
'(
+ 3
!
')"
+ 9
!
'!*(
=
1
27
+
1
3×27
+
1
9×27
/
01
/ +
/
2
+
/
3
= 0.0535
Both G1 and G2 are symmetric in the maze when the starting point is C1.
Thus: P[G2] = P[G1] = 0.0535
Step 3: Getting more approximate probability value by extending the tree.
It is clear from step 2 that P[G1] has this formula (there is no space to show
longer tree here):
P[G1] =
/
01
/ +
/
2
+
/
3
+
/
01
+
/
7/
+
/
082
+
/
103
+ ⋯ = :. :;;
Similarly, P[G2]= 0.055. Thus, P[G1]+P[G2]= 0.055+0.055 = 0.11
In this solution, we use the tree diagram.
Due to space constraint, we won’t be able to have an infinite
tree size. Hence, we calculate the required probability at
some converging point (~7 steps).
Step 1: create a tree diagram for the first 7 steps of the
chain (as shown in the tree diagram on the right).
Step 2: find the probability:
19

20. 1
2
3
4
5
6
7
8
9
10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
TransitionProbability
>> Q=[ 0, 1/3, 0, 0;
1/3, 0, 1/3, 1/3;
0, 1/3, 0, 0;
0, 1/3, 0, 0];
>> R=[ 0, 1/3, 1/3, 0, 0, 0;
0, 0, 0, 0, 0, 0;
1/3, 0, 0, 0, 0, 1/3;
0, 0, 0, 1/3, 1/3, 0];
>> P=[ Q , R ;
zeros(6,4), eye(6,6)];
>> mc=dtmc(P);
>> numstates = mc.NumStates
numstates = 10
>> graphplot(mc,'ColorEdges',true);
https://uk.mathworks.com/help/econ/dtmc.html
C1
R3
R2
C2
C3
C4
G1
G2
R1
R4
State 1: C4
State 2: C2
State 3: C3
State 4: C1
State 5: R1
State 6: R4
State 7: G1
State 8: R3
State 9: R2
State 10: G2
Note:
This solution is based on Matlab.
Step 1: Enter the transition matrix P and plot the state diagram (10 states)
20
Cont…

21. Step 2: Plotting Markov chain redistributions for n-steps (we use 20 steps here).
The number of steps should be large enough, so the n-step probabilities are converging.
Animated histogram of the redistributions. The vertical axis displays probability
mass, and the horizontal axis displays states.
>> mc=dtmc(P);
% the initial state distribution x0 = 4 (which is C1 as shown in the previous page)
>> X = redistribute(mc,numSteps,'X0',[0 0 0 1 0 0 0 0 0 0])
>> distplot(mc,X,'Type','histogram','FrameRate',4);
21
Cont…

22. Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.2
0.4
0.6
0.8
1
Probability
Step 0
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.2
0.4
0.6
0.8
1
Probability
Step 1
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.2
0.4
0.6
0.8
1
Probability
Step 3
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 5
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 6
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 7
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 8
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.2
0.4
0.6
0.8
1
Probability
Step 2
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 4
x-axis:
State 1: C4
State 2: C2
State 3: C3
State 4: C1
State 5: R1
State 6: R4
State 7: G1
State 8: R3
State 9: R2
State 10: G2
Step 0
Start at C1
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7
Step 8
Plot Markov chain
redistributions.
Cont…

23. Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 12
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 13
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 14
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 15
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 16
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 17
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 18
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 19
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 20
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 9
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 10
Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 11
Step 18
Step 19
Step 20
Step 15
Step 16
Step 17
Step 12
Step 13
Step 14
Step 9
Step 10
Step 11
The 20-step
probabilities are
converging.
Cont…

24. Distribution of States
1 2 3 4 5 6 7 8 9 10
State
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability
Step 20
G1 G1
0.0556
The probability that the dot will end in one of the
green squares= 0.0556 + 0.0556 = 0.1112
x-axis:
State 1: C4
State 2: C2
State 3: C3
State 4: C1
State 5: R1
State 6: R4
State 7: G1
State 8: R3
State 9: R2
State 10: G2
24
Step 20
Step 3: Finally, from the last histogram figure we can find the converging probability for G1 and
G2 given that C1 was the initial state.