Proof master theorem in Analysis of Algorithms

1. 1
Proof of Master Theorem
• The proof for the exact powers, n=bk
for k≥1.
• Lemma 4.2
– for T(n) = Θ(1) if n=1
– aT(n/b)+f(n) if n=bk
for k≥1
– where a ≥ 1, b>1, f(n) be a nonnegative function,
– Then
– T(n) = Θ(nlogba
)+ aj
f(n/bj
)
• Proof:
– By iterating the recurrence
– By recursion tree (See figure 4.3)
∑
j=0
logb
n-1

2. 2
Recursion tree for T(n)=aT(n/b)+f(n)

3. 3
Proof of Master Theorem (cont.)
• Lemma 4.3:
– Let a ≥ 1, b>1, f(n) be a nonnegative function defined on
exact power of b, then
– g(n)= aj
f(n/bj
) can be bounded for exact power of b as:
1. If f(n)=O(nlogb
a-ε) for some ε>0, then g(n)= O(nlogb
a
).
2. If f(n)= Θ(nlogb
a
), then g(n)= Θ(nlogb
a
lg n).
3. If f(n)= Ω(nlogb
a+ε) for some ε>0 and if af(n/b) ≤cf(n) for some
c<1 and all sufficiently large n ≥b, then g(n)= Θ(f(n)).
∑
j=0
logb
n-1

4. 4
Proof of Lemma 4.3
• For case 1: f(n)=O(nlogb
a-ε) implies f(n/bj
)=O((n /bj
)logb
a-ε), so
• g(n)= aj
f(n/bj
) =O( aj
(n /bj
)logb
a-ε)
• = O(nlogb
a-ε aj
/(blogb
a-ε)j
) = O(nlogb
a-ε aj
/(aj
(b-ε)j
))
• = O(nlogb
a-ε (bε)j
) = O(nlogb
a-ε(((bε)logbn
-1)/(bε
-1) )
• = O(nlogb
a-ε(((blogbn
)ε
-1)/(bε
-1)))=O(nlogb
a
n-ε
(nε
-1)/(bε
-1))
• = O(nlogb
a
)
∑
j=0
logb
n-1
∑
j=0
logb
n-1
∑
j=0
logb
n-1
∑
j=0
logb
n-1
∑
j=0
logb
n-1

5. 5
Proof of Lemma 4.3(cont.)
• For case 2: f(n)= Θ(nlogb
a
) implies f(n/bj
)= Θ((n /bj
)logb
a
), so
• g(n)= aj
f(n/bj
) = Θ( aj
(n /bj
)logb
a
)
• = Θ(nlogb
a
aj
/(blogb
a
)j
) = Θ(nlogb
a
1)
• = Θ(nlogb
a
logb
n) = Θ(nlogb
a
lg n)
∑
j=0
logb
n-1
∑
j=0
logb
n-1
∑
j=0
logb
n-1
∑
j=0
logb
n-1

6. 6
Proof of Lemma 4.3(cont.)
• For case 3:
– Since g(n) contains f(n), g(n) = Ω(f(n))
– Since af(n/b) ≤cf(n), aj
f(n/bj
) ≤cj
f(n) , why???
– g(n)= aj
f(n/bj
) ≤ cj
f(n) ≤ f(n) cj
– =f(n)(1/(1-c)) =O(f(n))
– Thus, g(n)=Θ(f(n))
∑
j=0
logb
n-1
∑
j=0
logb
n-1
∑
j=0
∞

7. 7
Proof of Master Theorem (cont.)
• Lemma 4.4:
– for T(n) = Θ(1) if n=1
– aT(n/b)+f(n) if n=bk
for k≥1
– where a ≥ 1, b>1, f(n) be a nonnegative function,
1. If f(n)=O(nlogb
a-ε) for some ε>0, then T(n)= Θ(nlogb
a
).
2. If f(n)= Θ(nlogb
a
), then T(n)= Θ(nlogb
a
lg n).
3. If f(n)=Ω(nlogb
a+ε) for some ε>0, and if af(n/b) ≤cf(n) for
some c<1 and all sufficiently large n, then T(n)=
Θ(f(n)).

8. 8
Proof of Lemma 4.4 (cont.)
• Combine Lemma 4.2 and 4.3,
– For case 1:
• T(n)= Θ(nlogb
a
)+O(nlogb
a
)=Θ(nlogb
a
).
– For case 2:
• T(n)= Θ(nlogb
a
)+Θ(nlogb
a
lg n)=Θ(nlogb
a
lg n).
– For case 3:
• T(n)= Θ(nlogb
a
)+Θ(f(n))=Θ(f(n)) because f(n)= Ω(nlogb
a+ε).

9. 9
Floors and Ceilings
• T(n)=aT(n/b)+f(n) and T(n)=aT(n/b)+f(n)
• Want to prove both equal to T(n)=aT(n/b)+f(n)
• Two results:
– Master theorem applied to all integers n.
– Floors and ceilings do not change the result.
• (Note: we proved this by domain transformation too).
• Since n/b≤n/b, and n/b≥ n/b, upper bound for
floors and lower bound for ceiling is held.
• So prove upper bound for ceilings (similar for lower
bound for floors).

10. 10
Upper bound of proof for T(n)=aT(n/b)+f(n)
• consider sequence n, n/b,  n/b/b,   n/b /b/b,
…
• Let us define nj as follows:
• nj= n if j=0
• = nj-1/b if j>0
• The sequence will be n0, n1, …, nlogb
n
• Draw recursion tree:

11. 11
Recursion tree of T(n)=aT(n/b)+f(n)

12. 12
The proof of upper bound for ceiling
– T(n) = Θ(nlogba
)+ aj
f(nj)
– Thus similar to Lemma 4.3 and 4.4, the upper
bound is proven.
∑
j=0
logb
n -1

13. 13
The simple format of master theorem
• T(n)=aT(n/b)+cnk
, with a, b, c, k are positive
constants, and a≥1 and b≥2,
O(nlogba
), if a>bk
.
• T(n) = O(nk
logn), if a=bk
.
O(nk
), if a<bk
.