04. Growth_Rate_AND_Asymptotic Notations_.pptx

Design and Analysis of Algorithms
GROWTH RATE AND ASYMPTOTIC NOTATIONS

Objectives
• Understanding Growth Rate
• Functions of Growth Rate
• Asymptotic Notations
• Analyzing Time Efficiency of algorithms and representing in appropriate asymptotic notation
• Examples

Analysis: Performance
DESIGN AND ANALYSIS OF ALGORITHMS 3
o We can determine the maximum number of
primitive/basic operations executed by an algorithm, as a
function of the input size.
Algorithm arrayMax(A, n)
# operations
currentMax  A[0] 2
for (i =1; i<n; i++) 2n
(i=1 once, i<n n times, i++ (n-1) times:post increment)
if A[i]  currentMax then 2(n  1)
currentMax  A[i] 2(n  1)
return currentMax 1
Total 6n 1

Analysis: Performance
Algorithm arrayMax executes 6n  1 primitive operations in the
worst case.
Define:
a = Time taken by the fastest primitive operation
b = Time taken by the slowest primitive operation
Let T(n) be worst-case time of arrayMax. Then
a (6n  1)  T(n)  b(6n  1 )
Hence, the running time T(n) is bounded by two linear functions

Growth Rate of Running Time
• Affects T(n) by a constant factor, but
• Does not alter the growth rate of T(n)
Changing the hardware/ software environment
• The linear growth rate of the running time T(n) is an intrinsic property of algorithm arrayMax
Intrinsic Property
• Growth rate simplifies the Time Efficiency analysis and mechanism to compare algorithms
Analysis and Comparison

Seven
Important
Functions
Constant  1
Logarithmic  log n
Linear  n
N-Log-N  n log n
Quadratic  n2
Cubic  n3
Exponential  2n
These seven functions that often appear in algorithm
analysis.
In a chart, the slope of the line corresponds to the growth
rate of the function
1E-1
1E+1
1E+3
1E+5
1E+7
1E+9
1E+11
1E+13
1E+15
1E+17
1E+19
1E+21
1E+23
1E+25
1E+27
1E+29
1E-1 1E+2 1E+5 1E+8
T(n)
n
Cubic
Quadratic
Linear

n lgn nlgn n2
n3
2n
0 0 0 1
1 0 0 1 1 2
2 1 2 4 8 4
4 2 8 16 64 16
8 3 24 64 512 256
16 4 64 256 4096 65536
32 5 160 1024 32768 4294967296
64 6 384 4096 262144 1.84467E+19
128 7 896 16384 2097152 3.40282E+38
256 8 2048 65536 16777216 1.15792E+77
512 9 4608 262144 134217728 1.3408E+154
1024 10 10240 1048576 1073741824
2048 11 22528 4194304 8589934592

Execution Times
(one time constant is 1 nano seconds)
n f(n) = lgn f(n) = n f(n) = nlgn f(n) = n2
f(n) = n3
f(n) = 2n
1 0 0.003 micro sec 0.01 micro sec 0.033 micro sec 0.1 micro sec 1 micro sec 1 micro sec
2 0 0.004 micro sec 0.02 micro sec 0.086 micro sec 0.4micro sec 8 micro sec 1 milli sec
3 0 0.005 micro sec 0.03 micro sec 0.147 micro sec 0.9 micro sec 27micro sec 1 sec
4 0 0.005 micro sec 0.04 micro sec 0.213 micro sec 1.6 micro sec 64 micro sec 18.3 min
5 0 0.006 micro sec 0.05 micro sec 0.282 micro sec 2.5 micro sec 125 micro sec 13 days
1 02
0.007 micro sec 0.10 micro sec 0.664 micro sec 10 micro sec 1 milli sec 4 exp 13 years
1 03
0.010 micro sec 1.00 micro sec 9.966 micro sec 1 milli sec 1 sec
1 04
0.013 micro sec 10 micro sec 130 micro sec 100 milli sec 16.7 min
1 05
0.017 micro sec 0.10 milli sec 1.67 milli sec 10 s 11.6 days
1 06
0.020 micro sec 1 milli sec 19.93 milli sec 16.7 min 31.7 years
1 07
0.023 micro sec 0.01sec 0.23 sec 1.16 days 31709 years
1 08
0.027 micro sec 0.10 sec 2.66 sec 115.7 days 3.17 exp 7 years
1 09
0.030 micro sec 1 sec 29.90 sec 31.7 years

Constant Factors
The growth rate is not affected by
Constant factors or lower-order terms
Select Only higher order term
Examples
102n + 105 is a linear function
105n2 + 108n is a quadratic function

Asymptotic Notations
• In Mathematics, The term asymptotic means approaching a value or curve arbitrarily
closely (i.e., as some sort of limit is taken).
• Asymptotic analysis, is a method of describing limiting behavior. As an illustration,
suppose that we are interested in the properties of a function f as n becomes very
large. i.e., growth rate.
• Big Oh (O)
• Omega ()
• Theta ()

Big-Oh Notation)
Given functions f(n) and g(n),
f(n) = O(g(n)) iff there exist two positive
constants c and n0 such that
f(n) <= cg(n) for all n >= n0
g(n) is an asymptotic upper bound on f(n).
Example: 2n + 10 is O(n)
◦ 2n + 10  cn
◦ (c  2) n  10
◦ n  10/(c  2)
◦ Pick c = 3 and n0 = 10
Tt
1 n
g(n)
f(n)
n0

Big-Oh Example
Example: the function n2 is
not O(n)
◦ n2  cn
◦ n  c
◦ The above inequality cannot
be satisfied since c must be a
constant
1
10
100
1,000
10,000
100,000
1,000,000
1 10 100 1,000
n
n^2
100n
10n
n

More Big-Oh Examples
 7n-2
7n-2 is O(n)
need c > 0 and n0  1 such that 7n-2  c•n for n  n0
this is true for c = 7 and n0 = 1
 3n3 + 20n2 + 5
3n3 + 20n2 + 5 is O(n3)
need c > 0 and n0  1 such that 3n3 + 20n2 + 5  c•n3 for n  n0
 3 log n + 5
3 log n + 5 is O(log n)
need c > 0 and n0  1 such that 3 log n + 5  c•log n for n  n0

Big-Oh and
Growth Rate
The big-Oh notation gives an upper
bound on the growth rate of a
function
The statement “f(n) is O(g(n))” means
that the growth rate of f(n) is no more
than the growth rate of g(n)
We can use the big-Oh notation to
rank functions according to their
growth rate
f(n) is
O(g(n))
g(n) is
O(f(n))
g(n) grows
more
Yes No
f(n) grows
more
No Yes
Same
growth
Yes Yes

Big-Oh Rules
Use the simplest expression of the class
Say “3n + 5 is O(n)” instead of “3n + 5 is O(3n)”
Use the smallest possible class of functions
Say “2n is O(n)” instead of “2n is O(n2)”
If is f(n) a polynomial of degree d, then f(n) is O(nd), i.e.,
Drop
•Lower-order terms
•Constant factors
Examples:
T(n) = O(2n2) -------- wrong
T(n) = O(n2 + n) ----- wrong

Big-Oh Rules
If T(x) is a polynomial of degree n, then
T(x) = (xn)
T1(n) * T2(n) = O( f(n) * g(n) )
If T1(n) = O(f(n)) and T2(n) = O(g(n))
Then T1(n) + T2(n) = Max(O(f(n)), O(g(n)))

Asymptotic
Algorithm
Analysis
The asymptotic analysis of an algorithm determines the running time
in big-Oh notation
To perform the asymptotic analysis
◦ We find the worst-case number of primitive operations executed as
a function of the input size (Total Running Time)
◦ We express this function with big-Oh notation
Example:
◦ We determine that algorithm arrayMax executes at most 8n  2
primitive operations
◦ We say that algorithm arrayMax “runs in O(n) time”
Since constant factors and lower-order terms are eventually dropped
anyhow, we can disregard them when counting primitive operations

Example: Computing Prefix Averages
We further illustrate asymptotic
analysis with two algorithms for
prefix averages
The i-th prefix average of an array
X is average of the first (i + 1)
elements of X:
A[i] = (X[0] + X[1] + … + X[i])/(i+1)
Computing the array A of prefix
averages of another array X has
applications to financial analysis
0
5
10
15
20
25
30
35
1 2 3 4 5 6 7
X
A

Prefix Averages (Quadratic)
The following algorithm computes prefix averages in quadratic time
Algorithm prefixAverages1(X, n)
Input array X of n integers
Output array A of prefix averages of X #operations
A  new array of n integers n
for i  0 to n  1 do n
s  X[0] n
for j  1 to i do 1 + 2 + …+ (n  1)
s  s + X[j] 1 + 2 + …+ (n  1)
A[i]  s / (i + 1) n
return A 1

Arithmetic Progression
The running time of
prefixAverages1 is
O(1 + 2 + …+ n)
The sum of the first n integers
is n(n + 1) / 2
◦ There is a simple visual proof of
this fact
Thus, algorithm
prefixAverages1 runs in O(n2)
time
0
1
2
3
4
5
6
7
1 2 3 4 5 6

Prefix Averages (Linear)
The following algorithm computes prefix averages in
linear time by keeping a running sum
Algorithm prefixAverages2(X, n)
Input array X of n integers
Output array A of prefix averages of X #operations
A  new array of n integers n
s  0 1
for i  0 to n  1 do n
s  s + X[i] n
A[i]  s / (i + 1) n
return A 1
Algorithm prefixAverages2 runs in O(n) time

Relatives of Big-Oh
big-Omega
 f(n) is (g(n)) iff there is a constant c > 0
and an integer constant n0  1 such that
f(n)  c•g(n) for n  n0
big-Theta
 f(n) is (g(n)) iff there are constants
c’ > 0 and c’’ > 0 and an integer constant
n0  1 such that
c’•g(n)  f(n)  c’’•g(n) for n  n0
n
n0
f(n)
g(n)
n
n0
c2g(n)
c1g(n)
f(n)

Intuition for Asymptotic
Notation
Big-Oh
 f(n) is O(g(n)) if f(n) is asymptotically
less than or equal to g(n)
big-Omega
 f(n) is (g(n)) if f(n) is asymptotically
greater than or equal to g(n)
big-Theta
 f(n) is (g(n)) if f(n) is asymptotically
equal to g(n)

Example Uses of the
Relatives of Big-Oh
f(n) is (g(n)) if it is (n2) and O(n2). We have already seen the former,
for the latter recall that f(n) is O(g(n)) if there is a constant c > 0 and an
integer constant n0  1 such that f(n) < c•g(n) for n  n0
Let c = 5 and n0 = 1
 5n2 is (n2)
f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0  1
such that f(n)  c•g(n) for n  n0
let c = 1 and n0 = 1
 5n2 is (n)
f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0  1
such that f(n)  c•g(n) for n  n0
let c = 5 and n0 = 1
 5n2 is (n2)

A. Given f(𝑛) = 3𝑛2 + 2𝑛 + 10, Prove that f(𝑛) = Θ(𝑛2)
Solution:
f(n) = 3𝑛2 + 2𝑛 + 10 ----------- (1)
g(n) = n2 ------------ (2)
We know that,
If c1.g(n) <= 𝑓(𝑛) <= 𝑐2.𝑔(𝑛) ------------(3)
∀ 𝑛 ≥ 𝑛0 ∧ c1,c2 > 0
Then, 𝑓(𝑛) = Θ(𝑔(𝑛))
R.H.S:
𝑓(𝑛) <= 𝑐.𝑔(𝑛) ------------(4)
∀ 𝑛 ≥ 𝑛0 ∧ c > 0
(4)=> 3𝑛2 + 2𝑛 + 10 <= c*n ----------------(5)
Let c=4, n0 = 5
(4)=> 95 <= 100
=> f(n) < c.g(n) ---------------- (6)
Hence Proved R.H.S.
L.H.S
𝑓(𝑛) >= 𝑐.𝑔(𝑛) ------------(7)
∀ 𝑛 ≥ 𝑛0 ∧ c > 0
(7)=> 3𝑛2 + 2𝑛 + 10 >= c*n ----------------(8)
Let c=3, n0 = 1
(8)=> 15 >= 3
=> f(n) > c.g(n) -------------- (9)
Hence Proved L.H.S.
Hence Proved, f(n) = Θ(g(n))

04. Growth_Rate_AND_Asymptotic Notations_.pptx

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie 04. Growth_Rate_AND_Asymptotic Notations_.pptx

Ähnlich wie 04. Growth_Rate_AND_Asymptotic Notations_.pptx (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

04. Growth_Rate_AND_Asymptotic Notations_.pptx