A successful maximum likelihood parameter estimation scheme using the continuation method (homotopy method) is introduced. This algorithm is particularly useful for the three-parameter skewed distributions including thresholds. Such three-parameter distributions are, for example, Weibull, log-normal, gamma and inverse Gaussian distributions. As the proposed algorithm can almost always obtain the local maximum likelihood estimates automatically, it is of considerable practical value. The Monte Carlo simulation study shows the effectiveness of the proposed method.

1. ICSIST_Tokyo_1994
Hideo Hirose
A SUCCESSFUL MAXIMUM LIKELIHOOD
PARAMETER ESTIMATION IN SKEWED
DISTRIBUTIONS USING THE
CONTINUATION METHOD

2. 3 Parameters Diverged in Weibull3 Parameters Diverged in Weibull
-15
γ
0
0
β
50
L
Kako.data #1
(H.Hirose)
Likelihood
to infinity
log L = −7.75

3. Local Maximum in FrechetLocal Maximum in Frechet
Likelihood
Kako.data #1
(H.Hirose)
β
γ
L
4
6
4 8
η=1.713
β=6.276
γ=5.078
log L = −7.49

4. Local Maximum in GEVLocal Maximum in GEV
Likelihood L
Kako.data #1
(H.Hirose)
-0.2 0.2
3.5
3.0
µ
k
k=-0.1593
µ=3.364
σ=0.2730
WeibullFrechet
Gumbel
log L = −7.49

5. Likelihood function of GEVLikelihood function of GEV
µ
k
σ
3.6
3.1 0.17
0.37
-0.36 0.04
k=-0.1593
µ=3.364
σ=0.2730
log L = −7.49

6. Typical Case in 3-P Weibull LikelihoodTypical Case in 3-P Weibull Likelihood
1.0 3.0
0.0
3.1
β
γ
local maximum
saddle point
corner point
Rockette et al
Data from
Likelihood L

7. Density function of the 3 parameter Weibull
distribution
• The 3 parameter Weibull distribution tends to the
Gumbel distribution
Gumbel
Weibull
x
f(x)

9. θi +1
= θi
− Ji
( )
−1
f θi
( ), i = 0, 1, L
Newton-Raphson
f (θ) = 0
Ji
: Jacobian
f = ( f1, f2,..., fm )T
θ = (θ1,θ2 ,...,θm )
Ji
( )(θi+1
− θi
) = f θi
( ), i = 0, 1, L
Solve

10. Continuation method
h :[0,1]× ℜm
→ ℜm
h 0,θ( ) = g θ( )
h 1,θ( ) = f θ( )
h t,θ( ) = tf θ( ) + (1− t) f θ( ) − f θ 0( )
( ){ }
g : ℜm
→ ℜm
(trivial smooth map)
g(θ) = 0 (known zero points)
θ(0)
(a solution when t=0)
h t,θ( ) = tf θ( ) + (1− t)g(θ)
g(θ) = f θ( ) − f θ 0( )
( )
(parameterized by t)

11. Climbing Up
Starting Point
ε
(Hessian:singular)
Inflection Point
δ<0
δ>0
Final Point
shape parameter
Likelihood Function
embedded model
(Weibull)(Frechet)
(Gumbel)

12. Turning Point
t=0 t=1
turning points
regular
2 Parameter
Weibull
Regular
3 Parameter
Weibull
Non-regular
bifurcation point
Ex.
Ex.

13. Continuation method
h t,θ( ) = tf θ( ) + (1− t) f θ( ) − f θ 0( )
( ){ }
c s( ) = t s( ),θ s( )( ) ∈h−1
0( )
Pursue a smooth curve
c(0) = (0,θ(0)), ˙c 0( ) = (˙t 0( ), ˙θ(0))
s : arc length

14. Continuation method
g(θ) ≡
∂ log L
∂θ
−
∂ logL
∂θ θ 0( )
⇓
g(θ 0( )
) = 0
f (θ) =
∂ logL
∂θ
= 0
Find a zero point
Trivial function
∀
θ 0( )

15. Continuation method
h = 0Differentiate
h' c s( )( )⋅ ˙c s( ) = 0
∂f1
∂t θ(0)
⋅
dt
ds
+
∂f1
∂θ1
dθ1
ds
+
∂f1
∂θ2
dθ2
ds
+L +
∂f1
∂θm
dθm
ds
= 0
∂f21
∂t θ(0)
⋅
dt
ds
+
∂f2
∂θ1
dθ1
ds
+
∂f2
∂θ2
dθ2
ds
+L +
∂f2
∂θm
dθm
ds
= 0
M
∂fm1
∂t θ(0)
⋅
dt
ds
+
∂fm
∂θ1
dθ1
ds
+
∂fm
∂θ2
dθ2
ds
+L +
∂fm
∂θm
dθm
ds
= 0

16. Continuation method
θ j+1( )
= θ j( )
− δ ⋅ J j( )
( )
−1
f θ 0( )
( ), j = 0, 1, L
dt / ds = δ
θi +1
= θi
− Ji
( )
−1
f θi
( ), i = 0, 1, L
Continuation
Newton-Raphson
( )

17. Trace of Parameters
-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2
-2e+09
-2e+09
-1e+09
-5e+08
0e+00
5e+08
s
-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2
2
4
6
8
10
12
14
16
18
s
-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2
-0.2
0.0
0.2
0.4
0.6
0.8
q2
q3
q1
s
shape
location
scale
Parameters
Determinant of
Hessian Matrix
Log-likelihood

19. Success Rates in Finding Local MLE's
in the GEV Model
0
200
400
600
800
1000
1 2 3 4 5 6 7 8 9 0
200
400
600
800
1000
1 2 3 4 5 6 7 8 9
0
200
400
600
800
1000
1 2 3 4 5 6 7 8 9
0
200
400
600
800
1000
1 2 3 4 5 6 7 8 9
# of success
# of success# of success
# of successContinuation
Continuation
Continuation
Continuation
N-R
N-R
N-R N-R
β β
β β
n=100 n=50
n=20 n=10

20. Success Rates in Finding MLE
in the Log-Normal
-200
0
200
400
600
800
1000
1200
-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9
-200
0
200
400
600
800
1000
1200
-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9
-200
0
200
400
600
800
1000
1200
-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9
-200
0
200
400
600
800
1000
1200
-.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9
λ
λλ
λ
Continuation
N-R
ContinuationContinuation
Continuation
N-R
N-R
N-R
# of success
# of success# of success
# of success
n=10
n=50n=100
n=20

23. Existence of A Saddle Point
0.80 0.85 0.90 0.95 1.00
16.79
16.80
16.81
16.82
16.83
saddle point
λ
Parameter σ is optimized.
local maximum point
Log-likelihood for the data in Dumonseaux and Antle (1973)
Existence of a Saddle Point

24. Breakdown Voltage Data
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
5
10
15
20
22.5
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Voltage Voltage Voltage
Voltage Voltage Voltage
Case 1 Case 2 Case 3
Case 4 Case 5 total

25. Breakdown Voltage Data
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
1
2
3
4
5
6
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Count
0
5
10
15
20
22.5
0 .5 1 1.5 2 2.5 3 3.5 4 4.5 5
Voltage Voltage Voltage
Voltage Voltage Voltage
Case 1 Case 2 Case 3
Case 4 Case 5 total

26. Breakdown Voltage Data
Count
0
1
2
3
4
5
6
7
0 5 10 15 20 25 30 35 40
Count
0
1
2
3
4
5
6
7
0 5 10 15 20 25 30 35 40
Count
0
1
2
3
4
5
6
7
0 5 10 15 20 25 30 35 40
Count
0
1
2
3
4
5
6
7
0 5 10 15 20 25 30 35 40
Count
0
1
2
3
4
5
6
7
0 5 10 15 20 25 30 35 40
Count
0
5
10
15
20
25
0 5 10 15 20 25 30 35 40
Voltage
Case 1
Voltage Voltage
Voltage Voltage Voltage
Case 2 Case 3
Case 4 Case 5 total

27. Multiple Local MLE's
-0.5 0.0 0.5 1.0
-12.83
-12.82
-12.81
-12.80
-12.79
λ
local maxima
σ and µ are optimized.
saddle points
Fig. 3 Log-likelihood function from simulated data

28. Likelihoood Function in the GEVLikelihoood Function in the GEV
µ
k
σ
.118
.160
.732
.884
.420 .448
k=0.8036
µ=0.4339
σ=0.1388
log L = 17.068
Data from Dumonseaux and Antle (1973)
Count
0
1
2
3
4
5
6
7
8
0 .2 .4 .6 .8 1 1.2

29. Basin of AttractionBasin of Attraction
σ
µ
k
.160
.118
.420 .448
.884
.732
Successful Initial Value Region in N-R
Julia Sets

30. Basin of AttractionBasin of Attraction
σ
µ
k
.160
.118
.420 .448
.884
.732
Successful Initial Value Region in N-R
Julia Sets

31. RegionRegion
σ
µ
k
.160
.118
.420 .448
.884
.732
1+k(x-µ)/σ > 01+k(x-µ)/σ > 0

32. Basin of AttractionBasin of Attraction
Successful Initial Value Region in N-R
0.0 100.0shape
0.0
100.0
scale
0.0
100.0
location

33. Basin of AttractionBasin of Attraction
Successful Initial Value Region in N-R
0.0 100.0shape
0.0
100.0
scale
0.0
100.0
location

34. Likelihoood function in the GEVLikelihoood function in the GEV
0.0 100.0shape
0.0
100.0
scale
0.0
100.0
location
3.11
3.61
-0.36 0.04
0.17
0.37

35. Basin of AttractionBasin of Attraction
Successful Initial Value Region in N-R
0.0 100.0shape
0.0
100.0
scale
0.0
100.0
location 0.37
0.17
0.04-0.36
3.61
3.11

36. Basin of AttractionBasin of Attraction
Successful Initial Value Region in N-R
0.0 100.0shape
0.0
100.0
scale
0.0
100.0
location
3.11
3.61
0.37
0.17
0.04-0.36

37. The 3 Parameter Weibull Distribution
F(x; η, β, γ)=1 − exp −
x − γ
η
β
x ≥ γ, η > 0, β > 0
η : scale parameter
β: shape parameter
γ: location parameter

38. G(x; a, b)=1 − exp − exp x − b
a
a > 0, −∞< b < ∞
W(x; η, β, γ)=1 − exp −
x − γ
η
β
x ≥ γ, η > 0, β > 0
F(x; η, β, γ)=1 − exp −
γ − x
η
-β
x ≤ γ, η > 0, β > 0
Weibull
Gumbel
Frechet
Generalized extreme-value
Generalized Extreme-value
distribution
H(x; σ, µ, k)=1 − exp − 1 + k
x − µ
σ
1/k
1 + k
x − µ
σ >0, σ > 0