SlideShare a Scribd company logo
1 of 27
Download to read offline
Non Linear Optimization: Finance Applications
Thilanka Munasinghe
Department of Mathematics
West Virginia University
February 24, 2015
Thilanka Munasinghe (WVU) Optimization Methods in Finance 1 / 27
Overview
1 Volatility estimation and ARCH and GARCH models
2 Line Search Methods
3 Newton’s Method
4 Steepest Descent Method
5 Golden Section Search Method
6 Conjugate Gradient Method
Thilanka Munasinghe (WVU) Optimization Methods in Finance 2 / 27
Volatility in Finance
Volatility
Volatility is a term used to describe how much the security prices,
market indices, interest rates, etc., move up and down around their
mean.
Volatility is measured by the standard deviation of the random variable
that represents the financial quantity we are interested in.
Most investors prefer low volatility to high volatility and therefore expect
to be rewarded with higher long-term returns for holding higher
volatility securities.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 3 / 27
Volatility in Finance
Volatility estimations
Many financial computations require volatility estimates.
Most volatility estimation techniques can be classified as either a
historical or an implied method.
One either uses historical time series to infer patterns and estimate the
volatility using a statistical technique
or considers the known prices of related securities such as options that
may reveal the market sentiment on the volatility of the security in
question.
GARCH models exemplify the first approach, while the implied
volatilities calculated from the Black, Scholes and Merton (BSM)
formulas are the best known examples of the second approach.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 4 / 27
ARCH and GARCH models
The basics
Let Y be a stochastic process indexed by natural numbers.
Its value at time t, Yt is an n-dimensional vector of random variables.
The behavior of these random variables is modeled as
Yt =
m
i=1
Φi · Yt−i + t .
Here m is a positive integer representing the number of periods we
look back in our model and t satisfies:
E[ t | 1, ..., t−1] = 0.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 5 / 27
ARCH and GARCH models
The model for the univariate case
We set:
ht = E[ 2
t | 1, ..., t−1].
Then we model the conditional time dependence as follows:
ht = c +
q
i=1
αi · 2
t−i +
p
j=1
βj · ht−j.
This model is called GARCH(p,q).
ARCH model corresponds to choosing p = 0.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 6 / 27
ARCH and GARCH models
The optimization problem in the univariate case
After choosing p and q, the objective is to determine the optimal
parameters Φi, αi, βj.
Usually, this is achieved via maximum likelihood estimation.
If we assume that Yt has a normal distribution conditional on the
historical observations, the log-likelihood function can be written as
follows:
−
T
2
· ln(2 · π) −
1
2
·
T
i=1
ln ht −
1
2
·
T
i=1
2
t
ht
.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 7 / 27
ARCH and GARCH models
The optimization problem in the univariate case
max(−
T
2
· ln(2 · π) −
1
2
·
T
i=1
ln ht −
1
2
·
T
i=1
2
t
ht
)
Yt =
m
i=1
Φi · Yt−i + t
E[ t | 1, ..., t−1] = 0
ht = E[ 2
t | 1, ..., t−1] ≥ 0, for all t
Thilanka Munasinghe (WVU) Optimization Methods in Finance 8 / 27
ARCH and GARCH models
Stationarity properties
An important issue in GARCH parameter estimations is the
stationarity properties of the resulting model.
It is unclear whether one can assume that the model parameters
for financial time series are stationary over the time.
However, the forecasting and estimation is easier on stationary
models.
Theorem
If αi’s, βj’s and the scalar c are strictly positive and
q
i=1
αi +
p
j=1
βj < 1,
then the univariate GARCH model is stationary.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 9 / 27
Black-Scholes-Merton (BSM) equations
BSM for European pricing option
dSt
St
= µ · dt + σ · dWt ,
where
St - the underlying security price at time t,
µ- drift,
σ- the constant volatility,
Wt - a random variable.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 10 / 27
Black-Scholes-Merton (BSM) equations
The solutions
C(K, T) = S0 · Φ(d1) − K · e−r·T
· Φ(d2),
P(K, T) = K · e−r·T
· Φ(−d2) − S0 · Φ(−d1),
where
d1 =
log(S0/K) + (r + σ2/2) · T)
σ ·
√
T
,
d2 = d1 − σ ·
√
T,
Φ()-cumulative distribution function for the standard normal
distribution,
r-continuously compounded risk-free interest rate (a constant
available in US markets),
Thilanka Munasinghe (WVU) Optimization Methods in Finance 11 / 27
Black-Scholes-Merton (BSM) equations
Implied volatility
In order to determine the price, we just need the value or a close
approximation to σ.
Conversely, given the market price for a particular European call or
put, one can determine the volatility (implied by the price), called
implied volatility, by solving these equations with the unknown σ.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 12 / 27
Linear Vs Non Linear
Definition of Linear and Non Linear
f(x) = x1 + 2 · x2 − (3.4) · x3 is linear in x = [x1, x2, x3]T
f(x) = x1 · x2 + 2 · x2 − (3.4) · x3 is Non Linear in x
f(x) = cos(x1) + 2 · x2 − (3.4) · x3 is Non Linear in x
Thilanka Munasinghe (WVU) Optimization Methods in Finance 13 / 27
Existence and Uniqueness of the an Optimum Solution
Existence and uniqueness of the solution
Usually can not guarantee that we have found the Global Optima.
There can be multiple solutions that exist.
For unconstrained problems, at the minimum, f(x∗) = 0
Calculas : at minimum, the Second Derivative is greater than zero
Vector Case : at minimum, Hessian is positive definite.
Remark Necessary and Sufficient conditions for a minimum for an
unconstrained problem :
Gradient must equal to zero, f(x∗) = 0
Hessian must be positive definite.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 14 / 27
Line Search Methods
The Formulation of the Line Search Method
Consider an unconstrained optimization problem,
min f(x)
x ∈ R
Assume the function f(x) is smooth and continuous.
The objective is to find a minimum of f(x).
Thilanka Munasinghe (WVU) Optimization Methods in Finance 15 / 27
Optimization Algorithm
Initial point
Optimization Algorithm starts by an initial point x0, and performs series
of iterations.
Optimal Point
Goal is to find the ”optimal point” x∗
Iteration equation / Iteration Scheme
xk+1 = xk + αk · dk
dk = ”search direction”.
αk = ”step-length”
step-length,αk determines how far to go on the ”search direction”, dk
Thilanka Munasinghe (WVU) Optimization Methods in Finance 16 / 27
Constraints
Definition
The function value of the new point, f(xk+1) should be less than or
equal to the previous point function value, f(xk ).
Constraint
f(xk+1) ≤ f(xk )
f(xk + αk · dk ) ≤ f(xk )
Remark Optimization Algorithm starts with an initial point, x0.
Find the decent ”search direction”, dk .
Determine the ”step-size”, αk .
Check the iteration criteria.
Check the stopping conditions.
Output the Optimal point, x∗
Thilanka Munasinghe (WVU) Optimization Methods in Finance 17 / 27
Newton’s Method
Definition
Newton’s method (also known as the Newton-Raphson method)
Uses to finding successively better approximations to the roots (or
zeroes) of a real-valued function defined on the interval [a, b]
f(x) = 0
Where,f(x) is continuous and differentiable.
Definition
Given a function f(x) is defined over the x ∈ R, and its first derivative is
f (x),
Calculation begins with a first guessing of the initial point x0 for a root
of the function f(x). A new, considerably a better approximation point,
x1
which obtained by,
x1 = x0 + f(x0)
f (x0)Thilanka Munasinghe (WVU) Optimization Methods in Finance 18 / 27
Newton’s Method
Iterative Scheme
As the iterative process repeats, current approximation xn is used to
derive the formula for a new, better approximation, xn+1
xn+1 = xn + f(xn)
f (xn)
Thilanka Munasinghe (WVU) Optimization Methods in Finance 19 / 27
Newton’s Method
Tangent Line
Suppose we have some current approximation xn. Then we can derive
the formula for a better approximation, xn+1. The equation of the
tangent line to the curve y = f(x) at the point x = xn is,
y = f (xn)(xn+1 − xn) + f(xn)
Definition
The x-intercept of this line (the value of x such that y = 0) is then used
as the next approximation to the root, xn+1. In other words, setting
y = 0 and x = xn+1 gives
0 = f (xn)(xn+1 − xn) + f(xn)
Thilanka Munasinghe (WVU) Optimization Methods in Finance 20 / 27
Steepest Descent Method
Steepest Descent Algorithm for Unconstrained Optimization
Consider an unconstrained optimization problem,
min f(x)
x ∈ Rn
Assume the function f(x) is smooth and continuous and differentiable.
If, x = ¯x is a given point, f(x) can be approximated by its linear
expansion
f(¯x + d) ≈ f(¯x) + f(¯x)T d
if, d is small, notice that the approximation above is good.
We choose the value of d so that the inner product, f(¯x)T d is small
as possible.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 21 / 27
Steepest Descent Method
Gradient vector
Let’s normalize d so that, d = 1.
Then among all directions, d with norm d = 1,
the direction,
˜d = − f(¯x)
f(¯x
makes the smallest inner product with the gradient, f(¯x).
This fact follows from the following inequalites:
f(¯x)T d ≥ − f(¯x) d = f(¯x)T (− f(¯x)
f(¯x ) = − f(˜d).
Direction of the Steepest Descent
Due to the above reason, Steepest Descent Direction :
¯d = − f(¯x)
Thilanka Munasinghe (WVU) Optimization Methods in Finance 22 / 27
Steepest Descent Algorithm
Steps of the SD Algorithm
Steps are
Step 1. Initialize x0 and machine accuracy ε , set k = 0
Step 2. dk = − f(xk ). If dk = 0, then stop.
Step 3. Solve minαf(xk + αk · dk ) for the stepsize αk
Step 4. Set xk+1 ← xk + αk · dk , k ← k + 1
Note from Step 3, the fact that dk = − f(xk ) is a descent direction,
which follows the condition of
f(xk + αk · dk ) ≤ f(xk ) .
Thilanka Munasinghe (WVU) Optimization Methods in Finance 23 / 27
Example of Steepest Descent Method
Using SD Method to minimize f(x)
Find the the first iteration value of x∗ using the Steepest Descent
Method :
min f(x) = 0.5x2
1 + 2.5x2
2
Gradient is given by : f(xk ) =
x1
5x2
Taking x0 =
5
1
, we have f(x0) =
5
5
Performing line search along negative gradient direction,
minα0
f(x0 − α0 f(xo))
Exact minimum along line is given by α0 = 1/3 ,
so next approximation isThilanka Munasinghe (WVU) Optimization Methods in Finance 24 / 27
Golden Section Search Method
Definition
Suppose f(x) is unimodal on [a, b] ,
let x1 and x2 be two points within [a, b] ,
where x1 < x2
Evaluating and comparing the values of f(x1) and f(x2), we can
discard either,
(x2, b] or [a, x1), with minimum known to lie in remaining subintervals.
In order to repeat the process, we need to compute only one new
function evaluation.
To reduce length of an interval by a fixed fraction at each iteration, each
new pair of points must have the same relationship with respect to new
interval that previous pair had with respect to that previous interval.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 25 / 27
Golden Section Search Algorithm
Consider τ = (
√
5 − 1)/2
x1 = a + (1 − τ)(b − a)
f1 = f(x1)
x2 = a + τ(b − a) ;
f2 = f(x2)
While
((b − a) > tol) Do
if (f1 > f2) then
a = x1
x1 = x2
f1 = f2
x2 = a + τ(b − a)
f2 = f(x2)
Else
b = x2
x2 = x1
x1 = a + (1 − τ)(b − a)
f1 = f(x1)Thilanka Munasinghe (WVU) Optimization Methods in Finance 26 / 27
Definition
Another method that does not require explicit second derivatives,
and does not even store approximation to the Hessian matrix is,
Conjugate Gradient (CG) method.
CG generates sequence of conjugate search directions, implicitly
accumulating information about Hessian matrix.
For quadratic objective function, CG is theoretically exact after at
most n iterations, where n is the dimension of the problem.
CG is effective for general unconstrained minimization.
Thilanka Munasinghe (WVU) Optimization Methods in Finance 27 / 27

More Related Content

What's hot

Lecture cochran
Lecture cochranLecture cochran
Lecture cochransabbir11
 
Derivatives- CALL AND PUT OPTIONS
Derivatives- CALL AND PUT OPTIONSDerivatives- CALL AND PUT OPTIONS
Derivatives- CALL AND PUT OPTIONSDinesh Kumar
 
Linear Programming (graphical method)
Linear Programming (graphical method)Linear Programming (graphical method)
Linear Programming (graphical method)Kamel Attar
 
Application of linear programming technique for staff training of register se...
Application of linear programming technique for staff training of register se...Application of linear programming technique for staff training of register se...
Application of linear programming technique for staff training of register se...Enamul Islam
 
Dynamic Programming
Dynamic ProgrammingDynamic Programming
Dynamic Programmingcontact2kazi
 
Introduction to derivatives
Introduction to derivativesIntroduction to derivatives
Introduction to derivativesAmeya Ranadive
 
An introduction to Game Theory
An introduction to Game TheoryAn introduction to Game Theory
An introduction to Game TheoryPaul Trafford
 
Introduction to the theory of optimization
Introduction to the theory of optimizationIntroduction to the theory of optimization
Introduction to the theory of optimizationDelta Pi Systems
 

What's hot (20)

Lecture cochran
Lecture cochranLecture cochran
Lecture cochran
 
Game theory
Game theoryGame theory
Game theory
 
Arch & Garch Processes
Arch & Garch ProcessesArch & Garch Processes
Arch & Garch Processes
 
Game theory ppt
Game theory pptGame theory ppt
Game theory ppt
 
Linear programing
Linear programingLinear programing
Linear programing
 
Derivatives- CALL AND PUT OPTIONS
Derivatives- CALL AND PUT OPTIONSDerivatives- CALL AND PUT OPTIONS
Derivatives- CALL AND PUT OPTIONS
 
Linear Programming (graphical method)
Linear Programming (graphical method)Linear Programming (graphical method)
Linear Programming (graphical method)
 
Decision theory
Decision theoryDecision theory
Decision theory
 
Sensitivity analysis
Sensitivity analysisSensitivity analysis
Sensitivity analysis
 
Optmization techniques
Optmization techniquesOptmization techniques
Optmization techniques
 
Optimization tutorial
Optimization tutorialOptimization tutorial
Optimization tutorial
 
Game theory
Game theoryGame theory
Game theory
 
Application of linear programming technique for staff training of register se...
Application of linear programming technique for staff training of register se...Application of linear programming technique for staff training of register se...
Application of linear programming technique for staff training of register se...
 
Dynamic Programming
Dynamic ProgrammingDynamic Programming
Dynamic Programming
 
Concept of Duality
Concept of DualityConcept of Duality
Concept of Duality
 
Introduction to derivatives
Introduction to derivativesIntroduction to derivatives
Introduction to derivatives
 
Game theory
Game theoryGame theory
Game theory
 
An introduction to Game Theory
An introduction to Game TheoryAn introduction to Game Theory
An introduction to Game Theory
 
Regression
RegressionRegression
Regression
 
Introduction to the theory of optimization
Introduction to the theory of optimizationIntroduction to the theory of optimization
Introduction to the theory of optimization
 

Viewers also liked

Grds international conference on pure and applied science (5)
Grds international conference on pure and applied science (5)Grds international conference on pure and applied science (5)
Grds international conference on pure and applied science (5)Global R & D Services
 
Grds international conference on pure and applied science (6)
Grds international conference on pure and applied science (6)Grds international conference on pure and applied science (6)
Grds international conference on pure and applied science (6)Global R & D Services
 
Higher-Order Conjugate Gradient Method (HCGM) For Solving Continuous Optimal ...
Higher-Order Conjugate Gradient Method (HCGM) For Solving Continuous Optimal ...Higher-Order Conjugate Gradient Method (HCGM) For Solving Continuous Optimal ...
Higher-Order Conjugate Gradient Method (HCGM) For Solving Continuous Optimal ...iosrjce
 
Applying Linear Optimization Using GLPK
Applying Linear Optimization Using GLPKApplying Linear Optimization Using GLPK
Applying Linear Optimization Using GLPKJeremy Chen
 
Lesson 4.3 First and Second Derivative Theory
Lesson 4.3 First and Second Derivative TheoryLesson 4.3 First and Second Derivative Theory
Lesson 4.3 First and Second Derivative TheorySharon Henry
 
Time series data mining
Time series data miningTime series data mining
Time series data miningO-Ta Kung
 
Neural network and GA approaches for dwelling fire occurrence prediction
Neural network and GA approaches for dwelling fire occurrence predictionNeural network and GA approaches for dwelling fire occurrence prediction
Neural network and GA approaches for dwelling fire occurrence predictionPhisan Shukkhi
 
นาย จตุรพัฒน์ ภัควนิตย์
นาย จตุรพัฒน์ ภัควนิตย์นาย จตุรพัฒน์ ภัควนิตย์
นาย จตุรพัฒน์ ภัควนิตย์Pornchanida Ansietà
 
01 introduction to data mining
01 introduction to data mining01 introduction to data mining
01 introduction to data miningphakhwan22
 
โครงงานคอมพิวเตอร์ เรื่อง สมุนไพรรางจืด
โครงงานคอมพิวเตอร์ เรื่อง สมุนไพรรางจืดโครงงานคอมพิวเตอร์ เรื่อง สมุนไพรรางจืด
โครงงานคอมพิวเตอร์ เรื่อง สมุนไพรรางจืดRungnaree Uun
 
Comparative study of algorithms of nonlinear optimization
Comparative study of algorithms of nonlinear optimizationComparative study of algorithms of nonlinear optimization
Comparative study of algorithms of nonlinear optimizationPranamesh Chakraborty
 
15 Machine Learning Multilayer Perceptron
15 Machine Learning Multilayer Perceptron15 Machine Learning Multilayer Perceptron
15 Machine Learning Multilayer PerceptronAndres Mendez-Vazquez
 
Parallel algorithms for solving linear systems with block-fivediagonal matric...
Parallel algorithms for solving linear systems with block-fivediagonal matric...Parallel algorithms for solving linear systems with block-fivediagonal matric...
Parallel algorithms for solving linear systems with block-fivediagonal matric...Ural-PDC
 
02.03 Artificial Intelligence: Search by Optimization
02.03 Artificial Intelligence: Search by Optimization02.03 Artificial Intelligence: Search by Optimization
02.03 Artificial Intelligence: Search by OptimizationAndres Mendez-Vazquez
 
Solving Poisson Equation using Conjugate Gradient Method and its implementation
Solving Poisson Equation using Conjugate Gradient Methodand its implementationSolving Poisson Equation using Conjugate Gradient Methodand its implementation
Solving Poisson Equation using Conjugate Gradient Method and its implementationJongsu "Liam" Kim
 

Viewers also liked (20)

Grds international conference on pure and applied science (5)
Grds international conference on pure and applied science (5)Grds international conference on pure and applied science (5)
Grds international conference on pure and applied science (5)
 
Grds international conference on pure and applied science (6)
Grds international conference on pure and applied science (6)Grds international conference on pure and applied science (6)
Grds international conference on pure and applied science (6)
 
Higher-Order Conjugate Gradient Method (HCGM) For Solving Continuous Optimal ...
Higher-Order Conjugate Gradient Method (HCGM) For Solving Continuous Optimal ...Higher-Order Conjugate Gradient Method (HCGM) For Solving Continuous Optimal ...
Higher-Order Conjugate Gradient Method (HCGM) For Solving Continuous Optimal ...
 
Applying Linear Optimization Using GLPK
Applying Linear Optimization Using GLPKApplying Linear Optimization Using GLPK
Applying Linear Optimization Using GLPK
 
Lesson 4.3 First and Second Derivative Theory
Lesson 4.3 First and Second Derivative TheoryLesson 4.3 First and Second Derivative Theory
Lesson 4.3 First and Second Derivative Theory
 
Thai Glossary Repository
Thai Glossary RepositoryThai Glossary Repository
Thai Glossary Repository
 
Time series data mining
Time series data miningTime series data mining
Time series data mining
 
Neural network and GA approaches for dwelling fire occurrence prediction
Neural network and GA approaches for dwelling fire occurrence predictionNeural network and GA approaches for dwelling fire occurrence prediction
Neural network and GA approaches for dwelling fire occurrence prediction
 
นาย จตุรพัฒน์ ภัควนิตย์
นาย จตุรพัฒน์ ภัควนิตย์นาย จตุรพัฒน์ ภัควนิตย์
นาย จตุรพัฒน์ ภัควนิตย์
 
My topic
My topicMy topic
My topic
 
Microsoft Azure day 1
Microsoft Azure day 1Microsoft Azure day 1
Microsoft Azure day 1
 
01 introduction to data mining
01 introduction to data mining01 introduction to data mining
01 introduction to data mining
 
โครงงานคอมพิวเตอร์ เรื่อง สมุนไพรรางจืด
โครงงานคอมพิวเตอร์ เรื่อง สมุนไพรรางจืดโครงงานคอมพิวเตอร์ เรื่อง สมุนไพรรางจืด
โครงงานคอมพิวเตอร์ เรื่อง สมุนไพรรางจืด
 
Comparative study of algorithms of nonlinear optimization
Comparative study of algorithms of nonlinear optimizationComparative study of algorithms of nonlinear optimization
Comparative study of algorithms of nonlinear optimization
 
15 Machine Learning Multilayer Perceptron
15 Machine Learning Multilayer Perceptron15 Machine Learning Multilayer Perceptron
15 Machine Learning Multilayer Perceptron
 
Parallel algorithms for solving linear systems with block-fivediagonal matric...
Parallel algorithms for solving linear systems with block-fivediagonal matric...Parallel algorithms for solving linear systems with block-fivediagonal matric...
Parallel algorithms for solving linear systems with block-fivediagonal matric...
 
02.03 Artificial Intelligence: Search by Optimization
02.03 Artificial Intelligence: Search by Optimization02.03 Artificial Intelligence: Search by Optimization
02.03 Artificial Intelligence: Search by Optimization
 
Media&tech2learn 001-Part 1
Media&tech2learn 001-Part 1Media&tech2learn 001-Part 1
Media&tech2learn 001-Part 1
 
勾配法
勾配法勾配法
勾配法
 
Solving Poisson Equation using Conjugate Gradient Method and its implementation
Solving Poisson Equation using Conjugate Gradient Methodand its implementationSolving Poisson Equation using Conjugate Gradient Methodand its implementation
Solving Poisson Equation using Conjugate Gradient Method and its implementation
 

Similar to Optimization Methods in Finance

Research internship on optimal stochastic theory with financial application u...
Research internship on optimal stochastic theory with financial application u...Research internship on optimal stochastic theory with financial application u...
Research internship on optimal stochastic theory with financial application u...Asma Ben Slimene
 
Presentation on stochastic control problem with financial applications (Merto...
Presentation on stochastic control problem with financial applications (Merto...Presentation on stochastic control problem with financial applications (Merto...
Presentation on stochastic control problem with financial applications (Merto...Asma Ben Slimene
 
Slides econometrics-2018-graduate-2
Slides econometrics-2018-graduate-2Slides econometrics-2018-graduate-2
Slides econometrics-2018-graduate-2Arthur Charpentier
 
Computing near-optimal policies from trajectories by solving a sequence of st...
Computing near-optimal policies from trajectories by solving a sequence of st...Computing near-optimal policies from trajectories by solving a sequence of st...
Computing near-optimal policies from trajectories by solving a sequence of st...Université de Liège (ULg)
 
STA003_WK4_L.pptx
STA003_WK4_L.pptxSTA003_WK4_L.pptx
STA003_WK4_L.pptxMAmir23
 
Reinforcement Learning: Hidden Theory and New Super-Fast Algorithms
Reinforcement Learning: Hidden Theory and New Super-Fast AlgorithmsReinforcement Learning: Hidden Theory and New Super-Fast Algorithms
Reinforcement Learning: Hidden Theory and New Super-Fast AlgorithmsSean Meyn
 
Time-Series Analysis on Multiperiodic Conditional Correlation by Sparse Covar...
Time-Series Analysis on Multiperiodic Conditional Correlation by Sparse Covar...Time-Series Analysis on Multiperiodic Conditional Correlation by Sparse Covar...
Time-Series Analysis on Multiperiodic Conditional Correlation by Sparse Covar...Michael Lie
 
Numerical Methods
Numerical MethodsNumerical Methods
Numerical MethodsTeja Ande
 
Approximation in Stochastic Integer Programming
Approximation in Stochastic Integer ProgrammingApproximation in Stochastic Integer Programming
Approximation in Stochastic Integer ProgrammingSSA KPI
 
Episode 50 : Simulation Problem Solution Approaches Convergence Techniques S...
Episode 50 :  Simulation Problem Solution Approaches Convergence Techniques S...Episode 50 :  Simulation Problem Solution Approaches Convergence Techniques S...
Episode 50 : Simulation Problem Solution Approaches Convergence Techniques S...SAJJAD KHUDHUR ABBAS
 
A Monte Carlo strategy for structure multiple-step-head time series prediction
A Monte Carlo strategy for structure multiple-step-head time series predictionA Monte Carlo strategy for structure multiple-step-head time series prediction
A Monte Carlo strategy for structure multiple-step-head time series predictionGianluca Bontempi
 
Strategies for Sensor Data Aggregation in Support of Emergency Response
Strategies for Sensor Data Aggregation in Support of Emergency ResponseStrategies for Sensor Data Aggregation in Support of Emergency Response
Strategies for Sensor Data Aggregation in Support of Emergency ResponseMichele Weigle
 
Maximum likelihood estimation of regularisation parameters in inverse problem...
Maximum likelihood estimation of regularisation parameters in inverse problem...Maximum likelihood estimation of regularisation parameters in inverse problem...
Maximum likelihood estimation of regularisation parameters in inverse problem...Valentin De Bortoli
 
Intro to Quant Trading Strategies (Lecture 7 of 10)
Intro to Quant Trading Strategies (Lecture 7 of 10)Intro to Quant Trading Strategies (Lecture 7 of 10)
Intro to Quant Trading Strategies (Lecture 7 of 10)Adrian Aley
 

Similar to Optimization Methods in Finance (20)

Research internship on optimal stochastic theory with financial application u...
Research internship on optimal stochastic theory with financial application u...Research internship on optimal stochastic theory with financial application u...
Research internship on optimal stochastic theory with financial application u...
 
Presentation on stochastic control problem with financial applications (Merto...
Presentation on stochastic control problem with financial applications (Merto...Presentation on stochastic control problem with financial applications (Merto...
Presentation on stochastic control problem with financial applications (Merto...
 
Slides econometrics-2018-graduate-2
Slides econometrics-2018-graduate-2Slides econometrics-2018-graduate-2
Slides econometrics-2018-graduate-2
 
Slides ACTINFO 2016
Slides ACTINFO 2016Slides ACTINFO 2016
Slides ACTINFO 2016
 
Computing near-optimal policies from trajectories by solving a sequence of st...
Computing near-optimal policies from trajectories by solving a sequence of st...Computing near-optimal policies from trajectories by solving a sequence of st...
Computing near-optimal policies from trajectories by solving a sequence of st...
 
Slides Bank England
Slides Bank EnglandSlides Bank England
Slides Bank England
 
STA003_WK4_L.pptx
STA003_WK4_L.pptxSTA003_WK4_L.pptx
STA003_WK4_L.pptx
 
Reinforcement Learning: Hidden Theory and New Super-Fast Algorithms
Reinforcement Learning: Hidden Theory and New Super-Fast AlgorithmsReinforcement Learning: Hidden Theory and New Super-Fast Algorithms
Reinforcement Learning: Hidden Theory and New Super-Fast Algorithms
 
Time-Series Analysis on Multiperiodic Conditional Correlation by Sparse Covar...
Time-Series Analysis on Multiperiodic Conditional Correlation by Sparse Covar...Time-Series Analysis on Multiperiodic Conditional Correlation by Sparse Covar...
Time-Series Analysis on Multiperiodic Conditional Correlation by Sparse Covar...
 
Numerical Methods
Numerical MethodsNumerical Methods
Numerical Methods
 
Approximation in Stochastic Integer Programming
Approximation in Stochastic Integer ProgrammingApproximation in Stochastic Integer Programming
Approximation in Stochastic Integer Programming
 
Episode 50 : Simulation Problem Solution Approaches Convergence Techniques S...
Episode 50 :  Simulation Problem Solution Approaches Convergence Techniques S...Episode 50 :  Simulation Problem Solution Approaches Convergence Techniques S...
Episode 50 : Simulation Problem Solution Approaches Convergence Techniques S...
 
Nested sampling
Nested samplingNested sampling
Nested sampling
 
report
reportreport
report
 
A Monte Carlo strategy for structure multiple-step-head time series prediction
A Monte Carlo strategy for structure multiple-step-head time series predictionA Monte Carlo strategy for structure multiple-step-head time series prediction
A Monte Carlo strategy for structure multiple-step-head time series prediction
 
Strategies for Sensor Data Aggregation in Support of Emergency Response
Strategies for Sensor Data Aggregation in Support of Emergency ResponseStrategies for Sensor Data Aggregation in Support of Emergency Response
Strategies for Sensor Data Aggregation in Support of Emergency Response
 
intro
introintro
intro
 
Maximum likelihood estimation of regularisation parameters in inverse problem...
Maximum likelihood estimation of regularisation parameters in inverse problem...Maximum likelihood estimation of regularisation parameters in inverse problem...
Maximum likelihood estimation of regularisation parameters in inverse problem...
 
Talk iccf 19_ben_hammouda
Talk iccf 19_ben_hammoudaTalk iccf 19_ben_hammouda
Talk iccf 19_ben_hammouda
 
Intro to Quant Trading Strategies (Lecture 7 of 10)
Intro to Quant Trading Strategies (Lecture 7 of 10)Intro to Quant Trading Strategies (Lecture 7 of 10)
Intro to Quant Trading Strategies (Lecture 7 of 10)
 

Recently uploaded

UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISINGUNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISINGlokeshwarmaha
 
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...AustraliaChapterIIBA
 
NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023Steve Rader
 
Live-Streaming in the Music Industry Webinar
Live-Streaming in the Music Industry WebinarLive-Streaming in the Music Industry Webinar
Live-Streaming in the Music Industry WebinarNathanielSchmuck
 
Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsIntellect Design Arena Ltd
 
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John MeulemansBCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John MeulemansBBPMedia1
 
Tata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerakTata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerakEditores1
 
Fabric RFID Wristbands in Ireland for Events and Festivals
Fabric RFID Wristbands in Ireland for Events and FestivalsFabric RFID Wristbands in Ireland for Events and Festivals
Fabric RFID Wristbands in Ireland for Events and FestivalsWristbands Ireland
 
Lecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toLecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toumarfarooquejamali32
 
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...Brian Solis
 
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...IMARC Group
 
Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)Lviv Startup Club
 
To Create Your Own Wig Online To Create Your Own Wig Online
To Create Your Own Wig Online  To Create Your Own Wig OnlineTo Create Your Own Wig Online  To Create Your Own Wig Online
To Create Your Own Wig Online To Create Your Own Wig Onlinelng ths
 
Entrepreneurship & organisations: influences and organizations
Entrepreneurship & organisations: influences and organizationsEntrepreneurship & organisations: influences and organizations
Entrepreneurship & organisations: influences and organizationsP&CO
 
Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access
 
Slicing Work on Business Agility Meetup Berlin
Slicing Work on Business Agility Meetup BerlinSlicing Work on Business Agility Meetup Berlin
Slicing Work on Business Agility Meetup BerlinAnton Skornyakov
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView
 
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for BusinessQ2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for BusinessAPCO
 
Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access
 
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)tazeenaila12
 

Recently uploaded (20)

UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISINGUNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
 
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
 
NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023
 
Live-Streaming in the Music Industry Webinar
Live-Streaming in the Music Industry WebinarLive-Streaming in the Music Industry Webinar
Live-Streaming in the Music Industry Webinar
 
Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking Applications
 
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John MeulemansBCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
 
Tata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerakTata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerak
 
Fabric RFID Wristbands in Ireland for Events and Festivals
Fabric RFID Wristbands in Ireland for Events and FestivalsFabric RFID Wristbands in Ireland for Events and Festivals
Fabric RFID Wristbands in Ireland for Events and Festivals
 
Lecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toLecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb to
 
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
 
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
 
Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)
 
To Create Your Own Wig Online To Create Your Own Wig Online
To Create Your Own Wig Online  To Create Your Own Wig OnlineTo Create Your Own Wig Online  To Create Your Own Wig Online
To Create Your Own Wig Online To Create Your Own Wig Online
 
Entrepreneurship & organisations: influences and organizations
Entrepreneurship & organisations: influences and organizationsEntrepreneurship & organisations: influences and organizations
Entrepreneurship & organisations: influences and organizations
 
Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024
 
Slicing Work on Business Agility Meetup Berlin
Slicing Work on Business Agility Meetup BerlinSlicing Work on Business Agility Meetup Berlin
Slicing Work on Business Agility Meetup Berlin
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
 
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for BusinessQ2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
 
Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024
 
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
 

Optimization Methods in Finance

  • 1. Non Linear Optimization: Finance Applications Thilanka Munasinghe Department of Mathematics West Virginia University February 24, 2015 Thilanka Munasinghe (WVU) Optimization Methods in Finance 1 / 27
  • 2. Overview 1 Volatility estimation and ARCH and GARCH models 2 Line Search Methods 3 Newton’s Method 4 Steepest Descent Method 5 Golden Section Search Method 6 Conjugate Gradient Method Thilanka Munasinghe (WVU) Optimization Methods in Finance 2 / 27
  • 3. Volatility in Finance Volatility Volatility is a term used to describe how much the security prices, market indices, interest rates, etc., move up and down around their mean. Volatility is measured by the standard deviation of the random variable that represents the financial quantity we are interested in. Most investors prefer low volatility to high volatility and therefore expect to be rewarded with higher long-term returns for holding higher volatility securities. Thilanka Munasinghe (WVU) Optimization Methods in Finance 3 / 27
  • 4. Volatility in Finance Volatility estimations Many financial computations require volatility estimates. Most volatility estimation techniques can be classified as either a historical or an implied method. One either uses historical time series to infer patterns and estimate the volatility using a statistical technique or considers the known prices of related securities such as options that may reveal the market sentiment on the volatility of the security in question. GARCH models exemplify the first approach, while the implied volatilities calculated from the Black, Scholes and Merton (BSM) formulas are the best known examples of the second approach. Thilanka Munasinghe (WVU) Optimization Methods in Finance 4 / 27
  • 5. ARCH and GARCH models The basics Let Y be a stochastic process indexed by natural numbers. Its value at time t, Yt is an n-dimensional vector of random variables. The behavior of these random variables is modeled as Yt = m i=1 Φi · Yt−i + t . Here m is a positive integer representing the number of periods we look back in our model and t satisfies: E[ t | 1, ..., t−1] = 0. Thilanka Munasinghe (WVU) Optimization Methods in Finance 5 / 27
  • 6. ARCH and GARCH models The model for the univariate case We set: ht = E[ 2 t | 1, ..., t−1]. Then we model the conditional time dependence as follows: ht = c + q i=1 αi · 2 t−i + p j=1 βj · ht−j. This model is called GARCH(p,q). ARCH model corresponds to choosing p = 0. Thilanka Munasinghe (WVU) Optimization Methods in Finance 6 / 27
  • 7. ARCH and GARCH models The optimization problem in the univariate case After choosing p and q, the objective is to determine the optimal parameters Φi, αi, βj. Usually, this is achieved via maximum likelihood estimation. If we assume that Yt has a normal distribution conditional on the historical observations, the log-likelihood function can be written as follows: − T 2 · ln(2 · π) − 1 2 · T i=1 ln ht − 1 2 · T i=1 2 t ht . Thilanka Munasinghe (WVU) Optimization Methods in Finance 7 / 27
  • 8. ARCH and GARCH models The optimization problem in the univariate case max(− T 2 · ln(2 · π) − 1 2 · T i=1 ln ht − 1 2 · T i=1 2 t ht ) Yt = m i=1 Φi · Yt−i + t E[ t | 1, ..., t−1] = 0 ht = E[ 2 t | 1, ..., t−1] ≥ 0, for all t Thilanka Munasinghe (WVU) Optimization Methods in Finance 8 / 27
  • 9. ARCH and GARCH models Stationarity properties An important issue in GARCH parameter estimations is the stationarity properties of the resulting model. It is unclear whether one can assume that the model parameters for financial time series are stationary over the time. However, the forecasting and estimation is easier on stationary models. Theorem If αi’s, βj’s and the scalar c are strictly positive and q i=1 αi + p j=1 βj < 1, then the univariate GARCH model is stationary. Thilanka Munasinghe (WVU) Optimization Methods in Finance 9 / 27
  • 10. Black-Scholes-Merton (BSM) equations BSM for European pricing option dSt St = µ · dt + σ · dWt , where St - the underlying security price at time t, µ- drift, σ- the constant volatility, Wt - a random variable. Thilanka Munasinghe (WVU) Optimization Methods in Finance 10 / 27
  • 11. Black-Scholes-Merton (BSM) equations The solutions C(K, T) = S0 · Φ(d1) − K · e−r·T · Φ(d2), P(K, T) = K · e−r·T · Φ(−d2) − S0 · Φ(−d1), where d1 = log(S0/K) + (r + σ2/2) · T) σ · √ T , d2 = d1 − σ · √ T, Φ()-cumulative distribution function for the standard normal distribution, r-continuously compounded risk-free interest rate (a constant available in US markets), Thilanka Munasinghe (WVU) Optimization Methods in Finance 11 / 27
  • 12. Black-Scholes-Merton (BSM) equations Implied volatility In order to determine the price, we just need the value or a close approximation to σ. Conversely, given the market price for a particular European call or put, one can determine the volatility (implied by the price), called implied volatility, by solving these equations with the unknown σ. Thilanka Munasinghe (WVU) Optimization Methods in Finance 12 / 27
  • 13. Linear Vs Non Linear Definition of Linear and Non Linear f(x) = x1 + 2 · x2 − (3.4) · x3 is linear in x = [x1, x2, x3]T f(x) = x1 · x2 + 2 · x2 − (3.4) · x3 is Non Linear in x f(x) = cos(x1) + 2 · x2 − (3.4) · x3 is Non Linear in x Thilanka Munasinghe (WVU) Optimization Methods in Finance 13 / 27
  • 14. Existence and Uniqueness of the an Optimum Solution Existence and uniqueness of the solution Usually can not guarantee that we have found the Global Optima. There can be multiple solutions that exist. For unconstrained problems, at the minimum, f(x∗) = 0 Calculas : at minimum, the Second Derivative is greater than zero Vector Case : at minimum, Hessian is positive definite. Remark Necessary and Sufficient conditions for a minimum for an unconstrained problem : Gradient must equal to zero, f(x∗) = 0 Hessian must be positive definite. Thilanka Munasinghe (WVU) Optimization Methods in Finance 14 / 27
  • 15. Line Search Methods The Formulation of the Line Search Method Consider an unconstrained optimization problem, min f(x) x ∈ R Assume the function f(x) is smooth and continuous. The objective is to find a minimum of f(x). Thilanka Munasinghe (WVU) Optimization Methods in Finance 15 / 27
  • 16. Optimization Algorithm Initial point Optimization Algorithm starts by an initial point x0, and performs series of iterations. Optimal Point Goal is to find the ”optimal point” x∗ Iteration equation / Iteration Scheme xk+1 = xk + αk · dk dk = ”search direction”. αk = ”step-length” step-length,αk determines how far to go on the ”search direction”, dk Thilanka Munasinghe (WVU) Optimization Methods in Finance 16 / 27
  • 17. Constraints Definition The function value of the new point, f(xk+1) should be less than or equal to the previous point function value, f(xk ). Constraint f(xk+1) ≤ f(xk ) f(xk + αk · dk ) ≤ f(xk ) Remark Optimization Algorithm starts with an initial point, x0. Find the decent ”search direction”, dk . Determine the ”step-size”, αk . Check the iteration criteria. Check the stopping conditions. Output the Optimal point, x∗ Thilanka Munasinghe (WVU) Optimization Methods in Finance 17 / 27
  • 18. Newton’s Method Definition Newton’s method (also known as the Newton-Raphson method) Uses to finding successively better approximations to the roots (or zeroes) of a real-valued function defined on the interval [a, b] f(x) = 0 Where,f(x) is continuous and differentiable. Definition Given a function f(x) is defined over the x ∈ R, and its first derivative is f (x), Calculation begins with a first guessing of the initial point x0 for a root of the function f(x). A new, considerably a better approximation point, x1 which obtained by, x1 = x0 + f(x0) f (x0)Thilanka Munasinghe (WVU) Optimization Methods in Finance 18 / 27
  • 19. Newton’s Method Iterative Scheme As the iterative process repeats, current approximation xn is used to derive the formula for a new, better approximation, xn+1 xn+1 = xn + f(xn) f (xn) Thilanka Munasinghe (WVU) Optimization Methods in Finance 19 / 27
  • 20. Newton’s Method Tangent Line Suppose we have some current approximation xn. Then we can derive the formula for a better approximation, xn+1. The equation of the tangent line to the curve y = f(x) at the point x = xn is, y = f (xn)(xn+1 − xn) + f(xn) Definition The x-intercept of this line (the value of x such that y = 0) is then used as the next approximation to the root, xn+1. In other words, setting y = 0 and x = xn+1 gives 0 = f (xn)(xn+1 − xn) + f(xn) Thilanka Munasinghe (WVU) Optimization Methods in Finance 20 / 27
  • 21. Steepest Descent Method Steepest Descent Algorithm for Unconstrained Optimization Consider an unconstrained optimization problem, min f(x) x ∈ Rn Assume the function f(x) is smooth and continuous and differentiable. If, x = ¯x is a given point, f(x) can be approximated by its linear expansion f(¯x + d) ≈ f(¯x) + f(¯x)T d if, d is small, notice that the approximation above is good. We choose the value of d so that the inner product, f(¯x)T d is small as possible. Thilanka Munasinghe (WVU) Optimization Methods in Finance 21 / 27
  • 22. Steepest Descent Method Gradient vector Let’s normalize d so that, d = 1. Then among all directions, d with norm d = 1, the direction, ˜d = − f(¯x) f(¯x makes the smallest inner product with the gradient, f(¯x). This fact follows from the following inequalites: f(¯x)T d ≥ − f(¯x) d = f(¯x)T (− f(¯x) f(¯x ) = − f(˜d). Direction of the Steepest Descent Due to the above reason, Steepest Descent Direction : ¯d = − f(¯x) Thilanka Munasinghe (WVU) Optimization Methods in Finance 22 / 27
  • 23. Steepest Descent Algorithm Steps of the SD Algorithm Steps are Step 1. Initialize x0 and machine accuracy ε , set k = 0 Step 2. dk = − f(xk ). If dk = 0, then stop. Step 3. Solve minαf(xk + αk · dk ) for the stepsize αk Step 4. Set xk+1 ← xk + αk · dk , k ← k + 1 Note from Step 3, the fact that dk = − f(xk ) is a descent direction, which follows the condition of f(xk + αk · dk ) ≤ f(xk ) . Thilanka Munasinghe (WVU) Optimization Methods in Finance 23 / 27
  • 24. Example of Steepest Descent Method Using SD Method to minimize f(x) Find the the first iteration value of x∗ using the Steepest Descent Method : min f(x) = 0.5x2 1 + 2.5x2 2 Gradient is given by : f(xk ) = x1 5x2 Taking x0 = 5 1 , we have f(x0) = 5 5 Performing line search along negative gradient direction, minα0 f(x0 − α0 f(xo)) Exact minimum along line is given by α0 = 1/3 , so next approximation isThilanka Munasinghe (WVU) Optimization Methods in Finance 24 / 27
  • 25. Golden Section Search Method Definition Suppose f(x) is unimodal on [a, b] , let x1 and x2 be two points within [a, b] , where x1 < x2 Evaluating and comparing the values of f(x1) and f(x2), we can discard either, (x2, b] or [a, x1), with minimum known to lie in remaining subintervals. In order to repeat the process, we need to compute only one new function evaluation. To reduce length of an interval by a fixed fraction at each iteration, each new pair of points must have the same relationship with respect to new interval that previous pair had with respect to that previous interval. Thilanka Munasinghe (WVU) Optimization Methods in Finance 25 / 27
  • 26. Golden Section Search Algorithm Consider τ = ( √ 5 − 1)/2 x1 = a + (1 − τ)(b − a) f1 = f(x1) x2 = a + τ(b − a) ; f2 = f(x2) While ((b − a) > tol) Do if (f1 > f2) then a = x1 x1 = x2 f1 = f2 x2 = a + τ(b − a) f2 = f(x2) Else b = x2 x2 = x1 x1 = a + (1 − τ)(b − a) f1 = f(x1)Thilanka Munasinghe (WVU) Optimization Methods in Finance 26 / 27
  • 27. Definition Another method that does not require explicit second derivatives, and does not even store approximation to the Hessian matrix is, Conjugate Gradient (CG) method. CG generates sequence of conjugate search directions, implicitly accumulating information about Hessian matrix. For quadratic objective function, CG is theoretically exact after at most n iterations, where n is the dimension of the problem. CG is effective for general unconstrained minimization. Thilanka Munasinghe (WVU) Optimization Methods in Finance 27 / 27