Rochester Cloud Camp 2012 Slides

1. AWS in Academic Finance
High Performance Modeling
Stephen J. Barr
stephen.barr@simon.rochester.edu
University of Rochester
March 24, 2012
. . . . . .

2. Welcome
Welcome!
Glad to be here at Rochester Cloud Camp 2012
. . . . . .

3. Welcome
Welcome!
Glad to be here at Rochester Cloud Camp 2012
I am a PhD student at the Simon School of Business,
University of Rochester
. . . . . .

4. Welcome
Welcome!
Glad to be here at Rochester Cloud Camp 2012
I am a PhD student at the Simon School of Business,
University of Rochester
Presenting a bit of my research, and how I use AWS daily for
financial models
. . . . . .

5. Quick Overview
First 5-7 minutes
My research question
Remaining time
. . . . . .

6. Quick Overview
First 5-7 minutes
My research question
Remaining time
How I use AWS daily in my research
. . . . . .

7. Research Question
My area of interest is financial capital structure, meaning
how firms use debt and equity issuance to finance
themselves
net debt
Leverage =
total assets
The main goal of this line of research is to explain the dynamics
of leverage and investment in the face of uncertainty.
. . . . . .

8. Research Question
My area of interest is financial capital structure, meaning
how firms use debt and equity issuance to finance
themselves
Main quantities of interest are leverage and investment rates
net debt
Leverage =
total assets
The main goal of this line of research is to explain the dynamics
of leverage and investment in the face of uncertainty.
. . . . . .

9. Research Question
Rather than talk, lets just put together a simple model.
Modeling Principles
Take as many simplifications as possible
. . . . . .

10. Research Question
Rather than talk, lets just put together a simple model.
Modeling Principles
Take as many simplifications as possible
Based on economic principles
. . . . . .

11. Research Question
Rather than talk, lets just put together a simple model.
Modeling Principles
Take as many simplifications as possible
Based on economic principles
Rather than forcing the firm to follow rules about investment
or leverage, the firm chooses the value-maximizing behaviour
given reasonable incentives
. . . . . .

12. Research Question
Rather than talk, lets just put together a simple model.
Modeling Principles
Take as many simplifications as possible
Based on economic principles
Rather than forcing the firm to follow rules about investment
or leverage, the firm chooses the value-maximizing behaviour
given reasonable incentives
Firm’s incentive: maximize equity value
. . . . . .

13. Research Question - Math - I
e(k, k , b, b , z) = (1)
zk θ Profit (2)
+ p − (1 − r (1 − τc )p) New debt + repay old (3)
+ k − (1 − δk) Investment (4)
+ A(k, k ) Capital Adjustment Cost (5)
The function e(·) shows the firm’s financing status.
{
e(·) ≥ 0 the firm make a distribution to shareholders
(6)
e(·) < 0 the firm is issuing equity
. . . . . .

14. Research Question - Math - II
But, to issue equity, the firm goes to an investment bank. This
isn’t free.
φ(e(·)) = λ1 e(·) + λ2 (e(·))2 (7)
The firm wants to make the choices of k and p to maximize
current plus expected future discounted returns. In math, this is
called a dynamic programming problem and is expressed as a
Bellman equation (recursively defined).
V (Kt , Pt , zt∗ ) = (8)
{
max e(Kt , Kt+1 , Pt , Pt+1 , zt∗ ) + φ(e(Kt , Kt+1 , Pt , Pt+1 , zt∗ ))
Kt+1 ,Pt+1 ∈{K×P}
(9)
∫ }
∗ ∗ ∗
+β V (Kt+1 , Pt+1 , z )dΓ(zt , dzt+1 )
. . . . . .

15. Phew!
. . . . . .

16. How do we solve this?
The math and algorithms to solve these programs fall under
“dynamic programming”.
HUGE ISSUE:
The curse of dimensionality!!!
. . . . . .

17. Why is this an issue? - I
For 3 k’s {5, 10, 15} and 2 p’s {2, 4}
   
5 5 5 5 5 5 5 5 10 10 15 15
5 5 5 5 5 5 5 5 10 10 15 15
   
10 10 10 10 10 10  15
k =  k = 5 5 10 10 15 
 5 15
10 10 10 10 10 10  5 10 10 15 
15 15 15 15 15 15 5 5 10 10 15 15
15 15 15 15 15 15 5 5 10 10 15 15
   
2 2 2 2 2 2 2 4 2 4 2 4
4 4 4 4 4 4 2 4 2 4 2 4
  
2 2 2 2 2 2  4
p=  p = 2 4 2 4 2 
4 4 4 4 4 4 2 4 2 4 2 4
   
2 2 2 2 2 2  2 4 2 4 2 4
4 4 4 4 4 4 2 4 2 4 2 4
. . . . . .

18. Why is this an issue - II
1. For sufficient granularity, I need 71 k’s and 29 p’s
. . . . . .

19. Why is this an issue - II
1. For sufficient granularity, I need 71 k’s and 29 p’s
2. The above leads to a 2059 × 2059 matrix
. . . . . .

20. Why is this an issue - II
1. For sufficient granularity, I need 71 k’s and 29 p’s
2. The above leads to a 2059 × 2059 matrix
3. I need one of those for each shock z
. . . . . .

21. Why is this an issue - II
1. For sufficient granularity, I need 71 k’s and 29 p’s
2. The above leads to a 2059 × 2059 matrix
3. I need one of those for each shock z
4. My z is a combination of two processes...I have 323 of them.
. . . . . .

22. Why is this an issue - II
1. For sufficient granularity, I need 71 k’s and 29 p’s
2. The above leads to a 2059 × 2059 matrix
3. I need one of those for each shock z
4. My z is a combination of two processes...I have 323 of them.
5. I have a (2059 × 323) × 2059 matrix of double precision
numbers sitting in memory
. . . . . .

23. Why is this an issue - II
1. For sufficient granularity, I need 71 k’s and 29 p’s
2. The above leads to a 2059 × 2059 matrix
3. I need one of those for each shock z
4. My z is a combination of two processes...I have 323 of them.
5. I have a (2059 × 323) × 2059 matrix of double precision
numbers sitting in memory
6. This is a 665057 × 2059 = 1, 369, 352, 363 element matrix.
. . . . . .

24. How to solve
{
max e(K , K , P, P , z ∗ ) + φ(e(K , K , P, P , z ∗ )) (10)
K ,P ∈{K×P}
∫ }
+ β V (K , P , z ∗ )dΓ(z ∗ , dz ∗ ) (11)
Algorithm: Value Function Iteration
1. Compute eq. (10)
. . . . . .

25. How to solve
{
max e(K , K , P, P , z ∗ ) + φ(e(K , K , P, P , z ∗ )) (10)
K ,P ∈{K×P}
∫ }
+ β V (K , P , z ∗ )dΓ(z ∗ , dz ∗ ) (11)
Algorithm: Value Function Iteration
1. Compute eq. (10)
2. Iteratively guess eq.(11)
. . . . . .

26. How to solve
{
max e(K , K , P, P , z ∗ ) + φ(e(K , K , P, P , z ∗ )) (10)
K ,P ∈{K×P}
∫ }
+ β V (K , P , z ∗ )dΓ(z ∗ , dz ∗ ) (11)
Algorithm: Value Function Iteration
1. Compute eq. (10)
2. Iteratively guess eq.(11)
3. Get a policy function, giving the value-maximizing choice for
every state
. . . . . .

27. How to solve
{
max e(K , K , P, P , z ∗ ) + φ(e(K , K , P, P , z ∗ )) (10)
K ,P ∈{K×P}
∫ }
+ β V (K , P , z ∗ )dΓ(z ∗ , dz ∗ ) (11)
Algorithm: Value Function Iteration
1. Compute eq. (10)
2. Iteratively guess eq.(11)
3. Get a policy function, giving the value-maximizing choice for
every state
4. The policy function is a 323 × 2059 matrix giving the best of
{k , p } for every {z, k, p}
. . . . . .

28. How to solve
{
max e(K , K , P, P , z ∗ ) + φ(e(K , K , P, P , z ∗ )) (10)
K ,P ∈{K×P}
∫ }
+ β V (K , P , z ∗ )dΓ(z ∗ , dz ∗ ) (11)
Algorithm: Value Function Iteration
1. Compute eq. (10)
2. Iteratively guess eq.(11)
3. Get a policy function, giving the value-maximizing choice for
every state
4. The policy function is a 323 × 2059 matrix giving the best of
{k , p } for every {z, k, p}
5. Use a cc2.8xlarge
. . . . . .

29. Software - Old Version
Original solver written in Python / Numpy
. . . . . .

30. Software - Old Version
Original solver written in Python / Numpy
Pros: beautifully general, modular,
. . . . . .

31. Software - Old Version
Original solver written in Python / Numpy
Pros: beautifully general, modular,
Cons: takes 15 mins to solve. Global interpreter lock makes
things hard to parallelize
. . . . . .

32. Software and Hardware - Version 2 - I
Reimplemented in C++ using Eigen - high performance
templating engine
. . . . . .

33. Software and Hardware - Version 2 - I
Reimplemented in C++ using Eigen - high performance
templating engine
Eigen links against Intel’s Math Kernel Library (optimized
BLAS, LAPACK)
. . . . . .

34. Software and Hardware - Version 2 - I
Reimplemented in C++ using Eigen - high performance
templating engine
Eigen links against Intel’s Math Kernel Library (optimized
BLAS, LAPACK)
Intel’s icpc C++ compiler, using -xavx
. . . . . .

35. Software and Hardware - Version 2 - I
Reimplemented in C++ using Eigen - high performance
templating engine
Eigen links against Intel’s Math Kernel Library (optimized
BLAS, LAPACK)
Intel’s icpc C++ compiler, using -xavx
cc2.8xlarge have Intel Xeon Sandy Bridge-EP’s, which have
AVX
. . . . . .

36. Software and Hardware - Version 2 - I
Reimplemented in C++ using Eigen - high performance
templating engine
Eigen links against Intel’s Math Kernel Library (optimized
BLAS, LAPACK)
Intel’s icpc C++ compiler, using -xavx
cc2.8xlarge have Intel Xeon Sandy Bridge-EP’s, which have
AVX
OpenMP to use all 32 threads
. . . . . .

37. Software and Hardware - Version 2 - I
Reimplemented in C++ using Eigen - high performance
templating engine
Eigen links against Intel’s Math Kernel Library (optimized
BLAS, LAPACK)
Intel’s icpc C++ compiler, using -xavx
cc2.8xlarge have Intel Xeon Sandy Bridge-EP’s, which have
AVX
OpenMP to use all 32 threads
Result....
. . . . . .

38. Model Solves in 90 seconds
.

39. Software and Hardware - Version 2 - II
cc2.8xlarge’s are fast
coding towards MKL / AVX makes things gives a substantial
speedup
at this scale, optimizations such as row-major vs column
major are trivial. Eigen makes this easy.
typedef Eigen : : Matrix<double ,
Eigen : : Dynamic , Eigen : : Dynamic ,
Eigen : : RowMajor> MatrixRMXd ;
I can iterate on a 665057 × 2059 about once per second
OpenMP is easy
. . . . . .

40. while ( ( err > iterpar . TOLERANCE )
&& ( itercount < iterpar . MAXIT ) ) {
EV_prev . noalias ( ) = beta ∗ TMAT ∗ V_prev ;
#pragma omp parallel for
for ( int i = 0 ; i < Nshocks ; i++) {
// p a r a l l e l copy
fullmat . middleRows ( i∗ nstates , nstates ) =
staticmat . middleRows ( i∗ nstates , nstates ) ;
// p a r a l l e l b r o a d c a s t
fullmat . middleRows ( i∗ nstates , nstates ) . rowwise ( )
+= EV_prev . row ( i ) ;
// p a r a l l e l new v a l u e f n
V_new . row ( i ) = fullmat . middleRows ( i∗ nstates , nstates )
. rowwise ( ) . maxCoeff ( ) . transpose ( ) ;
// p a r a l l e l g e t mins
diffs ( i ) = ( V_new . row ( i )
− V_prev . row ( i ) ) . array ( ) . abs ( ) . maxCoeff ( ) ;
lbounds ( i ) = ( V_new . row ( i )
− V_prev . row ( i ) ) . minCoeff ( ) ;
ubounds ( i ) = ( V_new . row ( i )
− V_prev . row ( i ) ) . maxCoeff ( ) ;
} // end p a r a l l e l f o r
. . . . . .

41. lbound = beta ∗ double ( lbounds . minCoeff ( ) ) ;
ubound = beta ∗ double ( ubounds . maxCoeff ( ) ) ;
double adjfactoriter = 0 . 5 ∗ ( ( beta /(1− beta ) )
∗ lbound + ( beta /(1− beta ) ) ∗ ubound ) ;
double diff = diffs . maxCoeff ( ) ;
#pragma omp parallel for
for ( int ii = 0 ; ii < Nshocks ; ii++) {
V_new . row ( ii ) . array ( ) += adjfactoriter ;
}
err = diff ;
// UPDATE ITER COUNT AND VALUE FUNCTION
cout << "i: " << itercount++ << ": " << err << endl ;
V_prev = V_new ;
} // END WHILE
. . . . . .

42. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
. . . . . .

43. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
. . . . . .

44. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
. . . . . .

45. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
2. Assign a UUID to Ωi
. . . . . .

46. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
2. Assign a UUID to Ωi
3. Push Ωi SQS
. . . . . .

47. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
2. Assign a UUID to Ωi
3. Push Ωi SQS
SOLVER AMI
. . . . . .

48. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
2. Assign a UUID to Ωi
3. Push Ωi SQS
SOLVER AMI
1. Get latest solver binary
. . . . . .

49. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
2. Assign a UUID to Ωi
3. Push Ωi SQS
SOLVER AMI
1. Get latest solver binary
2. Check SQS queue
. . . . . .

50. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
2. Assign a UUID to Ωi
3. Push Ωi SQS
SOLVER AMI
1. Get latest solver binary
2. Check SQS queue
3. While (queue nonempty)
. . . . . .

51. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
2. Assign a UUID to Ωi
3. Push Ωi SQS
SOLVER AMI
1. Get latest solver binary
2. Check SQS queue
3. While (queue nonempty)
3.1 Get Ωi and solve
. . . . . .

52. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
2. Assign a UUID to Ωi
3. Push Ωi SQS
SOLVER AMI
1. Get latest solver binary
2. Check SQS queue
3. While (queue nonempty)
3.1 Get Ωi and solve
3.2 Simulate and gather statistics
. . . . . .

53. How I Use AWS - Solving the Model A LOT
GOAL: See how the model behaves under random sets of
parameterizations
HOWTO: - Small Python (using mostly boto) Script
For i in 1:N
1. Generate random parameterization Ωi = [ri , θi , δi , P,i , ...]
2. Assign a UUID to Ωi
3. Push Ωi SQS
SOLVER AMI
1. Get latest solver binary
2. Check SQS queue
3. While (queue nonempty)
3.1 Get Ωi and solve
3.2 Simulate and gather statistics
3.3 Push all back to S3
. . . . . .

54. Identification!
.

55. No Identification
.

56. AWS Advantages
For each plot, each point takes at least 90 seconds to generate
. . . . . .

57. AWS Advantages
For each plot, each point takes at least 90 seconds to generate
Allows me to work in real time - start a group of cc2.8xlarge’s
. . . . . .

58. AWS Advantages
For each plot, each point takes at least 90 seconds to generate
Allows me to work in real time - start a group of cc2.8xlarge’s
Paid for by AWS in Education grant
. . . . . .

59. AWS Advantages
For each plot, each point takes at least 90 seconds to generate
Allows me to work in real time - start a group of cc2.8xlarge’s
Paid for by AWS in Education grant
cc2.8xlarge’s are FAST!
. . . . . .

60. Next Research Step - Simulated Method of Moments
MAIN IDEA:
Reality has some true parameterization Ω∗ which we don’t
know.
. . . . . .

61. Next Research Step - Simulated Method of Moments
MAIN IDEA:
Reality has some true parameterization Ω∗ which we don’t
know.
Reality has some statistics X ∗ which we do know.
. . . . . .

62. Next Research Step - Simulated Method of Moments
MAIN IDEA:
Reality has some true parameterization Ω∗ which we don’t
know.
Reality has some statistics X ∗ which we do know.
A model’s Ωi implies simulated statistics Xi .
Xi = f (Ωi )
. . . . . .

63. Next Research Step - Simulated Method of Moments
MAIN IDEA:
Reality has some true parameterization Ω∗ which we don’t
know.
Reality has some statistics X ∗ which we do know.
A model’s Ωi implies simulated statistics Xi .
Xi = f (Ωi )
To find Ω∗ , we can define
g (Ωi ) = sum(abs(X ∗ − f (Ωi )))
and then find the Ωi which minimizes g (Ωi ).
. . . . . .

64. Next Research Step - Simulated Method of Moments
MAIN IDEA:
Reality has some true parameterization Ω∗ which we don’t
know.
Reality has some statistics X ∗ which we do know.
A model’s Ωi implies simulated statistics Xi .
Xi = f (Ωi )
To find Ω∗ , we can define
g (Ωi ) = sum(abs(X ∗ − f (Ωi )))
and then find the Ωi which minimizes g (Ωi ).
This amounts to a multidimensional optimization, which is
well understood (e.g. Nelder Mead, Simulated Annealing)
. . . . . .

65. Next Research Step - Simulated Method of Moments
Pick a large amount of starting values of the optimizers
. . . . . .

66. Next Research Step - Simulated Method of Moments
Pick a large amount of starting values of the optimizers
Each cc2.8xlarge can handle 2 optimizers simultaneously
. . . . . .

67. Next Research Step - Simulated Method of Moments
Pick a large amount of starting values of the optimizers
Each cc2.8xlarge can handle 2 optimizers simultaneously
Should take about 500 iterations to find a solution
. . . . . .

68. Thank you
References:
My blog: www.econsteve.com
. . . . . .

69. Thank you
References:
My blog: www.econsteve.com
My code: www.bitbucket.com/stevejb
. . . . . .

70. Thank you
References:
My blog: www.econsteve.com
My code: www.bitbucket.com/stevejb
Twitter: @stevejb
. . . . . .

71. Thank you
References:
My blog: www.econsteve.com
My code: www.bitbucket.com/stevejb
Twitter: @stevejb
AWS in Eduaction: http://aws.amazon.com/education/
. . . . . .