This document outlines an approach to using machine learning algorithms like hidden Markov models to predict stock prices. It discusses how technical analysis and increasing computational power allows algorithms to analyze large datasets. Human analysis is still important for interpretation. The document then provides an overview of using a hidden Markov model to account for different strategies in the stock market and modeling price data more accurately over time.

1. 1

2. 2
• Almost all players on the market (brokerage firm,
banks, wall street) use technical analysis for the
study of stock price evolution, mostly as a
complement for fundamental analysis.
• Even an expert human has a limited ability in
analyzing large quantities of correlations between
useful data's without the help of a computer.
• Increasing computational power makes way for a
new breed of expert analysts: Machine Learning
algorithms.
• Of course, the human still plays an important role
in valuing, guiding and interpreting the results.

3. 3
• The human is not a good analyzing machine by
default:
•People have no conscious experience of most of what is
happening in their minds.
• We tend to perceive what we expect to perceive.
• Mind sets for quickly and are resisting to change.
• New information is assimilated to existing images.
• “When the facts change, I change. And what do you do,
sir?” Keynes

4. 4
 To offer you an image of what the stock market is
 To answer the question: Can I predict the price of
a stock and if yes, to what extent?
 To provide knowledge of some of the latest
machine learning methods you can use in order to
make profits on the stock market

5. 5
 The Stock Market
 Is it possible to predict the future price of a stock?

6. 6
 The Stock Market
 Is it possible to predict the future price of a stock?
 Stock Market Analysis: Schools
 Fundamental Analysis
 Technical Analysis

7. 7
 The Stock Market
 Is it possible to predict the future price of a stock?
 Stock Market Analysis: Schools
 Fundamental Analysis
 Technical Analysis
 Designing a Hidden Markov Model for this problem

8. 8
 The Stock Market
 Is it possible to predict the future price of a stock?
 Stock Market Analysis: Schools
 Fundamental Analysis
 Technical Analysis
 Designing a Hidden Markov Model for this problem
 SQL Server Analysis Services

9. 9
 The Stock Market
 Is it possible to predict the future price of a stock?
 Stock Market Analysis: Schools
 Fundamental Analysis
 Technical Analysis
 Designing a Hidden Markov Model for this problem
 SQL Server Analysis Services
 What’s next?
 Review

10. 10
• Earning Per Share
• (Free) Cash Flow per Share
• P/E Ratio
• Trends
• Inflation/Deflation
• Incidental transactions
• Demography
• Liquidity
• Volume
• Market sentiment
• Industry economic strength

11. 11
• Efficient Market Hypothesis
• Dow Theory
• Analysis

12. 12
• If the value of a firm can be related to its financial
characteristics (growth prospects, risk profile and
cash flows) then we can see if the stock is under or
overvalued.
• Assumptions:
• The relationship between value and underlying financial
factors can be measured
• The relationship is stable over time
• Deviations from the relationship are corrected in a
reasonable time period

13. 13
quot;We try to stick to businesses we believe we
understand. That means they must be relatively
simple and stable in character. If a business is
complex and subject to constant change, we're not
smart enough to predict future cash flows.“ W. B.

14. 14
• Assumptions
• Stock price reflects everything that has or could affect a
the company (including fundamental factors and market
psychology), removing the need to consider these factors
separately.
• Price movements follow established trends.
• History repeat itself.

15. 15
• Problems
• Patterns of financial time series are dynamic and
complex.
• We need to balance between long-term trends and
short-term sideways.
• We need to determine usefulness of information.

16. 16
•We need:
• A financial expert could apply different unknown
strategies. We can model this by considering a set of
states where each state is a unknown strategy
• We can make different observations regarding the time
series evolution with each strategy. We will need set of
observations K. Observations could be Rise, Drop, Large
Rise …
• Which strategy we will first apply? A vector of the initial
state distribution probability.
• Which strategy is most likely to be applied next? A
matrix that contain transition probabilities between states
(strategies).

17. 17

18. 18
• Problems
• One observation like the return rate, or LR is not
enough. We need to feed our machine with vectors of
observations containing for example price and volume.
•Also how do we model data more accurately? For each
state we could build more probability functions that
describe the data (from different views); also we need to
quantify all of this probability distributions and to estimate
each distribution’s weight into the final quantity.
• How to smooth the time series, do we eliminate noise
than can confuse the interpretation?
• More recent data should be more valuable, that is more
sensitive. But if the recent data is in highly volatile period
how do we treat this situation so the model wouldn’t be
affected in the training process?

19. 19
• To model data we will use a Gaussian mixture
model at each state. A mixture model combines
more probability density functions that have the
form of a Gaussian distribution.
• For each adjustable distribution we will assign
adjustable weights to give more credit to some
distribution, that, at the given time period, predicts
more accurately.

20. 20

21. 21

22. 22
• The Gaussian mixture must be multivariate in
order to accept a vector of observation with a length
higher the 1.
• We will use the Gaussian Mixture as a forecast
generator.
•What are the parameters for re-estimation?
• For each Gaussian distribution used in each mixture we
need to re-estimate the mean, covariance matrix and the
weight attached that describes it best, based on the
training cases
• A transition probability matrix between states

23. 23
• Now that we found a way to model more
accurately the data, we ask:
• At training, how do we capture recent trends?
• We use a training pool that drops the oldest data when
a new one is available
• In financial time series recent data has more value than
old data. How do we add more value to the recent data?
• We give her more weight.
• Stock market time series are highly volatile. How do we
smooth the series?
• Using an exponential moving average method.

24. 24
• Exponential Moving Average
• Solves the problem of the importance of recent
data and the problem of needing to smooth the time
series in order to eliminate misleading information
and concentrate on trends.
• The sensitivity of recent data can be controlled
using the Dow Jones Composite Index, which is a
less volatile index.

25. 25
• Re-estimation of parameters

26. 26
•
BI Dev Your Application
Studio
(Visual
Studio) OLE DB/ ADOMD/ XMLA
App
Deploy Data
Analysis Services Mining Model
Server
Data Mining Algorithm Data
Source

27. 27
• Try a different approach other than Gaussian
Mixture
• More work on the EM algorithm
• HMM versus Microsoft Time Series in SQL 2008
(ARIMA + ART)

28. 28
 Review
 Questions