These days a lot of data being generated is in the form of time series. From climate data to users post in social media, stock prices, neurological data etc. Discovering the temporal dependence between different time series data is important task in time series analysis. It finds its application in varied fields ranging from advertising in social media, finding influencers, marketing, share markets, psychology, climate science etc. Identifying the networks of dependencies has been studied in this report. In this report we have study how this problem has been studied in the field of econometrics. We will also study three different approaches for building causal networks between the time series and then see how this knowledge has been used in three completely different fields. At last some important issues are presented and areas in which this can be extended for further research.

1. Causality Detection in Time Series
Tushar Mehndiratta
IDD CSE ( V year)
10211026

2. Overview
 Introduction
 Detecting Causality
 Control Experimentation
 Granger Causality
 Building Causal Relationship Graphs
 Exhaustive Granger Method
 Lasso Granger Method
 Forward backward Granger Method
 Application of Causal Modelling
 Brain Imaging
 Topic Mining
 Anomaly Detection
 Conclusion

3. Why?
Why did the apple fall
down instead of going
up?
Why does average
temperature rise?
Why did the stock
market fall?
Why did a post go viral
on facebook?
CAUSE EFFECT RELATIONSHIPS

4. Cause - Effect Relationships
◦ Causality is defined as the relation between two events: cause and effect where
the effect occurs as a consequence of the cause.
◦ Effect is “What happened?” and Cause is “Why it happened?”
◦ e.g. In case of global warming, the increase in Greenhouse gases is the cause and
increase in average temperature is the effect.[1]

5. Characteristics of Causal Relationships
◦ Temporal Precedence: It states that the cause occurs prior to the effect. e.g. A person
must smoke first and then he gets lung cancer.
◦ Co-occurrence : Whenever cause happens, effect must also happen. Cause cannot be
isolated from the effect. e.g. Whenever there is a net force on a body, it will accelerate.
Is Causality same as Association then?

6. Correlation Vs Causation
◦ Correlation does not imply Causation
◦ Correlation only means that two events co-exist more often than ordinary chance.[2]

7. Physics
Econometrics
Types of Data: web metrics , stock prices, sales
(all time series)
Medicine
Types of Data: experiments result, gene
sequences(sequential data), brain signals(time
series)
Climate Science
Types of Data: weather conditions (spatio-temporal
or temporal data)
Fields of Study
HOW TO DETECT CAUSALITY?

8. Detecting Causality
To test if X causes Y

9. Control Experimentation
Aim: To find out what happens to a system when you interfere with it.
Divide subjects
randomly into two
groups: Test and
Control
Introduce X only in
the test group and
observe Y in both.
If X causes Y :
ܲ(ܻ=ݕ|݀݋(ܺ)) >
ܲ(ܻ=ݕ|!݀݋(ܺ))
implies Causality

10. Disadvantage of Control Experimentation
◦ Not possible to always carry out the experiment.
◦ Most time series data cannot be manipulated. e.g. Climate, Stock data
◦ Have to resort to statistical methods to determine causality.
HOW TO DO IT IN TIME SERIES?

11. Time Series
◦ A time series is a sequence of data points, measured typically at successive
points in time spaced at uniform time intervals.

12. Granger Causality
◦ Also known as Predictive Causality.
◦ Granger said that Causality could be reflected by measuring the ability of
predicting the future values of a time series using past values of another time
series.
◦ Two main principles:
 Cause must occur before the Effect.
 The Cause can be used to predict the of Effect i.e. Cause has some unique information
about the future values of the effect.

13. Granger Causality
Suppose X and Y are two time series and for X to cause Y :
푃[푌(푡 + 1)|훤 푡 ≠ 푃[푌(푡 + 1)| 훤−X 푡
훤 푡 and 훤−X 푡 denote the “information in the universe up to time t” and “information in
alternate universe up to time t in which X is excluded”.

14. Performing the Granger Causality test
◦ Model 1: Build model 1 by regressing on the past values of both X and Y
푚 훼푗푌푡−푗 + 푖=1
퐸(푌|푌푡−푘 , 푋푡−푘 ) 푌푡 = 푗=1
푛 훽푖 푋푡−푖 + 퐷푡 + 휀푡
◦ Model 2: Build model 2 by regressing on the past values of Y only
푚 훼푗푌푡−푗 + 퐷푡 + 휀푡
퐸(푌|푌푡−푘 ) 푌푡 = 푗=1
◦ Check whether the prediction accuracy has significantly increased by performing
F-test.[11]

15. Granger Causality
• CONS
 It does not take into account the effect of hidden common
causes(confounders)
 It assumes that all the relationships are linear in nature and does not account
for non-linear dependencies.
HOW TO DEAL WITH MULTIPLE TIME SERIES?

16. Relationship Graphs in Time Series
Extending the concept of Granger Causality to Mult iple Time Series

17. Relationship Graphs
◦ Relationship graph has all time series as nodes and an edge between any two
nodes denotes the direction of relationship between the two.
◦ Input:
 Matrix X of time series
 Xlag which is the lagged versions of time series matrix X.
◦ Output
◦ Relationship graph between the time series with nodes xi’s each edge from xi to xj if xi
causes xj.
xi xj

18. Exhaustive Graphical Granger method
◦ Algorithm:
◦ For every pair of nodes(xi,xj) perform the following
 Insert an edge xi → xj if Granger (xi,xj, Xlag) = ‘yes’ and Granger (xj,xi, Xlag) = ‘no’
 Insert an edge xi ← xj if Granger (xi,xj, Xlag)= ‘no’ and Granger (xj,xi, Xlag) = ‘yes’
 Insert an edge xi↔xj, if Granger (xi,xj, Xlag) = ‘yes’ and Granger (xj,xi, Xlag)= ‘yes’

19. Exhaustive Graphical Granger method
◦ Complexity
 A total of N time series with T lags each and P time stamps/sample size, makes the
complexity as O(N2P2T2).
◦ Shortcomings
 Not considering the effect of other time series.
 Computationally expensive.

20. The LASSO-Granger Method
◦ Uses variable selection in Causality Detection
◦ Aim is to identify the subset of time series on which xi is conditionally dependent
and on what lag is it dependent.
◦ Achieved by applying variable selection on the set of time series and the lags
◦ Variable selection is done by LASSO.
LASSO-Least Absolute Shrinkage and Selection Operator

21. LASSO
◦ A selection method for linear regression
◦ Selects a subset of variables subject to the following condition
푤 = 푚푖푛
1
n
(푤. 푥 − 푦)2+휆 푤
Here w is the vector of coefficients, y is the variable to be predicted.
◦ Aim is to minimize the OLS error and the sum of coefficients to prevent over
fitting.
◦ LARS(Least Angle Regression): best method to achieve LASSO.

22. LARS(Least Angle Regression Shrinkage)
Step 1: Start with û0=0
Step 2: The residual ŷ2-û0 has a
greater correlation with x1 than
with x2

23. LARS(Least Angle Regression Shrinkage)
Step 3: Move in the direction of x1

24. LARS(Least Angle Regression Shrinkage)
Step 4: First LARS estimate :
û1 = û0 + ƛx1
where the residual ŷ2-û1
has equal correlation with
both x1 and x2

25. LARS(Least Angle Regression Shrinkage)
Step 5: Move in the direction of
Angular bisector of x1 and x2

26. The Lasso-Granger Method
◦ Algorithm
 Obtain Xlag(the lagged version of the time series matrix X).
 For each xi in X,
 y= xi
 Performs LASSO (y,Xlag)
Wi : the set of time series for which the coefficients returned by are non-zero.
 Add edge (xj, xi) to the graph if xj is in Wj

27. The Lasso-Granger Method
◦ Complexity
 Using LARS to solve the lasso problem: O(PN2T2).
◦ Pros.
 Computationally less expensive.
 Can be used when number of series are quite large as compared to the number of data
points.
 Consistency: The probability of Lasso falsely including a non-neighboring feature in its
neighborhood is very small even when the number of features are very large.

28. Forward Backward Granger Causality
◦ Improvement on LASSO-Granger Algorithm
◦ Inspired from Physics
◦ Principle: Reverse time and all the relationships must remain same except for
change in direction, i.e. if xi causes xj with a time lag of k then on reversing time xj
will cause xi with time lag k.
◦ Apply LASSO-Granger on both the forward and backward time series and
combine the results of the two.

29. Application of Causal Modelling
BRAIN IMAGING TOPIC MINING ANOMALY DETECTION

30. Brain Imaging
◦ How different portions of the brain affect one another.
 Identify the direction and order of influence
◦ Apply Granger Causality to obtain the relationship between different components of
the brain.
 Obtain fMRI data from the brain corresponding to a stimulus and divided it into
independent components corresponding to different sections of the brain.
 Each independent component corresponds to a time series.
 Apply Exhaustive Granger test to obtain the relationship between different time
series.

31. Brain Imaging
◦ Advantages:
 No prior assumption about the nodes and their inter-connections.
 Measures not only the connections but also the time lags between interactions.
 Can work with a large number of regions.

32. Mining topics based on Causality
◦ Identification of topics that are causally related with the non textual data
iteratively.
◦ InCaToMi (Integrative Causal Topic Miner)
◦ Architecture:
Topic modelling
module
Causality Module
Feedback
Text Data

33. InCaToMi: Integrative Causal Topic Miner
◦ Topic Modelling Module:
 Takes text and number of topics as input.
 Creates topics based on word probabilities and the likelihood of each topic in the
document using PLSA algorithm.
 Time series of the topic formed by summation of likelihood of each word in the topic for
a day.

34. InCaToMi: Integrative Causal Topic Miner
◦ Causality Module:
 Perform the Granger Causality test for the time series for each topic and for each word in
the topic.
 Form new candidate topic by selecting the words which are most causally related with the
non textual series.
 Use this as prior for the next round of Topic Modelling.

35. Anomaly Detection
◦ Types of Anomalies:
 Univariate Anomalies
 Dependency Anomalies
◦ Given two sets of data sequences A(training) and B(test) each containing p time
series we have to find data points in B which significantly deviate from the
normal pattern of data sequence.
◦ Algorithm for finding dependency anomalies.

36. Anomaly Detection
Learning temporal causal graphs
by regularization
Finding the Anomaly
Score using Kullback-
Leibler (KL)
Divergence
Determining
Anomalies by
specifying a threshold
and finding the
underlying causes
Hypothesis: Causal Graphs of both remain the same

37. Anomaly Detection
Learning temporal causal graphs
by regularization
Finding the Anomaly
Score using Kullback-
Leibler (KL)
Divergence
Determining
Anomalies by
specifying a threshold
and finding the
underlying causes
Calculate the graph for A by LASSO Granger method.
When finding the causal graph for B we need to apply additional constraints. This can be done using two
methods:
a) Neighborhood Similarity: This implies imposing an additional constraint that the values of β(a) should
be zero or non-zero only when the value of β(b) are zero or non-zero. Here β(a) and β(b) are the coefficients
obtained by running Lasso Granger on set A and Set B respectively.
b) Coefficient similarity: The constraint is that the coefficients β(a) and β(b) should be similar.

38. Anomaly Detection
Learning temporal causal graphs
by regularization
Finding the Anomaly
Score using Kullback-
Leibler (KL)
Divergence
Determining
Anomalies by
specifying a threshold
and finding the
underlying causes
KL divergence is a measure of how much one distribution differs from another.
Obtain the distributions for the two time series and the anomaly score is calculated using the KL
formulae.

39. Anomaly Detection
Learning temporal causal graphs
by regularization
Finding the Anomaly
Score using Kullback-
Leibler (KL)
Divergence
Determining
Anomalies by
specifying a threshold
and finding the
underlying causes
• To set a threshold we calculate how a normal time series would score on the anomaly score.
• We slide the window through the reference data and calculate the anomaly scores for each window.
• We them use these to approximate the distribution of anomaly scores that a normal time series
should have.
• Given a significance level α, we set the α quantile of the distribution as threshold cutoff.

40. Conclusion
◦ Widespread application of causal relationships motivates the study.
◦ Completely data driven approach. So provides a new outlook in every field
without making any assumptions.
◦ Further Scope:
 Applying the model to different domains. e.g Climate and Social media
 Predicting anomalous behavior.

41. References
[1] Lashof, Daniel A., and Dilip R. Ahuja. "Relative contributions of greenhouse gas emissions to global warming." (1990): 529-531.
[2] Perry, Ronen. "Correlation versus Causality: Further Thoughts on the Law Review/Law School Liaison." Conn. L. Rev. 39 (2006): 77.
[3] Diks, Cees, and Valentyn Panchenko. Modified hiemstra-jones test for Granger non-causality. No. 192. Society for Computational Economics, 2004.
[4] Granger, Clive WJ. "Investigating causal relations by econometric models and cross-spectral methods." Econometrica: Journal of the Econometric Society (1969):
424-438.
[5] Arnold, Andrew, Yan Liu, and Naoki Abe. "Temporal causal modeling with graphical granger methods." Proceedings of the 13th ACM SIGKDD international
conference on Knowledge discovery and data mining. ACM, 2007
[6] Tibshirani, Robert. "Regression shrinkage and selection via the lasso." Journal of the Royal Statistical Society. Series B (Methodological) (1996): 267-288.
[7] Cheng, Dehua, Mohammad Taha Bahadori, and Yan Liu. "FBLG: a simple and effective approach for temporal dependence discovery from time series
data."Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2014.
[8] Smith, Delmas, Iwabuchi, Kirk. “Demonstrating causal links between fMRI time series using time-lagged correlation”.
[9] Kim, Hyun Duk, et al. "Incatomi: Integrative causal topic miner between textual and non-textual time series data." Proceedings of the 21st ACM international
conference on Information and knowledge management. ACM, 2012.
[10] Qiu, Liu, Subrahmanya, et al. "Granger Causality for Time-Series Anomaly Detection." Proceedings of the 12th IEEE international conference on data mining,
2012.
[11] Lomax, Richard G. (2007) Statistical Concepts: A Second Course, p. 10

42. Thanks!!
Any Questions?