1) ChronoSAGE is a topic modeling method that presents research trends in an academic paper corpus in a chronological and readable manner. 2) It does this by extracting the trending words for each time epoch and displaying them chronologically using a modified version of the SAGE topic modeling algorithm. 3) An evaluation shows ChronoSAGE outperforms LDA and can extract chronological topic trends as top words for each time period, providing an analysis of trends over time that traditional topic modeling cannot.

1. ChronoSAGEChronoSAGE:
Diversifying Topic Modeling
Chronologically
Tomonari MASADA
NAGASAKI University
masada@nagasaki-u.ac.jp

2. Solution
ProblemProblem
• Find research trends
• Present them in a readable manner
Solution
• Extract trending words at each epoch
• Display them chronologically

3. MethodMethod
•SAGE [Eisenstein+ 11]
–Represent each word probability
as a multiplication of factors

4. ChronoSAGE
• Use SAGE for our chronological
analysis of academic papers
• Represent each word probability
as a multiplication of four factors
ChronoSAGE
• Use SAGE for our chronological
analysis of time-stamped docs
• Represent each word probability
as a multiplication of four factors

5. corpus-wide
background
per-topic
background

6. per-epoch
background
per-topic
trends

7. words sorted by
per-epoch background probabilities (TDT4)
t=0 edt paralymp lebanon 32nd wild-card u.s china
t=1 kippur 10-13 lebanon china palestinian text join
t=2 10-14 10-16 10-18 10-15 10-19 10-17 10-20
t=3 10-24 10-23 10-22 10-25 10-21 10-26 10-27
t=4 10-29 10-28 10-31 10-30 11-3 leipzig lebanon
t=5 11-10 11-8 11-9 11-6 11-7 11-5 convuls
t=6 11-17 11-16 11-11 11-14 11-15 11-12 11-13
t=7 11-18 11-19 11-24 11-22 11-23 11-20 11-21
t=8 11-25 11-27 11-28 11-26 11-30 11-29 seclus

8. words sorted by
per-epoch background probabilities (TDT4)
t=9 12-8 12-6 12-5 12-7 12-3 537-vote 12-4
t=10 12-12 12-15 12-14 12-10 12-13 12-11 12-9
t=11 12-17 12-18 12-21 12-20 12-19 12-22 12-16
t=12 12-24 12-28 12-29 12-23 12-27 12-26 12-25
t=13 309 tabasco 2001 1-5 vy 12-0 free-agent
t=14 presid-elect’s 1-12 1-8 1-11 1-9 1-10 1-7
t=15 1-14 1-13 1-19 1-18 1-17 1-16 1-15
t=16 1-21 1-26 1-25 1-22 1-20 1-23 1-24
t=17 1-28 1-31 1-30 1-27 1-29 dawosi bhuj

9. Evaluation (1)
• SAGE and ChronoSAGE are better
than LDA in terms of PMI (point-
wise mutual information).
–We used the entire English
Wikipedia for PMI computation.

10. PMI
PMI 𝑤𝑖, 𝑤𝑗 = log
𝑝(𝑤 𝑖,𝑤 𝑗)
𝑝(𝑤 𝑖)𝑝(𝑤 𝑗)
,
where 𝑝 𝑤𝑖 = 𝑅𝑖/𝑅.

14. Evaluation (2)
• ChronoSAGE can extract
chronological trends for each topic
as top-K word lists.
–ChronoSAGE can do what SAGE can’t do.