lecture1_.pdf

lecture1_.pdf
Social Forecasting Week 1 Thomas Chadefaux Trinity College Dublin
Trinity College Dublin, The University of Dublin Syllabus • 2 assignments • 1 final paper • Main book: Shmueli, Galit, and Kenneth C. Lichtendahl Jr. Practical Time Series Forecasting with R: A Hands-on Guide (2nd ed.). Axelrod Schnall Publishers, 2018.
Trinity College Dublin, The University of Dublin “Forecast”= predict the future value of a time series
Trinity College Dublin, The University of Dublin Time Series Everywhere ForecastingBook.com
Trinity College Dublin, The University of Dublin 5 Tell us what the future holds, so we may know that you are gods. (Isaiah 41:23) Lycurgus Consulting the Pythia (1835/1845), as imagined by Eugène Delacroix (source: Wikipedia)
Trinity College Dublin, The University of Dublin Who generates forecasts? Governments NGOs Corporates Private sector Consulting firms Academia
Trinity College Dublin, The University of Dublin Narratives Aka, we’re terrible predictors
Trinity College Dublin, The University of Dublin 8 “The horse is here to stay but the automobile is only a novelty—a fad” 1903, the president of Michigan Savings Bank Stock prices have reached “what looks like a permanently high plateau… I believe the principle of the investment trusts is sound, and the public is justified in participating in them.” Irving Fisher, October 1929 “I think there is a world market for maybe five computers.” Thomas Watson, 1943
Trinity College Dublin, The University of Dublin 9
Trinity College Dublin, The University of Dublin 10 Source:https://www.analyticsvidhya.com/
Trinity College Dublin, The University of Dublin Smarter than a rat? 11
Trinity College Dublin, The University of Dublin
Trinity College Dublin, The University of Dublin We are poor predictors We like simple explanations We don’t correct We are overconfident We hate randomness
14
Trinity College Dublin, The University of Dublin How predictable is it?
Trinity College Dublin, The University of Dublin Free will (?)
Trinity College Dublin, The University of Dublin Chaotic world? Aka the butterfly effect 17 X -> 4x(1-x) Y -> x+y
Trinity College Dublin, The University of Dublin Randomness 1 2 3 4
20
Trinity College Dublin, The University of Dublin Social Sciences are worse First order chaotic systems Second order chaotic systems Observers observing observers who observe observers
Trinity College Dublin, The University of Dublin Fundamentally unpredictable? Multiple equilibria Mixed strategies
Trinity College Dublin, The University of Dublin Irreducible sources of error - Specification error: cannot include all variables - Include as much as you can? No! - Measurement error: some variables are particularly difficult to observe - Natural phenomena: Indian Ocean tsunami and violence in Aceh Source: Spagat et al. “Estimating War Deaths: An Arena of Contestation”
Trinity College Dublin, The University of Dublin Much is predictable Rules Strategies and equilibria Structural constraints Strong autocorrelation in: space, time
Trinity College Dublin, The University of Dublin Non-trivial questions Boring Unpredictable Just right Civil war in Switzerland in 2022? Black swans Rare events
Trinity College Dublin, The University of Dublin So what CAN we forecast? 26
Trinity College Dublin, The University of Dublin Which is easiest to forecast? 29 • Daily electricity demand in 3 days time • Timing of next Halley’s comet appearance • Time of sunrise this day next year • Google stock price tomorrow • Google stock price in 6 months time • Maximum temperature tomorrow • Exchange rate of $/€ next week • Total sales of drugs in Irish pharmacies next month
Trinity College Dublin, The University of Dublin How predictable? 30 Depends on: 1. how well we understand the factors that contribute to it 2.how much data is available 3.whether the forecasts can affect the thing we are trying to forecast. 4.the future is somewhat similar to the past 5.there is relatively low natural/unexplainable random variation.
Trinity College Dublin, The University of Dublin Improving forecasts… 31 …but social science forecasts are much harder
Trinity College Dublin, The University of Dublin How do we do it?
Trinity College Dublin, The University of Dublin Most don’t
Trinity College Dublin, The University of Dublin Experts
Trinity College Dublin, The University of Dublin Game theory Preferences Capabilities Saliency
Trinity College Dublin, The University of Dublin Wisdom of Crowds
Trinity College Dublin, The University of Dublin Statistics 37 N refugeest = f(casualties, unemployment, day of the week, error) N refugeest = f(Nrefugeest-1, Nrefugeest-2, Nrefugeest-3, …, error) N refugeest = f(Nrefugeest-1, casualties, unemployment, …, error)
Trinity College Dublin, The University of Dublin Statistics Learning Sample Test Sample Y increases by b when x increases by 1 (well, sort of) Predictions in test sample
Trinity College Dublin, The University of Dublin Machine learning algorithms Support vector machines
Trinity College Dublin, The University of Dublin Risks of Machine Learning approaches Over-fitting Too little data Don’t improve that much, if at all, over much simpler logits
Trinity College Dublin, The University of Dublin Forecasts are hard, especially about the future Aka: how poorly do we do?
Trinity College Dublin, The University of Dublin 42 3 -5 -4 -3 -2 -1 0 1 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2017 Financial year ended 2013 2016 2015 2014 2012 2011 Actual Grattan analysis of Commonwealth Budget Papers Commonwealth plans to drift back to surplus show the triumph of experience over hope Actual and forecast Commonwealth underlying cash balance per cent of GDP Forecast made in
Trinity College Dublin, The University of Dublin Important forecasting efforts Academic: — Political Instability Task Force 2002-present — DARPA ICEWS 2007-2015 — Peace Research Center Oslo (PRIO) and Uppsala University UCDP models — Uppsala ViEWS — Many others Governments: typically rely on experts, but some use large-N data: — Germany — Netherlands — EU — World Bank — US — Others, but often classified
Trinity College Dublin, The University of Dublin Beware of accuracy claims
Trinity College Dublin, The University of Dublin Overpredicting vs underpredicting True warnings False alarms
Trinity College Dublin, The University of Dublin How well: Experts Tetlock: 284 experts 20+ years of forecasts 1000s of forecasts
Trinity College Dublin, The University of Dublin Experts: results Overpredict rare events No better than dilettantes All humans far worse than simple algorithms Why so bad?
Trinity College Dublin, The University of Dublin Machine learning: performance No Conflict Conflict No conflict 1432 80 Conflict 58 374 48 Predicted Observed
Trinity College Dublin, The University of Dublin How well? Crowds IARPA competition: GJP the winner The top forecasters in the Good Judgement Project (Tetlock) are "reportedly 30% better than intelligence officers with access to actual classified information.” 49
Trinity College Dublin, The University of Dublin More complex models?
Trinity College Dublin, The University of Dublin More Data?
Trinity College Dublin, The University of Dublin Aggregating forecasts: ensemble models
Trinity College Dublin, The University of Dublin Fundamentals of Forecasting 53
Trinity College Dublin, The University of Dublin Time series vs. cross-sectional data ForecastingBook.com
Trinity College Dublin, The University of Dublin Prediction vs Forecasting • Prediction: estimate the outcomes of unseen data – E.g., predict the value of inflation in Burundi today based on the one in Rwanda today • Forecasting: predictions about the future – Predict the value of inflation in Burundi tomorrow based on the one in Rwanda today © 2012 Shmueli Some rights Reserved
Trinity College Dublin, The University of Dublin Basic Notation t=1,2,3… = time period index Yt = value of the series at time period t Ft+k = forecast for time period t+k, given data until time t et = forecast error for period t ForecastingBook.com
Trinity College Dublin, The University of Dublin Time series components Systematic part • Level • Trend • Seasonal patterns Non-systematic part • “Noise” Additive: Yt = Level + Trend + Seasonality + Noise Multiplicative: Yt = Level x Trend x Seasonality x Noise ForecastingBook.com
Trinity College Dublin, The University of Dublin Time series components

Check these out next

lecture1_.pdf

Recomendados

Más contenido relacionado

Similar a lecture1_.pdf(20)

Último(20)

lecture1_.pdf