Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

On the Dynamics of Machine Learning Algorithms and Behavioral Game Theory

6.787 Aufrufe

Veröffentlicht am

Presentation Material used in guest lecturing at University of Tsukuba on September 17, 2016.
Target audience is part-time PhD student working at a machine learning, data mining, or agent-based simulation project.

Veröffentlicht in: Wirtschaft & Finanzen
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

On the Dynamics of Machine Learning Algorithms and Behavioral Game Theory

  1. 1. On the Dynamics of Machine Learning Algorithms and Behavioral Game Theory Towards Effective Decision Making in Multi-Agent Environment Graduate School of Systems and Information Engineering University of Tsukuba Sep 17, 2016 Rikiya Takahashi, Ph.D. (SmartNews, Inc.) rikiya.takahashi@smartnews.com
  2. 2. About Myself ● Rikiya TAKAHASHI ( 高橋 力矢 ) ● Engineer in SmartNews, Inc., from 2015 to current ● Research Staff Member in IBM Research – Tokyo, from 2004 to 2015 ● Ph.D in Engineering from University of Tsukuba, 2014 – Dissertation: "Stable Fitting of Nonparametric Models to Predict Complex Human Behaviors" – Supervisor: Prof. Setsuya Kurahashi ● M.Sc (2004) & B.Eng (2002) from The University of Tokyo ● Research Interests: machine learning, reinforcement learning, cognitive science, behavioral economics, complex systems ● Descriptive models about real human behavior ● Prescriptive decision making by exploiting such descriptive models
  3. 3. References Choice and Social Interaction Why did you purchase Windows 10 XXX Edition? Because the price and quality of that OS were good? Or because your friends were using it? Or both reasons? Are you interested in quantifying each factor for better decision making? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  4. 4. References Decision Making under Social Uncertainty You can be either a player or a designer of the market. Players: consumers, firms competing with other brands Designers: politicians, platformer of auction or SNS In both scenarios you must optimize your decisions under uncertainty over other players’ decisions. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  5. 5. 0 5 10 15 20 25 30 35 0 0.25 0.5 0.75 1 Elapsed Time [days] Retention What I was doing during PhD: Stable Fitting of Power Law Models ● Heavy-tail distributions / long-range dependence ● Bounded rationality: incomplete information, cognitive bias ● Positive feedback: richer-get-richer, increasing survival prob. Travel time in road network Ebbinghaus forgetting curve Asset returns in finance Pageview of a video in YouTube Power-law decay by cascading word- of-mouth (Crane & Sornette, 2008) Heavy-tail by price crash (stimultaneous shorting) http://finance.yahoo.com Heavy-tail by huge traffic congestion http://en.wikipedia.org/ wiki/Traffic_congestion Power-law decay by interaction among short-, mid-, & long- term memories
  6. 6. What I was doing during PhD: Power-Law Models = Multi-Scale Nonparametrics ● Global optimization in fitting nonparametric models ● Non-linear modeling by linearly mixing local or multi-scale basis functions ● Convex optimization of the mixing weights ● Domain-specific design of fixed basis functions ElapsedTime Retention Value ProbabilityDensity Heavy-tail distribution as scale-mixture of Gaussians Power-law decay as scale- mixture of exponential decays
  7. 7. Agenda ● Irrationality and Disequilibrium: essential phenomena making social science challenging ● Failed Forecasts by Rational Economic Models: Irrational Disequilibrium or Multiple Equilibria? ● Frontiers in Mathematical Modeling of Irrationality: a transitional-state perspective ● For on-going PhD students: how to exploit your research experiences into Jobs
  8. 8. Intertemporal Decision Making ● $100 today = $100 * (1+interest rate) in the future ● Objective must be time-consistent. ● Exponential discounting (constant interest rate) ● Unchanged preference order over time V (t0)=V (t)exp(−λ (t−t0)) where t> t0, λ > 0
  9. 9. Real Human Discounts by Power Law ● Hyperbolic discounting (Ainslie, 1974) ● Time-inconsistent preference order V (t0)= V (t) (1+ λ (t−t0))α where t> t0, λ > 0,α > 0
  10. 10. Irrationality of Hyperbolic Discounting ● Discrepancy between thought and action ● Long-term-oriented when the decision time is distant. ● But suddently become myopic as the time reaches. "About 1 month ago, I was thinking I would study hard (=long- term large utility) in the last 1 week before the exam, but I did play video games (=short-term small utility) this week..." Long-term Option B is more prioritized than Short-term Option A at t=0, but its order is reversed at t=2.
  11. 11. Irrationality of Hyperbolic Discounting ● Money pumps (Cubit and Sugden, 2001) ● We can steal money from hyperbolic discounter without risks, while cannot steal from exponential discounter. At time t=0, we borrow Option B and exchange it with the target's Option A and $2.5 (=$15-$12.5). Then we earn interests on this $2.5. At time t=3, we exchange our Option A with the target's Option B and $10 (=$30-$20). Then return this Option B. We get ($2.5 * (1 + interests) + $10 – borrowing cost) without risks.
  12. 12. Known Counterarguments ● Hyperbolic discounting is rather rational, when the interest rates in the future is uncertain. ● E.g., (Azfar, 1999; Farmer & Geanakoplos, 2009) ● Meaningful particularly in financial decision making ● Integral on gamma prior distribution for interest rate (multi-scale mixture of exponential discountings)
  13. 13. Power-Law and Disequilibrium ● Power-law or fat tails in asset-return distributions ● See (Cont, 2001) for stylized facts. ● Short-term momentum generates outlying returns ● Positive autocorrelation in rare events (Sornette, 2004). http://finance.yahoo.com http://www.proba.jussieu.fr/pageperso/ramacont/papers/empirical.pdf
  14. 14. What Causes Fat Tails? ● Hypothesis #1. Interplay among momentum traders ● E.g., Log-Periodic Power-Law (LPPL) model (Johansen+, 1999) as an extension of rational bubble model (Blanchard & Watson, 1982) Sell ? Sell ! Sell More!! $$: market price http://arxiv.org/pdf/1107.3171.pdf si=sign(K ∑j∈N (i) sj+ ε i) si∈{−1,+ 1} K: strength of interaction N(i): set of neighbors for investor i epsiloni : investor i's own indiosyncratic prediction
  15. 15. What Causes Fat Tails? ● Hypothesis #2. Over-confidence on stability ● Leverage in low-volatility period (Thurner+, 2012) – Once a downward price fluctuation occurs, resulting margin call causes rushes of selling into an already falling market, amplifying the downward price movement. http://finance.yahoo.com Low-Volatility Period with Leverage Sudden Price Drop with Margin Calls
  16. 16. What Causes Fat Tails? ● Implications are obtained by explicitly modeling and simulating the dynamics in trading. ● Physical modeling using stochastic processes ● Transitionary states and disequilibrium play crucial roles. ● Do not think that the system is always in equilibrium. https://www.amazon.co.jp/dp/B009IRP3GW M. Buchanan, “Forecast: What Physics, Meteorology, and the Natural Sciences Can Teach Us About Economics,” A&C Black, 2013
  17. 17. Regarding Irrationality as Disequilibrium ● Assume that human plays a game in his mind. ● Then irrationality is regarded as an outcome from state dynamics in mental processes. ● Rationality = choose the strategy in stationary state ● Irrationality = choose a strategy in transitional state ● Possibility to formalize many social phenomena universally via explicit state dynamics ● For better understanding: play p-beauty contest
  18. 18. References Understand Dynamics by (2/3)-beauty Contest What are the numbers chosen by these n players? Each player i ∈{1, . . ., n} chooses an integer Yi ∈[0, 100]. Winner(s): player(s) whose Yi is closest to 2 3 1 n n j=1Yj . SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  19. 19. References Equilibrium of p-beauty Contest (Moulin, 1986) Nash Equilibrium when 0 ≤ p < 1: ∀i Yi = 0 1 Let C0 be a set of purely na¨ıve players, who choose from 0 to 100 at uniformly random. 2 Since E[ 1 |C0| i∈C0 Yi ] = 50, a slightly more strategic player in class C1 will choose round(50 × 2/3)=33. 3 Further strategic players in class C2 will choose round(33 × 2/3)=22. Players in class C3 will choose ... At convergence, every player should choose zero. However, do you believe such prediction? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  20. 20. References A Result of p-beauty Contest by Real Humans Mean is apart from 0 (Camerer et al., 2004; Ho et al., 2006). Table: Average Choice in (2/3)-beauty Contests Subject Pool Group Size Sample Size Mean[Yi ] Caltech Board 73 73 49.4 80 year olds 33 33 37.0 High School Students 20-32 52 32.5 Economics PhDs 16 16 27.4 Portfolio Managers 26 26 24.3 Caltech Students 3 24 21.5 Game Theorists 27-54 136 19.1 SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  21. 21. References Unreality of Nash Equilibrium Every player is homogeneous. All of them adopt the same thinking process. Every player has infinite forecasting horizon. Can all real humans think so intelligently? Such unrealistic assumption leads vulnerability to perturbation. What if one player does not understand the game rule? What if one player intends to punish “rational” others? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  22. 22. Analyzing More Complex Interactions ● Agent-based modeling ● Can be free from some assumptions: homogeneity, complete information, rationality, etc. ● True challenge: design of good agent models ● Often too many degrees of freedom in tuning J. M. Epstein, “Generative Social Science: Studies in Agent-Based Computational Modeling,“ Princeton University Press, 2012. S. F. Railsback and V. Grimm, “Agent-Based and Individual-Based Modeling: A Practical Introduction,“ Princeton University Press, 2011.
  23. 23. One Viewpoint for Good Agent Design ● Explicitly model human's bounded rationality. ● Irrationality is not the outcome of human's stupidity. ● Human does try optimization, but cannot reach the true optimum due to the lack of mental resources. – Finite memory about past events – Uncertainty over the future environment – Uncertainty over other agents' decisions ● Refer to Behavioral Game Theory ● Jewels in modeling bounded-rational agents
  24. 24. Short Summary ● Discussed irrationality in the real world. ● Observed that transitional states are often more realistic forecasts than equilibrium. ● Discussed direction for good agent models: hints for accurately modeling dynamics.
  25. 25. Agenda ● Irrationality and Disequilibrium: essential phenomena making social science challenging ● Failed Forecasts by Rational Economic Models: Irrational Disequilibrium or Multiple Equilibria? ● Frontiers in Mathematical Modeling of Irrationality: a transitional-state perspective ● For on-going PhD students: how to exploit your research experiences into Jobs
  26. 26. What is Rational / Irrational? ● Rationality = optimizing a consistent objective ● Irrationality = any behavior different from rationality ● Inconsistent optimization risks being manipulated by others. ● E.g., hyperbolic discounting: time-inconsistent preference order causes vulnerability of money pumps. ● Other forms of irrational decision making ● Choice from options whose coverage is manipulated by others
  27. 27. References Discrete Choice Modelling Goal: predict prob. of choosing an option from a choice set. Why solving this problem? For business: brand positioning among competitors For business: sales promotion (yet involving some abuse) To deeply understand how human makes decisions SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  28. 28. References Random Utility Theory as Rational Model Each human is a maximizer of a probabilistic utility. i’s choice from Si = arg max j∈Si fi (vj ) mean utility + εij random noise Si : choice set for i, vj : vector of j’s attributes, fi : i’s mean utility function Assuming independence among every option’s attractiveness For both mean and noise: (e.g., logit (McFadden, 1980)) For only mean: (e.g., nested logit (Williams, 1977)) SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  29. 29. References Context Effects: Complexity of Human’s Choice An example of choosing PC (Kivetz et al., 2004) Each subject chooses 1 option from a choice set A B C D E CPU [MHz] 250 300 350 400 450 Mem. [MB] 192 160 128 96 64 Choice Set #subjects {A, B, C} 36:176:144 {B, C, D} 56:177:115 {C, D, E} 94:181:109 Can random utility theory still explain the preference reversals? B C or C B? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  30. 30. References Similarity Effect (Tversky, 1972) Top-share choice can change due to correlated utilities. E.g., one color from {Blue, Red} or {Violet, Blue, Red}? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  31. 31. References Attraction Effect (Huber et al., 1982) Introduction of an absolutely-inferior option A− (=decoy) causes irregular increase of option A’s attractiveness. Despite the natural guess that decoy never affects the choice. If D A, then D A A− . If A D, then A is superior to both A− and D. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  32. 32. References Compromise Effect (Simonson, 1989) Moderate options within each chosen set are preferred. Different from non-linear utility function involving diminishing returns (e.g., √ inexpensiveness+ √ quality). SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  33. 33. Multiple Equilibria also Spoil Forecasts ● Pivotal mechanism (Clarke, 1971) to decide whether to start a public project ● Every player discloses a utility of the project outcome. ● If and only if sum(utilities) > 0, then project is started. ● Player i must pay tax amount abs(other players' utility sum), when sign of player i's utility and that of other players is opposite. ● For every player, honestly disclosing his true utility is optimal regardless other players' utilities. sum(utilities) = 1 decision = start disclosed utility -1 2 5 -3 -2 tax 2 0 0 4 3
  34. 34. Multiple Equilibria also Spoil Forecasts ● Failure of pivotal mechanism (Attiyeh+, 2000) ● Being rational is difficult because of too complex rules ● Even if rationality leading into an equilibrium exists, which equilibrium will be actually chosen? ● Each equilibrium has its own path from initial state. ● Identifying both of the path and finite time is hard. ● One promising way: converting transitional state in one game into an equilbrium of other game.
  35. 35. Short Summary ● Introduced more examples of irrational decision making by real humans. ● Irrationality spoils forecasting by standard economic models. ● Multiple equilibria further complicate the forecasting in addition to the irrational disequilibrium.
  36. 36. Agenda ● Irrationality and Disequilibrium: essential phenomena making social science challenging ● Failed Forecasts by Rational Economic Models: Irrational Disequilibrium or Multiple Equilibria? ● Frontiers in Mathematical Modeling of Irrationality: a transitional-state perspective ● For on-going PhD students: how to exploit your research experiences into Jobs
  37. 37. References Game with Heterogeneous Pay-Offs Which numbers will be chosen by these 3 players? Each player i ∈{1, . . ., n} chooses an integer Yi ∈[0, 10]. Player #1’s pay-off: 39 + 12Y1 − (Y1+Y2)2 Player #2’s pay-off: 47 + 20Y2 − (Y2+Y3)2 Player #3’s pay-off: 6Y3 − (Y3− 1 2 (Y1 + Y2))2 SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  38. 38. References An Idiot’s View of Game Theory If other players’ decisions Y i (Y1, . . . , Yi−1, Yi+1, . . . , Yn) are known, optimal decision Y ∗ i for player i is given by ∀i ∈{1, . . . , n} Y ∗ i |Y i = arg max Y ui (Y , Y i ). (1) ui : utility function of player i Game theory is merely solving a system of n equations by assuming ∀i Yi ≡ Y ∗ i in Eq. (1). Every player is assumed to be a utility maximizer. Variety of games just comes from the variable type of Yi . However, what if players are irrational or unpredictable? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  39. 39. References Equilibrium of Linearly-Solvable Games Maximization of concave-quadratic equation = linear equality 0 = 12 − 2(Y ∗ 1 +Y2) 0 = 20 − 2(Y ∗ 2 +Y3) 0 = 6 − 2(Y ∗ 3 − 1 2 (Y1 + Y2)) ∀i Yi ≡Y ∗ i leads a matrix-vector relationship   2 2 0 0 2 2 −1 −1 2     Y ∗ 1 Y ∗ 2 Y ∗ 3   =   12 20 6   . (Y ∗ 1 , Y ∗ 2 , Y ∗ 3 ) = (2, 4, 6) with Pay-offs = (27, 27, 27) SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  40. 40. References Belief Learning: Iterative Solving of Game Equilibrium is tractable only for limited classes of utility functions, while in general is iteratively computed as t =0: Initialize each player’s decision by some value. t > 0: Compute the t-step optimum given the (t−1)-step decisions by others. ∀i ∈{1, . . . , n} Y (t) i |Y (t−1) i = arg max Y ui (Y , Y (t−1) i ) Belief learning: classes of algorithms to iteratively compute the equilibrium. (t + 1)-step looking-ahead player beats the t-step-only players, (t + 2)-step player beats... How about using Y (t) i at finite t, instead of the one at t →∞? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  41. 41. How to Formalize Context Effects? ● What dynamics causes context effects? ● Hypothesis: a dynamical process to estimate utility function (Takahashi & Morimura, 2015). ● Irrational contextual effects are observed via regularized estimates of the utility function. ● Machine learning as a dynamical process ● Transitionary state in maximum-likelihood estimation ● Stationary state in Bayesian shrinkage estimation
  42. 42. References Gaussian Process Uncertainty Aversion (GPUA) A dual-personality model regarding utilities as samples in statistics (Takahashi and Morimura, 2015) Assumption 1: Utility function is partially disclosed to DMS. 1 UC computes the sample value of every option’s utility, and sends only these samples to DMS. 2 DMS statistically estimates the utility function. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  43. 43. References GPUA: Mental Conflict as Bayesian Shrinkage Assumption 2: DMS does Bayesian shrinkage estimation. i ∈{1, . . . , n}: context, yi ∈{1, . . . , m[i]}: final choice Xi (xi1 ∈RdX , . . . , xim[i]) : features of m[i] options Objective Data: values of random utilities vi (vi1, . . . , vim[i]) ∼N µi , σ2 Im[i] , vij = b+wφ φ (xij ) µi : Rm[i]: vec. of the true mean utility, σ2: noise level b: bias term, φ : RdX →Rdφ : mapping function. wφ: vec. of coefficients Subjective Prior: choice-set-dependent Gaussian process µi ∼ N 0m[i], σ2 K(Xi ) s.t. K(Xi ) = (K(xij , xij ))∈Rm[i]×m[i] µi ∈Rm[i]: vec. of random utilities, K(·, ·): similarity between options Final choice: based on (Posterior mean u∗ i + i.i.d. noise) as u∗ i = K(Xi ) Im[i]+K(Xi ) −1 b1m[i]+Φi wφ , yi = arg max j (u∗ ij + εij ) where ∀j εij ∼ Gumbel. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  44. 44. References GPUA: Irrationality by Bayesian Shrinkage Implication of (2): similarity-dependent discounting u∗ i = K(Xi ) Im[i]+K(Xi ) −1 shrinkage factor b1m[i]+Φi wφ vec. of utility samples . (2) Under RBF kernel K(x, x ) = exp(−γ x − x 2 ), an option dissimilar to others involves high uncertainty. Strongly shrunk into prior mean 0. Context effects as Bayesian uncertainty aversion 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1 2 3 4 FinalEvaluation X1=(5-X2) DA - A {A,D} {A,A- ,D} 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1 2 3 4FinalEvaluation X1=(5-X2) DCBA {A,B,C} {B,C,D} SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  45. 45. References GPUA: Convex Optimization using Posterior Mean Global fitting of the parameters using data (Xi , yi )n i=1 Fix the mapping and similarity functions during updates. Shrinkage factor Hi K(Xi )(Im[i] + K(Xi ))−1 is constant! Obtaining a MAP estimate is convex w.r.t. (b, wφ). max b,wφ n i=1 ( bHi 1m[i]+Hi Φi wφ Context−specific Hi is multiplied. , yi ) − c 2 wφ 2 Exploiting the log-concavity of multinomial logit (u∗ i , yi ) log exp(u∗ iyi ) m[i] j =1 exp(u∗ ij ) SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  46. 46. References GPUA: Experimental Settings Evaluates accuracy & log-likelihood for real choice data. Dataset #1: PC (n=1, 088, dX =2) Dataset #2: SP (n=972, dX =2) Subjects are asked of choosing a speaker. A B C D E Power [Watt] 50 75 100 125 150 Price [USD] 100 130 160 190 220 Choice Set #subjects {A, B, C} 45:135:145 {B, C, D} 58:137:111 {C, D, E} 95:155: 91 Dataset #3: SM (n=10, 719, dX =23) SwissMetro dataset (Antonini et al., 2007) Subjects are asked of choosing one transportation, either from {train, car, SwissMetro} or {train, SwissMetro}. Attribute of option: cost, travel time, headway, seat type, and type of transportation. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  47. 47. References GPUA: Cross-Validation Performances High predictability in addition to the interpretable mechanism. For SP, successfully detected combination of compromise effect & prioritization of power. 1st best for PC & SP. 2nd best for higher-dimensional SM: slightly worse than highly expressive nonparametric version of mixed multinomial logit (McFadden and Train, 2000). -1.1 -1 -0.9 -0.8 AverageLog-Likelihood Dataset PC SP SM LinLogit NpLogit LinMix NpMix GPUA 0.3 0.4 0.5 0.6 0.7 ClassificationAccuracy Dataset PC SP SM LinLogit NpLogit LinMix NpMix GPUA 2 3 4 100 150 200 Evaluation Price [USD] EDCBA Obj. Eval. {A,B,C} {B,C,D} {C,D,E} SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  48. 48. Linking ML with Game Theory (GT) via Shrinkage Principle Optimization without shrinkage Optimization with shrinkage ML GT Maximum-Likelihood estimation Bayesian estimation Transitional State or Quantal Response Equilibrium Nash Equilibrium Optimal for training data, but less generalization capability to test data Optimal for given game but less predictable to real- world decisions Shrinkage towards uniform probabilities causes suboptimality for the given game, but more predictable to real-world decisions Shrinkage towards prior causes suboptimality for training data, but more generalization capability to test data
  49. 49. References Quantitative Handling of Irrationality Iterative equilibrium computation lightens two natural ways. Early stopping at step k: Level-k thinking or Cognitive Hierarchy Theory (Camerer et al., 2004) Humans cannot predict the infinite future. Using non-stationary transitional state Randomization of utility via noise εit: Quantal Response Equilibrium (McKelvey and Palfrey, 1995) ∀i ∈{1, . . . , n} Y (t) i |Y (t−1) i = arg max Y fi (Y , Y (t−1) i ) + εit Both methods essentially work as regularization of rationality. Shrinkage into initial values or uniform choice probabilities Affinity to Bayesian regularization in ML SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  50. 50. References Logit Quantal Response Equilibrium (LQRE) A special form of QRE associated with RUT. If εit obeys the standard Gumbel distribution and Y (t) i |Y (t−1) i = arg max Y ∈S fi (Y , Y (t−1) i ) + εit/βi , then choice probability becomes closed-form as P(Y (t) i = y|Y (t−1) i ) = exp βi fi (y, Y (t−1) i ) y ∈S exp βi fi (y , Y (t−1) i ) . βi is called the degree of irrationality of player i. βi →0: uniform choice probability (na¨ıve) βi →∞: Nash equilibrium (deterministic & rational) SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  51. 51. Early Stopping and Regularization ML as a Dynamical System to find the optimal parameters GT as a Dynamical System to find the equilibrium Parameter #1 Parameter #2 Exact Maximum-likelihood estimate (e.g., OLS) Exact Bayesian estimate shrunk towards zero (e.g., Ridge regression) 0 t=10 t=20 t=30 t=50 An early-stopping estimate t=0 t=1 t →∞ t=2 ... mean = 50 mean = 34 mean = 15 mean = 0 Nash Equilibrium Level-2 Transitional State
  52. 52. References Towards Useful Decision Making by using QRE Economists discuss when utility functions {fi }n i=1 are known. QRE is analytically-intractable but can be simulated. E.g., ad-auction for irrational bidders (Rong et al., 2015) ML scientists should estimate unknown utility functions! Extension of statistical marketing research methods through rich functional approximation techniques in ML SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  53. 53. References Multi-Agent Extension of RUT in Marketing RUT in marketing research has already been data-oriented. Estimating utility functions from real data DCM such as Logit model (McFadden, 1980) Identical opt. objective to multinomial logistic regression Conjoint analysis (Green and Srinivasan, 1978) Special case of DCM by showing only 2 options Related with learning to rank problem: see (Chapelle and Harchaoui, 2005) Adding other-player-dependent terms into existing marketing research models yields a simulation model to compute QRE. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  54. 54. References Possible Formalisms & Algorithmic Studies Multi-agent generalization of DCM or learning to rank Simulation-based fitting (e.g., Approximate Bayesian Computation (Tavar´e et al., 1997)) Functional approximations (e.g., Gradient Boosting Decision Trees (Friedman, 2001), Deep Neural Network) with partially-observable other players’ decisions SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  55. 55. References A Future Forecast: Rise of Deep Belief Learning Belief Learning (BL) vs Reinforcement Learning (RL) BL: explicitly guessing other players’ thinking processes RL: choosing optimal actions purely from experiences Other players’ decision functions are implicitly parts of the environment While predictive accuracies would be similar, BL provides more white boxes than RL in terms of thinking processes AlphaGo is a successful application of Deep RL (e.g., (Mnih et al., 2013; van Hasselt et al., 2016)). What will be killer applications of Deep BL? SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  56. 56. Other Approaches for Irrationality ● Use quantum theory instead of probability ● Quantum Cognition – (Burza+, 2009; Mogiliansky+, 2009; de Barros & Suppes, 2009; Busemeyer & Burza, 2012) ● Key mechanism: double standard in quantum theory – During (unobserved) thinking: integrated over complex state space – In (observed) decision: classical probability by taking the absolute magnitude of state https://www.amazon.co.jp/Quantum-Models-Cognition-Decision-Busemeyer/dp/1107419883/
  57. 57. Short Summary ● Introduced recent advances on mathematical modeling of human's irrationality, for more accurate forecasts. ● Handled irrationality as transional states in both Machine Learning and Game Theory. ● Importing mathematical techniques from both ML and GT communities will serve better social decision making with more accurate forecasts.
  58. 58. Agenda ● Irrationality and Disequilibrium: essential phenomena making social science challenging ● Failed Forecasts by Rational Economic Models: Irrational Disequilibrium or Multiple Equilibria? ● Frontiers in Mathematical Modeling of Irrationality: a transitional-state perspective ● For on-going PhD students: how to exploit your research experiences into Jobs
  59. 59. WARNING The following pages exhibit the author's personal opinions on how to make a good research direction and/or identify a good area of business. Effectiveness of these ideas has not been scientifically proved. Read them at your own risk.
  60. 60. How PhD changed my life ● Before obtaining PhD ● Job: Research Staff Member in a large B2B enterprise ● Research Topic: Required to be sticked with one coherent research direction ● Seeking for problems that are solvable via my Machine Learning (ML) disciplines ● After obtaining PhD ● Job: Engineer in a small B2C startup ● Research Topic: Freedom to target more ambitious topics in broader area ● Integrating ML disciplines with multi-agent perspective obtained during schooling
  61. 61. Hope and Actuality in PhD Course ● What I was intending ● Exploit ML for automatically designing agents. ● Or learn the essence in manually designing agents, through seminar discussions. ● What actually occurred ● Still difficult to know how to design agents! – Why this paper's agent model is designed like this? ● Effective viewpoints on the design of agents came after finishing PhD.
  62. 62. Interplay between Research and Job ● Paid Job requires real-world decision making. ● Skin in the game: you cannot use models or approaches that you do not rely on. ● In order to be confident on your approach, make focus & apply Occam's razor strongly. ● Avoid using models #1 & #2 & #3 ... Combintion makes difficult of root-cause finding in failure. ● Define your unique optimization problem, which is directly solvable by one essential approach. – Also one-principle-based paper is easily publishable.
  63. 63. Interplay between Research and Job ● A case in job: how to create network externality? ● The key factor in successful platform business (e.g., Operating Systems, Social Network Services) ● You must have a good mechanism to incentivize users to use your platform. ● Do the existing mechanisms really incentivize users? ● Are they quantitative to enable real operations? ● Freeing from unrealistic assumptions and practicality requirement are natual sources of research ideas.
  64. 64. Some Tactics under Competitions ● Development of the truly universal approach = Red Ocean fought by the World's Top Talents ● Identify the minimum requirement. You create an approach at least universal in your area. ● Make an approach that competitors dislike to use. ● Such approach often causes disruptive innovation. ● Do not confuse simplicity with naïvety
  65. 65. Necessity of Ample Surveys ● Avoid reinventing wheels. Most industrial problems have already been partially solved. ● Respect & steal other players' ideas by reading. ● Remember that some prior work is written over- confidently; prior authors do not know conditions that spoil their approaches in your new problem. ● Key for success: good strategy to search for relevant papers and books
  66. 66. Encouraging Bottom-Up Learning ● Check the neighboring disciplines from yours; be in Optimum Stimulation Level (Berlyne, 1960) ● Your brain is strongly stimulated by insights in slightly distant areas from your expertise. ● Deep understanding on the very slight difference between two areas often clarifies the white space in your area. Machine Learnng StatisticsBiostatistics Econometrics Psychometrics Cognitive Science Neuroscience Behavioral Economics Behavioral Game Theory
  67. 67. Uncertainty is Your Friend ● Most people hate uncertainty, but you must love it. ● Further one tactic: beat the irrationality of your competitors! ● The more uncertain parts your research or business contains, the more competitors will be fooled by too much complexity. ● You: solve the entire problem by one critial solution. ● Competitors: solve each of the sub-problems by its specific method, and trapped by poor sub-optima. ● Optimism in face of uncertainty
  68. 68. Uncertainty is Your Friend ● Care the difference between risks and uncertainty. ● Risks: volatility calibrated from existing data ● Uncertainty: cannot be quantified from data ● Donald Rumsfeld's unknown unknowns. ● You do not have take high risks. But you should take high uncertainties. ● In big-data era, competitors rush into the areas with ample datasets, and become professed with risks. ● By contrast, the human's nature of hating uncertainty would remain, and it will be a source of your success.
  69. 69. References References I Ainslie, G. W. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21(3):485–489. Antonini, G., Gioia, C., Frejinger, E., and Th´emans, M. (2007). Swissmetro: description of the data. http://biogeme.epfl.ch/swissmetro/examples.html. Attiyeh, G., Franciosi, R., and Isaac, R. M. (2000). Experiments with the pivot process for providing public goods. Public Choice, 102:95–114. Azfar, O. (1999). Rationalizing hyperbolic discounting. Journal of Economic Behavior and Organization, 38(2):245–252. Blanchard, O. J. and Watson, M. W. (1982). Bubbles, rational expectations and financial markets. Crises in the Economic and Financial Structure, pages 295–316. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  70. 70. References References II Bruza, P., Kitto, K., Nelson, D., and McEvoy, C. (2009). Is there something quantum-like about the human mental lexicon? Journal of Mathematical Psychology, 53(5):362–377. Camerer, C. F., Ho, T. H., and Chong, J. (2004). A cognitive hierarchy model of games. Quarterly Journal of Economics, 119:861–898. Chapelle, O. and Harchaoui, Z. (2005). A machine learning approach to conjoint analysis. In Advances in Neural Information Processing Systems 17, pages 257–264. MIT Press, Cambridge, MA, USA. Clarke, E. H. (1971). Multipart pricing of public goods. Public Choice, 2:19–33. Cont, R. (2001). Empirical properties of asset returns: stylized facts and statistical issues. Quantitative Finance, 1(2):223–236. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  71. 71. References References III Cubitt, R. P. and Sugden, R. (2001). On money pumps. Games and Economic Behavior, 37(1):121–160. de Barros, J. A. and Suppes, P. (2009). Quantum mechanics, interference, and the brain. Journal of Mathematical Psychology, 53(5):306–313. Dotson, J. P., Lenk, P., Brazell, J., Otter, T., Maceachern, S. N., and Allenby, G. M. (2009). A probit model with structured covariance for similarity effects and source of volume calculations. http://ssrn.com/abstract=1396232. Farmer, J. D. and Geanakoplos, J. (2009). Hyperbolic discounting is rational: Valuing the far future with uncertain discount rates. Cowles Foundation Discussion Paper No. 1719. Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5):1189–1232. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  72. 72. References References IV Gonz´alez-Vallejo, C. (2002). Making trade-offs: A probabilistic and context-sensitive model of choice behavior. Psychological Review, 109:137–154. Green, P. and Srinivasan, V. (1978). Conjoint analysis in consumer research: Issues and outlook. Journal of Consumer Research, 5:103–123. Ho, T. H., Lim, N., and Camerer, C. F. (2006). Modeling the psychology of consumer and firm behavior with behavioral economics. Journal of Marketing Research, 43(3):307–331. Huber, J., Payne, J. W., and Puto, C. (1982). Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal of Consumer Research, 9:90–98. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  73. 73. References References V Johansen, A., Ledoit, O., and Sornette, D. (2000). Crashes as critical points. International Journal of Theoretical and Applied Finance, 3:219–255. Johansen, A. and Sornette, D. (1999). Critical crashes. Risk, 12(1):91–94. Johansen, A., Sornette, D., and Ledoit, O. (1999). Predicting financial crashes using discrete scale invariance. Journal of Risk, 1(4):5–32. Kivetz, R., Netzer, O., and Srinivasan, V. S. (2004). Alternative models for capturing the compromise effect. Journal of Marketing Research, 41(3):237–257. McFadden, D. and Train, K. (2000). Mixed MNL models for discrete response. Journal of Applied Econometrics, 15:447 –470. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  74. 74. References References VI McFadden, D. L. (1980). Econometric models of probabilistic choice among products. Journal of Business, 53(3):13–29. McKelvey, R. and Palfrey, T. (1995). Quantal response equilibria for normal form games. Games and Economic Behavior, 10:6–38. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. In NIPS Deep Learning Workshop. Mogiliansky, A. L., Zamir, S., and Zwirn, H. (2009). Type indeterminacy: A model of the KT (kahneman tversky)-man. Journal of Mathematical Psychology, 53(5):349–361. Moulin, H. (1986). Game Theory for the Social Sciences. NYU Press, second edition edition. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  75. 75. References References VII Roe, R. M., Busemeyer, J. R., and Townsend, J. T. (2001). Multialternative decision field theory: A dynamic connectionist model of decision making. Psychological Review, 108:370–392. Rong, J., Qin, T., and An, B. (2015). Computing quantal response equilibrium for sponsored search auctions. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), pages 1803–1804, Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems. Shenoy, P. and Yu, A. J. (2013). A rational account of contextual effects in preference choice: What makes for a bargain? In Proceedings of the Cognitive Science Society Conference. Simonson, I. (1989). Choice based on reasons: The case of attraction and compromise effects. Journal of Consumer Research, 16:158–174. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  76. 76. References References VIII Sornette, D. (2004). Why Stock Markets Crash: Critical Events in Complex Financial Systems. Princeton University Press. Takahashi, R. and Morimura, T. (2015). Predicting preference reversals via gaussian process uncertainty aversion. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTATS 2015), pages 958–967. Tavar´e, S., Balding, D. J., Griffiths, R. C., and Donnelly, P. (1997). Inferring coalescence times from DNA sequence data. Genetics, 145(2):505–518. Thurner, S., Farmer, J. D., and Geanakoplos, J. (2012). Leverage causes fat tails and clustered volatility. Quantitative Finance, 12(5):695–707. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  77. 77. References References IX Trueblood, J. S. (2014). The multiattribute linear ballistic accumulator model of context effects in multialternative choice. Psychological Review, 121(2):179– 205. Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79:281–299. Usher, M. and McClelland, J. L. (2004). Loss aversion and inhibition in dynamical models of multialternative choice. Psychological Review, 111:757– 769. van Hasselt, H., Guez, A., and Silver, D. (2016). Deep reinforcement learning with double Q-learning. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16). Wen, C.-H. and Koppelman, F. (2001). The generalized nested logit model. Transportation Research Part B, 35:627–641. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  78. 78. References References X Williams, H. (1977). On the formulation of travel demand models and economic evaluation measures of user benefit. Environment and Planning A, 9(3):285–344. Yai, T. (1997). Multinomial probit with structured covariance for route choice behavior. Transportation Research Part B: Methodological, 31(3):195–207. Yue, Y. and Guestrin, C. (2011). Linear submodular bandits and their application to diversified retrieval. In Shawe-taylor, J., Zemel, R., Bartlett, P., Pereira, F., and Weinberger, K., editors, Advances in Neural Information Processing Systems 24, pages 2483–2491. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  79. 79. References References XI Zhang, S. and Yu, A. J. (2013). Forgetful Bayes and myopic planning: Human learning and decision-making in a bandit setting. In Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K., editors, Advances in Neural Information Processing Systems 26, pages 2607–2615. Curran Associates, Inc. Ziegler, C.-N., McNee, S. M., Konstan, J. A., and Lausen, G. (2005). Improving recommendation lists through topic diversification. In Proceedings of the 14th international conference on World Wide Web (WWW 2005), pages 22–32. ACM. SmartNews Connecting Machine Learning Algorithms with Behavioral Game Theo
  80. 80. THANK YOU FOR ATTENDING!

×