This is a presentation on Probabilistic programming languages for "Advanced Funcional Languages" course of MSc in Computer Science @ University of Padua

1. Università degli studi di Padova
Anno accademico 2016/2017
Study of Probabilistic Programming Languages
MSc in Computer Science
Aspetti Avanzati dei Linguaggi di Programmazione
Stefano Campese 1127711

2. Index
Table of contents
● Introduction
● Main ideas
● Application cases
● Some languages and frameworks
● WebPPL examples
● References
Stefano Campese 2Anno accademico 2016/2017

3. Introduction
Introduction
● High-level languages (WebPPL, Church, Figaro, PyMC, Infer.net ...)
● Easy definition of Probability Models
● Specialized language primitives (random event, probabilistic variable..)
● Reduction of complexity of probabilistic inference
● Reduce time to understand data
● Abstract details of CPUs and memory
Stefano Campese 3Anno accademico 2016/2017

4. Introduction
What is a Probabilistic Programming System
● Programming Language
○ WebPPL (http://webppl.org/)
○ Curch (http://projects.csail.mit.edu/church/wiki/Church)
● Libraries or Frameworks
○ Figaro (https://github.com/p2t2/figaro)
○ PyMc (https://pymc-devs.github.io/)
○ Infer.net (http://infernet.azurewebsites.net/)
Stefano Campese 4Anno accademico 2016/2017

5. Introduction
Probabilistic Programming System Model
Stefano Campese 5Anno accademico 2016/2017
● Data can be modeled directly even with their imperfections
● Model can represent and use uncertain knowledge
● Low model complexity
● Efficient probabilistic inference
● Essential aspects of data

6. Main ideas
Data model with uncertain knowledge in ML
Stefano Campese 6Anno accademico 2016/2017
… Machine-learning way of problem approach is modeling the situation with random variables …

7. Main ideas
Decision Tree
Stefano Campese 7Anno accademico 2016/2017

8. Main ideas
Data model with uncertain knowledge in PPL
Stefano Campese 8Anno accademico 2016/2017
● PPL runtime environment run program both
forward and backward
● Forward is the simulation
● Inference is the backward
● Same background assumption
● Different approaches

9. Main ideas
Forward
Stefano Campese 9Anno accademico 2016/2017
… Instructions follow the same direction of the process that is going to be modeled ...

10. Main ideas
Backward
Stefano Campese 10Anno accademico 2016/2017
… Inverse problem, from result will be generated a configuration that can explain it ...

11. Case of application
Case of application
Stefano Campese 11Anno accademico 2016/2017
● Climate models
● Machine perceptions (Google’s self-driving cars)
● IoT with massively distributed sensors
● Recommendation systems (Reduce long tail matrix problem)
● Shorter and more humane clinical trials
● Reduction of side effects and more accurate outcomes
● Natural language processing
● Approximate computation
● Security
… all field where exists complex interactions with some levels of uncertain knowledge ...

12. Case of application
Why use PPL
Giacomo Manzoli 12Anno accademico 2016/2017
● Unifying anecdotal reasoning with reliable statistical approaches
● Improve the modeling with heterogeneous data as:
○ mixed dataset
○ relational dataset
○ computer vision
○ textual dataset
● Great fit in Artificial Intelligence fields
○ Google’s self driving car
○ IBM’s Watson
○ General predictions and classifications

13. Some languages and frameworks
Some languages and frameworks: FIGARO
Stefano Campese 13Anno accademico 2016/2017
Figaro framework:
● Scala library
● Born in 2004
● Open-source license
● Built-in reasoning algorithms
● Rich models supports
● Easy draws conclusion from models
● Possibility to integrate with Apache Spark
● Java interoperability (adapt to enterprise world)

14. Some languages and frameworks
Some languages and frameworks: PyMC
Stefano Campese 14Anno accademico 2016/2017
PyMC framework:
● Python module that works over numpy
● Born in 2003
● Academic Free License
● Bayesian statistical models
● MCMC supports
● Only high probability values are being computed
● Optimized probability distributions

15. Some languages and frameworks
Some languages and frameworks: CHURCH
Stefano Campese 15Anno accademico 2016/2017
Church programming language:
● Lisp-like family
● Born in 2008
● Founded and grants by Naval Research of US
● Derived from Scheme PPL
● Designed for expressive generation of probabilistic models
● Different implementation in JavaScript or Scheme
● Defined as “Universal probabilistic programming language”
● Used to implement “Model of Cognition”

16. Some languages and frameworks
Some languages and frameworks: WebPPL
Stefano Campese 16Anno accademico 2016/2017
WebPPL programming language:
● Called “Web people”
● Subset of purely functional JavaScript language
● Integrable both in Web pages and JavaScript server (NPM and Node)
● Built-in several inference algorithms (MCMC or HMC)
● Non-commercial license
● No loops (while, for…)
● CPS philosophy
● Supported by DARPA and Naval Research

17. WebPPL
Generic information about WebPPL
Stefano Campese 17Anno accademico 2016/2017
● Installable by using NodeJs (>= 6.x) and NPM
dependency management
● WebPPL file must have .wppl extension
● To execute from command line: webppl
myprogram.wppl
● Inclusion of native or other JavaScript function
is allowed
● Docker allows to have preinstalled and ready
environment

18. WebPPL
Simple Example WebPPL
Stefano Campese 18Anno accademico 2016/2017
● Not only a JavaScript subset
● Possible separate distribution models from
distributions
● Possibility to retrieve distribution sample
● Possibility to retrieve the log probability of a
sample value
● Possibility to retrieve the support of the
distribution

19. WebPPL
Simple Inference Example WebPPL
Stefano Campese 19Anno accademico 2016/2017
.. Inference is called also marginalization. Other built-in inference paradigm such as enumerate and MCMC ...

20. WebPPL
Other Examples
Stefano Campese 20Anno accademico 2016/2017
● Bayesian language comprehender
● Computer Vision

21. References
References
Stefano Campese 21Anno accademico 2016/2017
● The Design and Implementation of Probabilistic Programming Languages, Noah D. Goodman and Andreas
Stuhlmuller, 2016
● What is probabilistic programming?, Beau Cronin, 2013
● Probabilistic Programming: Why, What, How, When?, Beau Cronin, 2014
● What is probabilistic programming?, Michael Hicks, 2014
● The State of Probabilistic Programming, Mohammed AlQuraishi, 2015
● Lifted Inference for Probabilistic Programming, Department of CS of KU Leuven
● Probabilistic Programming, Department of CS of Cornell University, 2016
● The Principles and Practice of Probabilistic Programming, Noah D. Goodman - Stanford University, 2013
● Probabilistic-Programming.org
● Why Probabilistic Programming Matters, Rob Zinkov, 2012
● Probabilistic Programming for Advancing Machine Learning (PPAML), Dr. Jennifer Roberts
● What is Probabilistic Computing, Navia Systems, 2010