How Fast is AI in Pharo? Benchmarking Linear Regression IWST22

1. How Fast is AI in Pharo?
Benchmarking Linear Regression
1Arolla, Paris
2Inria, Univ. Lille, CNRS, Centrale Lille, UMR 9189 - CRIStAL
Oleksandr ZAITSEV1,2, Sebastian JORDAN MONTAÑO2, Stéphane DUCASSE 2
IWST22 — International Workshop on Smalltalk Technologies Novi Sad, Serbia
oleksandr.zaitsev@arolla.fr

2. 2
Fast
Very slow!
Why?
pharo-ai
Scikit-learn
(Python)
Training machine learning models

3. 3
Training machine learning models
Can we do the same in Pharo?

4. 4

5. 5
LAPACK SciPy scikit-learn
pharo-ai
PolyMath
LAPACK
Pharo
Python
Fast
Algebra
Math
Machine
Learning

6. 6
We want to provide a proof of concept that math & AI
algorithms in Pharo can be boosted by delegating time-
consuming operations to highly-optimised external libraries.
To do that, we build a prototype implementation of linear
regression based on the DGELSD routine of LAPACK

7. Linear regression
and how it is
implemented

8. 8
Linear Regression Problem

9. 9
Linear Regression Problem

10. 10
Linear Regression Problem

11. 11
Linear Regression Implementations
Implementation
Gradient Descent Least Squares
- Iterative
- Approximate
- One iteration
- Exact solution

12. 12
Implementation 1. Gradient Descent

13. 13
Need to minimise the errors
ei = ̂
yi − yi
Cost function
- mean squared errors
J(w) =
1
m
m
∑
i=1
e2
i
Implementation 1. Gradient Descent

14. 14
J(w) =
1
m
m
∑
i=1
( ̂
yi − yi)2
Implementation 1. Gradient Descent

15. 15
J(w) =
1
m
m
∑
i=1
( ̂
yi − yi)2
Implementation 1. Gradient Descent

16. 16
J(w) =
1
m
m
∑
i=1
( ̂
yi − yi)2
θ(new)
= θ(old)
− α
∂
∂θi
J(θ)
Implementation 1. Gradient Descent

17. 17
Implementation 2. Least Squares
̂
θ = argmin||y − Xθ||2
Orthogonal projection

18. 18
Implementation 2. Least Squares
Implemented as
DGELSD routine
In LAPACK
We can call it
through FFI

19. 19
Demo will come in the end
Wait for it!

20. Experiment

21. 21
Datasets

22. 22
Research Questions
- RQ.1 - Measuring LAPACK speedup.
How much time improvement can we achieve by calling
LAPACK from Pharo?
- RQ.2 - Comparing to scikit-learn.
How does Pharo & LAPACK implementation compare to the
one provided by scikit-learn?
- RQ.3 - Comparing pure Pharo with Python.
How does pure Pharo implementation of linear regression
compare to equivalent pure Python implementation?

23. 23
RQ.1 - Measuring LAPACK speedup

24. 24
Time in seconds (less is better)
Small
Medium
Pure Pharo vs Pharo & LAPACK
1820 times faster
2103 times faster
Pharo Pharo & LAPACK
19min40sec 1h44min
0.6sec 2.9sec
Help
RQ.1 - Measuring LAPACK speedup

25. 25
RQ.2 - Comparing to scikit-learn.

26. 26
RQ.2 - Comparing to scikit-learn.

27. 27
RQ.3 - Comparing pure Pharo with Python

28. 28
RQ.3 - Comparing pure Pharo with Python

29. Summary
‣ We propose a prototype implementation of Linear Regression based on LAPACK
‣ We show that LAPACK & Pharo is up to 2103 times faster than pure Pharo
‣ We also show that scikit-learn is 8-5 times faster than our prototype, depending on the
size of the data.
‣ Finally, we demonstrate that pure Pharo is up to 15 times faster than the equivalent
implementation in pure Python
‣ Those findings can lay the foundation for the future work in building fast numerical
libraries for Pharo and further using them in higher-level libraries such as pharo-ai.