1. What’s next in Jul_
Jiahao Chen
MIT Computer Science and Artificial Intelligence Laboratory
julialang.org
Alan EdelmanJeff Bezanson Stefan Karpinski Viral B. Shah

2. What’s the big deal about Julia ?
julialang.org/benchmarks

3. It bridges the divide between computer science
and computational science
What’s the big deal about Julia ?

4. It bridges the divide between computer science
and computational science
What’s the big deal about Julia ?
data abstraction
performance

5. It bridges the divide between computer science
and computational science
What’s the big deal about Julia ?
data abstraction
performance
What if you didn’t have to choose between
data abstraction and performance?

6. It’s a programming language designed for technical computing
What’s the big deal about Julia ?

7. It’s a programming language designed for technical computing
What’s the big deal about Julia ?
The key ingredients
Multi-methods (multiple dispatch)
Dataflow type inference
together allow for cost-efficient data abstraction

8. Object-oriented programming with classes
What can I do with/to a thing?

9. Object-oriented programming with classes
What can I do with/to a thing?

10. Object-oriented programming with classes
What can I do with/to a thing?
top up
pay fare
lose
buy

11. Object-oriented programming with classes
What can I do with/to a thing?
top up
pay fare
lose
buy
top up
pay fare
lose
buy

12. Object-oriented programming with classes
What can I do with/to a thing?
top up
pay fare
lose
buy
top up
pay fare
lose
buy

13. Object-oriented programming with classes
What can I do with/to a thing?
top up
pay fare
lose
buy
top up
pay fare
lose
buy
pay fare
lose
buy

14. methods objects
Object-oriented programming with classes
What can I do with/to a thing?
top up
pay fare
lose
buy
top up
pay fare
lose
buy
pay fare
lose
buy

15. methods objects
Object-oriented programming with classes
What can I do with/to a thing?
top up
pay fare
lose
buy
top up
pay fare
lose
buy
pay fare
lose
buy
class-based OO
!
classes are more
fundamental
than methods

16. Object-oriented programming with multi-methods
What can I do with/to a thing?
top up
pay fare
lose
buy
generic
function
objectsmethods

17. Object-oriented programming with multi-methods
What can I do with/to a thing?
top up
pay fare
lose
buy
generic
function
objectsmethods
multimethods
!
relationships between
objects and functions

18. Multi-methods with type hierarchy
top up
pay fare
lose
buy
generic
function
objectsmethods
rechargeable
subway
pass
single-use
subway
ticket
is a subtype of
subway
ticket
abstract object

19. generic
function
objectsmethods
rechargeable
subway
pass
single-use
subway
ticket
is a subtype of
subway
ticket
top up
pay fare
lose
buy
abstract object
Multi-methods with type hierarchy

20. generic
function
objectsmethods
rechargeable
subway
pass
single-use
subway
ticket
is a subtype of
subway
ticket
top up
pay fare
lose
buy
abstract object
Multi-methods with type hierarchy

21. The Julia codebase is compact
5,774 lines of Scheme
32,707 lines of C
59727 lines of Julia
!
+LLVM, BLAS, LAPACK, SuiteSparse, ARPACK,
Rmath, GMP, MPFR, FFTW,…

22. Data types as a lattice
Dana Scott, Data types as lattices, SIAM J. Comput. 5: 522-87. 1976
Real
Number
FloatingPoint Rational
Complex
Float64 BigFloat…
…
Integer
is a parameter of
is a subtype of
Signed BigInt
Any
Int64…
…

23. Signed
Multiple dispatch with type lattice traversal
Real
Number
FloatingPoint Rational
Float64 BigFloat…
Integer
is a parameter of
is a subtype of
BigInt
Any
Int64…
…

24. Signed
Multiple dispatch with type lattice traversal
Real
Number
FloatingPoint Rational
Float64 BigFloat…
Integer
is a parameter of
is a subtype of
BigInt
Any
Int64…
…

25. Signed
Multiple dispatch with type lattice traversal
Real
Number
FloatingPoint Rational
Float64 BigFloat…
Integer
is a parameter of
is a subtype of
BigInt
Any
Int64…
…

26. Signed
Multiple dispatch with type lattice traversal
Real
Number
FloatingPoint Rational
Float64 BigFloat…
Integer
is a parameter of
is a subtype of
BigInt
Any
Int64…
…
no method here
try supertype
super(Int64) = Signed

27. Signed
Multiple dispatch with type lattice traversal
Real
Number
FloatingPoint Rational
Float64 BigFloat…
Integer
is a parameter of
is a subtype of
BigInt
Any
Int64…
…
no method here
try supertype
super(Int64) = Signed
no method here either
super(Signed) = Integer

28. Signed
Multiple dispatch with type lattice traversal
Real
Number
FloatingPoint Rational
Float64 BigFloat…
Integer
is a parameter of
is a subtype of
BigInt
Any
Int64…
…
no method here
try supertype
super(Int64) = Signed
no method here either
super(Signed) = Integer
found a method

29. Signed
Multiple dispatch with type lattice traversal
Real
Number
FloatingPoint Rational
Float64 BigFloat…
Integer
is a parameter of
is a subtype of
BigInt
Any
Int64…
…

30. Signed
Multiple dispatch with type lattice traversal
Real
Number
FloatingPoint Rational
Float64 BigFloat…
Integer
is a parameter of
is a subtype of
BigInt
Any
Int64…
…

32. Julia
LLVM IR
Machine assembly

34. aggressive type specialization
!
compiler generates specialized
methods for different input types
to the same generic function

35. unitful computations with
essentially no runtime
overhead
Keno Fischer
Harvard
Physics and Mathematics

36. Type lattice of arrays
DenseArray{T, N}
AbstractArray{T, N}
Array{T, N}
…
is a subtype of
Any
Array{T,1}===Vector{T}
Array{T,2}===Matrix{T}
A single parametric type defines many types
of matrices
Matrix{Float64}, Matrix{Int64},
Matrix{BigFloat}, Matrix{Complex128},
Matrix{Rational}, Matrix{Quaternion{Float64}},…

37. Type lattice of arrays
DenseArray{T, N}
AbstractArray{T, N}
Array{T, N}
…
is a subtype of
Any
Array{T,1}===Vector{T}
Array{T,2}===Matrix{T}
A single parametric type defines many types
of matrices
Matrix{Float64}, Matrix{Int64},
Matrix{BigFloat}, Matrix{Complex128},
Matrix{Rational}, Matrix{Quaternion{Float64}},…

38. Type lattice of arrays
DenseArray{T, N}
AbstractArray{T, N}
Array{T, N}
…
is a subtype of
Any
Array{T,1}===Vector{T}
Array{T,2}===Matrix{T}
rotation matrices
structured matrices
distributed arrays
A single parametric type defines many types
of matrices
Matrix{Float64}, Matrix{Int64},
Matrix{BigFloat}, Matrix{Complex128},
Matrix{Rational}, Matrix{Quaternion{Float64}},…

39. Multi-methods for linear algebra
What can I do with/to a thing?
find eigenvalues and eigenvectors
find singular values
find singular values and vectors
find eigenvalues
generic
function
objectsmethods
general matrix
symmetric tridiagonal matrix
bidiagonal matrix
Methods can take advantage of special matrix structures
eigvals
eigfact
svdvals
svdfact
Matrix
SymTridiagonal
Bidiagonal

40. So how does this help us with linear algebra?

41. So how does this help us with linear algebra?
Multi-method dispatch on special matrix types

42. So how does this help us with linear algebra?
Multi-method dispatch on special matrix types

43. So how does this help us with linear algebra?
Multi-method dispatch on special matrix types
stev!{T<:BlasFloat} calls sgestv
dgestv
cgestv
zgestv
and handles workspace
memory allocation

44. So how does this help us with linear algebra?
Multi-method dispatch with generic fallbacks
Matrix operations on general rings

45. So how does this help us with linear algebra?
Multi-method dispatch with generic fallbacks
Matrix operations on general rings textbook algorithm

46. So how does this help us with linear algebra?
Multi-method dispatch with generic fallbacks
Matrix operations on general rings

47. Matrix factorization types

48. Iterative algorithms as iterators
for item in iterable
#body
end
!
#is equivalent to
!
state = start(iterable)
while !done(iterable, state)
item, state = next(iterable, state)
# body
end

49. Conjugate gradients (Hestenes-Stiefel)
http://en.wikipedia.org/wiki/Conjugate_gradient_method

50. Conjugate gradients (Hestenes-Stiefel)
http://en.wikipedia.org/wiki/Conjugate_gradient_method
Can we write this as an iterator?

51. Conjugate gradients (Hestenes-Stiefel)
http://en.wikipedia.org/wiki/Conjugate_gradient_method
Can we write this as an iterator?
state = start(iterable)
done(iterable, state)

52. Conjugate gradients (Hestenes-Stiefel)
http://en.wikipedia.org/wiki/Conjugate_gradient_method
state = start(iterable)
done(iterable, state)
state, dx = next(iterable, state)

53. immutable cg_hs #iterative solver
K :: KrylovSpace #wraps A, v0, k
t :: Terminator #termination criteria
end
immutable cg_hs_state
r :: Vector #residual
p :: Vector #search direction
rnormsq :: Float64 #Squared norm of previous residual
iter :: Int #iteration count
end
start(a::cg_hs) = cg_hs_state(a.K.v0,
zeros(size(a.K.v0,1)), Inf, 0)
function next(a::cg_hs, s::cg_hs_state)
rnormsq = dot(s.r, s.r)
p = s.r + (rnormsq/s.rnormsq)*s.p
Ap = a.K.A*p
α = rnormsq / dot(p, Ap)
α*p, cg_hs_state(s.r-α*Ap, p, rnormsq, s.iter+1)
end
done(a::cg_hs, s::cg_hs_state) = done(a.t, s)
!
K = method(KrylovSpace(A, b, k), Terminator())
x += reduce(+, K) # for dx in K; x += dx; end
Hestenes-Stiefel CG

54. immutable cg_hs #iterative solver
K :: KrylovSpace #wraps A, v0, k
t :: Terminator #termination criteria
end
immutable cg_hs_state
r :: Vector #residual
p :: Vector #search direction
rnormsq :: Float64 #Squared norm of previous residual
iter :: Int #iteration count
end
start(a::cg_hs) = cg_hs_state(a.K.v0,
zeros(size(a.K.v0,1)), Inf, 0)
function next(a::cg_hs, s::cg_hs_state)
rnormsq = dot(s.r, s.r)
p = s.r + (rnormsq/s.rnormsq)*s.p
Ap = a.K.A*p
α = rnormsq / dot(p, Ap)
α*p, cg_hs_state(s.r-α*Ap, p, rnormsq, s.iter+1)
end
done(a::cg_hs, s::cg_hs_state) = done(a.t, s)
!
K = method(KrylovSpace(A, b, k), Terminator())
x += reduce(+, K) # for dx in K; x += dx; end
Hestenes-Stiefel CG
Abstracts out termination check and solution update steps

55. Native parallelism constructs

56. Native parallelism constructs

57. Native parallelism constructs

58. Distributed arrays

59. IJulia: Julia in IPython Notebook

60. IJulia: Julia in IPython Notebook

61. JuMP: writing simple DSLs in Julia
Iain Dunning Miles Lubin
MIT
Operations Research

62. Andreas N. Jensen
U. Copenhagen
Economics
Alan EdelmanJeff Bezanson Stefan Karpinski Viral B. Shah
Carlo Baldassi
Poly. Torino
Neuroscience
Tim E. Holy
WUSTL
Anatomy
Douglas M. Bates
Wisconsin-Madison
Statistics
Steven G. Johnson
MIT
Mathematics