This document summarizes concurrency and parallel computing in Julia. It discusses tasks/coroutines, channels for inter-task communication, and parallel for loops and maps for distributing work across processes. Key points:
- Tasks allow suspending/resuming execution without context switching and can run in any order.
- Channels provide synchronous inter-task communication via put! and take!.
- The @spawn and @spawnat macros create remote tasks on other processes.
- @parallel distributes loop iterations or map tasks across processes for parallel execution.
- Orchestration involves coordinating local and remote asynchronous/parallel tasks.
7. Single task
julia> function print_to(n)
for i in 1:n
println(i)
end
end
print_to (generic function with 1 method)
julia> foo = @task print_to(100)
Task (runnable)
9. Use wait() to pause
julia> function print_to(n)
for i in 1:n
println(i)
wait() # set task state
to :waiting and pause
end
end
print_to (generic function with 1 method)
julia> foo = @task print_to(100)
Task (runnable)
10. Use wait() to pause
julia> yield(foo)
1
julia> yield(foo)
2
julia> yield(foo)
3
julia> foo
Task (runnable)
17. Concurrency
• A way of flow control
• A way to program
• Concurrency is about dealing with lots of things at once.
• Parallelism is about doing lots of things at once.
• Concurrency is not parallelism -- Rob Pike
21. Channel
• Inter-task communication
• Synchronous (blocking)
ch = Channel{Int64}(50)
put!(ch, 5)
x = take!(ch)
isready(ch) # test if anything in it
close(ch)
25. Channel can be accessed by multi-task
function producer(ch)
for i in 1:100
put!(ch, i)
yield()
end
end
function consumer(ch)
while true
x = take!(ch)
println(x)
yield()
end
end
ch = Channel{Int64}(100)
for _ in 1:10
@schedule producer(ch)
@schedule consumer(ch)
end
26. Concurrency example
function f1(ch1)
for i in 1:100
put!(ch1, i)
end
end
function f2(ch1, ch2)
while true
x = take!(ch1)
y = x^2 + 1
put!(ch2, y)
yield()
end
end
ch1 = Channel{Int64}(100)
ch2 = Channel{Int64}(100)
ch3 = Channel{Tuple}(100)
function f3(ch2, ch3)
while true
x = take!(ch2)
y = log(x)
put!(ch3, (x, y))
yield()
end
end
27. You can bind channel to task
• Binding the lifetime of channel to specific task
• When a channel is bound to multiple tasks, the first task to terminate
will close the channel
t1 = @schedule f1(ch1)
t2 = @schedule f2(ch1, ch2)
t3 = @schedule f3(ch2, ch3)
bind(ch1, t1) # Optional
bind(ch2, t2)
bind(ch3, t3)
28. Concurrency example
• @async
• wrap expression as a Task
• put it to the scheduling queue
• adds it to the nearest enclosing set that @sync
waits for
• @sync
• Wait for all enclosed uses of @async, @spawn,
@spawnat and @parallel are complete
@sync begin
@async f1(ch1)
@async f2(ch1, ch2)
@async f3(ch2, ch3)
end
29. macro tips
• @task
• wrap expression as a Task
• @schedule
• wrap expression as a Task
• put it to the scheduling queue
• @async
• wrap expression as a Task
• put it to the scheduling queue
• adds it to the nearest enclosing set that @sync waits for
30. Events
cond = Condition()
function waiter(cond)
while true
x = wait(cond) # task
change state to :waiting
# do something
yield()
end
end
function notifyer(cond)
while true
x = f()
notify(cond, x) # change
waiter state to queued and pass x
to waiter
yield()
end
end
35. Code accessibility
• Code should be accessible on execution processes.
• `include(“Module.jl”)`: only load file into current process
• `using Module`: load module to every process, but is brought into scope only on
current process
• `@everywhere`: directly define expressions on all processes
@everywhere using Module
36. Data movement
• Explicitly
• fetch
• Implicitly
• @spawn
r = @spawn rand(2,2)
s = @spawn 1 .+ fetch(r)
results = fetch(s)
@everywhere x = 10
37. @spawn API
• @spawnat p expr -> Future
• Create a closure around the expression and run it on specified process
• @spawn expr -> Future
• Create a closure around the expression and run it on automatically-chosen
process
38. Parallel for loop
• Equally distribute tasks into worker processes
• Suitable for lightweight task but huge amount of iterations
@parallel for i in 1:10000
# do something
end
result = @parallel (+) for i in 1:10000
y = f(i)
y
end
results = @parallel (vcat) for i in 1:10000
y = f(i)
[y]
end
39. Parallel map
• Run each heavy_work(args) in a process
args = [i for i in 1:1000]
result = pmap(heavy_work, args)
41. Parallel computing
• Only one process
• myid() == 1 is also a worker
• Multiple processes
• Worker
• myid() == 1 is not a worker process
42. Run tasks on worker processes
func = [f1, f2, f3]
@sync begin
for (i, f) in zip(workers(), func)
@async remotecall(f, i)
end
end
43. RemoteChannel is required
• Channel: use with in local process
• RemoteChannel: pass data to remote process
• Composed of RemoteRef and Channel
ch1 = RemoteChannel(() -> Channel{Int64}(32))
ch2 = RemoteChannel(() -> Channel{Int64}(32))
ch3 = RemoteChannel(() -> Channel{Int64}(32))
44. Future is a specialized RemoteChannel
id =
1
worker
Channel{Any}(1)
Channel{?}(?)
RemoteRef
RemoteRef
Future
RemoteChannel