AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
The art of concurrent programming
1. The Art of Concurrent
Programming
Past, Present and Future
Iskren Chernev
2. What is this talk about
● What is parallel, concurrent, asynchronous
● Brief history
● Run Loops
● Callbacks
● Promises
● Coroutines
3. Parallel, Concurrent, Asynchronous
● Parallel - 2 pieces of code running at the same time (threads/processes
running different CPUs, cores)
● Concurrent - the begining and end of 2 pieces of code overlap, but not
necessarily run at the same time (could be parralel, but not necessarily). So it
appears to overlap, but it doesn’t have to
● Asynchronous - an operation that will complete at a later time, but its
execution doesn’t block the thread/process
● We’ll cover Asynchronous programming in this lecture
4. Brief overview of the environment
● Obviously this talk is opinionated, so here’s what I have in mind
● server-side business logic applications
● little amount of computation
● A lot of IO (talking to other (micro)-services and databases)
● A huge volume of “easy” requests
● Client side is also mostly event driven with little computation so most of talk
applies there too
5. Brief History
● Single Threaded software
● Mutli-core brought with it Multi Threaded Software
○ (+) Its very close to single threaded software
○ (-) slow context switch, creation
○ (-) cache trashing
○ (-) need for synchronization primitives -- this is complicated
● With Node.js (2009) came the hype around asynchronous, single threaded
programming
● After that most other popular languages added asynchronous primitives and
libraries as standard, or popularized and extended existing asynchronous
primitives (Java, C#, Objective-C, Python)
6. So what is this Node.js all about
var server = http.createServer((request, response) => {
response.end( 'It Works!! Path Hit: ' + request.url);
})
server.listen(PORT, () => {
console.log("Server listening on: http://localhost:%s" , PORT);
})
7. Node.js - basic
function doSomethingSync(args) {
r = doSlow1(args);
return doSlow2(r)
}
function doSomething(args, done) {
doSlow1(args, (err, res) => {
if (err) return done(err);
doSlow2(res, done);
});
}
● Easy function creation with “closures” is a must
● Every async computation calls a given function after finished
8. What else - let’s try an if
function doSomethingSync(args) {
if (cond) {
r = doSlow1();
} else {
r = 5;
}
doFast(r);
return doSlow2();
}
WTF
function doSomething(args, done) {
var doRest = (r, done) => {
doFast(r)
doSlow2(done)
}
if (cond) {
return doSlow1((err, res) => {
if (err) return done(err)
doRest(arg, done)
})
}
doRest(5, done);
}
9. Key takeaways from Node.js
● Its great because it showed the world
what asynchronous code looks like and
how to do it with callbacks
● It sucks to actually code it (there is
async.js but its still pretty painful)
● Is here a better way?
● (pro) its very fast
● (pro) all libraries support callback async
interface
● (cons) writing correct async-callback
code is tricky to do right (easy to
double call the callback, or not call it at
all, or forget to return when calling done
● (cons) and its ugly (deep nesting for a
chain of async operations)
10. Run Loops
● Something like a thread manager
● Operations a run loop should implement
○ schedule(fn) -- this function only is enough
to implement basic concurrent systems
with multithreaded blocking IO (main
thread)
○ scheduleLater(when, fn) -- similar to
schedule but executes at a later point
○ addSocketWithEvents(sock, evmask, fn)
- register for socket events
○ NOTE: that linux support for async DISK io
is spotty, and is mostly wrapped with
thread pool under the hood
● Run loops are the basis of asynchronous
programming
● You can have one or many, and
communicate between them using
schedule
● Node.js has a run-loop built in, but other
languages added those later
● Libraries should explicitly support run
loops (if there are many), and most “old”
languages, “old” libraries are blocking
● You could shim any blocking operation
with a thread pool and a single main
run-loop
11. What is a Promise
● Called in different ways : Future,
CompletionStage (Java), Task (C#) etc
● Its an object representing the result of
an asynchronous operation
● It supports chaining
function doSomethingSync(args) {
if (cond) {
r = doSlow1();
} else {
r = 5;
}
doFast(r);
return doSlow2();
}
function doSomething(args) {
if (cond) {
r = doSlow1();
} else {
r = 5;
}
Promise.resole(r).then((r) => {
doFast(r);
return doSlow2();
});
}
12. Promises -- loops
function doSomethingConc(n) {
var futures = []
for (var i = 0; i < n; ++i) {
futures.push(doSlow(i));
}
// returns a new future, which
// resolves to an array of results
return Promise.all(futures);
}
function doSomethingSeries(n) {
var future = Promise.resolved(null);
var every = (i) => {
future = future.then(() => doSlow(i));
}
for (var i = 0; i < n; ++i) {
every(i);
}
return future;
}
13. Promisses - notes
● Error propagation
● If you need to exit early from a chain you have to use exceptions
● Compared to callbacks -- promisses guarantee listener is called at most once
● So callbacks are still necessary if you need to call listener many times
● In some languages (C++, some JS libs), promise and future are separate object
● In typed languages (C++, Java, C#) you need a few different APIs on the promise to handle async
and sync returns
● Normally the listener is called right after its produced, on the same thread, but you can change
that
● Promises are hard to implement -- use an official/stable lib, do NOT do it yourself
14. Coroutines
● Coroutines are mostly syntax sugar on top
of promises, but it pays off!
● The observation is that mostly you have a
linear chain of promises, so in that case
you can use a keyword await (C#, python)
/ yield (python)
● It handles linear code, ifs and serial
loops
● You can always fall back to promises if
what you’re doing is more complicated
(branches off, parralel loops)
● To implement you need support in the
language
async def doStuff(x):
if cond:
r = async doSlow(x)
else:
r = 5
doFast(r)
return async doSlow()
