This is a revamped Asynchronous Programming FTW talk, given at YAPC::NA 2013, Cluj.pm, and AmsterdamX.pm.

1. AsynchronousAsynchronous
ProgrammingProgramming
FTW!FTW!(Sawyer X)
@PerlSawyer
Theory
Practice

2. TheoryTheory
We think in non-blocking terms
We act in non-blocking terms
We program in blocking terms
An example? Sure!

3. The exampleThe example
use LWP::UserAgent;
my @urls = (
'https://duckduckgo.com',
'http://cluj.pm',
'http://telaviv.pm.org',
);
my $ua = LWP::UserAgent->new;
foreach my $url (@urls) {
my $res = $ua->get($url);
say $res->decoded_content;
}

4. Problem?Problem?
Wait on each URL
Ineffecient waste of time
Pointless
Senseless
CRIME AGAINST HUMANITY
(maybe just ineffecient)

5. How we thinkHow we think
Nothing to work on? Defer to later!
Like cooking!

6. Making dinnerMaking dinner
Boil water. Wait? No! Salad time!
(water gets deferred to later)
Cook potatoes. Wait? No! Sauce time!
(potatoes get deferred to later)
We think and act non-blocking

7. TIMTOWTDITIMTOWTDI
Forks
Threads
Event-loops <- this talk

8. Event loop?Event loop?
Single thread, single process
Like a big while(1) {}
Heavy usage of code references
No real race conditions
No locking, semaphores, etc.
Can yield better speed
But... nothing can be blocking! (I/O)

9. Event loops in PerlEvent loops in Perl
POE
Reflex
IO::Async
IO::Lambda
Qt, GTK, Tk, Glib, Cocoa::EventLoop, EV, FLTK, Irssi
AnyEvent <- this talk

10. AnyEventAnyEvent
Light-weight
Fast
Very fast
Slim interface
Good support for other event loops
Lots of modules

11. PracticePractice
Cooking with AnyEventCooking with AnyEvent
(timers)(timers)
use AnyEvent;
say '>> Water boiling...';
my $water = AnyEvent->timer( after => 5*60, cb => sub {
say '<< Water boiled!';
say '>> Potatos being prepared...';
my $potatoes; $potatoes = AnyEvent->timer( after => 10*60, cb => sub {
undef $potatoes; say '<< Potatos ready!';
} );
} );
say '>> Salad being prepared...';
my $salad = AnyEvent->timer( after => 6*60, cb => sub {
say '<< Salad ready!';
say '>> Sauce being prepared...';
my $sauce; $sauce = AnyEvent->timer( after => 12*60, cb => sub {
undef $sauce; say '<< Sauce ready!';
} );
} );

12. Water - 5 minutes
Potatos - 10 minutes after water
Salad - 6 minutes
Sauce - 12 minutes after salad
Water, salad, potatoes, sauce
>> Water boiling...
>> Salad being prepared...
<< Water boiled!
>> Potatos being prepared...
<< Salad ready!
>> Sauce being prepared...
<< Potatos ready!
<< Sauce ready!

13. Condition variablesCondition variables
A condition waiting to be fulfilled
Starting at false, hoping to get to true
Helps waiting for other events to finish
Call ->recv() to wait for stuff to finish
Call ->send() when finished doing your stuff
Without condition variables, we reach the end of programs
We reach the end == program closes
Program closes == our code doesn't get executed
Our code doesn't get executed == we get executed!

14. Simple exampleSimple example
use AnyEvent;
my $count = 1;
my $cv = AnyEvent->condvar;
my $timer = AnyEvent->timer(
interval => 1,
cb => sub {
say "Count: $count";
$count++ == 5 and $cv->send;
},
);
$cv->recv;
say 'We counted to 5!';

15. Bigger exampleBigger example
my $cv = AnyEvent->condvar;
my $water = AnyEvent->timer( after => 5*60, cb => sub {
my $potatoes; $potatoes = AnyEvent->timer( after => 10*60, cb => sub {
undef $potatoes;
} );
} );
my $salad = AnyEvent->timer( after => 6*60, cb => sub {
my $sauce; $sauce = AnyEvent->timer( after => 12*60, cb => sub {
undef $sauce;
$cv->send;
} );
} );
$cv->recv;

16. Better example :: LWPBetter example :: LWP
use LWP::UserAgent;
my @urls = (
'https://whatismyip.com',
'http://checkip.dyndns.org',
'http://wtfismyipaddress.org',
);
my $ua = LWP::UserAgent->new;
foreach my $url (@urls) {
my $res = $ua->get($url);
say $res->decoded_content;
}

17. Better example :: AnyEventBetter example :: AnyEvent
use AnyEvent;
use AnyEvent::HTTP;
my $cv = AnyEvent->condvar;
foreach my $url (@urls) {
$cv->begin;
http_get $url, sub {
my $body = shift;
say $body;
$cv->end;
}
}
$cv->recv;

18. POP QUIZ TIME!POP QUIZ TIME!
A few examples
Spot the error(s) and get a prize
Available prizes: handshake, high five, hug, kiss
(not necessarily from me)

19. Round 1Round 1
use AnyEvent;
use AnyEvent::HTTP;
http_post $url, $body, sub {
say 'Successfully made an API call to do something cool!';
};
# wait for all events to finish
AnyEvent->condvar->recv;

20. Round 2Round 2
use AnyEvent;
my $timer = AnyEvent->timer(
interval => 5,
cb => sub {
sleep 1;
say 'Bwahaha!';
}
);

21. Round 3Round 3
use AnyEvent;
my $cv = AnyEvent->condvar;
my $player1 = AnyEvent->timer(
after => 3.7,
cb => sub {
say 'Player 1 joins the game!';
$cv->send;
},
);
my $player2 = AnyEvent->timer(
after => 7.2,
cb => sub {
say 'Player 2 joins the game!';
$cv->send;
},
);
$cv->recv;

22. Round 4Round 4
use AnyEvent; use HTTP::Tiny;
my $cv = AnyEvent->condvar;
$cv->begin for 1 .. 2;
my $player1 = AnyEvent->timer(
after => 3.7,
cb => sub {
say 'Player 1 joins the game! Alerting server!';
my $res = HTTP::Tiny->new->get('http://server:3030/add/1');
$res->{'success'} or die 'Failed to insert player 1 to game!';
$cv->end;
},
);
my $player2 = AnyEvent->timer(
after => 7.2,
cb => sub { say 'Player 2 joins the game!'; $cv->end; },
);
$cv->recv;

23. Stuff to do with AnyEventStuff to do with AnyEvent
xkcd interface

24. WWW::xkcdWWW::xkcd
use WWW::xkcd;
my $xkcd = WWW::xkcd->new;
$xkcd->fetch( sub {
my ( $img, $comic ) = @_;
say "Today's comic is titled: ", $comic->{'title'};
...
} );

25. Stuff to do with AnyEventStuff to do with AnyEvent
xkcd interface
PSGI/Plack server

26. TwiggyTwiggy
use Twiggy::Server;
my $app = sub {
# PSGI app
};
my $server = Twiggy::Server->new(...);
$server->register_service($app);
AnyEvent->condvar->recv;

27. Stuff to do with AnyEventStuff to do with AnyEvent
xkcd interface
PSGI/Plack server
Async AWS EC2 interface

28. AnyEvent::EC2::TinyAnyEvent::EC2::Tiny
use AnyEvent::EC2::Tiny;
my $ec2 = AnyEvent::EC2::Tiny->new(...);
my $xml = $ec2->send(
'RegionName.1' => 'us-east-1',
Action => 'DescribeRegions',
success_cb => sub {
my $xml = shift;
# prints ec2.us-east-1.amazonaws.com
say $xml->{'regionInfo'}{'item'}[0]{'regionEndpoint'};
},
fail_cb => sub {
my $error = shift;
...
},
);

29. Stuff to do with AnyEventStuff to do with AnyEvent
xkcd interface
PSGI/Plack server
Async AWS EC2 interface
Monitoring framework

30. JunoJuno
use Juno;
my $juno = Juno->new(
hosts => [ 'server1', 'server2' ],
interval => 10,
checks => {
HTTP => {
headers => {
{ 'Host', 'example.com' },
},
on_result => sub {
my $result = shift;
...
},
},
},
);

31. Stuff to do with AnyEventStuff to do with AnyEvent
xkcd interface
PSGI/Plack server
Async AWS EC2 interface
Monitoring framework
Real-time network path resolution

32. SIP RouteSIP Route
use AnyEvent;
use Anyevent::Util 'fork_call';
use AnyEvent::Socket;
use AnyEvent::Handle;
use Data::Dump 'dump';
my $tcp_server_guard = tcp_server '127.0.0.1', 2020, sub {
my ($fh) = @_;
my $handle = AnyEvent::Handle->new( fh => $fh );
$handle->push_read( line => sub {...} );
};

33. my ( $hdl, $target ) = @_;
if ( ! defined $target ) {
$hdl->push_write($default_route);
$hdl->destroy;
return;
}
chomp $target;
if ( ! exists $data{$target} ) {
$hdl->push_write($default_route);
$hdl->destroy;
return;
}

34. my @sorted = sort { $data{$target}{$a} <=> $data{$target}{$b} }
keys %{ $data{$target} };
$hdl->push_write( $sorted[0] );
$hdl->destroy;

35. foreach my $target (@targets) {
foreach my $iface (@interfaces) {
my $cmd = "sipsak --sip-uri=sip:$target -D $timeout -X $iface";
push @timers, AE::timer $start_after, $check_every, sub {
my $time = AE::now;
fork_call {
system "$cmd >/dev/null 2>&1";
return $? >> 8;
} sub {
my $exit = shift;
( $exit == 0 || $exit == 1 )
or return delete $data{$target}{$iface};
$data{$target}{$iface} = AE::now - $time;
};
};
}
}
AE::cv->recv;

36. if ($verbose) {
push @timers, AE::timer 5, 5, sub { dump %data };
}

37. Stuff to do with AnyEventStuff to do with AnyEvent
xkcd interface
PSGI/Plack server
Async AWS EC2 interface
Monitoring framework
Real-time network path resolution
API interfaces to services

38. AnyEvent::RiakAnyEvent::Riak
use AnyEvent::Riak;
my $riak = AnyEvent::Riak->new(
host => 'http://127.0.0.1:8098',
path => 'riak',
);
$riak->list_bucket(
'mybucket',
{props => 'true', keys => 'false'},
sub {
my $res = shift;
...
}
);

39. AnyEvent::SNMPAnyEvent::SNMP
use AnyEvent::SNMP;
use Net::SNMP;
my $cv = AnyEvent->condvar;
Net::SNMP->session(
-hostname => "127.0.0.1",
-community => "public",
-nonblocking => 1
)->get_request( -callback => sub { $cv->send (@_) } );
my @result = $cv->recv;

40. AnyEvent::XMLRPCAnyEvent::XMLRPC
use AnyEvent::XMLRPC;
my $serv = AnyEvent::XMLRPC->new(
methods => {
'echo' => &echo,
},
);

41. AnyEvent::DNSAnyEvent::DNS
use AnyEvent::DNS;
my $cv = AnyEvent->condvar;
AnyEvent::DNS::a "www.google.ro", $cv;
# later...
my @addrs = $cv->recv;

42. AnyEvent::TwitterAnyEvent::Twitter
my $ua = AnyEvent::Twitter->new(
consumer_key => 'consumer_key',
consumer_secret => 'consumer_secret',
access_token => 'access_token',
access_token_secret => 'access_token_secret',
);
my $cv = AnyEvent->condvar;
# GET request
$cv->begin;
$ua->get( 'account/verify_credentials', sub {
my ($header, $response, $reason) = @_;
say $response->{screen_name};
$cv->end;
} );

43. AnyEvent::GraphiteAnyEvent::Graphite
my $graphite = AnyEvent::Graphite->new(
host => '127.0.0.1',
port => '2003',
);
$graphite->send("a.b.c.d", $value, $timestamp);

44. Stuff to do with AnyEventStuff to do with AnyEvent
xkcd interface
PSGI/Plack server
Async AWS EC2 interface
Monitoring framework
Real-time network path resolution
API interfaces to services
Find matching special actors in TV series

45. Actor matchesActor matches
use AnyEvent; use AnyEvent::HTTP;
my $base = 'http://www.imdb.com/title';
my %titles = (
tt1196946 => { name => 'Mentalist, the' },
tt1219024 => { name => 'Castle' },
tt0773262 => { name => 'Dexter' },
tt0491738 => { name => 'Psych' },
tt0247082 => { name => 'CSI Las Vegas' },
tt0458253 => { name => 'Closer, the' },
);
# DON'T PARSE HTML WITH REGEX!!!!1127
my $episode_regex = qr{...};
my $cast_regex = qr{...};

46. # Fetching number of seasons for each title
my $cv = AE::cv;
foreach my $title_id ( keys %titles ) {
my $title = $titles{$title_id}{'name'};
print "[$title] Fetching number of seasons.n";
my $url = "$base/$title_id/";
my $regex = qr{/title/$title_id/episodes?season=([0-9]+)};
$cv->begin;
http_get $url, sub {
my ( $body, $hdr ) = @_;
my @found = $body =~ /$regex/mg;
$titles{$title_id}{'episodes'}{$_} = [] for @found;
$cv->end;
};
}
$cv->recv;

47. # fetching the list of episodes for each season
foreach my $title_id ( keys %titles ) {
my $cv = AE::cv;
foreach my $season ( keys %{ $titles{$title_id}{'episodes'} } ) {
my $title = $titles{$title_id}{'name'};
print "[$title] [Season $season] fetching episodesn";
my $season_url = "$base/$title_id/episodes?season=$season";
$cv->begin;
http_get $season_url, sub {
my ( $body, $hdr ) = @_;
my @found = $body =~ /$episode_regex/mg;
while (@found) {
my $id = shift @found;
my $name = shift @found;

48. foreach my $title_id ( keys %titles ) {
my $title = $titles{$title_id}{'name'};
foreach my $season ( keys %{ $titles{$title_id}{'episodes'} } ) {
my @episodes = @{ $titles{$title_id}{'episodes'}{$season} };
print "[$title] [$season] Fetching cast for season $seasonn";
my $cv = AE::cv;
foreach my $episode_id (@episodes) {
print " -> [$title] [$season] $episode_idn";
my $url = "$base/$episode_id/fullcredits";
$cv->begin;
http_get $url, sub {
my ( $body, $hdr ) = @_;
my @found = $body =~ /$cast_regex/mg;

49. print "Populating actorsn";
my %actors = ();
foreach my $title_id ( keys %titles ) {
my $title = $titles{$title_id}{'name'};
foreach my $cast_sets ( @{ $titles{$title_id}{'cast'} } ) {
my ( $name, $role ) = @{$cast_sets};
$actors{$name}{$title} = $role;
}
}

50. print "Cross referencingn";
foreach my $name ( keys %actors ) {
scalar keys %{ $actors{$name} } > 1 or next;
my @where = ();
foreach my $title ( keys %{ $actors{$name} } ) {
my $role = $actors{$name}{$title};
push @where, "$title ($role)";
}
printf "Actor $name appeared in %sn", join ', ', @where;
}

51. Some dataSome data
6 TV series
42 total seasons
1,510 total episodes
6,484 total actors
7,493 total roles
380,520 total requests (6 * 42 * 1,510)
~ 2s/req = 761,040s = 12,684m = 211h = 8.8 days
Time to run (incl. analysis) with AnyEvent: 6 minutes
Matches found... 1,095!

52. Actor Michael Patrick McGill appeared in:
CSI Las Vegas (Ruben Caster)
Dexter (Troy)
Actor Robert Esser appeared in:
CSI Las Vegas (Waiter)
Castle (Officer #2 (uncredited))
Actor Andre Alexsen appeared in:
Closer, the (Adam)
Mentalist, the (CA State Govenor (uncredited))
Actor Becky O'Donohue appeared in:
CSI Las Vegas (Ava Rendell)
Mentalist, the (Sasha)
Psych (Molly Gogolack)

53. Mark PellegrinoMark Pellegrino
Tom Dempsey / Tom Dempsey III in Castle
Gavin Q. Baker III in The Closer
Bruno Curtis in CSI Last Vegas
Paul Bennett in Dexter
Von McBride in The Mentalist
Most accomplishedMost accomplished
cross-actorcross-actor

54. Stuff to do with AnyEventStuff to do with AnyEvent
xkcd interface
PSGI/Plack server
Async AWS EC2 interface
Monitoring framework
Real-time network path resolution
API interfaces to services
Find matching special actors in TV series
Whole lot of other awesome stuff
(which I'm not gonna show you now)

55. Thank youThank you