4. Очереди к ограниченным ресурсам необходимы!
● Пред-расчет данных
● Операции импорта/экспорта данных
● Обработка логов системы
● Маркетинговый анализ базы данных
● Почтовые рассылки
● Map/Reduce
● и многое другое...
5. Выполнение задачи в асинхронном режиме
Client Job Server Worker
add job
job handle
execute
success
7. About project Gearman
Gearman = Manager
2005: originally Perl implementation by Danga Interactive
2008: rewrite in C by Brian Aker
Used in:
Digg (45+ servers, 400K jobs/day)
Yahoo (60+ servers, 6M jobs/day)
Xing.com
Grooveshark
...
8. Преимущества и недостатки
● Высокая скорость ● Мониторинг, администрирование
● Масштабируемость ● Слабая документация
● Балансировка нагрузки ● Работа с памятью
● Параллельное и асинхронное ● Отличия API в различных версиях
выполнение задач
● Защита от потери очереди
● Multi-language API
9. Gearman server monitoring
Administrative Protocol
> telnet localhost 4730
> Trying ::1...
> Connected to localhost.
> Escape character is '^]'.
> wokers
FD IP-ADDRESS CLIENT-ID : FUNCTION
> status
FUNCTION TOTAL RUNNING AVAILABLE_WORKERS
GearUp: Gearman based monitoring service
https://launchpad.net/gearup
Mod Gearman
http://labs.consol.de/nagios/mod-gearman/
10. Client Client Client Client Client Client
Client API Client API
Gearman Gearman
Worker API Worker API
Worker Worker Worker Worker Worker
11. client.php
<?php
# create our object
$client= new GearmanClient();
# add the default server
$client->addServer();
$workload = array(
'email' => 'nobody@example.com' ,
'subject' => 'the subject' ,
'message' => 'the message'
);
# send email
$client->doBackground( "send_mail" , json_encode($workload));
if ($client->returnCode() != GEARMAN_SUCCESS) {
echo "bad return coden" ;
exit;
}
?>
12. worker.php
<?php
# create the worker
$worker = new GearmanWorker();
# add the default job server (localhost)
$worker->addServer();
# add the reverse function
$worker->addFunction( "send_mail" , "sendMail" );
# start to worker listening for job submissions
while ($worker->work());
function sendMail($job)
{
$arg = json_decode($job->workload(), true);
mail($arg[ 'email'], $arg[ 'subject' ], $arg[ 'message' ]);
echo "email has been sentn" ;
}
?>
14. worker.php
<?php
#...
function sendMail($job)
{
$arg = json_decode($job->workload(), true);
# connect to remote service
$mail = new Mail($arg['email'], $arg['subject'], $arg['message']);
# send email
if (!$mail->send()){
# it needs to be solved
} else {
echo "email has been sentn";
}
}
?>
16. worker.php
<?php
#...
function sendMail($job)
{
#...
# send email
if (!$mail->send()){
sleep(20);
$client= new GearmanClient();
$client->addServer();
$client->doBackground("send_mail", $job->workload());
echo "error has happened n";
} else {
echo "email has been sentn";
}
}
?>
17. Client Gearman Worker Task
doBackground() run task execute
sleep(20)
doBackground()
failure
run task execute
sleep(20)
doBackground()
failure
run task execute
success
18. worker.php
<?php
#...
function sendMail($job)
{
#...
# send email
if (!$mail->send()){
setCron("send_mail", $job->workload(), 20);
echo "error has happened n";
} else {
echo "email has been sentn";
}
}
?>
19. Client Gearman Worker Task Cron
doBackground() run task execute
failure do postponed
delay
doBackground()
run task execute
do postponed
failure
delay
doBackground()
run task execute
success do postponed
20. Solution
Credit:
— In ver. 0.20 added when_to_run tasks functionality
— But public API doesn't support that functionality
— Neither does PECL Gearman module
— Documentation also is silent about it
John Ewart
Software Engineer
21. worker.php
<?php
#...
function sendMail($job)
{
#...
# send email
if (!$mail->send()){
$client= new GearmanClient();
$client->addServer();
$time = time() + 20;
$client->doSuspended("send_mail", $job->workload(), NULL, $time);
echo "error has happened n" ;
} else {
echo "email has been sentn" ;
}
}
?>
22. Suspended task execution
Client Gearman Worker Task
doBackground() run task execute
doSuspended()
failure
delay
run task execute
doSuspended()
failure
delay
run task execute
success
