Os Whitaker

Keeping Your Workers In Line: ,[object Object],[object Object],use TheSchwartz;

Once upon a time... In a galaxy far far away Users wanted features Like subscription based notifications, Pinging external services, And other things which Love to tie up webserver processes And are generally too slow / blocking to execute synchronously with web requests

(Yes, that was really my best slide-foo, so bear with me)

So we needed to solve the problems these features create

Which can easily result in a mess

Contacting External Web Services ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Processing Uploaded Media ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Initial Solution: GhettoQueue ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Incoming data from lots of different transports ,[object Object],[object Object],[object Object],[object Object]

Initial Solution: Lots of Daemons! ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Events, Subscriptions, Notifications? ,[object Object],[object Object],[object Object],[object Object],[object Object]

Problems With These Approaches ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Implementation ,[object Object],[object Object],[object Object]

Topology ,[object Object],[object Object],[object Object]

Schwartz Database ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Schwartz Workers ,[object Object],[object Object],[object Object],[object Object]

Full Topology: With Application

Request Cycle ,[object Object],[object Object],[object Object],[object Object],[object Object]

Workers can define other per-worker behaviors ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Other fun features ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Using TheSchwartz in production ,[object Object]

Schwartz Database Configuration ,[object Object],[object Object],[object Object],[object Object],[object Object]

Schwartz Database Configuration … .. And so on, adding clusters as needed

Tools and monitoring ,[object Object],[object Object],[object Object],[object Object],[object Object]

Schwartzmon Example lj@ljadmin1:~$ schwartzmon --dsn=DBI:mysql:theschwartz_livejournalhost=10.191.90.101 --user=lj -f errors Thu Jul 26 19:22:13 2007 [2116902910]: Connection failed to domain 'imagemenagerie.com', MXes: [imagemenagerie.com] Thu Jul 26 19:22:14 2007 [2120335058]: Connection failed to domain 'thedashcat.net', MXes: [thedashcat.net] Thu Jul 26 19:22:15 2007 [2126277932]: Connection failed to domain 'cox.net', MXes: [mx.west.cox.net mx.east.cox.net] Thu Jul 26 19:22:16 2007 [2126007758]: Connection failed to domain 'cox.net', MXes: [mx.west.cox.net mx.east.cox.net] Thu Jul 26 19:22:16 2007 [2126309296]: Permanent failure TO [hey2a@cs.com]: 550 MAILBOX NOT FOUND Thu Jul 26 19:22:17 2007 [2126309446]: Permanent failure TO [joe_junkpan@hotmail.com]: 550 Requested action not taken: mailbox unavailable Thu Jul 26 19:22:18 2007 [2126308836]: Error during DATAEND phase to [ourxtrees@yahoo.com]: 451 go ahead Message temporarily deferred - [250] Thu Jul 26 19:22:18 2007 [2126188996]: Connection failed to domain 'buffyboarders.zzn.com', MXes: [c2mailmx.mailcentro.com c2mds.mailcentro.com] Thu Jul 26 19:22:21 2007 [2125661508]: Connection failed to domain 'cox.net', MXes: [mx.east.cox.net mx.west.cox.net]

Ljworkerctrl example ,[object Object]

Ljworkerctrl example ,[object Object],-

Questions? http://code.sixapart.com/svn/TheSchwartz/trunk

Os Whitaker

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (8)

Ähnlich wie Os Whitaker

Ähnlich wie Os Whitaker (20)

Mehr von oscon2007

Mehr von oscon2007 (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Os Whitaker