SlideShare ist ein Scribd-Unternehmen logo
1 von 27
CPAN realtime feed
       Tatsuhiko Miyagawa
   YAPC::Asia 2009 lightning talks
CPAN authors?
(upload your module now)
What happens
when a new module is
 uploaded to CPAN
PAUSE
Perl Author Upload Server
Author:
uploads a module
5s
PAUSE:
index the module
~30s
QA mirrors:
fetches the diff
~20s
QA mirrors:
 reindex
1 hour
Other mirrors:
  rsync diffs
1~24 hour
Users:
update CPAN.pm cache
24 hours
 (default)
Upload (10s)
PAUSE index (30s)
QA mirrors sync (20s)
QA reindex (1h)
rsync mirrors (~24h)
CPAN cache (~24h)
~48 hours
Not cool.
A solution:
CPAN Realtime feed
friendfeed.com/cpan
The bot
Fetching QA’s diffs
Upload (10s)
PAUSE index (30s)
QA mirrors sync (20s)
QA reindex (1h)
rsync mirrors (~24h)
CPAN cache (~24h)
FriendFeed:
XMPP, APIs, Feeds
 PubSubHubbub
cpanf
(App::CPAN::Fresh)
Install CPAN modules
  via real time feeds
DEMO
(might fail.)

Weitere ähnliche Inhalte

Was ist angesagt?

Async in .NET
Async in .NETAsync in .NET
Async in .NET
RTigger
 
Java 7 & 8
Java 7 & 8Java 7 & 8
Java 7 & 8
Ken Coenen
 

Was ist angesagt? (20)

MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
 
Introduction to GraalVM
Introduction to GraalVMIntroduction to GraalVM
Introduction to GraalVM
 
Native Java with GraalVM
Native Java with GraalVMNative Java with GraalVM
Native Java with GraalVM
 
C# 5 deep drive into asynchronous programming
C# 5 deep drive into asynchronous programmingC# 5 deep drive into asynchronous programming
C# 5 deep drive into asynchronous programming
 
Synchronizing parallel delivery flows in jenkins using groovy, build flow and...
Synchronizing parallel delivery flows in jenkins using groovy, build flow and...Synchronizing parallel delivery flows in jenkins using groovy, build flow and...
Synchronizing parallel delivery flows in jenkins using groovy, build flow and...
 
Flink Forward Berlin 2017: Piotr Wawrzyniak - Extending Apache Flink stream p...
Flink Forward Berlin 2017: Piotr Wawrzyniak - Extending Apache Flink stream p...Flink Forward Berlin 2017: Piotr Wawrzyniak - Extending Apache Flink stream p...
Flink Forward Berlin 2017: Piotr Wawrzyniak - Extending Apache Flink stream p...
 
Types - slice, map, new, make, struct - Gopherlabs
Types - slice, map, new, make, struct - Gopherlabs Types - slice, map, new, make, struct - Gopherlabs
Types - slice, map, new, make, struct - Gopherlabs
 
Async in .NET
Async in .NETAsync in .NET
Async in .NET
 
Async/Await Best Practices
Async/Await Best PracticesAsync/Await Best Practices
Async/Await Best Practices
 
Big data: Loading your data with flume and sqoop
Big data:  Loading your data with flume and sqoopBig data:  Loading your data with flume and sqoop
Big data: Loading your data with flume and sqoop
 
Scrum-ban: Time-boxes and flow
Scrum-ban: Time-boxes and flowScrum-ban: Time-boxes and flow
Scrum-ban: Time-boxes and flow
 
Java 7 & 8
Java 7 & 8Java 7 & 8
Java 7 & 8
 
Flink Forward Berlin 2017: Zohar Mizrahi - Python Streaming API
Flink Forward Berlin 2017: Zohar Mizrahi - Python Streaming APIFlink Forward Berlin 2017: Zohar Mizrahi - Python Streaming API
Flink Forward Berlin 2017: Zohar Mizrahi - Python Streaming API
 
What's coming in Airflow 2.0? - NYC Apache Airflow Meetup
What's coming in Airflow 2.0? - NYC Apache Airflow MeetupWhat's coming in Airflow 2.0? - NYC Apache Airflow Meetup
What's coming in Airflow 2.0? - NYC Apache Airflow Meetup
 
Quick test professional (qtp) online training
Quick test professional (qtp) online trainingQuick test professional (qtp) online training
Quick test professional (qtp) online training
 
Open Source Load Testing: JMeter, Gatling and Taurus
Open Source Load Testing: JMeter, Gatling and TaurusOpen Source Load Testing: JMeter, Gatling and Taurus
Open Source Load Testing: JMeter, Gatling and Taurus
 
Ceilometer juno-midpoint
Ceilometer juno-midpointCeilometer juno-midpoint
Ceilometer juno-midpoint
 
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
Flink Forward Berlin 2017: Andreas Kunft - Efficiently executing R Dataframes...
 
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipeline
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipelineKubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipeline
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipeline
 
Flink Forward Berlin 2017: Maciek Próchniak - TouK Nussknacker - creating Fli...
Flink Forward Berlin 2017: Maciek Próchniak - TouK Nussknacker - creating Fli...Flink Forward Berlin 2017: Maciek Próchniak - TouK Nussknacker - creating Fli...
Flink Forward Berlin 2017: Maciek Próchniak - TouK Nussknacker - creating Fli...
 

Andere mochten auch (9)

Wight: Phantom’s Perl friend - YAPC::Asia 2012
Wight: Phantom’s Perl friend - YAPC::Asia 2012Wight: Phantom’s Perl friend - YAPC::Asia 2012
Wight: Phantom’s Perl friend - YAPC::Asia 2012
 
ZeroMQ in PHP
ZeroMQ in PHPZeroMQ in PHP
ZeroMQ in PHP
 
Deploying Plack Web Applications: OSCON 2011
Deploying Plack Web Applications: OSCON 2011Deploying Plack Web Applications: OSCON 2011
Deploying Plack Web Applications: OSCON 2011
 
Remedie: Building a desktop app with HTTP::Engine, SQLite and jQuery
Remedie: Building a desktop app with HTTP::Engine, SQLite and jQueryRemedie: Building a desktop app with HTTP::Engine, SQLite and jQuery
Remedie: Building a desktop app with HTTP::Engine, SQLite and jQuery
 
No Hugging, No Learning
No Hugging, No LearningNo Hugging, No Learning
No Hugging, No Learning
 
Carton CPAN dependency manager
Carton CPAN dependency managerCarton CPAN dependency manager
Carton CPAN dependency manager
 
Network Programming With Anyevent
Network Programming With AnyeventNetwork Programming With Anyevent
Network Programming With Anyevent
 
Intro to PSGI and Plack
Intro to PSGI and PlackIntro to PSGI and Plack
Intro to PSGI and Plack
 
Tatsumaki
TatsumakiTatsumaki
Tatsumaki
 

Ähnlich wie CPAN Realtime feed

Running large scale Kafka upgrades at Yelp (Manpreet Singh,Yelp) Kafka Summit...
Running large scale Kafka upgrades at Yelp (Manpreet Singh,Yelp) Kafka Summit...Running large scale Kafka upgrades at Yelp (Manpreet Singh,Yelp) Kafka Summit...
Running large scale Kafka upgrades at Yelp (Manpreet Singh,Yelp) Kafka Summit...
confluent
 
Plack perl superglue for web frameworks and servers
Plack perl superglue for web frameworks and serversPlack perl superglue for web frameworks and servers
Plack perl superglue for web frameworks and servers
Tatsuhiko Miyagawa
 
Modern Commandline Tool
Modern Commandline ToolModern Commandline Tool
Modern Commandline Tool
Yuji Shimada
 

Ähnlich wie CPAN Realtime feed (20)

CPAN Module Maintenance
CPAN Module MaintenanceCPAN Module Maintenance
CPAN Module Maintenance
 
Oleksandr Tarasenko "Using Kafka in your python applications"
Oleksandr Tarasenko "Using Kafka in your python applications"Oleksandr Tarasenko "Using Kafka in your python applications"
Oleksandr Tarasenko "Using Kafka in your python applications"
 
Using Kafka in your python application - Python fwdays 2020
Using Kafka in your python application - Python fwdays 2020Using Kafka in your python application - Python fwdays 2020
Using Kafka in your python application - Python fwdays 2020
 
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...
Introduction to Apache Beam & No Shard Left Behind: APIs for Massive Parallel...
 
Revisiting ppm
Revisiting ppmRevisiting ppm
Revisiting ppm
 
Evolution of kube-proxy (Brussels, Fosdem 2020)
Evolution of kube-proxy (Brussels, Fosdem 2020)Evolution of kube-proxy (Brussels, Fosdem 2020)
Evolution of kube-proxy (Brussels, Fosdem 2020)
 
Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)
Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)
Adopt DevOps philosophy on your Symfony projects (Symfony Live 2011)
 
rhaco updates - php conference japan 2010
rhaco updates - php conference japan 2010rhaco updates - php conference japan 2010
rhaco updates - php conference japan 2010
 
Running large scale Kafka upgrades at Yelp (Manpreet Singh,Yelp) Kafka Summit...
Running large scale Kafka upgrades at Yelp (Manpreet Singh,Yelp) Kafka Summit...Running large scale Kafka upgrades at Yelp (Manpreet Singh,Yelp) Kafka Summit...
Running large scale Kafka upgrades at Yelp (Manpreet Singh,Yelp) Kafka Summit...
 
Kanban Method July 2018
Kanban Method July 2018Kanban Method July 2018
Kanban Method July 2018
 
WSO2 API Manager Community Call - November 2021
WSO2 API Manager Community Call - November 2021WSO2 API Manager Community Call - November 2021
WSO2 API Manager Community Call - November 2021
 
Flink Forward Berlin 2017: Aljoscha Krettek - Talk Python to me: Stream Proce...
Flink Forward Berlin 2017: Aljoscha Krettek - Talk Python to me: Stream Proce...Flink Forward Berlin 2017: Aljoscha Krettek - Talk Python to me: Stream Proce...
Flink Forward Berlin 2017: Aljoscha Krettek - Talk Python to me: Stream Proce...
 
Ad Server Optimization
Ad Server OptimizationAd Server Optimization
Ad Server Optimization
 
Open Source tools overview
Open Source tools overviewOpen Source tools overview
Open Source tools overview
 
Monitoring Kafka w/ Prometheus
Monitoring Kafka w/ PrometheusMonitoring Kafka w/ Prometheus
Monitoring Kafka w/ Prometheus
 
Mitigating Security Threats with Fastly - Joe Williams at Fastly Altitude 2015
Mitigating Security Threats with Fastly - Joe Williams at Fastly Altitude 2015Mitigating Security Threats with Fastly - Joe Williams at Fastly Altitude 2015
Mitigating Security Threats with Fastly - Joe Williams at Fastly Altitude 2015
 
Plack perl superglue for web frameworks and servers
Plack perl superglue for web frameworks and serversPlack perl superglue for web frameworks and servers
Plack perl superglue for web frameworks and servers
 
Modern Commandline Tool
Modern Commandline ToolModern Commandline Tool
Modern Commandline Tool
 
Talk Python To Me: Stream Processing in your favourite Language with Beam on ...
Talk Python To Me: Stream Processing in your favourite Language with Beam on ...Talk Python To Me: Stream Processing in your favourite Language with Beam on ...
Talk Python To Me: Stream Processing in your favourite Language with Beam on ...
 
Test-Driven Infrastructure with Puppet, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Puppet, Test Kitchen, Serverspec and RSpecTest-Driven Infrastructure with Puppet, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Puppet, Test Kitchen, Serverspec and RSpec
 

Mehr von Tatsuhiko Miyagawa

Asynchronous programming with AnyEvent
Asynchronous programming with AnyEventAsynchronous programming with AnyEvent
Asynchronous programming with AnyEvent
Tatsuhiko Miyagawa
 
Building a desktop app with HTTP::Engine, SQLite and jQuery
Building a desktop app with HTTP::Engine, SQLite and jQueryBuilding a desktop app with HTTP::Engine, SQLite and jQuery
Building a desktop app with HTTP::Engine, SQLite and jQuery
Tatsuhiko Miyagawa
 
Why Open Matters It Pro Challenge 2008
Why Open Matters It Pro Challenge 2008Why Open Matters It Pro Challenge 2008
Why Open Matters It Pro Challenge 2008
Tatsuhiko Miyagawa
 
20 modules i haven't yet talked about
20 modules i haven't yet talked about20 modules i haven't yet talked about
20 modules i haven't yet talked about
Tatsuhiko Miyagawa
 
Web Scraper Shibuya.pm tech talk #8
Web Scraper Shibuya.pm tech talk #8Web Scraper Shibuya.pm tech talk #8
Web Scraper Shibuya.pm tech talk #8
Tatsuhiko Miyagawa
 

Mehr von Tatsuhiko Miyagawa (20)

Plack at OSCON 2010
Plack at OSCON 2010Plack at OSCON 2010
Plack at OSCON 2010
 
cpanminus at YAPC::NA 2010
cpanminus at YAPC::NA 2010cpanminus at YAPC::NA 2010
cpanminus at YAPC::NA 2010
 
Plack at YAPC::NA 2010
Plack at YAPC::NA 2010Plack at YAPC::NA 2010
Plack at YAPC::NA 2010
 
PSGI/Plack OSDC.TW
PSGI/Plack OSDC.TWPSGI/Plack OSDC.TW
PSGI/Plack OSDC.TW
 
Plack - LPW 2009
Plack - LPW 2009Plack - LPW 2009
Plack - LPW 2009
 
Asynchronous programming with AnyEvent
Asynchronous programming with AnyEventAsynchronous programming with AnyEvent
Asynchronous programming with AnyEvent
 
Building a desktop app with HTTP::Engine, SQLite and jQuery
Building a desktop app with HTTP::Engine, SQLite and jQueryBuilding a desktop app with HTTP::Engine, SQLite and jQuery
Building a desktop app with HTTP::Engine, SQLite and jQuery
 
Remedie OSDC.TW
Remedie OSDC.TWRemedie OSDC.TW
Remedie OSDC.TW
 
Why Open Matters It Pro Challenge 2008
Why Open Matters It Pro Challenge 2008Why Open Matters It Pro Challenge 2008
Why Open Matters It Pro Challenge 2008
 
20 modules i haven't yet talked about
20 modules i haven't yet talked about20 modules i haven't yet talked about
20 modules i haven't yet talked about
 
Web::Scraper for SF.pm LT
Web::Scraper for SF.pm LTWeb::Scraper for SF.pm LT
Web::Scraper for SF.pm LT
 
Web Scraper Shibuya.pm tech talk #8
Web Scraper Shibuya.pm tech talk #8Web Scraper Shibuya.pm tech talk #8
Web Scraper Shibuya.pm tech talk #8
 
Web::Scraper
Web::ScraperWeb::Scraper
Web::Scraper
 
XML::Liberal
XML::LiberalXML::Liberal
XML::Liberal
 
Test::Base
Test::BaseTest::Base
Test::Base
 
Hacking Vox and Plagger
Hacking Vox and PlaggerHacking Vox and Plagger
Hacking Vox and Plagger
 
Plagger the duct tape of internet
Plagger the duct tape of internetPlagger the duct tape of internet
Plagger the duct tape of internet
 
Tilting Google Maps and MissileLauncher
Tilting Google Maps and MissileLauncherTilting Google Maps and MissileLauncher
Tilting Google Maps and MissileLauncher
 
Writing Pluggable Software
Writing Pluggable SoftwareWriting Pluggable Software
Writing Pluggable Software
 
How we build Vox
How we build VoxHow we build Vox
How we build Vox
 

Kürzlich hochgeladen

Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Kürzlich hochgeladen (20)

Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 

CPAN Realtime feed