SlideShare ist ein Scribd-Unternehmen logo
1 von 14
Downloaden Sie, um offline zu lesen
Making ZOPE faster
●   Without Squid
●   Without CacheFu
●   Without Varnish
The Problem
                (my problem)
●   The website uses LinguaPlone
    (what is that?!?)
    ... It uses Cookies for
    language selection

●   Aaaaagh !
Apache + Dynamic Rewrite Map
RewriteMap rpx prg:/home/admin/rpx.py
RewriteRule ^(.*)$ ${rpx:%{HTTP_COOKIE}<>$1<>%{QUERY_STRING}} [P,L]

RewriteMap rpx prg:/home/admin/rpx.py

RewriteRule ^(.*)$
   ${rpx:%{HTTP_COOKIE}
   <>$1
   <>%{QUERY_STRING}}
   [P,L]
The Dynamic Map Script

#!/usr/bin/python -u
import sys, os, time, urllib2, urllib, re

cacheroot = "/var/www/cache"
zoperoot =
  "http://slowzope:10000/VirtualHostBas
  e/http/fastrewriter:80/VirtualHostRoot"
logpath = "/home/admin/rpx.log"
The Real Dynamic Map Script
cookies, uri, qstring = request.split('<>')
lang = re.findall('I18N_LANGUAGE="([A_Za-z]+)"', cookies)
lang = lang and lang[0] or ''
locuri = uri + "/<>" + lang
if qstring:
    log("PASS %sn"%locuri)
    sys.stdout.write(zoperoot+uri+"n")
elif os.path.isfile(cacheroot + locuri):
    log("HIT %sn"%locuri)
    sys.stdout.write("http://faststatic/cache/%sn"%locuri)
else:
    log("MISS %sn"%locuri)
    sys.stdout.write(zoperoot+uri+"n")
The Cache Feeder
if line.startswith("MISS "):
     filepath = line[5:].strip()
     uri, extra = filepath.split("/<>")
     req = urllib2.Request(zoperoot+urllib.quote(uri),
              None, {'Cookie': 'I18N_LANGUAGE="%s"'%extra})
     o = urllib2.urlopen(req)
     data = o.read()
     datadir = cacheroot + "/" + uri
     if not os.path.isdir(datadir):os.makedirs(datadir)
     f = open(datadir + "/.htaccess", "w")
     f.write("ForceType %sn"%o.info().type)
     f.close()
     f = open(cacheroot + "/" + filepath,"w")
     f.write(data)
     f.close()
     log("FETCH %s <> %sn"%(filepath,extra))
The Cache Itself
admin@cache:~/cache$ find -type f | head
./services/locations-and-contacts/.htaccess
./services/locations-and-contacts/<>
./services/locations-and-contacts/<>fr
./services/locations-and-contacts/<>de
./capabilities-1/navigation-services-1/.htaccess
./capabilities-1/navigation-services-1/<>
./services/index_html/rss.gif/.htaccess
./services/index_html/rss.gif/<>fr
I swear the code is simple !

admin@cache:~$ wc -l rpx.py
74 rpx.py
Results




Watch to the lovely pictures !
In the unlikely event you want to know
      more about this really dirty hack ...
●   Jérôme Petazzoni
    <jerome.petazzoni@pilotsystems.net>

Weitere ähnliche Inhalte

Was ist angesagt?

Javascript Continues Integration in Jenkins with AngularJS
Javascript Continues Integration in Jenkins with AngularJSJavascript Continues Integration in Jenkins with AngularJS
Javascript Continues Integration in Jenkins with AngularJS
Ladislav Prskavec
 

Was ist angesagt? (19)

Raspberry pi Part 4
Raspberry pi Part 4Raspberry pi Part 4
Raspberry pi Part 4
 
serverstats
serverstatsserverstats
serverstats
 
Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)
 
File::CleanupTask
File::CleanupTaskFile::CleanupTask
File::CleanupTask
 
Cpsh sh
Cpsh shCpsh sh
Cpsh sh
 
Profiling with Xhprof
Profiling with XhprofProfiling with Xhprof
Profiling with Xhprof
 
Find and zip files
Find and zip filesFind and zip files
Find and zip files
 
Express Presentation
Express PresentationExpress Presentation
Express Presentation
 
mongodb-introduction
mongodb-introductionmongodb-introduction
mongodb-introduction
 
Compile kernel
Compile kernelCompile kernel
Compile kernel
 
Bash Scripting
Bash ScriptingBash Scripting
Bash Scripting
 
Ch3(working with file)
Ch3(working with file)Ch3(working with file)
Ch3(working with file)
 
App-o-Lockalypse now!
App-o-Lockalypse now!App-o-Lockalypse now!
App-o-Lockalypse now!
 
Gaelyk
GaelykGaelyk
Gaelyk
 
working with files
working with filesworking with files
working with files
 
Web Scrapping with Python
Web Scrapping with PythonWeb Scrapping with Python
Web Scrapping with Python
 
S3 & Glacier - The only backup solution you'll ever need
S3 & Glacier - The only backup solution you'll ever needS3 & Glacier - The only backup solution you'll ever need
S3 & Glacier - The only backup solution you'll ever need
 
Abusing Erlang compilation pipeline for Fun and Profit
Abusing Erlang compilation pipeline for Fun and ProfitAbusing Erlang compilation pipeline for Fun and Profit
Abusing Erlang compilation pipeline for Fun and Profit
 
Javascript Continues Integration in Jenkins with AngularJS
Javascript Continues Integration in Jenkins with AngularJSJavascript Continues Integration in Jenkins with AngularJS
Javascript Continues Integration in Jenkins with AngularJS
 

Andere mochten auch (6)

Neoppod, discloses first open source transactional NoSQL Object database for ...
Neoppod, discloses first open source transactional NoSQL Object database for ...Neoppod, discloses first open source transactional NoSQL Object database for ...
Neoppod, discloses first open source transactional NoSQL Object database for ...
 
DoSomething + Mobile
DoSomething + MobileDoSomething + Mobile
DoSomething + Mobile
 
Deutsche EuroShop "Acquisition of A10 Center Wildau"
Deutsche EuroShop "Acquisition of A10 Center Wildau"Deutsche EuroShop "Acquisition of A10 Center Wildau"
Deutsche EuroShop "Acquisition of A10 Center Wildau"
 
Computer Beginners
Computer BeginnersComputer Beginners
Computer Beginners
 
How Animals Move - Chapter 30
How Animals Move - Chapter 30How Animals Move - Chapter 30
How Animals Move - Chapter 30
 
Transformation digitale
Transformation digitaleTransformation digitale
Transformation digitale
 

Ähnlich wie Plone Conference 2008 Lightning Talk Static Zope Rpx

On secure application of PHP wrappers
On secure application  of PHP wrappersOn secure application  of PHP wrappers
On secure application of PHP wrappers
Positive Hack Days
 
4069180 Caching Performance Lessons From Facebook
4069180 Caching Performance Lessons From Facebook4069180 Caching Performance Lessons From Facebook
4069180 Caching Performance Lessons From Facebook
guoqing75
 
Facebook的缓存系统
Facebook的缓存系统Facebook的缓存系统
Facebook的缓存系统
yiditushe
 

Ähnlich wie Plone Conference 2008 Lightning Talk Static Zope Rpx (20)

ELK: a log management framework
ELK: a log management frameworkELK: a log management framework
ELK: a log management framework
 
On secure application of PHP wrappers
On secure application  of PHP wrappersOn secure application  of PHP wrappers
On secure application of PHP wrappers
 
4069180 Caching Performance Lessons From Facebook
4069180 Caching Performance Lessons From Facebook4069180 Caching Performance Lessons From Facebook
4069180 Caching Performance Lessons From Facebook
 
Facebook的缓存系统
Facebook的缓存系统Facebook的缓存系统
Facebook的缓存系统
 
Debugging: Rules & Tools
Debugging: Rules & ToolsDebugging: Rules & Tools
Debugging: Rules & Tools
 
Go Web Development
Go Web DevelopmentGo Web Development
Go Web Development
 
Debugging: Rules And Tools - PHPTek 11 Version
Debugging: Rules And Tools - PHPTek 11 VersionDebugging: Rules And Tools - PHPTek 11 Version
Debugging: Rules And Tools - PHPTek 11 Version
 
Heavy Web Optimization: Backend
Heavy Web Optimization: BackendHeavy Web Optimization: Backend
Heavy Web Optimization: Backend
 
Key features PHP 5.3 - 5.6
Key features PHP 5.3 - 5.6Key features PHP 5.3 - 5.6
Key features PHP 5.3 - 5.6
 
RESTing with the new Yandex.Disk API, Clemens Аuer
RESTing with the new Yandex.Disk API, Clemens АuerRESTing with the new Yandex.Disk API, Clemens Аuer
RESTing with the new Yandex.Disk API, Clemens Аuer
 
Using and scaling Rack and Rack-based middleware
Using and scaling Rack and Rack-based middlewareUsing and scaling Rack and Rack-based middleware
Using and scaling Rack and Rack-based middleware
 
Hacking with hhvm
Hacking with hhvmHacking with hhvm
Hacking with hhvm
 
GDG DevFest 2013 - PHP Web Apps on Google Cloud
GDG DevFest 2013 - PHP Web Apps on Google CloudGDG DevFest 2013 - PHP Web Apps on Google Cloud
GDG DevFest 2013 - PHP Web Apps on Google Cloud
 
AWS Hadoop and PIG and overview
AWS Hadoop and PIG and overviewAWS Hadoop and PIG and overview
AWS Hadoop and PIG and overview
 
PHP Backdoor: The rise of the vuln
PHP Backdoor: The rise of the vulnPHP Backdoor: The rise of the vuln
PHP Backdoor: The rise of the vuln
 
PHP CLI: A Cinderella Story
PHP CLI: A Cinderella StoryPHP CLI: A Cinderella Story
PHP CLI: A Cinderella Story
 
Puppet Camp 2012
Puppet Camp 2012Puppet Camp 2012
Puppet Camp 2012
 
PHP5.5 is Here
PHP5.5 is HerePHP5.5 is Here
PHP5.5 is Here
 
Twas the night before Malware...
Twas the night before Malware...Twas the night before Malware...
Twas the night before Malware...
 
Configuration Surgery with Augeas
Configuration Surgery with AugeasConfiguration Surgery with Augeas
Configuration Surgery with Augeas
 

Mehr von Paris, France

Django Hacking : Comment dresser les serpents guitaristes
Django Hacking : Comment dresser les serpents guitaristesDjango Hacking : Comment dresser les serpents guitaristes
Django Hacking : Comment dresser les serpents guitaristes
Paris, France
 
SeSQL : un moteur de recherche en Python et PostgreSQL
SeSQL : un moteur de recherche en Python et PostgreSQLSeSQL : un moteur de recherche en Python et PostgreSQL
SeSQL : un moteur de recherche en Python et PostgreSQL
Paris, France
 
Neopod: une base de données objet distribuée
Neopod: une base de données objet distribuéeNeopod: une base de données objet distribuée
Neopod: une base de données objet distribuée
Paris, France
 

Mehr von Paris, France (20)

Django Hacking : Comment dresser les serpents guitaristes
Django Hacking : Comment dresser les serpents guitaristesDjango Hacking : Comment dresser les serpents guitaristes
Django Hacking : Comment dresser les serpents guitaristes
 
SeSQL : un moteur de recherche en Python et PostgreSQL
SeSQL : un moteur de recherche en Python et PostgreSQLSeSQL : un moteur de recherche en Python et PostgreSQL
SeSQL : un moteur de recherche en Python et PostgreSQL
 
Neopod: une base de données objet distribuée
Neopod: une base de données objet distribuéeNeopod: une base de données objet distribuée
Neopod: une base de données objet distribuée
 
Conférence Sesql - DjangoCong 2011
Conférence Sesql - DjangoCong 2011Conférence Sesql - DjangoCong 2011
Conférence Sesql - DjangoCong 2011
 
API Cockpit : faites communiquer vos applications web
API Cockpit : faites communiquer vos applications webAPI Cockpit : faites communiquer vos applications web
API Cockpit : faites communiquer vos applications web
 
Indexation d'une base documentaire pour Liberation
Indexation d'une base documentaire pour LiberationIndexation d'une base documentaire pour Liberation
Indexation d'une base documentaire pour Liberation
 
Systematic/NEOPPOD dévoilent la première base de données objets NoSQL Open So...
Systematic/NEOPPOD dévoilent la première base de données objets NoSQL Open So...Systematic/NEOPPOD dévoilent la première base de données objets NoSQL Open So...
Systematic/NEOPPOD dévoilent la première base de données objets NoSQL Open So...
 
Cockpit, gestion de contacts et d'emailing
Cockpit, gestion de contacts et d'emailingCockpit, gestion de contacts et d'emailing
Cockpit, gestion de contacts et d'emailing
 
Plone, du site web à la gestion documentaire
Plone, du site web à la gestion documentairePlone, du site web à la gestion documentaire
Plone, du site web à la gestion documentaire
 
Les critères de choix d'un intranet
Les critères de choix d'un intranetLes critères de choix d'un intranet
Les critères de choix d'un intranet
 
Virtualisation Cloud Computing Saas Open Source
Virtualisation Cloud Computing Saas Open SourceVirtualisation Cloud Computing Saas Open Source
Virtualisation Cloud Computing Saas Open Source
 
Partenariat Exo Platform/Pilot Systems
Partenariat Exo Platform/Pilot SystemsPartenariat Exo Platform/Pilot Systems
Partenariat Exo Platform/Pilot Systems
 
Open World Forum 2009 Synthesis
Open World Forum 2009 SynthesisOpen World Forum 2009 Synthesis
Open World Forum 2009 Synthesis
 
Open World Forum 2009 Presentation
Open World Forum 2009 PresentationOpen World Forum 2009 Presentation
Open World Forum 2009 Presentation
 
Deliverance, la puissance de la gestion de contenu Plone appliquee aux themes...
Deliverance, la puissance de la gestion de contenu Plone appliquee aux themes...Deliverance, la puissance de la gestion de contenu Plone appliquee aux themes...
Deliverance, la puissance de la gestion de contenu Plone appliquee aux themes...
 
Deploiement de Plone et Deliverance avec Ubuntu VM Builder
Deploiement de Plone et Deliverance avec Ubuntu VM BuilderDeploiement de Plone et Deliverance avec Ubuntu VM Builder
Deploiement de Plone et Deliverance avec Ubuntu VM Builder
 
Deliverance, la puissance de la gestion de contenu Plone appliquée aux thèmes...
Deliverance, la puissance de la gestion de contenu Plone appliquée aux thèmes...Deliverance, la puissance de la gestion de contenu Plone appliquée aux thèmes...
Deliverance, la puissance de la gestion de contenu Plone appliquée aux thèmes...
 
Construire un module enfichable (pluggable) et dynamique pour Plone
Construire un module enfichable (pluggable) et dynamique pour PloneConstruire un module enfichable (pluggable) et dynamique pour Plone
Construire un module enfichable (pluggable) et dynamique pour Plone
 
La virtualisation dans le Datacenter
La virtualisation dans le DatacenterLa virtualisation dans le Datacenter
La virtualisation dans le Datacenter
 
Guide pratique des logiciels libres dans les administrations
Guide pratique des logiciels libres dans les administrationsGuide pratique des logiciels libres dans les administrations
Guide pratique des logiciels libres dans les administrations
 

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdf
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 

Plone Conference 2008 Lightning Talk Static Zope Rpx

  • 1. Making ZOPE faster ● Without Squid ● Without CacheFu ● Without Varnish
  • 2. The Problem (my problem) ● The website uses LinguaPlone (what is that?!?) ... It uses Cookies for language selection ● Aaaaagh !
  • 3. Apache + Dynamic Rewrite Map RewriteMap rpx prg:/home/admin/rpx.py RewriteRule ^(.*)$ ${rpx:%{HTTP_COOKIE}<>$1<>%{QUERY_STRING}} [P,L] RewriteMap rpx prg:/home/admin/rpx.py RewriteRule ^(.*)$ ${rpx:%{HTTP_COOKIE} <>$1 <>%{QUERY_STRING}} [P,L]
  • 4. The Dynamic Map Script #!/usr/bin/python -u import sys, os, time, urllib2, urllib, re cacheroot = "/var/www/cache" zoperoot = "http://slowzope:10000/VirtualHostBas e/http/fastrewriter:80/VirtualHostRoot" logpath = "/home/admin/rpx.log"
  • 5. The Real Dynamic Map Script cookies, uri, qstring = request.split('<>') lang = re.findall('I18N_LANGUAGE="([A_Za-z]+)"', cookies) lang = lang and lang[0] or '' locuri = uri + "/<>" + lang if qstring: log("PASS %sn"%locuri) sys.stdout.write(zoperoot+uri+"n") elif os.path.isfile(cacheroot + locuri): log("HIT %sn"%locuri) sys.stdout.write("http://faststatic/cache/%sn"%locuri) else: log("MISS %sn"%locuri) sys.stdout.write(zoperoot+uri+"n")
  • 6. The Cache Feeder if line.startswith("MISS "): filepath = line[5:].strip() uri, extra = filepath.split("/<>") req = urllib2.Request(zoperoot+urllib.quote(uri), None, {'Cookie': 'I18N_LANGUAGE="%s"'%extra}) o = urllib2.urlopen(req) data = o.read() datadir = cacheroot + "/" + uri if not os.path.isdir(datadir):os.makedirs(datadir) f = open(datadir + "/.htaccess", "w") f.write("ForceType %sn"%o.info().type) f.close() f = open(cacheroot + "/" + filepath,"w") f.write(data) f.close() log("FETCH %s <> %sn"%(filepath,extra))
  • 7. The Cache Itself admin@cache:~/cache$ find -type f | head ./services/locations-and-contacts/.htaccess ./services/locations-and-contacts/<> ./services/locations-and-contacts/<>fr ./services/locations-and-contacts/<>de ./capabilities-1/navigation-services-1/.htaccess ./capabilities-1/navigation-services-1/<> ./services/index_html/rss.gif/.htaccess ./services/index_html/rss.gif/<>fr
  • 8. I swear the code is simple ! admin@cache:~$ wc -l rpx.py 74 rpx.py
  • 9. Results Watch to the lovely pictures !
  • 10.
  • 11.
  • 12.
  • 13.
  • 14. In the unlikely event you want to know more about this really dirty hack ... ● Jérôme Petazzoni <jerome.petazzoni@pilotsystems.net>