SlideShare ist ein Scribd-Unternehmen logo

Growing pains - PosKeyErrors and other malaises

As a Plone project grows and changes it experiences growing pains. I will discuss some strategies to prevent and reduce these issues and treatments to cure them if your project is already infected.

1 von 36
Downloaden Sie, um offline zu lesen
GROWING PAINS
PosKeyErrors and other malaises
Philip Bauer - starzel.de - Plone Conference 2020
SYMPTOM 1: HUGE DATABASE
SYMPTOM 1: HUGE DATABASE
➤ Cause 1: Revisions / Versions
➤ Remedies
➤ Remove all versions and pack
hs = api.portal.get_tool('portal_historiesstorage')
zvcr = hs.zvc_repo
zvcr._histories.clear()
storage = hs._shadowStorage
storage._storage.clear()
➤ Manage/limit revisions. Use collective.revisionmanager
➤ Disable versioning of files
➤ Enable manual versioning instead of automatic
SYMPTOM 1: HUGE DATABASE
➤ Cause 2: No packing
➤ Remedies
➤ Pack!
➤ Use the script zeopack which is part of plone.recipe.zeoserver
➤ Add a cronjob for it
SYMPTOM 1: HUGE DATABASE
➤ Cause 3: Unused Content
➤ Remedies
➤ Delete it
➤ Find it first.
➤ You could use statistics.py from collective.migrationhelpers
SYMPTOM 1: HUGE DATABASE
➤ Cause 4: SearchableText is huge
➤ Remedies
➤ Use solr or elasticsearch and remove SearchableText index
➤ Don't index files

Recomendados

Anatomy of a PHP Request ( UTOSC 2010 )
Anatomy of a PHP Request ( UTOSC 2010 )Anatomy of a PHP Request ( UTOSC 2010 )
Anatomy of a PHP Request ( UTOSC 2010 )Joseph Scott
 
PuppetConf 2016: Device-Based Modules: Making Them as Simple as a Light Switc...
PuppetConf 2016: Device-Based Modules: Making Them as Simple as a Light Switc...PuppetConf 2016: Device-Based Modules: Making Them as Simple as a Light Switc...
PuppetConf 2016: Device-Based Modules: Making Them as Simple as a Light Switc...Puppet
 
PHP SA 2014 - Releasing Your Open Source Project
PHP SA 2014 - Releasing Your Open Source ProjectPHP SA 2014 - Releasing Your Open Source Project
PHP SA 2014 - Releasing Your Open Source Projectxsist10
 
Philipp Von Weitershausen Plone Age Mammoths, Sabers And Caveen Cant The...
Philipp Von Weitershausen   Plone Age  Mammoths, Sabers And Caveen   Cant The...Philipp Von Weitershausen   Plone Age  Mammoths, Sabers And Caveen   Cant The...
Philipp Von Weitershausen Plone Age Mammoths, Sabers And Caveen Cant The...Vincenzo Barone
 
The why and how of moving to php 5.4
The why and how of moving to php 5.4The why and how of moving to php 5.4
The why and how of moving to php 5.4Wim Godden
 
David Rey Lessons Learned Updating Content Licensing To Be Plone 3 Compat...
David Rey   Lessons Learned   Updating Content Licensing To Be Plone 3 Compat...David Rey   Lessons Learned   Updating Content Licensing To Be Plone 3 Compat...
David Rey Lessons Learned Updating Content Licensing To Be Plone 3 Compat...Vincenzo Barone
 
When dynamic becomes static: the next step in web caching techniques
When dynamic becomes static: the next step in web caching techniquesWhen dynamic becomes static: the next step in web caching techniques
When dynamic becomes static: the next step in web caching techniquesWim Godden
 

Más contenido relacionado

Was ist angesagt?

Advanced Debugging with WinDbg and SOS
Advanced Debugging with WinDbg and SOSAdvanced Debugging with WinDbg and SOS
Advanced Debugging with WinDbg and SOSSasha Goldshtein
 
톰캣 #05-배치
톰캣 #05-배치톰캣 #05-배치
톰캣 #05-배치GyuSeok Lee
 
Sql Injections With Real Life Scenarious
Sql Injections With Real Life ScenariousSql Injections With Real Life Scenarious
Sql Injections With Real Life ScenariousFrancis Alexander
 
The why and how of moving to php 5.4/5.5
The why and how of moving to php 5.4/5.5The why and how of moving to php 5.4/5.5
The why and how of moving to php 5.4/5.5Wim Godden
 
A new way to develop with WordPress!
A new way to develop with WordPress!A new way to develop with WordPress!
A new way to develop with WordPress!David Sanchez
 
Introduction to PhoneGap and PhoneGap Build
Introduction to PhoneGap and PhoneGap BuildIntroduction to PhoneGap and PhoneGap Build
Introduction to PhoneGap and PhoneGap BuildMartin de Keijzer
 
Generic Setup De-Mystified
Generic Setup De-MystifiedGeneric Setup De-Mystified
Generic Setup De-MystifiedClayton Parker
 
Unit tests for dummies
Unit tests for dummiesUnit tests for dummies
Unit tests for dummiescpsitgmbh
 
11 tools for your PHP devops stack
11 tools for your PHP devops stack11 tools for your PHP devops stack
11 tools for your PHP devops stackKris Buytaert
 
PHPUnit Episode iv.iii: Return of the tests
PHPUnit Episode iv.iii: Return of the testsPHPUnit Episode iv.iii: Return of the tests
PHPUnit Episode iv.iii: Return of the testsMichelangelo van Dam
 
PHP 5.6 New and Deprecated Features
PHP 5.6  New and Deprecated FeaturesPHP 5.6  New and Deprecated Features
PHP 5.6 New and Deprecated FeaturesMark Niebergall
 
Troubleshooting Plone
Troubleshooting PloneTroubleshooting Plone
Troubleshooting PloneRicado Alves
 
Экспресс-анализ вредоносов / Crowdsourced Malware Triage
Экспресс-анализ вредоносов / Crowdsourced Malware TriageЭкспресс-анализ вредоносов / Crowdsourced Malware Triage
Экспресс-анализ вредоносов / Crowdsourced Malware TriagePositive Hack Days
 
Debugging, Monitoring and Profiling in TYPO3
Debugging, Monitoring and Profiling in TYPO3Debugging, Monitoring and Profiling in TYPO3
Debugging, Monitoring and Profiling in TYPO3AOE
 
How not to develop with Plone
How not to develop with PloneHow not to develop with Plone
How not to develop with PloneLennart Regebro
 
Even Faster Web Sites at jQuery Conference '09
Even Faster Web Sites at jQuery Conference '09Even Faster Web Sites at jQuery Conference '09
Even Faster Web Sites at jQuery Conference '09Steve Souders
 

Was ist angesagt? (20)

Advanced Debugging with WinDbg and SOS
Advanced Debugging with WinDbg and SOSAdvanced Debugging with WinDbg and SOS
Advanced Debugging with WinDbg and SOS
 
톰캣 #05-배치
톰캣 #05-배치톰캣 #05-배치
톰캣 #05-배치
 
Sql Injections With Real Life Scenarious
Sql Injections With Real Life ScenariousSql Injections With Real Life Scenarious
Sql Injections With Real Life Scenarious
 
The why and how of moving to php 5.4/5.5
The why and how of moving to php 5.4/5.5The why and how of moving to php 5.4/5.5
The why and how of moving to php 5.4/5.5
 
A new way to develop with WordPress!
A new way to develop with WordPress!A new way to develop with WordPress!
A new way to develop with WordPress!
 
Introduction to PhoneGap and PhoneGap Build
Introduction to PhoneGap and PhoneGap BuildIntroduction to PhoneGap and PhoneGap Build
Introduction to PhoneGap and PhoneGap Build
 
Generic Setup De-Mystified
Generic Setup De-MystifiedGeneric Setup De-Mystified
Generic Setup De-Mystified
 
Unit tests for dummies
Unit tests for dummiesUnit tests for dummies
Unit tests for dummies
 
11 tools for your PHP devops stack
11 tools for your PHP devops stack11 tools for your PHP devops stack
11 tools for your PHP devops stack
 
PHPUnit Episode iv.iii: Return of the tests
PHPUnit Episode iv.iii: Return of the testsPHPUnit Episode iv.iii: Return of the tests
PHPUnit Episode iv.iii: Return of the tests
 
TYPO3 Scheduler
TYPO3 SchedulerTYPO3 Scheduler
TYPO3 Scheduler
 
PHP 5.6 New and Deprecated Features
PHP 5.6  New and Deprecated FeaturesPHP 5.6  New and Deprecated Features
PHP 5.6 New and Deprecated Features
 
Troubleshooting Plone
Troubleshooting PloneTroubleshooting Plone
Troubleshooting Plone
 
Deploy Flex with Apache Ant
Deploy Flex with Apache AntDeploy Flex with Apache Ant
Deploy Flex with Apache Ant
 
Экспресс-анализ вредоносов / Crowdsourced Malware Triage
Экспресс-анализ вредоносов / Crowdsourced Malware TriageЭкспресс-анализ вредоносов / Crowdsourced Malware Triage
Экспресс-анализ вредоносов / Crowdsourced Malware Triage
 
Debugging, Monitoring and Profiling in TYPO3
Debugging, Monitoring and Profiling in TYPO3Debugging, Monitoring and Profiling in TYPO3
Debugging, Monitoring and Profiling in TYPO3
 
Kyiv.py #17 Flask talk
Kyiv.py #17 Flask talkKyiv.py #17 Flask talk
Kyiv.py #17 Flask talk
 
How not to develop with Plone
How not to develop with PloneHow not to develop with Plone
How not to develop with Plone
 
Ant Build Tool
Ant Build ToolAnt Build Tool
Ant Build Tool
 
Even Faster Web Sites at jQuery Conference '09
Even Faster Web Sites at jQuery Conference '09Even Faster Web Sites at jQuery Conference '09
Even Faster Web Sites at jQuery Conference '09
 

Ähnlich wie Growing pains - PosKeyErrors and other malaises

When Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
When Good Code Goes Bad: Tools and Techniques for Troubleshooting PloneWhen Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
When Good Code Goes Bad: Tools and Techniques for Troubleshooting PloneDavid Glick
 
PuppetConf 2016: Getting to the Latest Puppet – Nate McCurdy & Elizabeth Witt...
PuppetConf 2016: Getting to the Latest Puppet – Nate McCurdy & Elizabeth Witt...PuppetConf 2016: Getting to the Latest Puppet – Nate McCurdy & Elizabeth Witt...
PuppetConf 2016: Getting to the Latest Puppet – Nate McCurdy & Elizabeth Witt...Puppet
 
Operating Docker
Operating DockerOperating Docker
Operating DockerJen Andre
 
Martin Aspeli Extending And Customising Plone 3
Martin Aspeli   Extending And Customising Plone 3Martin Aspeli   Extending And Customising Plone 3
Martin Aspeli Extending And Customising Plone 3Vincenzo Barone
 
Cloud Foundry Summit 2015: 10 common errors when pushing apps to cloud foundry
Cloud Foundry Summit 2015: 10 common errors when pushing apps to cloud foundryCloud Foundry Summit 2015: 10 common errors when pushing apps to cloud foundry
Cloud Foundry Summit 2015: 10 common errors when pushing apps to cloud foundryJack-Junjie Cai
 
Pwning mobile apps without root or jailbreak
Pwning mobile apps without root or jailbreakPwning mobile apps without root or jailbreak
Pwning mobile apps without root or jailbreakAbraham Aranguren
 
PuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of PuppetPuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of PuppetWalter Heck
 
PuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of PuppetPuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of PuppetOlinData
 
Fix me if you can - DrupalCon prague
Fix me if you can - DrupalCon pragueFix me if you can - DrupalCon prague
Fix me if you can - DrupalCon praguehernanibf
 
PuppetConf 2016: Puppet 4.x: The Low WAT-tage Edition – Nick Fagerlund, Puppet
PuppetConf 2016: Puppet 4.x: The Low WAT-tage Edition – Nick Fagerlund, PuppetPuppetConf 2016: Puppet 4.x: The Low WAT-tage Edition – Nick Fagerlund, Puppet
PuppetConf 2016: Puppet 4.x: The Low WAT-tage Edition – Nick Fagerlund, PuppetPuppet
 
Drupal, lessons learnt from real world security incidents
Drupal, lessons learnt from real world security incidentsDrupal, lessons learnt from real world security incidents
Drupal, lessons learnt from real world security incidentssydneydrupal
 
Badge Poser v3.0 - A DevOps Journey
Badge Poser v3.0 - A DevOps JourneyBadge Poser v3.0 - A DevOps Journey
Badge Poser v3.0 - A DevOps JourneyFabio Cicerchia
 
macOS Vulnerabilities Hiding in Plain Sight
macOS Vulnerabilities Hiding in Plain SightmacOS Vulnerabilities Hiding in Plain Sight
macOS Vulnerabilities Hiding in Plain SightCsaba Fitzl
 
Magento Imagine 2011 - Magento Debugging - Erik Hansen, Classy Llama Studios
Magento Imagine 2011 - Magento Debugging - Erik Hansen, Classy Llama StudiosMagento Imagine 2011 - Magento Debugging - Erik Hansen, Classy Llama Studios
Magento Imagine 2011 - Magento Debugging - Erik Hansen, Classy Llama StudiosErik Hansen
 
Neo4j Stored Procedure Training Part 1
Neo4j Stored Procedure Training Part 1Neo4j Stored Procedure Training Part 1
Neo4j Stored Procedure Training Part 1Max De Marzi
 
Dolibarr - information for developers and partners - devcamp lyon 2019
Dolibarr - information for developers and partners - devcamp lyon 2019Dolibarr - information for developers and partners - devcamp lyon 2019
Dolibarr - information for developers and partners - devcamp lyon 2019Laurent Destailleur
 
1192020 Tutorial 3httpslinuxzoo.netpagewildcards_tu.docx
1192020 Tutorial 3httpslinuxzoo.netpagewildcards_tu.docx1192020 Tutorial 3httpslinuxzoo.netpagewildcards_tu.docx
1192020 Tutorial 3httpslinuxzoo.netpagewildcards_tu.docxaulasnilda
 
Where's the source, Luke? : How to find and debug the code behind Plone
Where's the source, Luke? : How to find and debug the code behind PloneWhere's the source, Luke? : How to find and debug the code behind Plone
Where's the source, Luke? : How to find and debug the code behind PloneVincenzo Barone
 

Ähnlich wie Growing pains - PosKeyErrors and other malaises (20)

When Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
When Good Code Goes Bad: Tools and Techniques for Troubleshooting PloneWhen Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
When Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
 
PuppetConf 2016: Getting to the Latest Puppet – Nate McCurdy & Elizabeth Witt...
PuppetConf 2016: Getting to the Latest Puppet – Nate McCurdy & Elizabeth Witt...PuppetConf 2016: Getting to the Latest Puppet – Nate McCurdy & Elizabeth Witt...
PuppetConf 2016: Getting to the Latest Puppet – Nate McCurdy & Elizabeth Witt...
 
Operating Docker
Operating DockerOperating Docker
Operating Docker
 
Martin Aspeli Extending And Customising Plone 3
Martin Aspeli   Extending And Customising Plone 3Martin Aspeli   Extending And Customising Plone 3
Martin Aspeli Extending And Customising Plone 3
 
Cloud Foundry Summit 2015: 10 common errors when pushing apps to cloud foundry
Cloud Foundry Summit 2015: 10 common errors when pushing apps to cloud foundryCloud Foundry Summit 2015: 10 common errors when pushing apps to cloud foundry
Cloud Foundry Summit 2015: 10 common errors when pushing apps to cloud foundry
 
Pwning mobile apps without root or jailbreak
Pwning mobile apps without root or jailbreakPwning mobile apps without root or jailbreak
Pwning mobile apps without root or jailbreak
 
PuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of PuppetPuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of Puppet
 
PuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of PuppetPuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of Puppet
 
Fix me if you can - DrupalCon prague
Fix me if you can - DrupalCon pragueFix me if you can - DrupalCon prague
Fix me if you can - DrupalCon prague
 
PuppetConf 2016: Puppet 4.x: The Low WAT-tage Edition – Nick Fagerlund, Puppet
PuppetConf 2016: Puppet 4.x: The Low WAT-tage Edition – Nick Fagerlund, PuppetPuppetConf 2016: Puppet 4.x: The Low WAT-tage Edition – Nick Fagerlund, Puppet
PuppetConf 2016: Puppet 4.x: The Low WAT-tage Edition – Nick Fagerlund, Puppet
 
Drupal, lessons learnt from real world security incidents
Drupal, lessons learnt from real world security incidentsDrupal, lessons learnt from real world security incidents
Drupal, lessons learnt from real world security incidents
 
Badge Poser v3.0 - A DevOps Journey
Badge Poser v3.0 - A DevOps JourneyBadge Poser v3.0 - A DevOps Journey
Badge Poser v3.0 - A DevOps Journey
 
macOS Vulnerabilities Hiding in Plain Sight
macOS Vulnerabilities Hiding in Plain SightmacOS Vulnerabilities Hiding in Plain Sight
macOS Vulnerabilities Hiding in Plain Sight
 
Magento Imagine 2011 - Magento Debugging - Erik Hansen, Classy Llama Studios
Magento Imagine 2011 - Magento Debugging - Erik Hansen, Classy Llama StudiosMagento Imagine 2011 - Magento Debugging - Erik Hansen, Classy Llama Studios
Magento Imagine 2011 - Magento Debugging - Erik Hansen, Classy Llama Studios
 
Neo4j Stored Procedure Training Part 1
Neo4j Stored Procedure Training Part 1Neo4j Stored Procedure Training Part 1
Neo4j Stored Procedure Training Part 1
 
DAC
DACDAC
DAC
 
How to reset TCP/IP Stack
How to reset TCP/IP StackHow to reset TCP/IP Stack
How to reset TCP/IP Stack
 
Dolibarr - information for developers and partners - devcamp lyon 2019
Dolibarr - information for developers and partners - devcamp lyon 2019Dolibarr - information for developers and partners - devcamp lyon 2019
Dolibarr - information for developers and partners - devcamp lyon 2019
 
1192020 Tutorial 3httpslinuxzoo.netpagewildcards_tu.docx
1192020 Tutorial 3httpslinuxzoo.netpagewildcards_tu.docx1192020 Tutorial 3httpslinuxzoo.netpagewildcards_tu.docx
1192020 Tutorial 3httpslinuxzoo.netpagewildcards_tu.docx
 
Where's the source, Luke? : How to find and debug the code behind Plone
Where's the source, Luke? : How to find and debug the code behind PloneWhere's the source, Luke? : How to find and debug the code behind Plone
Where's the source, Luke? : How to find and debug the code behind Plone
 

Mehr von Philip Bauer

Migrations migrations migrations
Migrations migrations migrationsMigrations migrations migrations
Migrations migrations migrationsPhilip Bauer
 
Plone ♥︎ Python 3
Plone ♥︎ Python 3Plone ♥︎ Python 3
Plone ♥︎ Python 3Philip Bauer
 
Mosaic - The Layout Solution You Always Wanted
Mosaic - The Layout Solution You Always WantedMosaic - The Layout Solution You Always Wanted
Mosaic - The Layout Solution You Always WantedPhilip Bauer
 
Upgrade to Plone 5
Upgrade to Plone 5Upgrade to Plone 5
Upgrade to Plone 5Philip Bauer
 
Migrations, Upgrades and Relaunches
Migrations, Upgrades and RelaunchesMigrations, Upgrades and Relaunches
Migrations, Upgrades and RelaunchesPhilip Bauer
 
It's the way of the present - Why you should use plone.app.contenttypes
It's the way of the present - Why you should use plone.app.contenttypesIt's the way of the present - Why you should use plone.app.contenttypes
It's the way of the present - Why you should use plone.app.contenttypesPhilip Bauer
 
Alles in Allem. Wie man mit Deliverance existierende Inhalte oder Anwendungen...
Alles in Allem. Wie man mit Deliverance existierende Inhalte oder Anwendungen...Alles in Allem. Wie man mit Deliverance existierende Inhalte oder Anwendungen...
Alles in Allem. Wie man mit Deliverance existierende Inhalte oder Anwendungen...Philip Bauer
 
Plone-Content-Typen mit Dexterity
Plone-Content-Typen mit DexterityPlone-Content-Typen mit Dexterity
Plone-Content-Typen mit DexterityPhilip Bauer
 

Mehr von Philip Bauer (10)

Migrations migrations migrations
Migrations migrations migrationsMigrations migrations migrations
Migrations migrations migrations
 
Plone ♥︎ Python 3
Plone ♥︎ Python 3Plone ♥︎ Python 3
Plone ♥︎ Python 3
 
pdb like a pro
pdb like a propdb like a pro
pdb like a pro
 
Mosaic - The Layout Solution You Always Wanted
Mosaic - The Layout Solution You Always WantedMosaic - The Layout Solution You Always Wanted
Mosaic - The Layout Solution You Always Wanted
 
Upgrade to Plone 5
Upgrade to Plone 5Upgrade to Plone 5
Upgrade to Plone 5
 
Migrations, Upgrades and Relaunches
Migrations, Upgrades and RelaunchesMigrations, Upgrades and Relaunches
Migrations, Upgrades and Relaunches
 
Pimp my Plone
Pimp my PlonePimp my Plone
Pimp my Plone
 
It's the way of the present - Why you should use plone.app.contenttypes
It's the way of the present - Why you should use plone.app.contenttypesIt's the way of the present - Why you should use plone.app.contenttypes
It's the way of the present - Why you should use plone.app.contenttypes
 
Alles in Allem. Wie man mit Deliverance existierende Inhalte oder Anwendungen...
Alles in Allem. Wie man mit Deliverance existierende Inhalte oder Anwendungen...Alles in Allem. Wie man mit Deliverance existierende Inhalte oder Anwendungen...
Alles in Allem. Wie man mit Deliverance existierende Inhalte oder Anwendungen...
 
Plone-Content-Typen mit Dexterity
Plone-Content-Typen mit DexterityPlone-Content-Typen mit Dexterity
Plone-Content-Typen mit Dexterity
 

Último

"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...ISPMAIndia
 
killing camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdfkilling camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdfssuser82c38d
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkTimothy Spann
 
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A..."Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...ISPMAIndia
 
maximum subarray ppt for killing camp students
maximum subarray ppt for killing camp studentsmaximum subarray ppt for killing camp students
maximum subarray ppt for killing camp studentsssuser82c38d
 
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTSi-engage
 
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)GDSCNiT
 
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...emili denli
 
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...confluent
 
P1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 SmartsheetP1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 SmartsheetMatthewTHawley
 
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이ssuser82c38d
 
Essence of Requirements Engineering: Pragmatic Insights for 2024
Essence of Requirements Engineering: Pragmatic Insights for 2024Essence of Requirements Engineering: Pragmatic Insights for 2024
Essence of Requirements Engineering: Pragmatic Insights for 2024Asher Sterkin
 
sql ppt for students who preparing for sql
sql ppt for students who preparing for sqlsql ppt for students who preparing for sql
sql ppt for students who preparing for sqlbharatjanadharwarud
 
Les02 Restricting and Sorting Data using SQL.ppt
Les02 Restricting and Sorting Data using SQL.pptLes02 Restricting and Sorting Data using SQL.ppt
Les02 Restricting and Sorting Data using SQL.pptDrZeeshanBhatti
 
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdfAUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdfAutokey
 
Sql server types of joins with example.pptx
Sql server types of joins with example.pptxSql server types of joins with example.pptx
Sql server types of joins with example.pptxsameer gaikwad
 
AI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit BendigiriAI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit BendigiriISPMAIndia
 
App Builder - Hierarchical Data Apps.pptx
App Builder - Hierarchical Data Apps.pptxApp Builder - Hierarchical Data Apps.pptx
App Builder - Hierarchical Data Apps.pptxPoojitha B
 
Manual de la Mezcladora SoundCraft Notepad -12Fx
Manual de la Mezcladora SoundCraft Notepad -12FxManual de la Mezcladora SoundCraft Notepad -12Fx
Manual de la Mezcladora SoundCraft Notepad -12Fxjavierdavidvelasco17
 

Último (20)

"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
 
killing camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdfkilling camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdf
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
 
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A..."Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
 
maximum subarray ppt for killing camp students
maximum subarray ppt for killing camp studentsmaximum subarray ppt for killing camp students
maximum subarray ppt for killing camp students
 
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
 
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
 
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
 
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
 
P1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 SmartsheetP1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 Smartsheet
 
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
 
Essence of Requirements Engineering: Pragmatic Insights for 2024
Essence of Requirements Engineering: Pragmatic Insights for 2024Essence of Requirements Engineering: Pragmatic Insights for 2024
Essence of Requirements Engineering: Pragmatic Insights for 2024
 
sql ppt for students who preparing for sql
sql ppt for students who preparing for sqlsql ppt for students who preparing for sql
sql ppt for students who preparing for sql
 
Les02 Restricting and Sorting Data using SQL.ppt
Les02 Restricting and Sorting Data using SQL.pptLes02 Restricting and Sorting Data using SQL.ppt
Les02 Restricting and Sorting Data using SQL.ppt
 
Features of IETM Software -Code and Pixels
Features of IETM Software -Code and PixelsFeatures of IETM Software -Code and Pixels
Features of IETM Software -Code and Pixels
 
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdfAUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
 
Sql server types of joins with example.pptx
Sql server types of joins with example.pptxSql server types of joins with example.pptx
Sql server types of joins with example.pptx
 
AI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit BendigiriAI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit Bendigiri
 
App Builder - Hierarchical Data Apps.pptx
App Builder - Hierarchical Data Apps.pptxApp Builder - Hierarchical Data Apps.pptx
App Builder - Hierarchical Data Apps.pptx
 
Manual de la Mezcladora SoundCraft Notepad -12Fx
Manual de la Mezcladora SoundCraft Notepad -12FxManual de la Mezcladora SoundCraft Notepad -12Fx
Manual de la Mezcladora SoundCraft Notepad -12Fx
 

Growing pains - PosKeyErrors and other malaises

  • 1. GROWING PAINS PosKeyErrors and other malaises Philip Bauer - starzel.de - Plone Conference 2020
  • 2. SYMPTOM 1: HUGE DATABASE
  • 3. SYMPTOM 1: HUGE DATABASE ➤ Cause 1: Revisions / Versions ➤ Remedies ➤ Remove all versions and pack hs = api.portal.get_tool('portal_historiesstorage') zvcr = hs.zvc_repo zvcr._histories.clear() storage = hs._shadowStorage storage._storage.clear() ➤ Manage/limit revisions. Use collective.revisionmanager ➤ Disable versioning of files ➤ Enable manual versioning instead of automatic
  • 4. SYMPTOM 1: HUGE DATABASE ➤ Cause 2: No packing ➤ Remedies ➤ Pack! ➤ Use the script zeopack which is part of plone.recipe.zeoserver ➤ Add a cronjob for it
  • 5. SYMPTOM 1: HUGE DATABASE ➤ Cause 3: Unused Content ➤ Remedies ➤ Delete it ➤ Find it first. ➤ You could use statistics.py from collective.migrationhelpers
  • 6. SYMPTOM 1: HUGE DATABASE ➤ Cause 4: SearchableText is huge ➤ Remedies ➤ Use solr or elasticsearch and remove SearchableText index ➤ Don't index files
  • 7. SYMPTOM 1: HUGE DATABASE ➤ Cause 5: Large blobs ➤ Remedies ➤ Limit upload size ➤ Get stats and remove/replace too large items
  • 8. SYMPTOM 1: HUGE DATABASE ➤ Cause 6: Aborted uploads ➤ Remedies ➤ Check IAnnotations(portal).get('file_upload_map')
  • 10. SYMPTOM 2: SLOW SITE ➤ Cause 1: Unneeded full renders of content ➤ Remedies ➤ Use Python in page templates ➤ tal:define="foo context/foo" ➤ tal:define="foo python:context.foo"
  • 11. SYMPTOM 2: SLOW SITE ➤ Cause 2: Wake up many objects ➤ Remedies ➤ Always use brains and metadata ➤ Listing 3000 brains: 0.2 seconds ➤ Listing 3000 objects: 2 seconds ➤ Same for Volto when you use search-endpoint with fullobjects
  • 12. SYMPTOM 2: SLOW SITE ➤ Cause 3: No caching ➤ Remedies ➤ Switch on built-in caching ➤ Add varnish ➤ Manage zeocache ➤ Use memoize in your code
  • 13. SYMPTOM 2: SLOW SITE ➤ Cause 4: Hardware ➤ Remedies ➤ Don't be cheap ➤ Buy enough ram to keep the DB in memory
  • 14. SYMPTOM 2: SLOW SITE ➤ Cause 5: Slow code ➤ Remedies ➤ Learn and use profiling ➤ https://pypi.org/project/py-spy ➤ sudo py-spy top --pid 57425 ➤ Don't call methods multiple times from templates
  • 15. SYMPTOM 2: SLOW SITE ➤ Cause 6: Slow data sources ➤ Remedies ➤ Decouple (e.g. using redis/celery) ➤ Async ➤ Lazyload
  • 17. SYMPTOM 3: CONFLICT ERRORS* ➤ Cause 1: Conflict resolving is not enabled ➤ Remedy ➤ Add application code to the zeoserver [zeoserver] eggs = ${buildout:eggs} *) ConflictErrors happen when two transactions try to modify the same object at once. Example: transaction1 (t1) reads a obj and wants to change it but it takes a while. Meanwhile (after t1 has started) transaction2 (t2) reads the same obj und changes it quickly before t2 is done. When t1 is finally done and its change to obj are written to the database the state of obj is no longer the same as when it was read at the beginning of t1. This raises a ConflictError. If a ConflictError occurs Zope will attempt to replay a transaction up to three times.
  • 18. SYMPTOM 3: CONFLICT ERRORS ➤ Cause 2: Long running requests change data ➤ Remedies ➤ Prevent writes ➤ Do intermediate commits ➤ Prevent Crossfire ➤ Disable cronjobs and editors ➤ Async
  • 20. SYMPTOM 4: POSKEYERRORS ➤ Cause 1: Missing blobs ➤ Remedies ➤ Copy all blobs :) ➤ Use experimental.gracefulblobmissing ➤ Find and delete afflicted content
  • 24. SYMPTOM 5: BROKEN DATA ➤ Cause 1: Code to unpickle data is missing ➤ Remedies: 1. Ignore the errors 2. Fix it with a rename_dict 3. Work around with a alias_module patch 4. Find our what and where broken objects are and then fix or remove them safely
  • 25. SYMPTOM 5: BROKEN DATA ➤ Remedy 2: zodbupdate_rename_dict ➤ In setup.py: entry_points={'zodbupdate': ['renames = mypackage:rename_dict']} or in setup.cfg: [options.entry_points] zodbupdate = renames = mypackage:rename_dict ➤ In __init__.py of mypackage: iface = "zope.interface Interface" rename_dict = { "App.interfaces IPersistentExtra": iface, "App.interfaces IUndoSupport": iface, "Products.ResourceRegistries.interfaces.settings IResourceRegistriesSettings": iface} ➤ See zest.zodbupdate
  • 26. SYMPTOM 5: BROKEN DATA ➤ Remedy 3: Work around with a alias_module patch ➤ In __init__.py: from OFS.SimpleItem import SimpleItem from plone.app.upgrade.utils import alias_module class BBB(object): pass SimpleBBB = SimpleItem try: from collective.solr import interfaces except ImportError: alias_module('collective.solr.indexer.SolrIndexProcessor', BBB) try: from collective.easyslideshow.descriptors import SlideshowDescriptor except ImportError: alias_module('collective.easyslideshow.descriptors.SlideshowDescriptor', SimpleBBB)
  • 27. SYMPTOM 5: BROKEN DATA ➤ Remedy 4: Find our what and where broken objects are and then fix or remove them safely 1. Use zodbverify to get all broken objects 2. Pick one error-type at a time 3. Use zodbverify with -o <OID> -D to inspect one object and find out where that object is referenced 4. Remove or fix the object
  • 28. SYMPTOM 5: BROKEN DATA ➤ Step 1: Use zodbverify to get all broken objects ➤ Checkout zodbverify ➤ Use https://github.com/plone/zodbverify/pull/8 ➤ ./bin/zodbverify -f var/filestorage/Data.fs or ./bin/instance zodbverify
  • 29. 2020-12-08 12:26:57,602 INFO [zodbverify:48][MainThread] Done! Scanned 154351 records. Found 43 records that could not be loaded. Exceptions, how often they happened and which oids are affected: AttributeError: module 'plone.app.upgrade.atcontentypes_bbb' has no attribute 'MetadataElementPolicy': 12 0x0126 0x0127 0x0128 0x0129 0x012a 0x012b 0x012c 0x012d 0x012e 0x012f 0x0130 0x0131 ModuleNotFoundError: No module named 'fourdigits': 8 0x28030f 0x280310 0x280311 0x280312 0x280313 0x280314 0x280315 0x280316 AttributeError: module 'plone.app.upgrade.atcontentypes_bbb' has no attribute 'ElementSpec': 7 0x011f 0x0120 0x0121 0x0122 0x0123 0x0124 0x0125 ModuleNotFoundError: No module named 'Products.Archetypes': 5 0x0e00eb 0x0e00ee 0x0e00ef 0x0e00f0 0x0e00f1 ModuleNotFoundError: No module named 'Products.ATContentTypes': 4 0x0e00e9 0x0e011a 0x0e01b3 0x0e0cb3 AttributeError: module 'plone.app.event.interfaces' has no attribute 'IEventSettings': 3 0x2a712b 0x2a712c 0x2a712d AttributeError: module 'plone.app.upgrade.atcontentypes_bbb' has no attribute 'MetadataSchema': 2 0x25 0x011e ModuleNotFoundError: No module named 'plone.app.controlpanel': 1 0x0f4c2b
  • 30. SYMPTOM 5: BROKEN DATA ➤ Step 2: Pick one error-type at a time ➤ You will forget what you did! ➤ Make notes ➤ Write upgrade-steps ➤ Keep the terminal log!
  • 31. SYMPTOM 5: BROKEN DATA ➤ Step 3: Inspect one broken object ➤ ./bin/zodbverify -f var/filestorage/Data.fs -o 0x280311 -D ➤ Find out what it is ➤ Look at the error ➤ Look at the obj ➤ Look at the pickle ➤ Find out where it is ➤ See where it is referenced
  • 32. SYMPTOM 5: BROKEN DATA ➤ Step 4: Fix it ➤ Hack it away ➤ Then write a repeatable upgrade-step
  • 33. SYMPTOM 5: BROKEN DATA ➤ Step 5: Check that is is gone ➤ The broken object still exists ➤ OID should not be referenced any more ➤ Pack your database ➤ Done : )
  • 35. SYMPTOM 6: BAD CODE ➤ Unreadable Code ➤ Untested Code ➤ Unused Code ➤ Undocumented Code ➤ Unmaintained Code ➤ Complicated Code ➤ Overtly Complex Code ➤ Too Much Code
  • 36. THANK YOU * yes, we are for hire