This is a run down of the mistakes we made and the lessons we learnt trying to scale a big Plone site with un-anticipated traffic once the site had gone live and the budget all been spent. Find out what we did wrong, and how we fixed it.

2. Matt Hamilton
&
Matt Sital-Singh
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

3. 0
Plone Conference 2011
San Francisco
Background
Matt Hamilton
Matt Sital-Singh

4. Use Case
• Learning Management-type system
• A lot of content
• Users ‘editing’ content, but not necessarily
aware of it
• Lots of ‘clustering’ of load on resources
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

5. Scale
• Plone 3
• User accounts: 30,000
• Messages: 160,000
• Course questions: 260,000
• Other: 60,000
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

6. 1 Don’t Create Ghost
Content
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

7. • User perception of speed
• If they expect a ‘read’ they want a ‘read’
- user signups
- course starting
• Will accept a delay if you are actually
processing something
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

8. 2 Use Optimization
Products
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

9. • experimental.catalogqueryplan
• experimental.contentcreation
• experimental.daterangeindexoptimisations
• experimental.aggressiveopaquespeedup
• collective.indexing
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

10. 3 One Plone site or
Multiple?
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

11. • One Plone site per ‘site’
• Multiple ‘sites’ per Plone site
• Common content?
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

12. 4 Large Catalogs are
Slow
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

13. • We had 260,000 questions
• each had 2-4 answers
• = 520,000 - 1,400,000 answers
All in portal_catalog!
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

14. 5
Optimize your Catalogs
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

15. • Pre-load Catalog Queryplan
• Balance your BTrees
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

16. effective date index:
1308585257
1319126057
1319730857
1319817170
1319817198
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

17. 1308585257
1319126057
1319730857
1319817170
1319817198
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

18. • Used catalog multiplexer to assign ‘Answer’
content type to dummy catalog
• Moved messages to separate
message_catalog
• Monkey-patch items out of the catalog?
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

19. 6 If You Know Where
Something Is - Don’t
Search for it
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

20. • Got into bad habit of using the catalog
ALL the time
• Don’t just blindly use catalog when
unrestrictedTraverse / contentValues will
do
• RAM is cheap, user patience is NOT
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

21. 7 Use
unrestrictedSearchResults
if possible
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

22. • unrestrictedSearchResults does not check:
• effective index
• allowed_roles_and_users index
➡ A lot less work for the catalog to do
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

23. 8Is Plone the Right Tool?
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

24. • “When you have a hammer, everything
looks like a nail...”
• A lot of this data is relational...
• Not much of it is CMSish
• Maybe should have used Pyramid?
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

25. 9 Use Event Driven
Workflow
(Don’t recalculate)
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

26. • User rating / Course ranking
• Use workflow event subscribers, don’t re-
calculate on view every time
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

27. 10 Some things don’t need
to be immediate
Plone Conference 2011
San Francisco
Matt Hamilton
Matt Sital-Singh

28. • “You have 10 new messages”
• Not everything has to be updated all the
time in every location on the site
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh

29. Thanks!
Plone Conference 2011 Matt Hamilton
San Francisco Matt Sital-Singh