Benchmark OpenERP : testez les performances et la robustesse d'OpenERP sur vos volumes de données. Le benchmark technique d'OpenERP propose des scripts automatiques pour tester la charge d'OpenERP vis-à-vis de vos données et de votre activité.

1. OPENERP BENCHMARK
How to test performance and
robustness against your own volumes ?
Cyril MORISSE ( @cmorisse )

2. Introduction
 At Audaxis, we decided to invest in a methodology
to answer 3 kinds of questions:
 Is OpenERP able to manage my business
 Is this hardware configuration able to sustain our
operations load ? Until when ?
 What hardware should we buy to run OpenERP for the
next 3 years ?

3. What to measure ?
 You must find the most challenging business processes and
conditions then express them in terms of what you can measure
 Eg. Can OpenERP handle 240.000 orders per years with 40 users ?
 You can’t benchmark on this base
 240.000 orders per year is 2,5 orders per hour per user (on a 240
working days of 10 hours per year base)
 Note: that this is one order every 36s too !
 What you can measure is:
 Can my system maintains response time < 1 s per transaction over 10
hours with :
 40 users entering 1 order every 16 minutes during 10 (peak hours intensity)
 That’s 1 order every 24s

4. What to measure (continued)
 What does «good response time» technically means ?
 JSON-RPC requests response time must be kept around
1s for the system to be felt as responsive.
 Funkload measure responsiveness for you using APDEX
OpenERP
Database
OpenERP
Application
Server
User OpenERP
Web Client
Enter fields,
Press
buttons
JSON-RPC SQL
Responsiveness
1 Sale Order with 5 lines, “Confirmed” then “Invoiced”
~ 20 ~ 150 ~ 6700

5. How to measure it
 Use Funkload
 Develop a test case which simulates user interactions (via XML-RPC…)
 Install « Funkload monitor » on your servers
 Define and run Funkload benchmarks with:
 A test case
 List of cycle defining the number of concurrent users
 Eg. 100,100,100,100
 Cycle duration
 Frequency (as a period : one test every x seconds)
 You will get a (quite) fancy report with:
 Test execution time
 Individual request execution time
 Main server metrics (CPU, RAM, Network Load)

6. Funkload Sample Report (continued)

7. Funkload Sample Report (continued)

8. Funkload Sample Report (end)

9. Database size really matters
 You must bench with real volume of data !!!
0
1
2
3
4
5
6
7
8
0 100000 200000 300000 400000 500000
OpenERP Response Time evolution over Sales Order
count
Response Time (s)

10. Database size really matters(continued)
 In fact, you will need several benchmarks:
 Bench-filler
 Use these benchmarks to fill up your database (customers,
products, …)
 Bench-reference
 2 hours with target activity level
 Bench-robustness
 10 (several) hours with target activity level
 Bench-limit
 We run this one at the end of the bench to determine how many
users more the system will support at target activity level

11. Plan your architecture
but focus on the database
 Configure your database ! At least use pgtune !!!
 ORM may be your enemy ; think about the roundtrips involved by your code
 Don’t split Application and Database servers too early !
 If your database is not too big : think SSD (From 20x to 250x more IOPS than HDD)*
OpenERP Database
OpenERP
Application Server
OpenERP
Web Client
JSON-RPC SQL
Horizontal Scaling
VerticalScaling

12. Tools & References
 Funkload ( http://funkload.nuxeo.org )
 Benchmark (and Test) your OpenERP Server and others
 Muppy ( http://bitbucket.org/cmorisse/muppy )
 “Make your OpenERP Servers behave as puppets”
 Install OpenERP platforms everywhere (in really alpha stage)
 Pgbadger ( http://dalibo.github.io/pgbadger )
 Analyse SQL requests processed by your PostgreSQL Server
 Pgtune ( https://github.com/gregs1104/pgtune )
 Helps you to configure your PostgreSQL (basically)
 Pg_activity (https://github.com/julmon/pg_activity )
 Top like application for PostgreSQL Server
 OpenERP buildout Recipe ( https://pypi.python.org/pypi/anybox.recipe.openerp )
 A set of Ugly scripts to automate the launch of Funkload Filler benchmarks
 Theses tools allows to measure performance over database size
 http://www.apdex.org
 SQL Databases Don’t Scale by Adam WIGGINS
 http://adam.heroku.com/past/2009/7/6/sql_databases_dont_scale/
 HDD vs SSD by IBM
 http://www-03.ibm.com/systems/power/hardware/peripherals/ssd/hdd_ssd.html

13. More information?
www.audaxis.com
Contact us
openerp@audaxis.com
13
Visit our booth at OpenERP Community Days in Hall K