Overview of performance profiling and testing instruments, we use at OroCommerce

1. Performance profiling and
testing of a Symfony
application

3. About me
● Developer Advocate
● Release Manager
● Trainer
● Team Leader
● Software Engineer
at

4. Performance profiling
Finding bottlenecks in the code to make it work faster

5. Performance testing
- The process of evaluating the quality or capacity of a
product.
- Performance testing is vital in the software development
lifecycle.

6. The goal
To measure code quality

7. Profiling without testing
- Possible only when your project will not be changed.
- If the code is left tested, even one line can break
everything with the damage discovered by clients already
in production.

8. When do we need profiling?
- The website is down because of overload
- Website load takes more than 2 seconds (performance is
a mandatory feature of any website)
- How One Second Could Cost Amazon $1.6 Billion In
Sales

9. When we don’t need profiling?
- The customer said there would have a significant load in production without
giving any proof.
- Premature optimization is almost always a bad idea.
This doesn’t mean you should use $repo->findAll()->paginate(),
This means: don’t do profiling when there are no performance issues.

10. Step #1
At ORO we started with:
- A time limit for Behat tests
- A network tab in the Chrome Dev Toolbar

11. Issue #1
Time is a good but unreliable metric.
How to know when time goes up because of:
- VM load
- CPU throttling
- Network issues
- Unreliable external API
- Or bad code ?

12. If the website has slowed down, you can blame:
- A software developer
- A system administrator
- The Hardware
Blame someone else but don’t fix it

13. How to blame the PHP code?
Don’t measure the time, measure the code quality.
The quality from performance perspective, not OOD

14. Why is a PHP application slow?
- PHP
- PHP 7 twice as fast as the same code on PHP 5.6
- Framework
- Symfony 4 twice as fast as the same code on Symfony 3.4

15. because of
SQL

16. Step #2
Let’s measure bad SQL with Symfony toolbar extension
- Hydration time
- Slow with a lot of joins
- Queries count
- Duplicate queries count
- same queries with different params
- Identical queries count
- Same queries
- Performance tab
- time based

17. Performance tab metrics

18. Extend performance tab metrics

19. Extend performance tab metrics
@debug.stopwatch

20. Step #2.1
Repeat the same for all network operations if they affect
performance:
- External API calls count
- Redis cache requests count
- Filesystem access count
- Memory usage by application or better the method
- etc. count

21. Issue #2
Developers don’t check Toolbar

22. Step #3
Integrate query metrics with test framework

23. Test query metrics only on real or real-life data.

24. Issue #3
Now PHP and Framework usage is a bottleneck

25. Symfony Toolbar Main Issue
Symfony developer toolbar adds overhead by itself and
can’t be used in production
So it can’t be used for real precise profiling

26. Step #4
Choose the tool for profiling PHP code.
- xDebug
- good start, but it’s not a profiler
- Xhprof
- very basic functionality, only time and memory
- Blackfire
- fork of Xhprof, widely used, a lot of integrations
- free plan has all xhprof features in a fancy interface, paid plans bring a lot of features
- Tideways
- fork of Xhprof
- Very similar to blackfire by the feature list but not so common

27. Blackfire pros
- Low overhead *
- Can be used in production
- A lot of ways to use, from CURL to Player
- Profile comparing
- Sharing
- Cross platform support
- Testing by all the metrics
- Periodic builds on production
- Profile every N request on overloaded application

28. Blackfire cons
- Most of features only available with paid plans.
- The big overhead on new PHP versions.
- Interface not always intuitive.
- *Requires prod-like environment.

29. Blackfire graph view

30. Blackfire Timeline View

31. Blackfire Recorder

32. Blackfire Metrics

33. Defining custom metrics at Blackfire

34. Blackfire Tests
Can replace some generic Symfony profiler based tests

35. Blackfire SDK

36. PhpUnit integration
BlackfireBridgePhpUnitTestCaseTrait;

37. Behat Integration

38. Validate environment configuration
- bin/symfony_requirements
- bin/console doctrine:ensure-production-settings
- bin/console oro:check-requirements
- Blackfire configs

39. What else?

40. Xdebug extension to check Garbage Collector
efficiency

41. Logging
Monolog Processors to enrich log messages
● Elapsed memory
● Current memory
● Peak memory
● Time taken
● Channels

42. Reach log message

43. Collecting Logs

44. Logs Monitoring

45. To summarize
Metrics can be used for:
- Local profiling
- CI tests
- Production testing
- Load testing

46. To summarize
To make a developer follow metrics:
- Make every metric explicit and as small as possible
- Describe the value of all the metrics and how to fix them
- Don’t rely on time or other unstable metrics
- Don’t add tests to common metrics that you don’t know
how to fix

47. To summarize
- Symfony Toolbar and Blackfire are best friends of
profiling
- xDebug could help too but be carefull
- Logging helps to “profile” in production

48. Questions