A talk I gave at WordCamp Sofa 2016 on measuring and optimizing memory usage, dealing with memory related errors, as well as monitoring server memory health.

1. Keep Your Memory
Under Control
Konstantin Kovshenin
konstantin.blog

3. Fast

4. Fast
Expensive

5. Fast
Expensive
Never enough

6. Memory Usage
free
top, htop, ...

7. free

8. free

9. top

10. top

11. PHP / WordPress

12. Zend Memory Manager

13. Zend Memory Manager
OS Kernel
foo.php
$a = 5;
malloc, mmap, ...

14. memory_get_usage()
memory_get_peak_usage()
Debug Bar, Query Monitor
Memory Usage

15. $before = memory_get_usage();
// Do something
$after = memory_get_usage();
echo $after - $before;

16. Environment loaded
Init action
Shutdown action
12.9 MB
14.1 MB
14.4 MB

17. 12.9 MB
14.1 MB
14.4 MB
12.9 MB
27.3 MB
28.6 MB
Environment loaded
Init action
Shutdown action

18. Fatal error: Allowed memory size
of 134217728 bytes exhausted

19. ... tried to allocate 8552 bytes in
/path/to/wp-includes/wp-db.php
on line 1775

20. ... tried to allocate 8552 bytes in
/path/to/wp-includes/wp-db.php
on line 1775
1775: while ( $row = mysqli_fetch_object( ...

21. ... tried to allocate 8552 bytes in
/path/to/wp-includes/wp-db.php
on line 1775
1775: while ( $row = mysqli_fetch_object( ...

22. Google...

23. ini_set( 'memory_limit', '2048M' );

25. Stop

26. ini_set( 'memory_limit', '2048M' );

27. 

script.php
script.php

28. 

Zend
Memory
Manager
script.php
2GB
2GB
script.php

29. 

Zend
Memory
Manager
Kernel
script.php
2GB
2GB
2x2GB
script.php

30. OOM Killer

31. OOM Killer
mysqld
nginx
sshd
postfix
fail2ban

32. 1. Fine-tune the OOM

33. 1. Fine-tune the OOM
2. Add more Swap memory

34. 1. Fine-tune the OOM
2. Add more Swap memory
3. Figure things out

35. ... tried to allocate 8552 bytes in
/path/to/wp-includes/wp-db.php
on line 1775

36. xdebug
xhprof, tideways
memprof
Memory Profiling

37. Call Stack:
34032184 6. do_action()
.../htdocs/wp-settings.php:393
34925000 7. call_user_func_array:{}()
.../wp-includes/plugin.php:525
34925104 8. my_plugin_init()
.../wp-includes/plugin.php:525
134139664 9. wpdb->query()
.../wp-content/plugins/memory.php:25
134172496 10. mysqli_fetch_object()
.../wp-includes/wp-db.php:1775

38. XDEBUG_TRACE
tracefile-analyser.php
Memory Profiling

39. vim trace.xt

40. Why?

41. posts_per_page => -1

42. // Pretty bad
$posts = get_posts( array(
'posts_per_page' => -1,
// ...
) );
foreach ( $posts as $post ) {
// ...
}

43. // A little better
$paged = 1;
while ( $posts = get_posts( array(
'posts_per_page' => 100,
'paged' => $paged++,
) ) ) {
foreach ( $posts as $post ) {
// ...
}
}

44. // Good
wp_suspend_cache_addition( true );
$paged = 1;
while ( $posts = get_posts( array(
'posts_per_page' => 100,
'paged' => $paged++,
) ) ) {
// foreach ...
}
wp_suspend_cache_addition( false );

45. file_get_contents()

46. readfile()
fopen/fread/feof
X-Accel-Redirect

47. ob_start()

48. // Not so good
ob_start( 'ob_callback' );
// Much better
ob_start( 'ob_callback', 1024*1024*2 );
function ob_callback( $content, $phase ) {
// ...
}

49. Context

50. is_admin()
is_network_admin()
is_front_page()
is_singular()
...
template_redirect
wp_head
wp_footer
shutdown
cron
Context

51. Monitoring
top, free
cat /proc/meminfo
cat /proc/pid/{status,maps}

52. add_action( 'shutdown', function() {
$line = json_encode( array(
'timestamp' => time(),
'request_uri' => $_SERVER['REQUEST_URI'],
'peak_memory' => memory_get_peak_usage(),
) );
file_put_contents( '/tmp/php-memory.log',
$line . PHP_EOL, FILE_APPEND | LOCK_EX );
});

53. Kibana
Elasticsearch
Logstash
php-memory.log

54. Munin

55. 1. Choose the right context

56. 2. Profile with large datasets

57. 3. Keep consumption
under control

58. konstantin.blog/wcsof2016

59. Attributions:
Success Kid (Sammy Griner): https://en.wikipedia.org/wiki/Success_Kid
Ninja character: http://www.vecteezy.com/vector-art/88129-ninja-vector-set
Blood spatter CC-Attr-ShareALike 4.0: http://www.freevector.com/splattered-blood-graphics-set
User icons by FontAwesome, SIL OFL 1.1: http://fontawesome.io