Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Node.js memory limitations

181 Aufrufe

Veröffentlicht am

Node.js has memory limitations that you can hit quite easily in production. You'll know this if you ever tried to load a large data file into your Node.js application.

But where exactly are the limits of memory in Node.js? In this short talk we'll push Node.js to it's limits and find out where those limits are. We'll also cover some practical techniques you can use to work around the memory limitations and get your data to fit into memory.

A talk by Ashley Davis for the Brisbane JavaScript meetup.

To see blog post and video relating to these slides please go to The Data Wrangler:


Veröffentlicht in: Software
  • Als Erste(r) kommentieren

Node.js memory limitations

  1. 1. Node.js memory limitations How to blow up your app in 100 easy steps
  2. 2. Out of memory! <--- Last few GCs ---> [43108:00000272EA114EF0] 2209 ms: Mark-sweep 1391.0 (1399.7) -> 1390.9 (1396.7) MB, 224.2 / 0.0 ms last resort GC in old space requested [43108:00000272EA114EF0] 2460 ms: Mark-sweep 1390.9 (1396.7) -> 1390.9 (1396.7) MB, 251.0 / 0.0 ms last resort GC in old space requested ==== JS stack trace ========================================= Security context: 00000189F87A5EE1 <JSObject> 1: alloc [C:DropboxProjectsJSDWGithubnodejs-memory-testindex.js:~9] [pc=000000A10D205D61](this=0000006F6330BF99 <JSGlobal Object>,size=10240000) 3: allocToMax [C:DropboxProjectsJSDWGithubnodejs-memory-testindex.js:41] [bytecode=00000142282CFFB1 offset=160](this=0000006F6330BF99 <JSGlobal Object>) 5: /* anonymous */ [C:DropboxProjectsJSDWGithubnodejs-memory-testind... FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory 1: node_module_register 2: v8::internal::FatalProcessOutOfMemory 3: v8::internal::FatalProcessOutOfMemory 4: v8::internal::Factory::NewUninitializedFixedArray
  3. 3. Data Wrangling with JavaScript
  4. 4. Why?
  5. 5. Why? ● Loading too much data ● Memory leak
  6. 6. Node.js memory architecture ● Code ● Stack ● Heap
  7. 7. Node.js heap ● New space ● Old space
  8. 8. Allocate unto death while (true) { Allocate some memory const mu = process.memoryUsage(); Print memory allocated }
  9. 9. Node.js memory architecture Allocated since start 1.26 GB Allocated since start 1.27 GB Allocated since start 1.28 GB Allocated since start 1.29 GB Allocated since start 1.3 GB Allocated since start 1.31 GB Allocated since start 1.32 GB Allocated since start 1.33 GB Allocated since start 1.34 GB Allocated since start 1.35 GB Allocated since start 1.35 GB
  10. 10. Live demo Who wants to see me blow up Node.js? (if time)
  11. 11. The dirty hack node --max-old-space-size=6000 index.js
  12. 12. Memory to burn Allocated since start 5.74 GB Allocated since start 5.75 GB Allocated since start 5.76 GB Allocated since start 5.77 GB Allocated since start 5.78 GB Allocated since start 5.79 GB Allocated since start 5.8 GB Allocated since start 5.81 GB Allocated since start 5.82 GB Allocated since start 5.83 GB Allocated since start 5.84 GB Allocated since start 5.85 GB
  13. 13. How far? node --max-old-space-size=??? index.js
  14. 14. Fitting more in memory ● Don’t load your entire data set ● Avoid memory leaks ● Monitor memory usage ● Separate processes
  15. 15. Resources / Q&A Memory limit testing code: https://github.com/Data-Wrangling-with-JavaScript/nodejs-memory-test Interesting articles: https://blog.codeship.com/understanding-garbage-collection-in-node-js/ http://jayconrod.com/posts/55/a-tour-of-v8-garbage-collection https://blog.risingstack.com/finding-a-memory-leak-in-node-js/