1. WiredTiger configura/on variables -
Looking under the hood
Antonios Giannopoulos
Database Administrator – Rackspace
Percona Live Europe 2016

2. Agenda
We are going to discuss :
MongoDB WiredTiger Configura/on Variables
- What they mean?
- How to configure/change?
- Benchmarks

3. Benchmark specs
Rackspace OnMetal Cloud Server v2
12 cores
32GB RAM
2x800GB SSDs (RAID 1)
MongoDB 3.2.9
Sysbench for MongoDB by Tim Callaghan using default
se[ngs

4. Benchmark specs
Sysbench:
1) Creates 16 collec/ons 10 million documents each
2) Executes a mixed workload for 10 minutes
Load: Comple/on /me for (1)
TPS CUM: Average number of Sysbench transac/ons
TPS INT: Number of Sysbench transac/ons
Samples are collected every 10 seconds

5. WT configura/on/Files
Three ways to change WT variables:
mongod.conf: engineConfig.configString
run/me: db.adminCommand( { setParameter:…})
configura/on file(s): (WiredTiger.basecfg, WiredTiger.config)
The order of configura/on is:
WiredTiger.basecfg file
wired/ger_open configura/on string argument,
WiredTiger.config file,
WIREDTIGER_CONFIG environment variable

6. WT configura/on/mongod.conf
engineConfig:
cacheSizeGB: <number>
journalCompressor: <string>
directoryForIndexes: <boolean>
configString: <string> , <string> …
collec/onConfig:
blockCompressor: <string>
indexConfig:
prefixCompression: <boolean>
setParameter:
wiredTigerConcurrentReadTransac/ons: <number>
wiredTigerConcurrentWriteTransac/ons: <number>

7. WT configura/on/mongod.conf
engineConfig:
cacheSizeGB: <number>
journalCompressor: <string>
directoryForIndexes: <boolean>
configString: <string> , <string> …
collec/onConfig:
blockCompressor: <string>
indexConfig:
prefixCompression: <boolean>
setParameter:
wiredTigerConcurrentReadTransac/ons: <number>
wiredTigerConcurrentWriteTransac/ons: <number>

8. Journal Benchmark
Default: Snappy
Load: 10 minutes and 58 seconds
Without compression:
Load: 10 minutes and 21 seconds
Without journal:
Load: 8 minutes and 35 seconds

9. Journal (cum tps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
without compression
without journal

10. Journal (int tps)
0
200
400
600
800
1000
1200
1400
0 100 200 300 400 500 600 700
default
without compression
without journal

11. WT configura/on/mongod.conf
engineConfig:
cacheSizeGB: <number>
journalCompressor: <string>
directoryForIndexes: <boolean>
configString:<string> , <string> …
collec/onConfig:
blockCompressor: <string>
indexConfig:
prefixCompression: <boolean>
setParameter:
wiredTigerConcurrentReadTransac/ons: <number>
wiredTigerConcurrentWriteTransac/ons: <number>

12. Compression Algorithms
snappy: it gathers data up to a maximum of 32KB,
compresses it, and if compression is successful, writes the
block rounded up to the nearest 4KB.
zlib: it gathers more data and compress enough to fill a
32KB block on disk.
Configurable per collec/on:
db.createCollec/on("foo", { storageEngine: { wiredTiger:
{ configString: "block_compressor=zlib" } } })

13. blockCompressor Benchmark
Snappy:
Load: 10 minutes and 58 seconds
Zlib:
Load: 18 minutes and 21 seconds
Without compression:
Load: 10 minutes and 45 seconds

14. blockCompressor (cum tps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
snappy
zlib
no compression

15. blockCompressor (int tps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
snappy
zlib
none

16. WT configura/on/mongod.conf
engineConfig:
cacheSizeGB: <number>
journalCompressor: <string>
directoryForIndexes: <boolean>
configString:<string> , <string> …
collec/onConfig:
blockCompressor: <string>
indexConfig:
prefixCompression: <boolean>
setParameter:
wiredTigerConcurrentReadTransac/ons: <number>
wiredTigerConcurrentWriteTransac/ons: <number>

17. Prefix compression
use
used
useful
usefully
usefulness
useless
uselessly
uselessness

18. Prefix compression
use
used
useful
usefully
usefulness
useless
uselessly
uselessness
0: use
1: 0d
2: 0ful
3: 0fully
4: 0less
5: 0lessly
6: 0lessness

19. Prefix compression
use
used
useful
usefully
usefulness
useless
uselessly
uselessness
0: use
1: 0d
2: 0ful
3: 0fully
4: 0less
5: 0lessly
6: 0lessness
0: use
1: 0d
2: 0ful
3: 2ly
4: 0less
5: 4ly
6: 4ness

20. Prefix compression (load /mes)
Default: Snappy compression:
Load: 10 minutes and 58 seconds
None: Without compression:
Load: 09 minutes and 51 seconds

21. Prefix compression (cum tps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
none

22. Prefix compression (int tps)
0
200
400
600
800
1000
1200
1400
0 100 200 300 400 500 600 700
default
none

23. WT configura/on/mongod.conf
engineConfig:
cacheSizeGB: <number>
journalCompressor: <string>
directoryForIndexes: <boolean>
configString:<string> , <string> …
collec/onConfig:
blockCompressor: <string>
indexConfig:
prefixCompression: <boolean>
setParameter:
wiredTigerConcurrentReadTransacKons: <number>
wiredTigerConcurrentWriteTransacKons: <number>

24. Concurrent Transac/ons
Specify the maximum number of concurrent read/write transac/ons.
Default is 128
mongod.conf
setParameter:
wiredTigerConcurrentReadTransac/ons: <number>
RunKme: db.adminCommand( { setParameter: 1,
wiredTigerConcurrentWriteTransac/ons: <num> } )

25. Concurrent Transac/ons Benchmark
64 Trx: 64 read/write transcac/ons
Load: 11 minutes and 06 seconds
128 Trx: Default - 128 read/write transcac/ons
Load: 10 minutes and 58 seconds
256 Trx: 256 read/write transcac/ons
Load: 10 minutes and 46 seconds

26. Concurrent Transac/ons (cum tps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
trx-128
trx-64
trx-256

27. Concurrent Transac/ons (int tps)
0
200
400
600
800
1000
1200
1400
0 100 200 300 400 500 600 700
trx-128
trx-64
trx-256

28. WT configura/on/mongod.conf
engineConfig:
cacheSizeGB: <number>
journalCompressor: <string>
directoryForIndexes: <boolean>
configString: <string> , <string> …
collec/onConfig:
blockCompressor: <string>
indexConfig:
prefixCompression: <boolean>
setParameter:
wiredTigerConcurrentReadTransac/ons: <number>
wiredTigerConcurrentWriteTransac/ons: <number>

29. WT configura/on
Every WT variable can be assigned to configString
We are going to examine a subset:
- evic/on_target
- evic/on_trigger
- evic/on_dirty_target
- evic/on_dirty_trigger
- evic/on.threads_min
- evic/on.threads_max
- direct_IO

30. Evic/on
Resident Objects Shared cache

31. Evic/on
Cache saves objects (working set)

32. Evic/on
Evic/on Triggers

33. Evic/on
Evic/on Candidates

34. Evic/on
Evic/on Target

35. evic/on_trigger
evicKon_trigger is the occupied percentage of the total cache size that
causes evic/on to start.
Default value: 95%
Acceptable values: Between 10 and 99
RunKme: db.adminCommand({setParameter:1,
wiredTigerEngineRun/meConfig:"evic/on_trigger=71”})
mongod.conf : evic/on_trigger=71
WiredTiger.config: evic/on_trigger=71

36. evic/on_target
evicKon_target is the overall target for evic/on, expressed as a
percentage of total cache size.
Default value: 80%
Acceptable range: Between 10 and 99 (lower than trigger)
RunKme: db.adminCommand({setParameter:1,
wiredTigerEngineRun/meConfig:"evic/on_target=70"})
mongod.conf : evic/on_target=70
WiredTiger.config: evic/on_target=70

37. Evic/on_trigger/target Benchmark
Default: trigger/target: 95/80
Load: 11 minutes and 06 seconds
95/90: trigger/target: 95/90
Load: 11 minutes and 41 seconds
90/89: trigger/target: 90/89
Load: 10 minutes and 16 seconds
95/94: trigger/target: 95/94
Load: 12 minutes and 26 seconds

38. evic/on_trigger/target (cum tps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
95 and 90
90 and 89
95 and 94

39. Evic/on_trigger/target (int tps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
default
95 and 90
90 and 89
95 and 94

40. evic/on_dirty_trigger
evicKon_dirty_trigger: trigger evic/on when the cache is using this
much memory for dirty content, as a percentage of the total cache size.
Default value: 95%
Acceptable range: Between 5 and 99
RunKme: db.adminCommand({setParameter:1,
wiredTigerEngineRun/meConfig:"evic/on_dirty_trigger=71"})
mongod.conf : evic/on_dirty_trigger=71
WiredTiger.config: evic/on_dirty_trigger=71

41. evic/on_dirty_target
evicKon_dirty_target: con/nue evic/ng un/l the cache has less dirty
pages than this (as a percentage).
Default: 80%
Acceptable range: Between 5 and 99 (lower than trigger)
RunKme: db.adminCommand({setParameter:1,
wiredTigerEngineRun/meConfig:"evic/on_dirty_target=70"})
mongod.conf : evic/on_dirty_target=70
WiredTiger.config: evic/on_dirty_target=70

42. dirty_trigger/target Benchmark
Default: trigger/target 95/80
Load: 10 minutes and 58 seconds
20/5: trigger/target 20/5
Load: 31 minutes and 20 seconds
80/70: trigger/target 80/70
Load: 10 minutes and 06 seconds

43. dirty_trigger/target (cum tps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
20 and 5
80 and 70

44. dirty_trigger/target (int tps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
default
20 and 5
80 and 70

45. evic/on threads
evicKon.threads_min & evicKon.threads _max : the minimum and maximum
number of addi/onal evic/on threads
Default: 1 and 4
Acceptable range: between 1 and 20
RunKme:
db.adminCommand({setParameter:1,
wiredTigerEngineRun/meConfig:"evic/on=(threads_min=2)"} )
mongod.conf : evic/on=(threads_max=2)
WiredTiger.config: evic/on=(threads_max=2)

46. evict_threads_min/max Benchmark
Default: min/max 1/4
Load: 10 minutes and 58 seconds
4 and 6: min/max 4/6
Load: 10 minutes and 31 seconds
2 and 8: min/max 2/8
Load: 10 minutes and 11 seconds
4 and 4: min/max 4/4
Load: 10 minutes and 21 seconds

47. evict_threads_min/max (cum tps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
4 and 6
2 and 8
4 and 4

48. evict_threads_min/max (int tps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
default
4 and 6
2 and 8
4 and 4

49. direct_io
STORAGE STORAGE
FS cache WT cache WT cache FS cache
RAM RAM

50. direct_io
• minimize the opera/ng system cache effects of I/O to and
from WiredTiger's buffer cache
• avoid double-buffering of blocks in WiredTiger's cache and
the opera/ng system buffer cache
• avoid stalling underlying solid-state drives by wri/ng a large
number of dirty blocks.
WiredTiger.config: direct_io=(data)
mongod.conf : storage.wiredTiger.engineConfig.configString:
direct_io=(data)

51. Direct IO Benchmark
Default: Without Direct IO
Load: 10 minutes and 58 seconds
Direct IO: With Direct IO
Load: 11 minutes and 25 seconds

52. Direct IO (cum tps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
direct IO

53. Direct IO (int tps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
default
direct IO

54. Ques/ons?
Thank you!!!
antonios.giannopoulos@rackspace.co.uk
#iamantonios