The slide deck describes options available in Postgre

1. /
Scaling Postgres
Denish Patel
Database Architect
https://twitter.com/DenishPatel
Wednesday, September 18, 13

2. 1
Wednesday, September 18, 13

3. OmniTI
1
Wednesday, September 18, 13

4. OmniTI
1
• Helping customers
navigate explosive growth
with technology.
Wednesday, September 18, 13

5. OmniTI
1
• Helping customers
navigate explosive growth
with technology.
100MM+ users
$1B+ gross online sales
Wednesday, September 18, 13

6. OmniTI
1
• Helping customers
navigate explosive growth
with technology.
100MM+ users
$1B+ gross online sales
Open and closed source thought leaders,
experts and authors
Wednesday, September 18, 13

7. OmniTI
1
• Helping customers
navigate explosive growth
with technology.
100MM+ users
$1B+ gross online sales
Open and closed source thought leaders,
experts and authors
Wednesday, September 18, 13

8. 2
Wednesday, September 18, 13

9. Talk Outline
2
Wednesday, September 18, 13

10. Talk Outline
• Scalability
• Database Scaling needs, costs, methods
• Scaling Postgres
• Vertically
• Horizontally
• Obstacles to Scalability
• Beyond Postgres
2
Wednesday, September 18, 13

11. 3
Wednesday, September 18, 13

12. What is Scalability?
3
Wednesday, September 18, 13

13. What is Scalability?
3
A service is said to be scalable if when we
increase the resources in a system, it results in
increased performance in a manner proportional
to resources added.
Wednesday, September 18, 13

14. 4
Wednesday, September 18, 13

15. Why to scale databases?
4
Wednesday, September 18, 13

16. Why to scale databases?
• Support a higher volume of users
4
Wednesday, September 18, 13

17. Why to scale databases?
• Support a higher volume of users
4
Wednesday, September 18, 13

18. Why to scale databases?
• Support a higher volume of users
4
Wednesday, September 18, 13

19. Why to scale databases?
• Support a higher volume of users
4
Wednesday, September 18, 13

20. Why to scale databases?
• Provide better performance for existing users
• Support a higher volume of users
4
Wednesday, September 18, 13

21. Why to scale databases?
• Provide better performance for existing users
• Store a larger volume of data
• Support a higher volume of users
4
Wednesday, September 18, 13

22. Why to scale databases?
• Provide better performance for existing users
• Store a larger volume of data
• Improve system availability
• Support a higher volume of users
4
Wednesday, September 18, 13

23. Why to scale databases?
• Provide better performance for existing users
• Store a larger volume of data
• Improve system availability
• Geographic dispersion
• Support a higher volume of users
4
Wednesday, September 18, 13

24. 5
Wednesday, September 18, 13

25. Why is database scalability so hard?
5
Wednesday, September 18, 13

26. Why is database scalability so hard?
5
Wednesday, September 18, 13

27. Why is database scalability so hard?
• Search
5
Wednesday, September 18, 13

28. Why is database scalability so hard?
• Search
• Concurrency
5
Wednesday, September 18, 13

29. Why is database scalability so hard?
• Search
• Concurrency
• Consistency
5
Wednesday, September 18, 13

30. Why is database scalability so hard?
• Search
• Concurrency
• Consistency
• Speed
5
Wednesday, September 18, 13

31. Why is database scalability so hard?
• Search
• Concurrency
• Consistency
• Speed
• Cost
5
Wednesday, September 18, 13

32. Why is database scalability so hard?
• Search
• Concurrency
• Consistency
• Speed
• Cost
• Cost of hardware
5
Wednesday, September 18, 13

33. Why is database scalability so hard?
• Search
• Concurrency
• Consistency
• Speed
• Cost
• Cost of hardware
• Cost deployment effort
5
Wednesday, September 18, 13

34. Why is database scalability so hard?
• Search
• Concurrency
• Consistency
• Speed
• Cost
• Cost of hardware
• Cost deployment effort
• Cost Ongoing maintenance
5
Wednesday, September 18, 13

35. 6
Wednesday, September 18, 13

36. Scaling Needs & Methods
6
Wednesday, September 18, 13

37. Scaling Needs & Methods
6
Wednesday, September 18, 13

38. Scaling Needs & Methods
• Data growth
6
Wednesday, September 18, 13

39. Scaling Needs & Methods
• Data growth
• Read requests
6
Wednesday, September 18, 13

40. Scaling Needs & Methods
• Data growth
• Read requests
• Write requests
6
Wednesday, September 18, 13

41. Scaling Needs & Methods
• Data growth
• Read requests
• Write requests
• Vertical Scaling
• Horizontal Scaling
6
Wednesday, September 18, 13

42. Scaling Needs & Methods
• Data growth
• Read requests
• Write requests
• Vertical Scaling
• Horizontal Scaling
6
Wednesday, September 18, 13

43. Scaling Needs & Methods
• Data growth
• Read requests
• Write requests
• Vertical Scaling
• Horizontal Scaling
6
Wednesday, September 18, 13

44. 7
Wednesday, September 18, 13

45. Vertical Scaling (Scale up)
7
Wednesday, September 18, 13

46. Vertical Scaling (Scale up)
7
Wednesday, September 18, 13

47. Vertical Scaling (Scale up)
• Pros
7
Wednesday, September 18, 13

48. Vertical Scaling (Scale up)
• Pros
• Simple to implement
7
Wednesday, September 18, 13

49. Vertical Scaling (Scale up)
• Pros
• Simple to implement
• Ease of maintenance
7
Wednesday, September 18, 13

50. Vertical Scaling (Scale up)
• Pros
• Simple to implement
• Ease of maintenance
• Cons
7
Wednesday, September 18, 13

51. Vertical Scaling (Scale up)
• Pros
• Simple to implement
• Ease of maintenance
• Cons
• Cost of hardware
7
Wednesday, September 18, 13

52. Vertical Scaling (Scale up)
• Pros
• Simple to implement
• Ease of maintenance
• Cons
• Cost of hardware
• Sometimes SPOFs
7
Wednesday, September 18, 13

53. 8
Wednesday, September 18, 13

54. Horizontal Scaling (Scale Out)
8
Wednesday, September 18, 13

55. Horizontal Scaling (Scale Out)
8
Wednesday, September 18, 13

56. Horizontal Scaling (Scale Out)
8
Wednesday, September 18, 13

57. Horizontal Scaling (Scale Out)
• Pros
8
Wednesday, September 18, 13

58. Horizontal Scaling (Scale Out)
• Pros
• Cheaper in hardware cost
8
Wednesday, September 18, 13

59. Horizontal Scaling (Scale Out)
• Pros
• Cheaper in hardware cost
• Flexibility
8
Wednesday, September 18, 13

60. Horizontal Scaling (Scale Out)
• Pros
• Cheaper in hardware cost
• Flexibility
• Higher fault tolerance
8
Wednesday, September 18, 13

61. Horizontal Scaling (Scale Out)
• Pros
• Cheaper in hardware cost
• Flexibility
• Higher fault tolerance
• Cons
8
Wednesday, September 18, 13

62. Horizontal Scaling (Scale Out)
• Pros
• Cheaper in hardware cost
• Flexibility
• Higher fault tolerance
• Cons
• Complex to implement
8
Wednesday, September 18, 13

63. Horizontal Scaling (Scale Out)
• Pros
• Cheaper in hardware cost
• Flexibility
• Higher fault tolerance
• Cons
• Complex to implement
• Expensive to maintain
8
Wednesday, September 18, 13

64. Horizontal Scaling (Scale Out)
• Pros
• Cheaper in hardware cost
• Flexibility
• Higher fault tolerance
• Cons
• Complex to implement
• Expensive to maintain
• Bigger footprint in the Data Center
8
Wednesday, September 18, 13

65. Horizontal Scaling (Scale Out)
• Pros
• Cheaper in hardware cost
• Flexibility
• Higher fault tolerance
• Cons
• Complex to implement
• Expensive to maintain
• Bigger footprint in the Data Center
• No built in support in databases
8
Wednesday, September 18, 13

66. 9
Wednesday, September 18, 13

67. Spec’ing Hardware
9
Wednesday, September 18, 13

68. Spec’ing Hardware
• CPU
9
Wednesday, September 18, 13

69. Spec’ing Hardware
• CPU
• 8+ cores
9
Wednesday, September 18, 13

70. Spec’ing Hardware
• CPU
• 8+ cores
• RAM
9
Wednesday, September 18, 13

71. Spec’ing Hardware
• CPU
• 8+ cores
• RAM
• 64GB+
9
Wednesday, September 18, 13

72. Spec’ing Hardware
• CPU
• 8+ cores
• RAM
• 64GB+
• Disks
9
Wednesday, September 18, 13

73. Spec’ing Hardware
• CPU
• 8+ cores
• RAM
• 64GB+
• Disks
• SSDs
9
Wednesday, September 18, 13

74. Spec’ing Hardware
• CPU
• 8+ cores
• RAM
• 64GB+
• Disks
• SSDs
• RAID 10
9
Wednesday, September 18, 13

75. Spec’ing Hardware
• CPU
• 8+ cores
• RAM
• 64GB+
• Disks
• SSDs
• RAID 10
• Network
9
Wednesday, September 18, 13

76. Spec’ing Hardware
• CPU
• 8+ cores
• RAM
• 64GB+
• Disks
• SSDs
• RAID 10
• Network
• min Gigbit, 10Gigbit
9
Wednesday, September 18, 13

77. 10
Wednesday, September 18, 13

78. Tune Postgres/memory parameters
10
Wednesday, September 18, 13

79. Tune Postgres/memory parameters
• shared_buffers
10
Wednesday, September 18, 13

80. Tune Postgres/memory parameters
• shared_buffers
• effective_cache_size
10
Wednesday, September 18, 13

81. Tune Postgres/memory parameters
• shared_buffers
• effective_cache_size
• checkpoint_completion_target
10
Wednesday, September 18, 13

82. Tune Postgres/memory parameters
• shared_buffers
• effective_cache_size
• checkpoint_completion_target
• checkpoint_segments
10
Wednesday, September 18, 13

83. Tune Postgres/memory parameters
• shared_buffers
• effective_cache_size
• checkpoint_completion_target
• checkpoint_segments
• max_connections
10
Wednesday, September 18, 13

84. Tune Postgres/memory parameters
• shared_buffers
• effective_cache_size
• checkpoint_completion_target
• checkpoint_segments
• max_connections
• work_mem
10
Wednesday, September 18, 13

85. Tune Postgres/memory parameters
• shared_buffers
• effective_cache_size
• checkpoint_completion_target
• checkpoint_segments
• max_connections
• work_mem
• maintenance_work_mem
10
Wednesday, September 18, 13

86. Tune Postgres/memory parameters
• shared_buffers
• effective_cache_size
• checkpoint_completion_target
• checkpoint_segments
• max_connections
• work_mem
• maintenance_work_mem
http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server
10
Wednesday, September 18, 13

87. 11
Wednesday, September 18, 13

88. Tune Postgres/logging parameters
11
Wednesday, September 18, 13

89. Tune Postgres/logging parameters
11
Wednesday, September 18, 13

90. Tune Postgres/logging parameters
11
Wednesday, September 18, 13

91. Tune Postgres/logging parameters
• logging_collector => 'on'
11
Wednesday, September 18, 13

92. Tune Postgres/logging parameters
• logging_collector => 'on'
• log_destination => 'stderr'
11
Wednesday, September 18, 13

93. Tune Postgres/logging parameters
• logging_collector => 'on'
• log_destination => 'stderr'
• log_filename => 'postgresql-%Y-%m-%d_%H%M%S.log'
11
Wednesday, September 18, 13

94. Tune Postgres/logging parameters
• logging_collector => 'on'
• log_destination => 'stderr'
• log_filename => 'postgresql-%Y-%m-%d_%H%M%S.log'
• log_line_prefix => '%m [%r] [%p]: [%l-1] user=%u,db=%d,e=%e '
11
Wednesday, September 18, 13

95. Tune Postgres/logging parameters
• logging_collector => 'on'
• log_destination => 'stderr'
• log_filename => 'postgresql-%Y-%m-%d_%H%M%S.log'
• log_line_prefix => '%m [%r] [%p]: [%l-1] user=%u,db=%d,e=%e '
• log_min_duration_statement => 1000ms
11
Wednesday, September 18, 13

96. Tune Postgres/logging parameters
• logging_collector => 'on'
• log_destination => 'stderr'
• log_filename => 'postgresql-%Y-%m-%d_%H%M%S.log'
• log_line_prefix => '%m [%r] [%p]: [%l-1] user=%u,db=%d,e=%e '
• log_min_duration_statement => 1000ms
• log_autovacuum_min_duration => '0'
11
Wednesday, September 18, 13

97. 12
Wednesday, September 18, 13

98. Tune Postgres/logging parameters
12
Wednesday, September 18, 13

99. Tune Postgres/logging parameters
12
Wednesday, September 18, 13

100. Tune Postgres/logging parameters
12
Wednesday, September 18, 13

101. Tune Postgres/logging parameters
• log_lock_waits => 'on'
12
Wednesday, September 18, 13

102. Tune Postgres/logging parameters
• log_lock_waits => 'on'
• log_temp_files => '0'
12
Wednesday, September 18, 13

103. Tune Postgres/logging parameters
• log_lock_waits => 'on'
• log_temp_files => '0'
• log_checkpoints => 'on'
12
Wednesday, September 18, 13

104. Tune Postgres/logging parameters
• log_lock_waits => 'on'
• log_temp_files => '0'
• log_checkpoints => 'on'
• log_connections => 'on'
12
Wednesday, September 18, 13

105. Tune Postgres/logging parameters
• log_lock_waits => 'on'
• log_temp_files => '0'
• log_checkpoints => 'on'
• log_connections => 'on'
• log_disconnections => 'on'
12
Wednesday, September 18, 13

106. Tune Postgres/logging parameters
• log_lock_waits => 'on'
• log_temp_files => '0'
• log_checkpoints => 'on'
• log_connections => 'on'
• log_disconnections => 'on'
• log_min_error_statement => 'warning'
12
Wednesday, September 18, 13

107. Tune Postgres/logging parameters
• log_lock_waits => 'on'
• log_temp_files => '0'
• log_checkpoints => 'on'
• log_connections => 'on'
• log_disconnections => 'on'
• log_min_error_statement => 'warning'
• log_min_messages => 'warning'
12
Wednesday, September 18, 13

108. Tune Postgres/logging parameters
• log_lock_waits => 'on'
• log_temp_files => '0'
• log_checkpoints => 'on'
• log_connections => 'on'
• log_disconnections => 'on'
• log_min_error_statement => 'warning'
• log_min_messages => 'warning'
• log_statement => 'ddl'
12
Wednesday, September 18, 13

109. 13
Wednesday, September 18, 13

110. Optimize Queries/report slow queries
13
Wednesday, September 18, 13

111. Optimize Queries/report slow queries
13
Wednesday, September 18, 13

112. Optimize Queries/report slow queries
13
Wednesday, September 18, 13

113. 14
Wednesday, September 18, 13

114. Optimize Queries/Extensive monitoring
14
Wednesday, September 18, 13

115. Optimize Queries/Extensive monitoring
14
Wednesday, September 18, 13

116. 15
Wednesday, September 18, 13

117. Optimize Queries /Explain Analyze
15
Wednesday, September 18, 13

118. Optimize Queries /Explain Analyze
explain (analyze,buffers) select col1,col2 from demo_ios where col2 between 0.01 and 0.02;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------
Index Only Scan using idx_demo_ios on demo_ios (cost=0.00..35330.93 rows=993633 width=16) (actual time=58.100..3250.589
rows=1000392 loops=1)
Index Cond: ((col2 >= 0.01::double precision) AND (col2 <= 0.02::double precision))
Heap Fetches: 0
Buffers: shared hit=923073 read=3848
Total runtime: 4297.405 ms
15
Wednesday, September 18, 13

119. Optimize Queries /Explain Analyze
explain (analyze,buffers) select col1,col2 from demo_ios where col2 between 0.01 and 0.02;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------
Index Only Scan using idx_demo_ios on demo_ios (cost=0.00..35330.93 rows=993633 width=16) (actual time=58.100..3250.589
rows=1000392 loops=1)
Index Cond: ((col2 >= 0.01::double precision) AND (col2 <= 0.02::double precision))
Heap Fetches: 0
Buffers: shared hit=923073 read=3848
Total runtime: 4297.405 ms
15
Wednesday, September 18, 13

120. 16
Wednesday, September 18, 13

121. Optimize Queries /track functions
16
Wednesday, September 18, 13

122. Optimize Queries /track functions
16
Wednesday, September 18, 13

123. Optimize Queries /track functions
• track_functions = pl # none, pl, all
16
Wednesday, September 18, 13

124. Optimize Queries /track functions
• track_functions = pl # none, pl, all
• reload online
16
Wednesday, September 18, 13

125. Optimize Queries /track functions
• track_functions = pl # none, pl, all
• reload online
• select * from pg_stat_user_functions;
16
Wednesday, September 18, 13

126. Optimize Queries /track functions
• track_functions = pl # none, pl, all
• reload online
• select * from pg_stat_user_functions;
16
Wednesday, September 18, 13

127. Optimize Queries /track functions
• track_functions = pl # none, pl, all
• reload online
• select * from pg_stat_user_functions;
funcid | schemaname | funcname | calls | total_time | self_time
16
Wednesday, September 18, 13

128. Optimize Queries /track functions
• track_functions = pl # none, pl, all
• reload online
• select * from pg_stat_user_functions;
funcid | schemaname | funcname | calls | total_time | self_time
16
Wednesday, September 18, 13

129. 17
Wednesday, September 18, 13

130. Partitioning
17
Wednesday, September 18, 13

131. Partitioning
• As table size grows, queries eventually slows down, even with indexing
17
Wednesday, September 18, 13

132. Partitioning
• As table size grows, queries eventually slows down, even with indexing
• Allows data added, removed and queried fast
17
Wednesday, September 18, 13

133. Partitioning
• As table size grows, queries eventually slows down, even with indexing
• Allows data added, removed and queried fast
• Partitioning pruning queries
17
Wednesday, September 18, 13

134. Partitioning
• As table size grows, queries eventually slows down, even with indexing
• Allows data added, removed and queried fast
• Partitioning pruning queries
• Manage partitions
17
Wednesday, September 18, 13

135. 18
Wednesday, September 18, 13

136. Partitioning /Postgres
18
Wednesday, September 18, 13

137. Partitioning /Postgres
18
Wednesday, September 18, 13

138. Partitioning /Postgres
• Postgres partitioning
18
Wednesday, September 18, 13

139. Partitioning /Postgres
• Postgres partitioning
• Trigger based
18
Wednesday, September 18, 13

140. Partitioning /Postgres
• Postgres partitioning
• Trigger based
• Rule based
18
Wednesday, September 18, 13

141. Partitioning /Postgres
• Postgres partitioning
• Trigger based
• Rule based
• Lack of built-in Postgres partition management
18
Wednesday, September 18, 13

142. Partitioning /Postgres
• Postgres partitioning
• Trigger based
• Rule based
• Lack of built-in Postgres partition management
• Postgres partition management extension: pg_partman
18
Wednesday, September 18, 13

143. Partitioning /Postgres
• Postgres partitioning
• Trigger based
• Rule based
• Lack of built-in Postgres partition management
• Postgres partition management extension: pg_partman
• http://pgxn.org/dist/pg_partman/doc/pg_partman.html
18
Wednesday, September 18, 13

144. Partitioning /Postgres
• Postgres partitioning
• Trigger based
• Rule based
• Lack of built-in Postgres partition management
• Postgres partition management extension: pg_partman
• http://pgxn.org/dist/pg_partman/doc/pg_partman.html
• Tomorrow’s session at 11:30AM (When Postgres can’t....)
18
Wednesday, September 18, 13

145. 19
Wednesday, September 18, 13

146. Partitioning /functional
19
Wednesday, September 18, 13

147. Partitioning /functional
Configuration Data Transaction data Session data
Configuration Tools Reporting Tools Monitoring Tools
Web Applications Other Applications
19
Wednesday, September 18, 13

148. Partitioning /functional
Configuration Data Transaction data Session data
Configuration Tools Reporting Tools Monitoring Tools
Web Applications Other Applications
• Partition data based on functionality
19
Wednesday, September 18, 13

149. Partitioning /functional
Configuration Data Transaction data Session data
Configuration Tools Reporting Tools Monitoring Tools
Web Applications Other Applications
• Partition data based on functionality
• Separate Postgres clusters
19
Wednesday, September 18, 13

150. Partitioning /functional
Configuration Data Transaction data Session data
Configuration Tools Reporting Tools Monitoring Tools
Web Applications Other Applications
• Partition data based on functionality
• Separate Postgres clusters
• Start with Separate schemas
19
Wednesday, September 18, 13

151. Partitioning /functional
Configuration Data Transaction data Session data
Configuration Tools Reporting Tools Monitoring Tools
Web Applications Other Applications
• Partition data based on functionality
• Separate Postgres clusters
• Start with Separate schemas
• No relationship between data
19
Wednesday, September 18, 13

152. Partitioning /functional
Configuration Data Transaction data Session data
Configuration Tools Reporting Tools Monitoring Tools
Web Applications Other Applications
• Partition data based on functionality
• Separate Postgres clusters
• Start with Separate schemas
• No relationship between data
• Help to spread the load across server
19
Wednesday, September 18, 13

153. Partitioning /functional
Configuration Data Transaction data Session data
Configuration Tools Reporting Tools Monitoring Tools
Web Applications Other Applications
• Partition data based on functionality
• Separate Postgres clusters
• Start with Separate schemas
• No relationship between data
• Help to spread the load across server
• Less complex compare to sharding!
19
Wednesday, September 18, 13

154. 20
Wednesday, September 18, 13

155. pgbouncer
20
Wednesday, September 18, 13

156. pgbouncer
20
Wednesday, September 18, 13

157. pgbouncer
20
Wednesday, September 18, 13

158. pgbouncer
20
Wednesday, September 18, 13

159. pgbouncer
• A lightweight connection pooler
20
Wednesday, September 18, 13

160. pgbouncer
• A lightweight connection pooler
• Helps to reduce # of newly created connections on DB server
20
Wednesday, September 18, 13

161. pgbouncer
• A lightweight connection pooler
• Helps to reduce # of newly created connections on DB server
• Abstracts DBs from App
20
Wednesday, September 18, 13

162. pgbouncer
• A lightweight connection pooler
• Helps to reduce # of newly created connections on DB server
• Abstracts DBs from App
• Helps to instrument smooth and easy failover
20
Wednesday, September 18, 13

163. pgbouncer
• A lightweight connection pooler
• Helps to reduce # of newly created connections on DB server
• Abstracts DBs from App
• Helps to instrument smooth and easy failover
• Connection pooling Options
20
Wednesday, September 18, 13

164. pgbouncer
• A lightweight connection pooler
• Helps to reduce # of newly created connections on DB server
• Abstracts DBs from App
• Helps to instrument smooth and easy failover
• Connection pooling Options
• Session, Transaction, Statement pooling options
20
Wednesday, September 18, 13

165. pgbouncer
• A lightweight connection pooler
• Helps to reduce # of newly created connections on DB server
• Abstracts DBs from App
• Helps to instrument smooth and easy failover
• Connection pooling Options
• Session, Transaction, Statement pooling options
• Beware! Transaction pooling doesn’t support prepared transactions
20
Wednesday, September 18, 13

166. 21
Wednesday, September 18, 13

167. Caching
21
Wednesday, September 18, 13

168. Caching
21
Wednesday, September 18, 13

169. Caching
• Memcached
21
Wednesday, September 18, 13

170. Caching
• Memcached
• Open source, High-performance distributed memory object
caching system
21
Wednesday, September 18, 13

171. Caching
• Memcached
• Open source, High-performance distributed memory object
caching system
• Speeds up dynamic web applications by alleviating database load.
21
Wednesday, September 18, 13

172. Caching
• Memcached
• Open source, High-performance distributed memory object
caching system
• Speeds up dynamic web applications by alleviating database load.
• An in-memory key-value store for small chunks of arbitrary data
21
Wednesday, September 18, 13

173. Caching
• Memcached
• Open source, High-performance distributed memory object
caching system
• Speeds up dynamic web applications by alleviating database load.
• An in-memory key-value store for small chunks of arbitrary data
• Redis
21
Wednesday, September 18, 13

174. Caching
• Memcached
• Open source, High-performance distributed memory object
caching system
• Speeds up dynamic web applications by alleviating database load.
• An in-memory key-value store for small chunks of arbitrary data
• Redis
• Open source, advanced key-value store.
21
Wednesday, September 18, 13

175. Caching
• Memcached
• Open source, High-performance distributed memory object
caching system
• Speeds up dynamic web applications by alleviating database load.
• An in-memory key-value store for small chunks of arbitrary data
• Redis
• Open source, advanced key-value store.
• Works with an in-memory & persistent dataset
21
Wednesday, September 18, 13

176. 22
Wednesday, September 18, 13

177. Replication /built-in
22
Wednesday, September 18, 13

178. Replication /built-in
• Cluster Level Replication (Binary)
22
Wednesday, September 18, 13

179. Replication /built-in
• Cluster Level Replication (Binary)
• Streaming Replication
22
Wednesday, September 18, 13

180. Replication /built-in
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
22
Wednesday, September 18, 13

181. Replication /built-in
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

182. Replication /built-in
Master
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

183. Replication /built-in
Master
Failover• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

184. Replication /built-in
Master
Failover
Read
Salve 1
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

185. Replication /built-in
Master
Failover
Read
Salve 1
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

186. Replication /built-in
Master
Failover
Read
Salve 1
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

187. Replication /built-in
Master
Failover
Read
Salve 1
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

188. Replication /built-in
Master
Failover
Read
Salve 1
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

189. Replication /built-in
Master
Failover
Read
Salve 1
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

190. Replication /built-in
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

191. Replication /built-in
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

192. Replication /built-in
• Pros:
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

193. Replication /built-in
• Pros:
• Built-in
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

194. Replication /built-in
• Pros:
• Built-in
• Allows to open replicated database in read-only mode
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

195. Replication /built-in
• Pros:
• Built-in
• Allows to open replicated database in read-only mode
• Cons:
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

196. Replication /built-in
• Pros:
• Built-in
• Allows to open replicated database in read-only mode
• Cons:
• All or none
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

197. Replication /built-in
• Pros:
• Built-in
• Allows to open replicated database in read-only mode
• Cons:
• All or none
• Doesn’t allow write on replicated database
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

198. Replication /built-in
• Pros:
• Built-in
• Allows to open replicated database in read-only mode
• Cons:
• All or none
• Doesn’t allow write on replicated database
• Doesn’t work across major version
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

199. Replication /built-in
• Pros:
• Built-in
• Allows to open replicated database in read-only mode
• Cons:
• All or none
• Doesn’t allow write on replicated database
• Doesn’t work across major version
• Postgres 9.2 (primary) does NOT replicate to 9.3 (secondary)
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

200. Replication /built-in
• Pros:
• Built-in
• Allows to open replicated database in read-only mode
• Cons:
• All or none
• Doesn’t allow write on replicated database
• Doesn’t work across major version
• Postgres 9.2 (primary) does NOT replicate to 9.3 (secondary)
• Postgres 9.2.1 (primary) can replicate to 9.2.4 (secondary)
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

201. Replication /built-in
• Pros:
• Built-in
• Allows to open replicated database in read-only mode
• Cons:
• All or none
• Doesn’t allow write on replicated database
• Doesn’t work across major version
• Postgres 9.2 (primary) does NOT replicate to 9.3 (secondary)
• Postgres 9.2.1 (primary) can replicate to 9.2.4 (secondary)
Master
Failover
Read
Salve 1
PITR!
PITR!
PITR!
Streaming Replication
https://wiki.postgresql.org/wiki/Binary_Replication_Tools
• Cluster Level Replication (Binary)
• Streaming Replication
• WAL-only replication
• Hybrid replication
22
Wednesday, September 18, 13

202. 23
Wednesday, September 18, 13

203. Replication /built-in
23
Wednesday, September 18, 13

204. Replication /built-in
Read
Salve 1
23
Wednesday, September 18, 13

205. Replication /built-in
Read
Salve 1
DW
System
23
Wednesday, September 18, 13

206. Replication /built-in
Read
Salve 1
DW
System
23
Wednesday, September 18, 13

207. Replication /built-in
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

208. Replication /built-in
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

209. Replication /built-in
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

210. Replication /built-in
• postgres_fdw
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

211. Replication /built-in
• postgres_fdw
• Postgres 9.3 feature
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

212. Replication /built-in
• postgres_fdw
• Postgres 9.3 feature
• Allows to access data stored in external PostgreSQL
servers
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

213. Replication /built-in
• postgres_fdw
• Postgres 9.3 feature
• Allows to access data stored in external PostgreSQL
servers
• cross version queries
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

214. Replication /built-in
• postgres_fdw
• Postgres 9.3 feature
• Allows to access data stored in external PostgreSQL
servers
• cross version queries
• Postgres 9.3 could query Postgres 9.1
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

215. Replication /built-in
• postgres_fdw
• Postgres 9.3 feature
• Allows to access data stored in external PostgreSQL
servers
• cross version queries
• Postgres 9.3 could query Postgres 9.1
• Application
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

216. Replication /built-in
• postgres_fdw
• Postgres 9.3 feature
• Allows to access data stored in external PostgreSQL
servers
• cross version queries
• Postgres 9.3 could query Postgres 9.1
• Application
• Run query remotely on slave db
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

217. Replication /built-in
• postgres_fdw
• Postgres 9.3 feature
• Allows to access data stored in external PostgreSQL
servers
• cross version queries
• Postgres 9.3 could query Postgres 9.1
• Application
• Run query remotely on slave db
• Data warehouse data refreshes
Read
Salve 1
DW
System
postgres_fdw
23
Wednesday, September 18, 13

218. 24
Wednesday, September 18, 13

219. Replication /third-party tools
24
Wednesday, September 18, 13

220. Replication /third-party tools
24
Wednesday, September 18, 13

221. Replication /third-party tools
24
Wednesday, September 18, 13

222. Replication /third-party tools
• Table level Replication Tools (Trigger based)
24
Wednesday, September 18, 13

223. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
24
Wednesday, September 18, 13

224. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
24
Wednesday, September 18, 13

225. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
24
Wednesday, September 18, 13

226. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
• Pros:
24
Wednesday, September 18, 13

227. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
• Pros:
• Allows to open replicated database in read-
write mode
24
Wednesday, September 18, 13

228. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
• Pros:
• Allows to open replicated database in read-
write mode
• Allows table/database level replication
24
Wednesday, September 18, 13

229. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
• Pros:
• Allows to open replicated database in read-
write mode
• Allows table/database level replication
• Allows rolling upgrade
24
Wednesday, September 18, 13

230. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
• Pros:
• Allows to open replicated database in read-
write mode
• Allows table/database level replication
• Allows rolling upgrade
• cross version replication is allowed
24
Wednesday, September 18, 13

231. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
• Pros:
• Allows to open replicated database in read-
write mode
• Allows table/database level replication
• Allows rolling upgrade
• cross version replication is allowed
• Multi-master replication
24
Wednesday, September 18, 13

232. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
• Pros:
• Allows to open replicated database in read-
write mode
• Allows table/database level replication
• Allows rolling upgrade
• cross version replication is allowed
• Multi-master replication
• Cons:
24
Wednesday, September 18, 13

233. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
• Pros:
• Allows to open replicated database in read-
write mode
• Allows table/database level replication
• Allows rolling upgrade
• cross version replication is allowed
• Multi-master replication
• Cons:
• Complicated to setup
24
Wednesday, September 18, 13

234. Replication /third-party tools
• Table level Replication Tools (Trigger based)
• Slony
• Bucardo
• Mimeo: http://pgxn.org/dist/mimeo/
• Pros:
• Allows to open replicated database in read-
write mode
• Allows table/database level replication
• Allows rolling upgrade
• cross version replication is allowed
• Multi-master replication
• Cons:
• Complicated to setup
• Unknown territory
24
Wednesday, September 18, 13

235. 25
Wednesday, September 18, 13

236. Sharding
25
Wednesday, September 18, 13

237. Sharding
25
Wednesday, September 18, 13

238. Sharding
25
Wednesday, September 18, 13

239. Sharding
• Sharding is the process of splitting up your data so it resides in
different tables or often different physical databases.
25
Wednesday, September 18, 13

240. Sharding
• Sharding is the process of splitting up your data so it resides in
different tables or often different physical databases.
• Application aware sharding
25
Wednesday, September 18, 13

241. Sharding
• Sharding is the process of splitting up your data so it resides in
different tables or often different physical databases.
• Application aware sharding
• Application transparent sharding
25
Wednesday, September 18, 13

242. 26
Wednesday, September 18, 13

243. Application aware sharding
26
Wednesday, September 18, 13

244. Application aware sharding
26
Wednesday, September 18, 13

245. Application aware sharding
26
Wednesday, September 18, 13

246. Application aware sharding
• http://instagram-engineering.tumblr.com/
post/10853187575/sharding-ids-at-
instagram
26
Wednesday, September 18, 13

247. Application aware sharding
• http://instagram-engineering.tumblr.com/
post/10853187575/sharding-ids-at-
instagram
• Postgres allows “logical” shards through
Schema
26
Wednesday, September 18, 13

248. Application aware sharding
• http://instagram-engineering.tumblr.com/
post/10853187575/sharding-ids-at-
instagram
• Postgres allows “logical” shards through
Schema
• Easy to move to “physical” shard later
26
Wednesday, September 18, 13

249. 27
Wednesday, September 18, 13

250. Application transparent sharding
27
Wednesday, September 18, 13

251. Application transparent sharding
27
Wednesday, September 18, 13

252. 28
Wednesday, September 18, 13

253. Sharding Challenges
28
Wednesday, September 18, 13

254. Sharding Challenges
• Reliability
28
Wednesday, September 18, 13

255. Sharding Challenges
• Reliability
• Distributed queries
28
Wednesday, September 18, 13

256. Sharding Challenges
• Reliability
• Distributed queries
• Cross-shard join
28
Wednesday, September 18, 13

257. Sharding Challenges
• Reliability
• Distributed queries
• Cross-shard join
• Auto-increment key management
28
Wednesday, September 18, 13

258. Sharding Challenges
• Reliability
• Distributed queries
• Cross-shard join
• Auto-increment key management
• Choosing shard key
28
Wednesday, September 18, 13

259. Sharding Challenges
• Reliability
• Distributed queries
• Cross-shard join
• Auto-increment key management
• Choosing shard key
• Shard schemes
28
Wednesday, September 18, 13

260. 29
Wednesday, September 18, 13

261. Obstacles for Scaling Postgres
29
Wednesday, September 18, 13

262. Obstacles for Scaling Postgres
29
Wednesday, September 18, 13

263. Obstacles for Scaling Postgres
• Postgres table bloat
29
Wednesday, September 18, 13

264. Obstacles for Scaling Postgres
• Postgres table bloat
• FKs relationships
29
Wednesday, September 18, 13

265. Obstacles for Scaling Postgres
• Postgres table bloat
• FKs relationships
• Insufficient logging
29
Wednesday, September 18, 13

266. Obstacles for Scaling Postgres
• Postgres table bloat
• FKs relationships
• Insufficient logging
• Insufficient Caching
29
Wednesday, September 18, 13

267. Obstacles for Scaling Postgres
• Postgres table bloat
• FKs relationships
• Insufficient logging
• Insufficient Caching
• Insufficient Monitoring and Metrics
29
Wednesday, September 18, 13

268. Obstacles for Scaling Postgres
• Postgres table bloat
• FKs relationships
• Insufficient logging
• Insufficient Caching
• Insufficient Monitoring and Metrics
• ORMs
29
Wednesday, September 18, 13

269. Obstacles for Scaling Postgres
• Postgres table bloat
• FKs relationships
• Insufficient logging
• Insufficient Caching
• Insufficient Monitoring and Metrics
• ORMs
• Single Point of Failure
29
Wednesday, September 18, 13

270. Obstacles for Scaling Postgres
• Postgres table bloat
• FKs relationships
• Insufficient logging
• Insufficient Caching
• Insufficient Monitoring and Metrics
• ORMs
• Single Point of Failure
• Lack of communications between teams
29
Wednesday, September 18, 13

271. 30
Wednesday, September 18, 13

272. Beyond Postgres
30
Wednesday, September 18, 13

273. Beyond Postgres
30
Wednesday, September 18, 13

274. Beyond Postgres
• Avoid serialization in application code
30
Wednesday, September 18, 13

275. Beyond Postgres
• Avoid serialization in application code
• Feature Flags
30
Wednesday, September 18, 13

276. Beyond Postgres
• Avoid serialization in application code
• Feature Flags
• Browse only mode (Read only mode)
30
Wednesday, September 18, 13

277. Beyond Postgres
• Avoid serialization in application code
• Feature Flags
• Browse only mode (Read only mode)
• Don’t use database for Queuing
30
Wednesday, September 18, 13

278. Beyond Postgres
• Avoid serialization in application code
• Feature Flags
• Browse only mode (Read only mode)
• Don’t use database for Queuing
• RabbitMQ
30
Wednesday, September 18, 13

279. Beyond Postgres
• Avoid serialization in application code
• Feature Flags
• Browse only mode (Read only mode)
• Don’t use database for Queuing
• RabbitMQ
• Reconsider options for Full Text Search
30
Wednesday, September 18, 13

280. Beyond Postgres
• Avoid serialization in application code
• Feature Flags
• Browse only mode (Read only mode)
• Don’t use database for Queuing
• RabbitMQ
• Reconsider options for Full Text Search
• tsearch provided by Postgres
30
Wednesday, September 18, 13

281. Beyond Postgres
• Avoid serialization in application code
• Feature Flags
• Browse only mode (Read only mode)
• Don’t use database for Queuing
• RabbitMQ
• Reconsider options for Full Text Search
• tsearch provided by Postgres
• Solr, Lucene
30
Wednesday, September 18, 13

282. 31
Wednesday, September 18, 13

283. Further reading . . .
31
Wednesday, September 18, 13

284. Further reading . . .
• Scalable Internet Architectures - Theo Schlossnagle
• Web Operations: Keeping the Data On Time - John Allspaw , Jesse
Robbins
• PostgreSQL 9.0 High Performance - Greg Smith
31
Wednesday, September 18, 13

285. 32
Wednesday, September 18, 13

286. References
32
Wednesday, September 18, 13

287. References
32
Wednesday, September 18, 13

288. References
• http://www.postgresql.org/docs/
• http://www.circonus.com
• http://pgbouncer.projects.pgfoundry.org/doc/usage.html
• http://pgxn.org/dist/pg_partman/doc/pg_partman.html
• http://dalibo.github.io/pgbadger/
• https://github.com/omniti-labs/omnipitr
• http://instagram-engineering.tumblr.com/post/10853187575/
sharding-ids-at-instagram
• http://memcached.org
• http://redis.io
• http://postgres-xc.sourceforge.net/docs/1_1/
32
Wednesday, September 18, 13

289. Thanks
• PostgresOpen Conference Committee
• OmniTI
• You!!
33
Wednesday, September 18, 13