Hot standby and streaming replication move the needle forward for high availability and scaling for a wide number of applications. Continuent Tungsten has earlier supported PostgreSQL clustering using warm standby. In this talk we will describe how to build clusters using the PostgreSQL 9 features and give our report from the trenches. This talk will cover how PostgreSQL 9 hot standby and streaming replication work from a user perspective, then dive into a description of how Tungsten really makes them to shine. We'll cover the following issues: * Configuration of warm standby and streaming replication * Provisioning new standby instances * Strategies for balancing reads across primary and standby database * Managing failover Please join us for an enlightening presentation a set of PostgreSQL 9 features that are interesting to a wide range of PostgreSQL users.

1. Linas Virbalas
Continuent, Inc.
© Continuent 2010

2. / Introductions
/ What is Tungsten?
/ PG 9: Moving from Warm to Hot Standby
/ Tungsten + PostgreSQL Hot Standby
and Streaming Replication
/ Tungsten’s Feature: MySQL to PostgreSQL
Replication
/ Demo!
/ Questions and Answers
© Continuent 2010

3. © Continuent 2010

4. / Our Value: Data reliability for data-driven businesses
• Enterprise capability
• Open source cost
• Integrated solution
/ Our Solution: Tungsten Database Clustering
/ Our Company:
• Founded in Finland, HQ in USA
• Venture-backed
• US/European presence
• Deep experience in databases and distributed systems
© Continuent 2010

5. Data-Driven Businesses
and Their Discontents
© Continuent 2010

6. Challenge 1: Business Growth
/ Growing business = growing transaction and data
volumes
/ Non-linear labor cost increases
/ Limits in the amount of data and machines that you
can practically manage
© Continuent 2010

7. Challenge 2: True High Availability
Fault tolerance alone is no longer
sufficient for high availability
The number one cause of downtime
in many applications is scheduled
database maintenance and upgrades
© Continuent 2010

8. Challenge 3: Complex Data Management
/ Data-driven businesses have “big league” topologies
/ Multi-master and cross-site flows are standard
Shared Data
Shared Data
Tenant and Tenant Tenant
Shared Data Apps Apps
Tenant Tenant
Data Data
Tenant Tenant
Analytics Analytics
© Continuent 2010

9. What is Tungsten and How
Does It Work?
© Continuent 2010

10. What Is Tungsten?
/ Tungsten creates integrated data services
using of-the-shelf PostgreSQL/MySQL
databases to:
• Protect data
• Keep applications constantly available
• Improve resource utilization
• Raise performance
© Continuent 2010

11. What’s Inside Tungsten?
/ Replication - Making copies
• Tungsten Replicator -- Database-neutral, platform independent
master/slave replication
/ Connectivity -- Finding databases
• Tungsten Connector -- Fast MySQL/PostgreSQL client to JDBC
proxying
• Tungsten SQL Router --JDBC wrapper for high-performance and
transparent failover, load-balancing, and partitioning (no proxy
required)
/ Management -- Administering the database
• Tungsten Manager -- Distributed administration with autonomic,
rule-based configuration and no single point of failure
• Tungsten Monitor -- Track resource status and
© Continuent 2010

12. Apache/Mod_PHP Apache/Mod_PHP
Manager Connector Connector Manager
Replicator Replicator Replicator
Manager Manager Manager
© Continuent 2010

13. Technology: Replication Pipelines
© Continuent 2010

14. Technology: SQL Routing
Java App Server PHP Application
Tungsten SQL Router libmysqlclient.a
MySQL JDBC Driver
Tungsten
Connector
Clustered Databases
© Continuent 2010

15. Technology: Distributed Management
Admin Client
Manager
Manager
(Coordinator)
Admin Client
Manager
Admin Client
© Continuent 2010

16. © Continuent 2010

17. PostgreSQL 8.4 Warm Standby
© Continuent 2010

18. Limitations of Warm Standby
1. Utilization -- Cannot open the standby
• To bring up the standby for queries you must end recovery
• Standby hardware is idle
• Difficult to track state of recovery since you cannot query log
position
2. Data Loss -- Warm standby transfers only full WAL
files
• Can bound loss using archive_timeout
• Low values create large numbers of WAL files; complicate point-
in-time recovery
• Workarounds using DRBD, etc. are complex
© Continuent 2010

19. Introducing Hot Standby
/ Allows users to connect to standby in read-only mode
• Allowed: SELECT, SET, LOAD, COMMIT/ROLLBACK
• Disallowed: INSERT, UPDATE, DELETE, CREATE, 2PC,
SELECT … FOR SHARE/UPDATE, nextval(), LISTEN, LOCK,
• No admin commands: ANALYZE, VACUUM, REINDEX, GRANT
/ Can come out of recovery while queries are running
© Continuent 2010

20. Introducing Log Streaming
© Continuent 2010

21. Configuration and Usage
/ Log streaming layers on top of existing warm standby
log shipping
/ Multiple standby servers allowed
/ Failure of one standby does not affect others
/ Manual management is not simple - must coordinate
provisioning & WAL shipping to set up/restart
© Continuent 2010

22. © Continuent 2010

23. Moving Tungsten to PostgreSQL
/ Problem: We can’t read PostgreSQL logs (yet)
Tungsten solution is to manage:
/ Warm Standby + WAL Shipping
(for PostgreSQL <9)
• Good basic availability/fast failover
• Slaves open up for reads only after failover
• No load balancing/scaling
/ Hot Standby + Streaming Replication
(for PostgreSQL >=9)
• Slaves opened up for reads = Tungsten scaling facilities work
• Add Streaming Replication = minimal delay in replicating data
• With these features available, Tungsten brings true scale-out
solution for PostgreSQL 9. With Tungsten PostgreSQL users gain
high availability and scalability MySQL users have been able to
enjoy over the years
© Continuent 2010

24. What is Tungsten’s Added Value To
PostgreSQL?
/ 15 minute cluster installation
/ Single commands to:
• View cluster status
• Provision a new standby
• Confirm liveness of replication
• Switch servers safely for maintenance
• Failover a dead server to most current replica
/ Automatic discovery of new database replicas
/ Automatic failover when databases fail
/ Simple procedures for provisioning
/ Transparent application routing
/ Easy scaling
© Continuent 2010

25. How Tungsten covers all this?
Tungsten Manager
Replicator JMX Interface Monitor
Replication State Model
Open Script
Plugin
© Continuent 2010

26. © Continuent 2010

27. Master Slave
Replicator Replicator
Transaction Transaction
History Log History Log
Filters Filters
MySQL PostgreSQL
Extractor Applier
© Continuent 2010

28. © Continuent 2010

29. © Continuent 2010

30. HQ and Americas EMEA and APAC
560 S. Winchester Blvd., Suite 500 Lars Sonckin kaari 16
San Jose, CA 95128 02600 Espoo, Finland
Tel (866) 998-3642 Tel +358 50 517 9059
Fax (408) 668-1009 Fax +358 9 863 0060
e-mail: sales@continuent.com
Continuent Web Site:
http://www.continuent.com
© Continuent 2010