Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

How to Replicate PostgreSQL Database

74 Aufrufe

Veröffentlicht am

How to Replicate PostgreSQL Database

Veröffentlicht in: Daten & Analysen
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

How to Replicate PostgreSQL Database

  1. 1. Service Platform Architect Brandon Kang sangjinn@gmail.com https://tech.brandonkang.net July 2020 How to Replicate PostgreSQL Database / / 0 2 / . 2 : / 2 . / A @ /: / / /
  2. 2. Replication Overall Primary Server • A server for both READ and WRITE Standby Server = Slave server, Backup server, Replica, etc. • A server where the data is kept in sync with the master continuously • A warm standby server can’t be connected until it is promoted to primary • A hot standby server accepts connections and serves read-only queries • Data is written to the master server and propagated to the slave servers • When primary server has an issue, a standby server will take over and continue to take ‘WRITE” process
  3. 3. WAL Shipping Replication WAL(Write-Ahead Logging) • A log file where all the modifications to the database • WAL is used for recovery after a database crash, ensuring data integrity • WAL is used in database systems to achieve atomicity and durability WAL based Replication ­ Streaming WAL Records • Write-ahead log records are used to keep the data in sync • WAL record chunks are streamed by database servers to synchronize • Standby server connects to the master to receive the WAL chunks • WAL records are streamed as they are generated • The streaming of WAL records need not wait for the WAL file to be filled
  4. 4. Streaming Replication PRIMARY Server Standby Server WAL Sender WAL Receiver WAL Record READ ONLYREAD/WRITE pg_wal directory WAL File pg_wal directory WAL File Synchronize - WAL Record1 - WAL Record2 - WAL Record3 ……
  5. 5. Replication Configuration Primary Node ­ pg_hba.conf /PG_HOME/DATA/pg_hba.conf # Allow replication connections from localhost, # by a user with the replication privilege. # TYPE DATABASE USER ADDRESS METHOD host replication repl_user IPaddress(CIDR) md5 Restart PostgreSQL service on the primary node Internal IP ranges Firewall Open for this.
  6. 6. Replication Configuration Primary Node ­/PG_HOME/DATA/postgres.conf # Possible values are replica|minimal|logical wal_level = hot_standby archive_mode = on #Syncronize with Primany node using wal archive archive_command = ‘cp %p /PG_TBS/backup/archive/%f’ #psql_superuser archive backup # required for pg_rewind capability when standby goes out of sync with master wal_log_hints = on # sets the maximum number of concurrent connections from the standby servers. max_wal_senders = 2 # The below parameter is used to tell the master to keep the minimum number of # segments of WAL logs so that they are not deleted before standby consumes them. # each segment is 16MB wal_keep_segments = 16
  7. 7. Replication Configuration Secondary Node ­/PG_HOME/DATA/postgres.conf hot_standby = on Secondary Node ­/PG_HOME/DATA/recovery.conf standby_mode = ‘on’ restore_command = ‘cp %p /PG_TBS/backup/archive/%f “%p”’ Primary_conninfo = ‘host=… ports=… user=postgres password= application_name=slave1’ # Explaining a few options used for pg_basebackup utility # -X option is used to include the required transaction log files (WAL files) in the # backup. When you specify stream, this will open a second connection to the server # and start streaming the transaction log at the same time as running the backup. # -c is the checkpoint option. Setting it to fast will force the checkpoint to be # created soon. # -W forces pg_basebackup to prompt for a password before connecting # to a database. pg_basebackup -D <data_directory> -h <master_host> -X stream -c fast -U repl_user -W
  8. 8. Replication Configuration Secondary Node ­ pg_basebackup utility # Explaining a few options used for pg_basebackup utility # -X option is used to include the required transaction log files (WAL files) in the # backup. When you specify stream, this will open a second connection to the server # and start streaming the transaction log at the same time as running the backup. # -c is the checkpoint option. Setting it to fast will force the checkpoint to be # created soon. # -W forces pg_basebackup to prompt for a password before connecting # to a database. pg_basebackup -D <data_directory> -h <master_host> -X stream -c fast -U repl_user -W Example) $pg_basebackup ­h localhost ­p 5170 -u postgres ­D /PG_TBS/cluster -checkpoint=fast --progress
  9. 9. Replication Configuration Secondary Node ­ Replication configuration file # Specifies whether to start the server as a standby. In streaming replication, # this parameter must be set to on. standby_mode = ‘on’ # Specifies a connection string which is used for the standby server to connect # with the primary/master. primary_conninfo = ‘host=<master_host> port=<postgres_port> user=<replication_user> password=<password> application_name=”host_name”’ # Specifies recovering to a particular timeline. The default is to recover along the # same timeline that was current when the base backup was taken. # Setting this to latest recovers to the latest timeline found # in the archive, which is useful in a standby server. recovery_target_timeline = ‘latest’ Start PostgreSQL service on the Standby node
  10. 10. Replication Testing postgres=# select * from pg_stat_replication ; -[ RECORD 1 ]----+--------------------------------- pid | 1114 usesysid | 16384 usename | repuser application_name | walreceiver client_addr | 127.0.0.1 client_hostname | client_port | 52444 backend_start | 08-MAR-20 19:54:05.535695 -04:00 state | streaming .. sync_priority | 0 sync_state | async ps ­ef | grep postgres [Master] wal sender process check [Standby] wal receiver & startup process check SELECT * FROM pg_stat_replication; Query execution on the Primary/Standby nodes
  11. 11. Replication Testing pg_is_in_recovery(): A function to check if standby server is still in recovery mode or not postgres=# select pg_is_in_recovery(); pg_is_in_recovery ------------------- t (1 row) pg_last_xact_replay_timestamp(): A function shows time stamp of last replica transaction postgres=# select pg_last_xact_replay_timestamp(); pg_last_xact_replay_timestamp ---------------------------------- 08-MAR-20 20:54:27.635591 -04:00 (1 row) SELECT CASE WHEN pg_last_xlog_receive_location() = pg_last_xlog_replay_location() THEN 0 ELSE EXTRACT (EPOCH FROM now() - pg_last_xact_replay_timestamp()) END AS log_delay; Calculating logs in seconds
  12. 12. Scaling PostgreSQL PRIMARY Server Standby Server WAL Sender WAL Receiver WAL Record WRITE Synchronize HAProxy / NGINX / F5 READ WRITE READ pgBouncer primary_db standby_db Application
  13. 13. - Thank You. - Service Platform Architect Brandon Kang sangjinn@gmail.com https://tech.brandonkang.net

×