ProxySQL is a MySQL protocol proxy that provides high availability, scalability, and security for MySQL database systems. It allows clients to connect to ProxySQL, which then evaluates requests and performs actions like routing queries to backend databases, caching reads, connection pooling, and load balancing across servers. ProxySQL's main features include query routing, firewalling, real-time statistics, monitoring, and management of large numbers of backend servers. The presentation discusses using ProxySQL's query routing and rewriting capabilities to mask sensitive data when replicating databases for development environments. It also covers using the REST API and Prometheus integration to configure ProxySQL and monitor metrics without direct SQL access.

1. ProxySQL and the
Tricks Up Its Sleeve
Ideas on How to Manage
Your Database Systems

2. Who am I?
Jesmar Cannaò
• COO at ProxySQL LLC
• ProxySQL Consultant &
Supporter
• MySQL DBA

3. ProxySQL LLC
We provide services to help build, support as well as improve the
performance & reliability of your Cloud-Based or On-Premise MySQL
infrastructure:
● ProxySQL Development
● Remote DBRE Consulting
● ProxySQL Support Services

4. We are hiring!
● Experience coding in C/C++?
● MySQL DBA / Development?
● DevOps / Automation?
● Working remotely?

5. Application and Database
layers
APPLICATIONS
DATABASES

6. APPLICATIONS
Database as a Service
(layered)
DATABASES + MANAGER(s)
DAAS – REVERSE PROXY

7. What is ProxySQL?
MySQL protocol aware data gateway
– Clients connect to ProxySQL
– Requests are evaluated
– Various actions are performed
Visit https://proxysql.com for more information

8. Main features
● High Availability and Scalability
● seamless failover
● firewall
● query throttling
● query timeout
● query mirroring
● runtime reconfiguration
● Scheduler
● Support for Async Replication, Galera/PXC, Group Replication, Aurora

9. Main features
● on-the-fly rewrite of queries
● caching reads outside the database
● connection pooling and multiplexing
● complex query routing and r/w split
● load balancing
● real time statistics
● monitoring
● Data masking
● Management of hundreds of backend servers
● Native Clustering

10. Agenda
• Query routing with thousands of schemas;
• ProxySQL Query firewalling;
• Rebuild environment keeping sensitive data safe;
• Stop having hanging transactions;
• Trigger ProxySQL configuration without writing any
SQL;
• MySQL Replication using ProxySQL;
• ProxySQL integrated Prometheus Exporter;
• Shield your database from spikes in created

11. Thousands of schemas
and Query Routing
● How to perform fast routing

12. Routing on user/schema
mysql_query_rules offers routing based on username and schemaname:
INSERT INTO mysql_query_rules (rule_id, username, schemaname,
destination_hostgroup, apply) VALUES
(11, user1, schema1, 10, 1),
(12, user1, schema2, 10, 1),
(13, user1, schema3, 20, 1),
(14, user1, schema4, 20, 1),
(15, user1, schema5, 20, 1);

13. Routing on user/schema
Same schemas on multiple hostgroups for different users:
INSERT INTO mysql_query_rules (rule_id, username, schemaname,
destination_hostgroup, apply) VALUES
(21, user2, schema1, 30, 1),
(22, user2, schema2, 30, 1),
(23, user2, schema3, 40, 1),
(24, user2, schema4, 40, 1),
(25, user2, schema5, 40, 1);

14. Routing on user/schema
Does it scale
with thousands
of rules?
Note: Average vs
Maximum latency

15. mysql_query_rules_fast_routing
CREATE TABLE mysql_query_rules_fast_routing (
username VARCHAR NOT NULL,
schemaname VARCHAR NOT NULL,
flagIN INT NOT NULL DEFAULT 0,
destination_hostgroup INT CHECK (destination_hostgroup >= 0) NOT NULL,
comment VARCHAR NOT NULL,
PRIMARY KEY (username, schemaname, flagIN) )

16. mysql_query_rules_fast_routing
INSERT INTO mysql_query_rules_fast_routing
(username, schemaname, destination_hostgroup)
VALUES
(user1,schema1,10),(user1,schema2,10),
(user1,schema3,20),(user1,schema4,20),(user1,schema5,20),
(user2,schema1,30),(user2,schema2,30),
(user2,schema3,40),(user2,schema4,40),(user2,schema5,40);

17. mysql_query_rules_fast_routing
Does it scale with
thousands of rules?
Note: Average vs
Maximum latency

18. Routing performance comparison

19. Fast routing and R/W split
INSERT INTO mysql_query_rules(rule_id,match_digest,flagOUT) VALUES
(1,’^SELECT.*FROM tablenameA’,1);
INSERT INTO mysql_query_rules_fast_routing
(username, schemaname, flagIN, destination_hostgroup)
VALUES
(user1,schema1,0,10),(user1,schema1,1,11),
(user1,schema2,0,10),(user1,schema2,1,11),
(user1,schema3,0,20),(user1,schema4,0,20),(user1,schema5,0,20),
(user1,schema3,1,21),(user1,schema4,1,21),(user1,schema5,1,21);

20. mysql_query_rules_fast_routing
Username is optional:
If a matching username/schemaname is not found, it searches for empty username + schemaname.
INSERT INTO mysql_query_rules_fast_routing
(schemaname, destination_hostgroup) VALUES (schema01,50),
(schema02,50),(schema03,50),(schema04,40),(schema05,60);
Pros: a lot less rows = less memory usage
Cons: double searches

21. ProxySQL Query Firewalling
● How to block specific queries
● How to build your own Firewall Whitelist

22. How to block specific queries?
mysql_query_rules offers error_msg:
CREATE TABLE mysql_query_rules (
rule_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 0,
…
error_msg VARCHAR,
…

23. How to build your own Firewall Whitelist
Whitelist tables:
• mysql_firewall_whitelist_users
• mysql_firewall_whitelist_rules

24. mysql_firewall_whitelist_users
CREATE TABLE mysql_firewall_whitelist_users (
active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 1,
username VARCHAR NOT NULL,
client_address VARCHAR NOT NULL,
mode VARCHAR CHECK (mode IN
('OFF','DETECTING','PROTECTING')) NOT NULL DEFAULT ('OFF'),
comment VARCHAR NOT NULL,
PRIMARY KEY (username, client_address) )

25. mysql_firewall_whitelist_users
Mode:
• OFF : allows all queries
• DETECTING: allows all queries, but not whitelisted queries are
logged in error log
• PROTECTING: allows only queries explicitly whitelisted
Where whitelisted?
mysql_firewall_whitelist_rules

26. Record traffic stats at runtime
CREATE TABLE stats_mysql_query_digest (
hostgroup INT,
schemaname VARCHAR NOT NULL,
username VARCHAR NOT NULL,
client_address VARCHAR NOT NULL,
digest VARCHAR NOT NULL,
digest_text VARCHAR NOT NULL,
count_star INTEGER NOT NULL,
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL,
sum_time INTEGER NOT NULL,
min_time INTEGER NOT NULL,
max_time INTEGER NOT NULL,
sum_rows_affected INTEGER NOT NULL,
sum_rows_sent INTEGER NOT NULL,
PRIMARY KEY(hostgroup, schemaname, username, client_address, digest))

27. mysql_firewall_whitelist_rules
CREATE TABLE mysql_firewall_whitelist_rules (
active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 1,
username VARCHAR NOT NULL,
client_address VARCHAR NOT NULL,
schemaname VARCHAR NOT NULL,
flagIN INT NOT NULL DEFAULT 0,
digest VARCHAR NOT NULL,
comment VARCHAR NOT NULL,
PRIMARY KEY (username, client_address, schemaname, flagIN, digest)
)

28. Record traffic stats on disk
CREATE TABLE history_mysql_query_digest (
dump_time INT,
hostgroup INT,
schemaname VARCHAR NOT NULL,
username VARCHAR NOT NULL,
client_address VARCHAR NOT NULL,
digest VARCHAR NOT NULL,
digest_text VARCHAR NOT NULL,
count_star INTEGER NOT NULL,
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL,
sum_time INTEGER NOT NULL,
min_time INTEGER NOT NULL,
max_time INTEGER NOT NULL,
sum_rows_affected INTEGER NOT NULL,
sum_rows_sent INTEGER NOT NULL)

29. Record traffic stats on disk
● Manually: SAVE MYSQL DIGEST TO DISK
● Automatically:
admin-stats_mysql_query_digest_to_disk

30. 30
Configure firewall users
INSERT INTO mysql_firewall_whitelist_users
(active, username, client_address, mode)
SELECT DISTINCT 1, username, '', 'DETECTING', ''
FROM mysql_users;

31. 31
Configure firewall rules
INSERT INTO mysql_firewall_whitelist_rules
(active, username, client_address, schemaname, flagIN, digest,
comment)
SELECT DISTINCT 1, username, client_address, schemaname, 0,
digest, ''
FROM stats_history.history_mysql_query_digest;

32. 32
Firewall commands
● LOAD MYSQL FIREWALL TO RUNTIME
● SAVE MYSQL FIREWALL TO DISK
● LOAD MYSQL FIREWALL FROM DISK
● SAVE MYSQL FIREWALL FROM RUNTIME

33. 33
Enable firewall globally
SET
mysql-firewall_whitelist_enabled=1;
LOAD MYSQL VARIABLES TO RUNTIME;
SAVE MYSQL VARIABLES TO DISK;

34. What error shall we send back to the
client?!
The choice is your!
“You shall not pass!” is my preferred, though
System Variable Name mysql-firewall_whitelist_errormsg
Dynamic Yes
Permitted Values Type String
Default Firewall blocked this query

35. Rebuild environment keeping
sensitive data safe
mysql_query_rules and mysqldump

36. How the mysql_query_rules would come
handy?
We could use ProxySQL mysql_query_rules to
shield the dataset from accessing sensitive
information

37. When we are in Dev/Staging/PreProd
environments, we tend to have less stringent
checks and sometime data is saved on
machine with different product owners and less
resources are being spent there…
So maybe it would be better to mask our data
already before making them available to

38. Solution?!
Again using ProxySQL mysql_query_rules …
but at source!
mysqldump or mydumper for example have very
simple and fixed form of queries like:
SELECT /*!40001 SQL_NO_CACHE */ * FROM `mytable`;

39. Query rewrite
mysql> SELECT * FROM mysql_query_rules WHERE rule_id=...G
*************************** 1. row ***************************
[..]
match_pattern: ^SELECT * FROM customer
[..]
replace_pattern: SELECT
customer_id,store_id,CONCAT(left(first_name,2),'xxxxxxx')
first_name,CONCAT(left(last_name,2),'xxxxxxxx')
last_name,CONCAT(left(email,2),'xxxxx@xxx',right(email,5))
email,address_id,active,create_date,last_update FROM customer
[..]
1 row in set (0.00 sec)

40. But what about if I do not want all the
rows?
mysql> SELECT * FROM mysql_query_rules WHERE rule_id=...G
*************************** 1. row ***************************
[..]
match_pattern: $
[..]
replace_pattern: LIMIT 500
[..]
1 row in set (0.00 sec)

41. Stop having hanging
transactions
mysql-max_transaction_time
mysql-max_transaction_idle_time

42. mysql-max_transaction_time
It defines the maximum running time for an active transactions: any transaction running for more than this time is
going to be killed.
System Variable Name mysql-max_transaction_time
Dynamic Yes
Permitted Values Type Integer (milliseconds)
Default 14400000 (4 hours)
Minimum 1000 (1 second)
Maximum 1728000000 (20 days)

43. mysql-max_transaction_idle_time
It defines the maximum idle time for an active transactions: any transaction remaining idle for more than this
time is going to be killed.
System Variable Name mysql-max_transaction_idle_time
Dynamic Yes
Permitted Values Type Integer (milliseconds)
Default 14400000 (4 hours)
Minimum 1000 (1 second)
Maximum 1728000000 (20 days)

44. Trigger ProxySQL
configuration without writing
any SQL
Via the API interface

45. Trigger ProxySQL configuration without
writing any SQL
Rest API variables involved
+-----------------------+----------------+
| variable_name | variable_value |
+-----------------------+----------------+
| admin-restapi_enabled | true |
| admin-restapi_port | 6070 |
+-----------------------+----------------+

46. Examples:
● Add user
● Flush cache
● Change server status
● Load users from a mysql server
● Kill all idle backend connections
● Scrape mysql query digest; etc.

47. What do I need to configure
CREATE TABLE restapi_routes (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 1,
timeout_ms INTEGER CHECK (timeout_ms>=100 AND
timeout_ms<=100000000) NOT NULL,
method VARCHAR NOT NULL CHECK (UPPER(method) IN ('GET','POST')),
uri VARCHAR NOT NULL,
script VARCHAR NOT NULL,
comment VARCHAR NOT NULL DEFAULT '')

48. INSERT INTO restapi_routes (active, timeout_ms,
method, uri, script, comment) values
(1,1000,'POST','change_host_status','./scripts/chang
e_host_status.sh','comm');
INSERT INTO restapi_routes (active, timeout_ms,
method, uri, script, comment) VALUES
(1,1000,'GET','flush_query_cache','./scripts/flush_q
uery_cache.sh','comm');

49. LOAD RESTAPI TO RUNTIME;
SAVE RESTAPI TO DISK;
LOAD RESTAPI FROM DISK;
SAVE RESTAPI FROM RUNTIME;

50. Who you gonna call?
Flush Query Cache:
curl -i -X GET
http://localhost:6070/sync/flush_query_cache

51. Who you gonna call?
Change host status:
Assuming local ProxySQL:
curl -i -X POST -d '{ "hostgroup_id": "0", "hostname":
"127.0.0.1", "port": 13306, "status": "OFFLINE_HARD" }'
http://localhost:6070/sync/change_host_status
Specifying server:
curl -i -X POST -d '{ "admin_host": "127.0.0.1", "admin_port":
"6032", "admin_user": "radmin", "admin_pass": "radmin",
"hostgroup_id": "0", "hostname": "127.0.0.1", "port": 13306,
"status": "OFFLINE_HARD" }'
http://localhost:6070/sync/change_host_status

52. Not going to go through them, but please, have a
look at some of those examples at
https://github.com/sysown/proxysql/tree/v2.x
/scripts

53. MySQL Replication &
ProxySQL

54. MySQL Replication vs. ProxySQL
MySQL Replication doesn’t really work when
you want to pass through ProxySQL and let
ProxySQL handle that session and also
considering also the nature of the MySQL
Replication protocol, it is not compatible with
ProxySQL’s functionalities like multiplexing,
query parsing, etc.

55. … so why you’re telling me
to use ProxySQL?
To make sure you are always connected to
the right node all the time.
To achieve that, configure
mysql_users.fast_forward=1
for the specific user you want to connect
with.

56. Use cases
• Replication across clusters;
• Debezium;
• Binlog backups;

57. ProxySQL integrated
Prometheus Exporter
Via the API interface

58. Prometheus via ProxySQL RestAPI
Once the RestAPI is enables, ProxySQL will
start exposing stats on
proxysql_address:6070/metrics
curl -i -X GET proxysql_address:6070/metric

59. admin-prometheus_memory_metrics_interval
While some metrics are collected and refreshed in real time, some metrics (currently only memory metrics) are
collected and refreshed only at regular intervals defined by admin-prometheus_memory_metrics_interval
System Variable Name admin-prometheus_memory_metrics_interval
Dynamic Yes
Permitted Values Type Integer (seconds)
Default 61
Minimum 0
Maximum 61

60. Shield your database from
spikes in created connections

61. Why? When? How?
• Massive application restart;
• Application peaks due to
events/promotions/etc;

62. Solution?
• Throttle your API requests; or
• Throttle your connections

63. mysql-throttle_connections_per_sec_to_hostgroup
System Variable Name mysql-throttle_connections_per_sec_to_hostgroup
Dynamic Yes
Permitted Values Type Integer
Default 1000000
Minimum 1
Maximum 1000000

64. ProxySQL 2.4
is out!!

65. Feedback & Thank you!
Visit: https://proxysql.com
Github: https://github.com/sysown/proxysql/
https://github.com/proxysql/
Mailing list: https://groups.google.com/g/proxysql
Twitter: @proxysql
Drop us an email: info@proxysql.com