讀寫分離實測
ProxySQL官方demo演示了三種讀寫分離的方式:使用不同的端口進行讀寫分離、使用正則表達式進行通用的讀寫分離、使用正則和digest進行更智能的讀寫分離。最后一種是針對特定業務進行的優化調整,也可將其歸結為第二種方式,下邊分別進行測試。
基于端口的讀寫分離
環境準備
- MySQL里創建訪問用戶,監控用戶
SQL #創建監控賬號 create user monitor@'192.168.31.%' identified with mysql_native_password by '******'; #授權 grant replication client on *.* to monitor@'192.168.31.%' identified by '******' ;
#創建訪問用戶賬號 ??? create user proxyadmin@'192.168.31.%' identified with mysql_native_password by '******'; #開放權限 grant all privileges on *.* to proxyadmin@'192.168.31.%';
flush privileges; |
- ProxySQL配置mysql節點
SQL ##hostgroup_id, hostname, port 組成一個主鍵 mysql> ?select * from mysql_servers; Empty set (0.00 sec)
mysql> insert into mysql_servers(hostgroup_id,hostname,port) ?values(10,'192.168.31.128',5310); Query OK, 1 row affected (0.01 sec)
mysql> insert into mysql_servers(hostgroup_id,hostname,port) ?values(20,'192.168.31.130',5310); Query OK, 1 row affected (0.01 sec) mysql> ?select * from mysql_servers\G *************************** 1. row *************************** ???????hostgroup_id: 10 ???????????hostname: 192.168.31.128 ???????????????port: 5310 ??????????gtid_port: 0 ?????????????status: ONLINE ?????????????weight: 1 ????????compression: 0 ????max_connections: 1000 max_replication_lag: 0 ????????????use_ssl: 0 ?????max_latency_ms: 0 ????????????comment: ??????? *************************** 2. row *************************** ???????hostgroup_id: 20 ???????????hostname: 192.168.31.130 ???????????????port: 5310 ??????????gtid_port: 0 ?????????????status: ONLINE ?????????????weight: 1 ????????compression: 0 ????max_connections: 1000 max_replication_lag: 0 ????????????use_ssl: 0 ?????max_latency_ms: 0 ????????????comment: 2 rows in set (0.00 sec)
##加載到內存 mysql> load mysql servers to runtime; Query OK, 0 rows affected (0.04 sec) ##持久化到磁盤 mysql> save mysql servers to disk; Query OK, 0 rows affected (0.08 sec) |
- 使用用戶配置,配置mysql_users表
SQL mysql> insert into mysql_users(username,password,default_hostgroup) values('proxyadmin','123qwe',10); Query OK, 1 row affected (0.00 sec) mysql> load mysql users to runtime; Query OK, 0 rows affected (0.01 sec)
mysql> save mysql users to disk; Query OK, 0 rows affected (0.02 sec)
mysql> select * from mysql_users\G *************************** 1. row *************************** ??????????????username: proxyadmin ??????????????password: 123qwe ????????????????active: 1 ???????????????use_ssl: 0 ?????default_hostgroup: 10 ????????default_schema: NULL ?????????schema_locked: 0 transaction_persistent: 1 ??????????fast_forward: 0 ???????????????backend: 1 ??????????????frontend: 1 ???????max_connections: 10000 ????????????attributes: ???????????????comment: 1 row in set (0.01 sec) |
- 監控用戶配置
Monitor模塊就會開始監控后端的read_only值,當監控到read_only值后,就會按照read_only的值將某些節點自動移動到讀/寫組
SQL mysql> set mysql-monitor_username='monitor'; Query OK, 1 row affected (0.00 sec)
mysql> set mysql-monitor_password='123qwe'; Query OK, 1 row affected (0.00 sec)
mysql> load mysql variables to runtime; Query OK, 0 rows affected (0.02 sec)
mysql> save mysql variables to disk; Query OK, 158 rows affected (0.02 sec) |
- 配置基于端口的路由規則
寫端口通過6401端口,讀端口通過6402端口,從而通過不同端口來實現讀寫分離功能。
SQL INSERT INTO mysql_query_rules (rule_id,active,proxy_port,destination_hostgroup,apply) VALUES (1,1,6401,10,1), (2,1,6402,20,1);
mysql> load mysql query rules to runtime; Query OK, 0 rows affected (0.01 sec)
mysql> save mysql query rules to disk; Query OK, 0 rows affected (0.03 sec) |
- 設置讀寫分離端口
因端口的修改不能動態加載,需重啟ProxySQL實例
SQL mysql> SET mysql-interfaces='0.0.0.0:6401;0.0.0.0:6402'; Query OK, 1 row affected (0.00 sec)
mysql> load mysql variables to runtime; Query OK, 0 rows affected (0.00 sec)
mysql> save mysql variables to disk; Query OK, 158 rows affected (0.01 sec) |
驗證
SQL [user@server1 proxysql]$ mysql -uproxyadmin -p***** -P6401 -h127.0.0.1 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. ?Commands end with ; or \g. Your MySQL connection id is 6 Server version: 5.5.30 (ProxySQL)
Copyright (c) 2000, 2023, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> select @@hostname; +------------+ | @@hostname | +------------+ | server1 ???| +------------+ 1 row in set (0.01 sec)
[user@server1 proxysql]$ mysql -uproxyadmin -p***** -P6402 -h127.0.0.1 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. ?Commands end with ; or \g. Your MySQL connection id is 6 Server version: 5.5.30 (ProxySQL)
Copyright (c) 2000, 2023, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> select @@hostname; +------------+ | @@hostname | +------------+ | server2 ???| +------------+ 1 row in set (0.01 sec) |
測試驗證
- 所有來自于6401端口連接的查詢都將被發送到hostgroup為10的組中;
- 所有來自于6402端口連接的查詢都將被發送到hostgroup為20的組中;
配置缺點
- 該配置需要應用有內置的讀寫分離功能,以便區分讀和寫;
- 通常應用只配置一個單獨的入口來連接ProxySQL,這對很多業務來說是不可接受的。
- 信息查詢
#read_only日志此時也為空(正常來說,新環境配置時,這個只讀日志是為空的)
Plaintext MySQL [(none)]> select * from mysql_server_read_only_log; |
#replication_lag的監控日志為空
Plaintext MySQL [(none)]> select * from mysql_server_replication_lag_log; |
#注意:可能會有很多connect_error,這是因為沒有配置監控信息時的錯誤,配置后如果connect_error的結果為NULL則表示正常。
Plaintext MySQL [(none)]> select * from mysql_server_connect_log; |
#以下是對心跳信息的監控(對ping指標的監控)
Plaintext MySQL [(none)]> select * from mysql_server_ping_log; |
查看路由的信息,可查詢stats庫中的stats_mysql_query_digest表。
Plaintext mysql> select* from stats_mysql_query_digest; mysql>SELECT hostgroup hg, sum_time, count_star, digest_text FROM stats_mysql_query_digest ORDER BY |
- 開啟web統計功能
查看路由的信息,可查詢stats庫中的stats_mysql_query_digest表。
SQL mysql> update global_variables set variable_value='true' where variable_name='admin-web_enabled'; Query OK, 1 row affected (0.01 sec)
mysql> LOAD ADMIN VARIABLES TO RUNTIME; Query OK, 0 rows affected (0.02 sec)
mysql> ?SAVE ADMIN VARIABLES TO DISK; Query OK, 49 rows affected (0.01 sec)
mysql> select * from global_variables where variable_name LIKE 'admin-web%' or variable_name LIKE 'admin-stats%'; +----------------------------------------+----------------+ | variable_name ?????????????????????????| variable_value | +----------------------------------------+----------------+ | admin-stats_credentials ???????????????| stats:***** ???| | admin-stats_mysql_connections ?????????| 60 ????????????| | admin-stats_mysql_connection_pool ?????| 60 ????????????| | admin-stats_mysql_query_cache ?????????| 60 ????????????| | admin-stats_mysql_query_digest_to_disk | 0 ?????????????| | admin-stats_system_cpu ????????????????| 60 ????????????| | admin-stats_system_memory ?????????????| 60 ????????????| | admin-web_enabled ?????????????????????| true ??????????| | admin-web_port ????????????????????????| 6080 ??????????| | admin-web_verbosity ???????????????????| 0 ?????????????| +----------------------------------------+----------------+ 10 rows in set (0.00 sec) |
訪問http://192.168.31.128:6080并使用stats:****登錄即可查看一些統計信息。

基于正則規則的讀寫分離
開啟eventlog
eventslog可以記錄用戶在通過ProxySQL訪問時的所有SQL語句,可以用其觀察具體的路由規則,以下為開啟方法:
SQL ## 設置eventslog的文件名稱 mysql> SET mysql-eventslog_filename='queries.log'; Query OK, 1 row affected (0.00 sec) ## 設置eventslog的格式為json mysql> ?SET mysql-eventslog_format=2; Query OK, 1 row affected (0.00 sec) ## 加載到runtime層并保存到持久層 mysql> load mysql variables to runtime; Query OK, 0 rows affected (0.00 sec)
mysql> save mysql variables to disk; Query OK, 158 rows affected (0.01 sec) |
清除之前創建的查詢規則:
SQL mysql> delete from mysql_query_rules; Query OK, 2 rows affected (0.00 sec)
mysql> SET mysql-interfaces='0.0.0.0:6033';; Query OK, 1 row affected (0.00 sec)
mysql> load mysql variables to runtime; Query OK, 0 rows affected (0.00 sec)
mysql> save mysql variables to disk; Query OK, 158 rows affected (0.01 sec)
## 延續上面配置,添加查詢規則記錄eventslog,如下表示對所有的操作都記錄 INSERT INTO mysql_query_rules(active, match_digest, log,apply) VALUES(1,'.',1,0);
## 添加基于正則的路由規則 INSERT INTO mysql_query_rules(active,match_digest,destination_hostgroup,apply) VALUES(1,'^SELECT.*FOR UPDATE$',10,1),(1,'^SELECT',20,1);
## 加載配置到運行層并保存配置到持久層 LOAD MYSQL QUERY RULES TO RUNTIME; SAVE MYSQL QUERY RULES TO DISK; |
測試預測
- 所有的SELECT FOR UPDATE語句將被路由到hostgroup為10的組中;
- 其他的SELECT語句將被路由到hostgroup為 20的組中;
- 除以上外的其他語句,將被路由到連接用戶默認的hostgroup組中;
測試驗證
SQL ## 小寫查詢(select請求忽略大小寫,查看eventslog,該SQL路由到讀組20) mysql> select * from sbtest1 limit 1;
## 大寫查詢(select請求忽略大小寫,查看eventslog,該SQL路由到讀組20) mysql> SELECT * from sbtest1 limit 1;
## 大小寫混合查詢(select請求忽略大小寫,查看eventslog,該SQL路由到讀組20) mysql> SelECT * from sbtest1 limit 1;
## 包含空格的查詢(ProxySQL會對包含空格的SQL進行處理,查看eventslog,該SQL路由到讀組20) mysql> ???SelECT * from sbtest1 limit 1;
## 小寫查詢(select請求忽略大小寫,查看eventslog,該SQL路由到寫組10) mysql> select * from sbtest1 limit 1 for update; |
ProxySQL集群
proxysql-1:192.168.31.128
proxysql-2:192.168.31.130
- 更改所有ProxySQL節點的配置文件
SQL # vim /etc/proxysql.cnf #修改 admin_variables= { ????????admin_credentials="admin:admin;;cluster1:******" # ??????mysql_ifaces="127.0.0.1:6032;/tmp/proxysql_admin.sock" ????????mysql_ifaces="0.0.0.0:6032" # ??????refresh_interval=2000 # ??????debug=true #集群用戶名稱,與最上面的相同 ????????cluster_username="cluster1" #集群用戶密碼,與最上面的相同 ??????? ????????cluster_password="*******" ????????cluster_check_interval_ms=200 ????????cluster_check_status_frequency=100 ????????cluster_mysql_query_rules_save_to_disk=true ????????cluster_mysql_servers_save_to_disk=true ????????cluster_mysql_users_save_to_disk=true ????????cluster_proxysql_servers_save_to_disk=true ????????cluster_mysql_query_rules_diffs_before_sync=3 ????????cluster_mysql_servers_diffs_before_sync=3 ????????cluster_mysql_users_diffs_before_sync=3 ????????cluster_proxysql_servers_diffs_before_sync=3 } #新增 proxysql_servers= ( ????????{ ????????????????hostname="192.168.31.128" ????????????????port=6032 ????????????????weight=1 ????????????????comment="ProxySQL-node1" ????????}, ????????{ ????????????????hostname="192.168.31.130" ????????????????port=6032 ????????????????weight=1 ????????????????comment="ProxySQL-node2" ????????} ) |
- 啟動兩個節點的ProxySQL服務
注意:
如果存在如果存在"proxysql.db"文件(在/var/lib/proxysql目錄下),則ProxySQL服務只有在第一次啟動時才會去讀取proxysql.cnf文件并解析;后面啟動會就不會讀取proxysql.cnf文件了!如果想要讓proxysql.cnf文件里的配置在重啟proxysql服務后生效(即想要讓proxysql重啟時讀取并解析proxysql.cnf配置文件),則需要先刪除/var/lib/proxysql/proxysql.db數據庫文件,然后再重啟proxysql服務。這樣就相當于初始化啟動proxysql服務了,會再次生產一個純凈的proxysql.db數據庫文件(如果之前配置了proxysql相關路由規則等,則就會被抹掉)。
- 觀察集群狀況 (兩個節點上都可以查看,無特殊說明均在192.168.31.128上操作))
SQL mysql> select * from proxysql_servers; +----------------+------+--------+----------------+ | hostname ??????| port | weight | comment ???????| +----------------+------+--------+----------------+ | 192.168.31.128 | 6032 | 1 ?????| ProxySQL-node1 | | 192.168.31.130 | 6032 | 1 ?????| ProxySQL-node2 | +----------------+------+--------+----------------+ 2 rows in set (0.00 sec) mysql> select * from stats_proxysql_servers_metrics; +----------------+------+--------+----------------+------------------+----------+---------------+---------+------------------------------+----------------------------+ | hostname ??????| port | weight | comment ???????| response_time_ms | Uptime_s | last_check_ms | Queries | Client_Connections_connected | Client_Connections_created | +----------------+------+--------+----------------+------------------+----------+---------------+---------+------------------------------+----------------------------+ | 192.168.31.130 | 6032 | 1 ?????| ProxySQL-node2 | 0 ???????????????| 0 ???????| 46226673 ?????| 0 ??????| 0 ???????????????????????????| 0 ?????????????????????????| | 192.168.31.128 | 6032 | 1 ?????| ProxySQL-node1 | 9 ???????????????| 141 ?????| 18414 ????????| 0 ??????| 0 ???????????????????????????| 0 ?????????????????????????| +----------------+------+--------+----------------+------------------+----------+---------------+---------+------------------------------+----------------------------+ 2 rows in set (0.00 sec)
mysql> select hostname,port,comment,Uptime_s,last_check_ms from stats_proxysql_servers_metrics; +----------------+------+----------------+----------+---------------+ | hostname ??????| port | comment ???????| Uptime_s | last_check_ms | +----------------+------+----------------+----------+---------------+ | 192.168.31.130 | 6032 | ProxySQL-node2 | 0 ???????| 46285888 ?????| | 192.168.31.128 | 6032 | ProxySQL-node1 | 201 ?????| 17332 ????????| +----------------+------+----------------+----------+---------------+ 2 rows in set (0.00 sec)
mysql> select hostname,name,checksum,updated_at from stats_proxysql_servers_checksums; +----------------+-------------------+--------------------+------------+ | hostname ??????| name ?????????????| checksum ??????????| updated_at | +----------------+-------------------+--------------------+------------+ | 192.168.31.130 | admin_variables ??| ???????????????????| 0 ?????????| | 192.168.31.130 | mysql_query_rules | ???????????????????| 0 ?????????| | 192.168.31.130 | mysql_servers ????| ???????????????????| 0 ?????????| | 192.168.31.130 | mysql_users ??????| ???????????????????| 0 ?????????| | 192.168.31.130 | mysql_variables ??| ???????????????????| 0 ?????????| | 192.168.31.130 | proxysql_servers ?| ???????????????????| 0 ?????????| | 192.168.31.128 | admin_variables ??| 0x6A7FE5E0EFF3CC92 | 1693908720 | | 192.168.31.128 | mysql_query_rules | 0x0000000000000000 | 1693908720 | | 192.168.31.128 | mysql_servers ????| 0x0000000000000000 | 1693908720 | | 192.168.31.128 | mysql_users ??????| 0x0000000000000000 | 1693908720 | | 192.168.31.128 | mysql_variables ??| 0x3F6C52D7C3EE3ADD | 1693908720 | | 192.168.31.128 | proxysql_servers ?| 0x7F9BB0FB6C104089 | 1693908720 | +----------------+-------------------+--------------------+------------+ 12 rows in set (0.00 sec) |
- 觀察ProxySQL集群中實例之間的數據同步,新增mysql節點
SQL #原有數據 mysql> select * from mysql_servers; Empty set (0.01 sec)
# 新增一個后端MySQL主機信息 mysql> insert into mysql_servers(hostgroup_id,hostname,port,comment) values (10,'192.168.31.128',5310,'master_mysql'); Query OK, 1 row affected (0.00 sec) #當前主機查看mysql_servers信息
mysql> select * from mysql_servers\G *************************** 1. row *************************** ???????hostgroup_id: 10 ???????????hostname: 192.168.31.128 ???????????????port: 5310 ??????????gtid_port: 0 ?????????????status: ONLINE ?????????????weight: 1 ????????compression: 0 ????max_connections: 1000 max_replication_lag: 0 ????????????use_ssl: 0 ?????max_latency_ms: 0 ????????????comment: master_mysql 1 row in set (0.01 sec)
#另一臺主機查看為空
# 持久化,并加載到運行環境中 mysql> save mysql servers to disk; mysql> load mysql servers to runtime;
# 再次到另一臺proxysql主機上查看,可以看到新插入的數據,發現有這個后端MySQL主機信息,已經被更新到192.168.31.130實例中的memory和runtime環境中。 mysql> select * from mysql_servers\G *************************** 1. row *************************** ???????hostgroup_id: 10 ???????????hostname: 192.168.31.128 ???????????????port: 5310 ??????????gtid_port: 0 ?????????????status: ONLINE ?????????????weight: 1 ????????compression: 0 ????max_connections: 1000 max_replication_lag: 0 ????????????use_ssl: 0 ?????max_latency_ms: 0 ????????????comment: master_mysql 1 row in set (0.01 sec)
# 查看另一臺proxysql主機日志,會看到同步的具體信息 2023-09-05 20:20:22 [INFO] Cluster: Loading to runtime MySQL Servers from peer 192.168.31.128:6032 2023-09-05 20:20:22 [INFO] Dumping mysql_servers_incoming +--------------+----------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+--------------+ | hostgroup_id | hostname ??????| port | gtid_port | weight | status | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment ?????| +--------------+----------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+--------------+ | 10 ??????????| 192.168.31.128 | 5310 | 0 ????????| 1 ?????| 0 ?????| 0 ??????????| 1000 ???????????| 0 ??????????????????| 0 ??????| 0 ?????????????| master_mysql | +--------------+----------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+--------------+ |
此時,兩節點的proxysql cluster集群搭建完畢
ProxySQL其他功能參考https://proxysql.com/documentation/
作者簡介
丁鵬,中國移動云能力中心數據庫產品部分布式數據庫高級開發工程師,負責云原生數據庫海山(He3DB)分布式版架構設計,擁有豐富的數據庫內核開發經驗和實踐知識。