应用MySQL读写分离以提高MySQL服务器的读写性能

  读写分离是借助MySQL中间件 ProxySQL 实现的
  ProxySQL 有两个版本:官方版和percona版,percona版是基于官方版基础上修改C++语言开发,轻量级但性能优异(支持处理千亿级数据)具有中间件所需的绝大多数功能,包括:

  1. 多种方式的读/写分离
  2. 定制基于用户、基于schema、基于语句的规则对SQL语句进行路由
  3. 缓存查询结果
  4. 后端节点监控

准备

  实现读写分离前,先实现主从复制

  注意:slave节点需要设置read_only=1
主机 IP地址 类型
CentOS7.6 192.168.36.101 Master
CentOS7.6 192.168.36.103 Slave
CentOS7.6 192.168.36.104 ProxySQL
注:实验之前为保障实验顺利进行,请关闭主机的selinux以及防火墙服务

开始搭建

Master节点修改数据库配置文件

[[email protected] ~]#cat /etc/my.cnf
[mysqld]
server_id=1     # 为Master节点设置一个全局唯一的ID号
binlog_format=row       # 基于行复制的数据库语句
log-bin=/data/bin/mysql-bin     # 启用二进制日志

重新启动数据库服务

[[email protected] ~]#service mysqld restart
Restarting mysqld (via systemctl):                         [  OK  ]

Master节点上创建带有复制权限的用户账号

MariaDB [(none)]> grant replication slave on *.* to [email protected]‘192.168.36.%‘ identified by ‘centos‘;
Query OK, 0 rows affected (0.00 sec)

查看Master的日志位置信息

MariaDB [mysql]> show master logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000001 |    912372 |
+------------------+-----------+
1 row in set (0.00 sec)

Slave节点修改配置文件

[[email protected] ~]#cat /etc/my.cnf
[mysqld]
server_id=2         # Slave节点设置全局唯一的ID号
read_only           # 只读

重新启动数据库服务

[[email protected] ~]#systemctl restart mariadb

使用Master创建的复制权限的用户账号进行同步

MariaDB [(none)]> CHANGE MASTER TO
    -> MASTER_HOST=‘192.168.36.101‘,
    ->  MASTER_USER=‘repluser‘,
    ->  MASTER_PASSWORD=‘centos‘,
    ->  MASTER_PORT=3306,
    ->  MASTER_LOG_FILE=‘mysql-bin.000001‘,
    ->  MASTER_LOG_POS=245;
Query OK, 0 rows affected (0.01 sec)

启动Slave线程

MariaDB [(none)]> slave start;
Query OK, 0 rows affected (0.00 sec)

查看线程是否启动

*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.36.101
                  Master_User: repluser
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000001
          Read_Master_Log_Pos: 7389
               Relay_Log_File: mariadb-relay-bin.000002
                Relay_Log_Pos: 7673
        Relay_Master_Log_File: mysql-bin.000001
             Slave_IO_Running: Yes      # 从节点的IO线程
            Slave_SQL_Running: Yes      # 从节点的SQL线程
....
        Seconds_Behind_Master: 0        # Master与SLave服务器差别延迟
.....
             Master_Server_Id: 1
1 row in set (0.00 sec)

检查数据同步情况

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| hellodb            |
| mysql              |
| performance_schema |
| test               |
+--------------------+
5 rows in set (0.00 sec)

配置ProxySQL的YUM仓库

[[email protected] ~]#cat <<EOF | tee /etc/yum.repos.d/proxysql.repo
> [proxysql_repo]
> name= ProxySQL YUM repository
> baseurl=http://repo.proxysql.com/ProxySQL/proxysql-1.4.x/centos/\$releasever
> gpgcheck=1
> gpgkey=http://repo.proxysql.com/ProxySQL/repo_pub_key
> EOF
[proxysql_repo]
name= ProxySQL YUM repository
baseurl=http://repo.proxysql.com/ProxySQL/proxysql-1.4.x/centos/$releasever
gpgcheck=1
gpgkey=http://repo.proxysql.com/ProxySQL/repo_pub_key

安装ProxySQL

[[email protected] ~]#yum install -y proxysql mariadb

启动ProxySQL

[[email protected] ~]#rpm -ql proxysql
/etc/init.d/proxysql
...

由于proxysql启动脚本在init.d文件中,所以需要使用service启动
[[email protected] ~]#service proxysql start
Starting ProxySQL: 2019-05-08 17:58:16 [INFO] Using config file /etc/proxysql.cnf
DONE!

端口查看

[[email protected] ~]#netstat -at
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:mysql           0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:6032            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:6033            0.0.0.0:*               LISTEN
# 6033端口是接收远程用户的连接、6032端口是连接,管理接口

连接proxysql管理接口

[[email protected] ~]#mysql -uadmin -padmin -P6032 -h127.0.0.1
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.5.30 (ProxySQL Admin Module)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

MySQL [(none)]>
ProxySQL相当于小型的MySQL,自带:用户名admin、密码admin

添加主从节点的地址

# 添加Master节点
MySQL [(none)]> insert into mysql_servers(hostgroup_id,hostname,port) values(10,‘192.168.36.101‘,3306);
Query OK, 1 row affected (0.00 sec)

# 添加Slave节点
MySQL [(none)]> insert into mysql_servers(hostgroup_id,hostname,port) values(10,‘192.168.36.103‘,3306);
Query OK, 1 row affected (0.00 sec)

# 查看添加信息
MySQL [(none)]> select *from mysql_servers;
+--------------+----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname       | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 10           | 192.168.36.101 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0  |         |
| 10           | 192.168.36.103 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0  |         |
+--------------+----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
2 rows in set (0.00 sec)

在Master上创建ProxySQL管理的账号

MariaDB [(none)]> grant replication client on *.* to [email protected]‘192.168.36.%‘ identified by ‘magedu‘;
Query OK, 0 rows affected (0.00 sec)

ProxySQL配置监控的用户名和密码设置,使其自动连接主从节点进行调整

MySQL [(none)]> set mysql-monitor_username=‘monitor‘;
Query OK, 1 row affected (0.00 sec)

MySQL [(none)]> set mysql-monitor_password=‘magedu‘;
Query OK, 1 row affected (0.00 sec)

使配置加载到内存中生效,并保存到磁盘中

MySQL [(none)]> load mysql variables to runtime;
Query OK, 0 rows affected (0.00 sec)

MySQL [(none)]> save mysql variables to disk;
Query OK, 97 rows affected (0.01 sec)

查看监控连接是否正常

MySQL [(none)]> select *from mysql_server_connect_log;
+----------------+------+------------------+-------------------------+-------------------------------------------------------------------------+
| hostname       | port | time_start_us    | connect_success_time_us | connect_error                                                           |
+----------------+------+------------------+-------------------------+-------------------------------------------------------------------------+
| 192.168.36.101 | 3306 | 1557312316296707 | 0                       | Access denied for user ‘monitor‘@‘192.168.36.104‘ (using password: YES) |
| 192.168.36.101 | 3306 | 1557312557263893 | 0                       | Access denied for user ‘monitor‘@‘192.168.36.104‘ (using password: YES) |
| 192.168.36.101 | 3306 | 1557312616308042 | 0                       | Access denied for user ‘monitor‘@‘192.168.36.104‘ (using password: YES) |
| 192.168.36.103 | 3306 | 1557312617121004 | 0                       | Access denied for user ‘monitor‘@‘192.168.36.104‘ (using password: YES) |
| 192.168.36.103 | 3306 | 1557312676308396 | 0                       | Access denied for user ‘monitor‘@‘192.168.36.104‘ (using password: YES) |
| 192.168.36.101 | 3306 | 1557312676936371 | 0                       | Access denied for user ‘monitor‘@‘192.168.36.104‘ (using password: YES) |
| 192.168.36.101 | 3306 | 1557312694163848 | 2228                    | NULL                                                                    |
| 192.168.36.103 | 3306 | 1557312695077512 | 4613                    | NULL                                                                    |
| 192.168.36.103 | 3306 | 1557312754164398 | 1370                    | NULL                                                                    |
| 192.168.36.103 | 3306 | 1557312874168899 | 2204                    | NULL                                                                    |
| 192.168.36.101 | 3306 | 1557312874890981 | 2939                    | NULL                                                                    |
+----------------+------+------------------+-------------------------+-------------------------------------------------------------------------+
22 rows in set (0.00 sec)

设置分组信息

MySQL [(none)]> insert into mysql_replication_hostgroups values(10,20,"test");
Query OK, 1 row affected (0.00 sec)

查看读写组信息

MySQL [(none)]> select *from mysql_replication_hostgroups;
+------------------+------------------+---------+
| writer_hostgroup | reader_hostgroup | comment |
+------------------+------------------+---------+
| 10               | 20               | test    |
+------------------+------------------+---------+
1 row in set (0.00 sec)

配置生效并保存到磁盘

MySQL [(none)]> save mysql servers to disk;
Query OK, 0 rows affected (0.02 sec)

查看信息

MySQL [(none)]> select *from mysql_servers;
+--------------+----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname       | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 10           | 192.168.36.101 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0  |         |
| 20           | 192.168.36.103 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0  |         |
+--------------+----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
2 rows in set (0.00 sec)

在master上进行创建设置账户,让客户端连接中间的调度器

MariaDB [(none)]> grant all on *.* to [email protected]‘192.168.36.%‘ identified by ‘magedu‘;
Query OK, 0 rows affected (0.00 sec)

ProxySQL添加记录,将用户添加到mysql_users表中

MySQL [(none)]> insert into mysql_users(username,password,default_hostgroup) values(‘sqluser‘,‘magedu‘,10);
Query OK, 1 row affected (0.00 sec)

查看是否添加成功

MySQL [(none)]> select *from mysql_users;
+----------+----------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+
| username | password | active | use_ssl | default_hostgroup | default_schema | schema_locked | transaction_persistent | fast_forward | backend | frontend | max_connections|
+----------+----------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+
| sqluser  | magedu   | 1      | 0       | 10                | NULL           | 0             | 1                      | 0            | 1       | 1        | 10000|
+----------+----------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+
1 row in set (0.00 sec)

保存配置

MySQL [(none)]> load mysql users to runtime;
Query OK, 0 rows affected (0.00 sec)

MySQL [(none)]> save mysql users to disk;
Query OK, 0 rows affected (0.01 sec)

启用一个客户端连接ProxySQL进行测试

[[email protected] ~]# mysql -usqluser -pmagedu -h192.168.36.104 -P6033
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.30 (ProxySQL)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

mysql>

查看连接到哪个主机

mysql> select @@server_id;
+-------------+
| @@server_id |
+-------------+
|           1 |
+-------------+
1 row in set (0.00 sec)

ProxySQL上定义调度规则

MySQL [(none)]> insert into mysql_query_rules
    -> (rule_id,active,match_digest,destination_hostgroup,apply) values
    -> (1,1,‘^select.*from update$‘,10,1),(2,1,‘^select‘,20,1);
Query OK, 2 rows affected (0.00 sec)

生效并存盘

MySQL [(none)]> load mysql query rules to runtime;
Query OK, 0 rows affected (0.00 sec)

MySQL [(none)]> save mysql query rules to disk;
Query OK, 0 rows affected (0.01 sec)

客户端进行读写分离测试

[[email protected] ~]# mysql -usqluser -pmagedu -h192.168.36.104 -P6033 -e ‘select @@server_id;‘
+-------------+
| @@server_id |
+-------------+
|           2 |
+-------------+

[[email protected] ~]# mysql -usqluser -pmagedu -h192.168.36.104 -P6033 -e ‘begin;use db1;insert t1 values(1);select @@server_id;‘
+-------------+
| @@server_id |
+-------------+
|           1 |
+-------------+
完工

原文地址:https://blog.51cto.com/12980155/2391271

时间: 2024-08-26 07:44:16

应用MySQL读写分离以提高MySQL服务器的读写性能的相关文章

redis作为mysql的缓存服务器(读写分离,通过mysql触发器实现数据同步)

一.redis简介Redis是一个key-value存储系统.和Memcached类似,为了保证效率,数据都是缓存在内存中.区别的是redis会周期性的把更新的数据写入磁盘或者把修改操作写入追加的记录文件,并且在此基础上实现了master-slave(主从)同步.在部分场合可以对关系数据库起到很好的补充作用.它提供了Java,C/C++(hiredis),C#,PHP,JavaScript,Perl,Object-C,Python,Ruby等客户端,使用很方便. 二.架构图<ignore_js_

Mysql读写分离方案-MySQL Proxy环境部署记录

Mysql的读写分离可以使用MySQL Proxy和Amoeba实现,其实也可以使用MySQL-MMM实现读写分离的自动切换.MySQL Proxy有一项强大功能是实现"读写分离",基本原理是让主数据库处理写方面事务,让从库处理SELECT查询:Amoeba for MySQL是一款优秀的中间件软件,同样可以实现读写分离,负载均衡等功能.下面重点说下Mysql Proxy: MySQL Proxy处于客户端应用程序和MySQL服务器之间,通过截断.改变并转发客户端和后端数据库之间的通信

Mysql数据库优化一:集群(读写分离)之主从服务器的安装与配置

Mysql数据库的集群(读写分离),说白了就是将读操作和写操作分开在不同的服务器上实现,以达到提高效率的目的. 大致原理如下: 数据库中的所有操作都是有日志记录的(前提是要打开这个日志记录功能) 1.master,主服务器(进行写操作的服务器)的所有记录都保存到二进制日志(binary log)中,这些记录叫做二进制日志事件(binary log events): 2.slave,从服务器(进行读操作的服务器)将mater的binart log拷贝到自己的中继日志(relay log),然后根据

Mysql主从配置,实现数据读写分离(amoeba+mysql) .

一.配置Master主服务器 1.创建新用户,并授权给slave数据库,192.168.0.%表示从数据库IP地址,'mysql'表示密码 mysql>create user repl; //创建新用户mysql> GRANT REPLICATION SLAVE ON . TO 'repl'@'192.168.0.%' IDENTIFIED BY 'mysql'; 2.找到MySQL安装文件夹修改my.Ini文件,在[mysqld]下面增加下面几行代码 server-id=1 //给数据库服务

mysql读写分离的操作动作依据(读写分离基本依据)

读的操作: 1.select 2.show 3.explain explain显示了MySQL如何使用索引来处理select语句以及连接表.可以帮助选择更好的索引和写出更优化的查询语句. 4.desc 4.use 写的操作: 2.delete 3.update 4.insert 5.drop 6.truncate 清空表数据 7.creat 8     .rename命令用于修改表名. rename命令格式:rename table 原表名 to 新表名; 9 alter . 表结构的修改 (1

MySQL查询缓存设置 提高MySQL查询性能

首先看看MSYQL逻辑框架:图片来自高性能mysql 如果使用了QueryCache,当查询接收到一个和之前同样的查询,服务器将会从查询缓存中检索结果,而不是再次分析和执行相同的查询.这样就能大大提高查询性能. 打开查询缓存,要通过几个步骤来设置: 虽然你设置mysql允许查询缓存,但是如果你设置的查询缓存大小为了0,这和没有允许没什么区别. 所以必须是几个步骤的设置才能真正打开查询缓存这个功能. 下面演示最常用的设置查询缓存 一. query_cache_type 使用查询缓存的方式 一般,我

MySQL查询缓存设置提高MySQL查询性能

首先看看MSYQL逻辑框架:图片来自高性能mysql 如果使用了QueryCache,当查询接收到一个和之前同样的查询,服务器将会从查询缓存中检索结果,而不是再次分析和执行相同的查询.这样就能大大提高查询性能. 打开查询缓存,要通过几个步骤来设置: 虽然你设置mysql允许查询缓存,但是如果你设置的查询缓存大小为了0,这和没有允许没什么区别. 所以必须是几个步骤的设置才能真正打开查询缓存这个功能. 下面演示最常用的设置查询缓存 一. query_cache_type 使用查询缓存的方式 一般,我

2-18,19 搭建MySQL主从服务器并并通过mysql-proxy实现读写分离

MySQL主从服务器 实现方式: MySQL  REPLICATION Replication可以实现将数据从一台数据库服务器(master)复制到一台或多台数据库服务器(slave) 默认情况下这种情况属于异步复制,无需维持长连接 通过配置,可以复制所有库或者几个库,甚至库中的一些表 它是MySQL内建的,自带 Replication的原理 主服务器master将数据库的改变写入二进制日志文件,从服务器slave同步这些二进制日志,并生成中继日志,从服务器根据中继日志,执行这些改变 DML:S

MySQL Proxy 实现 MySQL 读写分离提高并发负载

工作拓扑: MySQL Proxy有一项强大功能是实现"读写分离",基本原理是让主数据库处理写方面事务,让从库处理SELECT查询. Amoeba for MySQL是一款优秀的中间件软件,同样可以实现读写分离,负载均衡等功能,并且稳定性也高于MySQL Proxy,有兴趣的可以测试一下. 环境描述: 操作系统:CentOS6.3_x64 主服务器Master:192.168.0.202 从服务器Slave:192.168.0.203 调度服务器MySQL-Proxy:192.168.