MyCat 学习笔记 第十一篇.数据分片 之 分片事务处理

1 环境说明

VM 模拟3台MYSQL 5.6 服务器

　　 VM1 192.168.31.187：3307

　　 VM2 192.168.31.212：3307

　　 VM3 192.168.31.150:  3307

MYCAT 1.5 服务部署在宿主机上

　　MYCAT 192.168.31.207 :8806【SQL执行端口】 / 9066【管理端口】

2 应用场景

2.0 MYCAT配置

schema.xml

<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">
　　<table name="t_demo_travel_record" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />
　　<table name="t_demo_travel_record_child" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />
</schema>

<dataNode name="dn1" dataHost="vm1" database="test" />
<dataNode name="dn2" dataHost="vm2" database="test" />
<dataNode name="dn3" dataHost="vm3" database="test" />

<dataHost name="vm1" maxCon="1000" minCon="10" balance="0"
　　writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
　　<heartbeat>select user()</heartbeat>
　　<writeHost host="vm1M1" url="192.168.31.187:3307" user="root" password="root123"></writeHost>
</dataHost>

<dataHost name="vm2" maxCon="1000" minCon="10" balance="0"
　　writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
　　<heartbeat>select user()</heartbeat>
　　<writeHost host="vm2M1" url="192.168.31.212:3307" user="root" password="root123"></writeHost>

</dataHost>

<dataHost name="vm3" maxCon="1000" minCon="10" balance="0"
　　writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
　　<heartbeat>select user()</heartbeat>
　　<writeHost host="vm3M1" url="192.168.31.150:3307" user="root" password="root123"></writeHost>

</dataHost>

rule.xml

<tableRule name="auto-sharding-long">
　　<rule>
　　　　<columns>id</columns>
　　　　<algorithm>rang-long</algorithm>
　　</rule>
</tableRule>

<function name="rang-long"
　　class="org.opencloudb.route.function.AutoPartitionByLong">
　　<property name="mapFile">autopartition-long.txt</property>
　　<property name="defaultNode">0</property>
</function>

autopartition-long.txt

# range start-end ,data node index
# K=1000,M=10000.
0-500M=0
500M-1000M=1
1000M-1500M=2

2.1 手动关闭事，跨数据库进行更新操作。

用一套简单点的JAVA来做这次数据验证

　　DataSource ds = jdbc.getDataSource();

　　conn = ds.getConnection();

　　conn.setAutoCommit(false);

　　pstm1 = conn.prepareStatement("insert into t_demo_travel_record (id,context) values (?,?)");

　　pstm1.setInt(1, 5000);

　　pstm1.setString(2, "这条记录应该进行DN1");

　　pstm1.addBatch();

　　pstm1.setInt(1, 5999999);

　　pstm1.setString(2, "这条记录应该进行DN2");

　　pstm1.addBatch();

　　pstm1.setInt(1, 14991499);　　

　　pstm1.setString(2, "这条记录应该进行DN3");

　　pstm1.addBatch();

　　pstm1.executeBatch();

　　conn.commit();

MYCAT 192.168.31.207:  8806  查询已插入的记录

mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);

+----------+-------------+

| id       | context     |

+----------+-------------+

|     5000 | ????????DN1 |

| 14991499 | ????????DN3 |

|  5999999 | ????????DN2 |

+----------+-------------+

3 rows in set (0.01 sec)

VM1 192.168.31.187：3307  物理库

mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);

+------+-------------+

| id   | context     |

+------+-------------+

| 5000 | ????????DN1 |

+------+-------------+

1 row in set (0.01 sec)

VM2 192.168.31.212：3307

mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);

+---------+-------------+

| id      | context     |

+---------+-------------+

| 5999999 | ????????DN2 |

+---------+-------------+

1 row in set (0.00 sec)

VM3 192.168.31.150:  3307

mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);

+----------+-------------+

| id       | context     |

+----------+-------------+

| 14991499 | ????????DN3 |

+----------+-------------+

1 row in set (0.00 sec)

2.2 异常操作，验证下数据是否回滚

模拟数据如下，最后一条主键重复。

pstm1.setInt(1, 5001);

pstm1.setString(2, "这条记录应该进行DN1");

pstm1.addBatch();

pstm1.setInt(1, 5999998);

pstm1.setString(2, "这条记录应该进行DN2");

pstm1.addBatch();

pstm1.setInt(1, 14991499);

pstm1.setString(2, "这条记录应该进行DN3");

pstm1.addBatch();

程序执行报错：

2016-02-03 00:36:10,229 INFO : [] com.alibaba.druid.pool.DruidDataSource [DruidDataSource.java:669] - {dataSource-1} inited

java.sql.BatchUpdateException: Duplicate entry ‘14991499‘ for key ‘PRIMARY‘

...

再看下数据库里并没有执行前2句SQL语句，mycat 在数据分片插入不同的数据库时已给我们做了一定的数据库事务控制。

mysql> select * from t_demo_travel_record where id in (5001,5999998,14991499);

+----------+-------------+

| id       | context     |

+----------+-------------+

| 14991499 | ????????DN3 |

+----------+-------------+

1 row in set (0.01 sec)

再来看下MYCAT日志，根据分片原则，分析将SQL语句发向不同的数据结点

02/03 00:36:10.279 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5001,‘这条记录应该进行DN1‘)
02/03 00:36:10.279 DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5001,‘这条记录应该进行DN1‘), route={
1 -> dn1{insert into t_demo_travel_record (id,context) values (5001,‘这条记录应该进行DN1‘)}
} rrs
02/03 00:36:10.281 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5999998,‘这条记录应该进行DN2‘)
02/03 00:36:10.281 DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5999998,‘这条记录应该进行DN2‘), route={
1 -> dn2{insert into t_demo_travel_record (id,context) values (5999998,‘这条记录应该进行DN2‘)}
} rrs
02/03 00:36:10.281 DEBUG [$_NIOREACTOR-1-RW] (MySQLConnection.java:445) -con need syn ,total syn cmd 1 commands SET autocommit=0;schema change:false con:MySQLConnection [id=28, lastTime=1454430970281, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=true, attachment=dn2{insert into t_demo_travel_record (id,context) values (5999998,‘这条记录应该进行DN2‘)}, respHandler=SingleNodeHandler [node=dn2{insert into t_demo_travel_record (id,context) values (5999998,‘这条记录应该进行DN2‘)}, packetId=0], host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.284 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘)
02/03 00:36:10.284 DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘), route={
1 -> dn3{insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘)}
} rrs

准备同步数据库操作，发现SQL语句在DN3上编译失败

02/03 00:36:10.285 DEBUG [$_NIOREACTOR-1-RW] (MySQLConnection.java:445) -con need syn ,total syn cmd 1 commands SET autocommit=0;schema change:false con:MySQLConnection [id=8, lastTime=1454430970285, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=true, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘)}, respHandler=SingleNodeHandler [node=dn3{insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘)}, packetId=0], host=192.168.31.150, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.286 WARN [$_NIOREACTOR-0-RW] (SingleNodeHandler.java:222) -execute sql err : errno:1062 Duplicate entry ‘14991499‘ for key ‘PRIMARY‘ con:MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘)}, respHandler=SingleNodeHandler [node=dn3{insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘)}, packetId=1], host=192.168.31.150, port=3307, [email protected]8537945, writeQueue=0, modifiedSQLExecuted=true] frontend host:127.0.0.1/50644/test

再下来开始回滚所有数据结点上的更新操作。
02/03 00:36:10.305 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]rollback
02/03 00:36:10.305 DEBUG [$_NIOREACTOR-1-RW] (RollbackNodeHandler.java:71) -rollback job run for MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘)}, respHandler=SingleNodeHandler [node=dn3{insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘)}, packetId=1], host=192.168.31.150, port=3307, [email protected]8537945, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.305 DEBUG [$_NIOREACTOR-1-RW] (RollbackNodeHandler.java:71) -rollback job run for MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=dn1{insert into t_demo_travel_record (id,context) values (5001,‘这条记录应该进行DN1‘)}, respHandler=SingleNodeHandler [node=dn1{insert into t_demo_travel_record (id,context) values (5001,‘这条记录应该进行DN1‘)}, packetId=1], host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.306 DEBUG [$_NIOREACTOR-1-RW] (RollbackNodeHandler.java:71) -rollback job run for MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=dn2{insert into t_demo_travel_record (id,context) values (5999998,‘这条记录应该进行DN2‘)}, respHandler=SingleNodeHandler [node=dn2{insert into t_demo_travel_record (id,context) values (5999998,‘这条记录应该进行DN2‘)}, packetId=1], host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:361) -clear session resources [email protected]

释放数据库链接
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,‘这条记录应该进行DN3‘)}, r[email protected]394047fa, host=192.168.31.150, port=3307, [email protected]8537945, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=dn1{insert into t_demo_travel_record (id,context) values (5001,‘这条记录应该进行DN1‘)}, r[email protected]394047fa, host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=dn2{insert into t_demo_travel_record (id,context) values (5999998,‘这条记录应该进行DN2‘)}, r[email protected]394047fa, host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (RollbackReleaseHandler.java:58) -autocomit is false,but no commit or rollback ,so mycat rollbacked backend conn MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=null, resp[email protected]2b358b73, host=192.168.31.150, port=3307, [email protected]8537945, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=null, host=192.168.31.150, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (RollbackReleaseHandler.java:58) -autocomit is false,but no commit or rollback ,so mycat rollbacked backend conn MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=null, resp[email protected]4856412e, host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=null, host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (RollbackReleaseHandler.java:58) -autocomit is false,but no commit or rollback ,so mycat rollbacked backend conn MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=null, resp[email protected]16d1581c, host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=null, host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02

OK，通过2个场景验证了MYCAT的事务一致性。

再补一句，大家可以通过MYSQL权威指南文档了解到其实MYCAT实现一种弱XA的事务控制，怎么理解这个弱XA事务控制。

举例有3个结点，MYCAT根据分片规则依次在不同物理库中预执行，如果当中任何一个发现问题则回滚所有。

若预执行成功，3个结点分别进行commit操作时发生异常，则mycat无法回滚已提交的事务。当然，这种机率很小很小~~~

另外一个问题在于，如果发锁升级导致单结点无法操作时，会对整体应用产生比较大的影响。

本篇完

时间： 2024-10-05 00:19:38

MyCat 学习笔记 第十一篇.数据分片 之 分片事务处理的相关文章

MyCat 学习笔记 第十一篇.数据分片 之 分片数据查询 ( select * from table_name limit 100000,100 )

1 环境说明 VM 模拟3台MYSQL 5.6 服务器 VM1 192.168.31.187:3307 VM2 192.168.31.212:3307 VM3 192.168.31.150:  3307 MYCAT 1.5 服务部署在宿主机上 MYCAT 192.168.31.207 :8806[SQL执行端口] / 9066[管理端口] 2 应用场景 2.0 MYCAT配置 schema.xml <schema name="TESTDB" checkSQLschema=&quo

MyCat 学习笔记 第十篇.数据分片 之 ER分片

1 应用场景 这篇来说下mycat中自带的er关系分片,所谓er关系分片即可以理解为有关联关系表之间数据分片.类似于订单主表与订单详情表间的分片存储规则. 本文所说的er分片分为两种: a. 依据主键进行数据分片,验证发现主表数据保存在第1个datanode中,子表数据根据分片规则存储. b. 依据分片关键字段进行分片,验证发现主表与子表根据分片规则存储,且保存在相同的分片内. 接下来,可以下实际配置与数据验证 2 环境说明 参考  <MyCat 学习笔记>第六篇.数据分片 之 按月数据分片 

MyCat 学习笔记 第十三篇.数据分片 之 通过HINT执行存储过程

1 环境说明 VM 模拟3台MYSQL 5.6 服务器 VM1 192.168.31.187:3307 VM2 192.168.31.212:3307 VM3 192.168.31.150:  3307 MYCAT 1.5 服务部署在宿主机上 MYCAT 192.168.31.207 :8806[SQL执行端口] / 9066[管理端口] 2 应用场景 2.0 MYCAT配置 schema.xml <schema name="TESTDB" checkSQLschema=&quo

《Mycat学习笔记》 第二篇. MySql 读写分离与日志分析——主从多结点

1    环境说明 接上篇环境   <Mycat学习笔记> 第一篇. MySql 读写分离与日志分析——主从单结点 http://www.cnblogs.com/kaye0110/p/5134588.html 增加一套 mysql 实例,端口为3308 ,通过Binlog方式同步主机情况 localhost : 3306 主机,    在mycat 中配置为 writehost 1 localhost : 3307 从机 a ,在mycat 中配置为 readhost localhost :

《Mycat学习笔记》 第三篇. MySql 主从同步异常后，主从切换

1)系统环境说明 MySql 5.5 主从节点 127.0.0.1:3306   主结点,为验证主从切换效果,手动停止服务 127.0.0.1: 3307    从结点 1 127.0.0.1:338     从结点 2 ,为验证主从切换效果,在主结点停止后,新增两个记录. MyCat 1.5 schema.xml 配置 具体配置说明,参考上篇: <Mycat学习笔记> 第二篇. MySql 读写分离与日志分析——主从多结点 <dataHost name="localhost1

《MyCat 学习笔记》第八篇.数据分片 之 求摸运算分片

1 应用场景 Mycat 自带了多套数据分片的机制,其实根据数值取摸应该是最简单的一种. 优点:数据离散概率较为平均,可以有效的提高应用的数据吞吐. 缺点:比较明显,后期数据运维与迁移比较困难.好在Mycat有对应的解决方案,具体后期验证或可直接参考Mycat权威指南相应章节. 2 环境说明 参考  <MyCat 学习笔记>第六篇.数据分片 之 按月数据分片  http://www.cnblogs.com/kaye0110/p/5160826.html 3 参数配置 3.1 server.xm

Android学习笔记（十一）——从意图返回结果

从意图返回结果 startActivity()方法调用另一个活动,但并没有返回结果给当前活动.此时如想从一个活动中回传数据,就要使用startActivityForResult()方法. 点此获取完整代码~                                                                  1.使用上一篇中创建的项目,在secondactivity.xml文件中添加文本框和按钮,代码如下: <TextView android:layout_width

嵌入式学习笔记008-裸奔篇之串口

串口是个好东西,前几篇裸奔程序由于没有串口,自己调试都是有led等来表示的,比较"苦逼",终于可以用串口了~~~,这里主要采用上一篇博文(嵌入式学习笔记007-裸奔篇之定时器),也就是串口也是用中断实现的,而且也只是在前一篇博文增加串口的初始化uart0_init(),以及在中断处理函数增加对串口的处理.只要稍微改造前一篇博文就是一个通用的中断处理程序! 这里主要实现在串口输入一个字符,接受后+2再发送到串口,所以在串口输入a 会返回c---. 由于code都有相应的注释,读者自行查看

Java快速教程--vamei 学习笔记（基础篇）

链接:http://www.cnblogs.com/vamei/archive/2013/03/31/2991531.html java快速教程第1课 从HelloWorld到面向对象 学习网址:http://www.cnblogs.com/vamei/archive/2013/03/14/2958654.html java快速教程第2课 方法与数据成员 学习网址:http://www.cnblogs.com/vamei/archive/2013/03/25/2964430.html java快