一百万数据索引实例測试--mysql

推荐书籍:http://pan.baidu.com/s/1sjJIyRV

任务描写叙述:

如果一高频查询例如以下

SELECT * FROM user WHERE area=‘amoy‘ AND sex=0 ORDER BY last_login DESC limit 30;

怎样建立索引?描写叙述考虑的过程

user表例如以下:

初始化100W条数据,当中。area要通过IP查询生成,sex为 0,1 随机

CREATE TABLE user (

id int(10) NOT NULL AUTOINCREMENT COMMENT ‘自增编号‘,

username varchar(30) NOT NULL DEFAULT ‘0‘ COMMENT ‘用户名‘,

password varchar(30) NOT NULL DEFAULT ‘0‘ COMMENT ‘密码‘,

area varchar(30) NOT NULL COMMENT ‘地址‘,

sex int(10) NOT NULL COMMENT ‘性别0,男;1,女。

‘,

last_login int(10) NOT NULL COMMENT ‘近期一次登录时间戳‘,

PRIMARY KEY (id)

) ENGINE=InnoDB AUTOINCREMENT=892013 DEFAULT CHARSET=latin1

终于我的索引

(last_login,area)

数据例如以下:http://pan.baidu.com/s/1eQy0eQI

測试结果:http://pan.baidu.com/s/1jGn2AcY

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvbGFtcF93YXRlcg==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast" >

索引原则:

1.where和order by等的字段建立索引

2.使用唯一索引:对于last_login,area等字段反复的次数比較少,能够使用索引;而sex无非就两个值:性别1。男;2。不值得索引

3.多列索引:不要为每个列单独建立索引。这样并不能将mysql索引的效率最大化。使用“索引合并策略”

4.选择合理的索引列顺序:索引列的顺序意味着索引首先依照最左列进行排序。然后是第二列,以此类推。如(lastlogin,area)会先依照 lastlogin 进行排序。然后才是area。

5.将选择性最高的索引放到前面。也就是会所依照这个条件搜索到的数据最少,选择性就越高。比方选择性:last_login> area> sex。

6.索引不是越多越好。适合的索引能够提高查询效率。可是会减少写入效率。依据项目保持两者的平衡性最好了。

总结上面,首先sex不适合建立索引,有没有索引对于效率的提升意义不大,其次索引会依照最左列进行排序,因此将last_login放到最前面

測试过程:

user表

没有不论什么索引的查询相关日志:

SELECT * FROM user WHERE area=‘美国ATT用户‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.57s

SELECT * FROM user WHERE area=‘泰国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.56s

SELECT * FROM user WHERE area=‘台湾省台湾大宽频‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.55s

SELECT * FROM user WHERE area=‘美国弗吉尼亚州‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.59s

SELECT * FROM user WHERE area=‘德国奔驰汽车‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.55s

SELECT * FROM user WHERE area=‘台湾省中华电信‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.55s

SELECT * FROM user WHERE area=‘韩国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.57s

SELECT * FROM user WHERE area=‘拉美地区‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.58s

SELECT * FROM user WHERE area=‘美国纽约(Prudential)‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.57s

SELECT * FROM user WHERE area=‘印度尼西亚‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.57s

共花费时间:5.66s

建立索引area:

ALTER TABLE user ADD INDEX index_area (area)
;

SELECT * FROM user WHERE area=‘美国ATT用户‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.06s

SELECT * FROM user WHERE area=‘泰国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.02s

SELECT * FROM user WHERE area=‘台湾省台湾大宽频‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘美国弗吉尼亚州‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.10s

SELECT * FROM user WHERE area=‘德国奔驰汽车‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.04s

SELECT * FROM user WHERE area=‘台湾省中华电信‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.02s

SELECT * FROM user WHERE area=‘韩国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.11s

SELECT * FROM user WHERE area=‘拉美地区‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.20s

SELECT * FROM user WHERE area=‘美国纽约(Prudential)‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.07s

SELECT * FROM user WHERE area=‘印度尼西亚‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.04s

共花费时间:0.66s

可见。建立area以后对性能的影响是巨大的(5.66/0.66 约为8.5758倍)

删除索引:ALTER TABLE user DROP INDEX index_area;

删除area索引发现时间又变成了0.57s

建立lastlogin索引:

SELECT * FROM user WHERE area=‘美国ATT用户‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.03s

SELECT * FROM user WHERE area=‘泰国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.09s

SELECT * FROM user WHERE area=‘台湾省台湾大宽频‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.51s

SELECT * FROM user WHERE area=‘美国弗吉尼亚州‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.01s

SELECT * FROM user WHERE area=‘德国奔驰汽车‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.04s

SELECT * FROM user WHERE area=‘台湾省中华电信‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.07s

SELECT * FROM user WHERE area=‘韩国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.01s

SELECT * FROM user WHERE area=‘拉美地区‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.01s

SELECT * FROM user WHERE area=‘美国纽约(Prudential)‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.04s

SELECT * FROM user WHERE area=‘印度尼西亚‘ AND sex=0 ORDER BY last_login DESC limit 30; 0.06s

共花费时间:0.87s

相同可以提升性能(5.66/0.87 约为6.5057倍)

建立sex索引:

ALTER TABLE user ADD INDEX index_sex (sex)
;

SELECT * FROM user WHERE area=‘美国ATT用户‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.87s

SELECT * FROM user WHERE area=‘泰国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.87s

SELECT * FROM user WHERE area=‘台湾省台湾大宽频‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.87s

SELECT * FROM user WHERE area=‘美国弗吉尼亚州‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.89s

SELECT * FROM user WHERE area=‘德国奔驰汽车‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.88s

SELECT * FROM user WHERE area=‘台湾省中华电信‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.87s

SELECT * FROM user WHERE area=‘韩国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.86s

SELECT * FROM user WHERE area=‘拉美地区‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.88s

SELECT * FROM user WHERE area=‘美国纽约(Prudential)‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.87s

SELECT * FROM user WHERE area=‘印度尼西亚‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.87s

共花费时间:8.73s

相同可以提升性能(5.66s/8.73 约为0.6483倍)效率反而减少了??求解?

建立这个sex索引还不如不建。

删除索引:

ALTER TABLE user DROP INDEX index_sex;

发现时间又变成了0.57s左右,

建立两个单独的索引:

ALTER TABLE user

ADD INDEX index_area (area)
,

ADD INDEX index_last_login (last_login)
;

SELECT * FROM user WHERE area=‘美国ATT用户‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.09s

SELECT * FROM user WHERE area=‘泰国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.33s

SELECT * FROM user WHERE area=‘台湾省台湾大宽频‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.21s

SELECT * FROM user WHERE area=‘美国弗吉尼亚州‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.01s

SELECT * FROM user WHERE area=‘德国奔驰汽车‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.28s

SELECT * FROM user WHERE area=‘台湾省中华电信‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.02s

SELECT * FROM user WHERE area=‘韩国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.02s

SELECT * FROM user WHERE area=‘拉美地区‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.02s

SELECT * FROM user WHERE area=‘美国纽约(Prudential)‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.03s

SELECT * FROM user WHERE area=‘印度尼西亚‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.67s

发现建立两个单独的索引还不如仅仅建立一个索引

删除索引:

发现时间又变成了0.57s左右。

建立一个的联合索引:

ALTER TABLE user

ADD INDEX index_last_login_area (last_login,area)
,

SELECT * FROM user WHERE area=‘美国ATT用户‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘泰国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘台湾省台湾大宽频‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘美国弗吉尼亚州‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘德国奔驰汽车‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘台湾省中华电信‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘韩国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘拉美地区‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘美国纽约(Prudential)‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

SELECT * FROM user WHERE area=‘印度尼西亚‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.00s

额。第二条数据这是怎么了,我測试了5次都在这附近晃悠哈!

这尼玛。找对索引啦。就该这么建立,查询不出来须要的时间啦!预计就是我们须要的索引啦!

!!

删除索引:

发现时间又变成了0.57s左右,

建立一个的联合索引:

ALTER TABLE user

ADD INDEX index_sex_last_login_area (sex,last_login,area)

SELECT * FROM user WHERE area=‘美国ATT用户‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.18s

SELECT * FROM user WHERE area=‘泰国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.17s

SELECT * FROM user WHERE area=‘台湾省台湾大宽频‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.81s

SELECT * FROM user WHERE area=‘美国弗吉尼亚州‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.01s

SELECT * FROM user WHERE area=‘德国奔驰汽车‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.02s

SELECT * FROM user WHERE area=‘台湾省中华电信‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.04s

SELECT * FROM user WHERE area=‘韩国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.01s

SELECT * FROM user WHERE area=‘拉美地区‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.01s

SELECT * FROM user WHERE area=‘美国纽约(Prudential)‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.03s

SELECT * FROM user WHERE area=‘印度尼西亚‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.04s

sex怎么总是你在拖后腿啊!

把你调整到索引的最后一个吧。

删除索引:

发现时间又变成了0.57s左右,

建立一个的联合索引:

ALTER TABLE user

ADD INDEX index_last_login_area_sex (area,last_login,sex)

SELECT * FROM user WHERE area=‘美国ATT用户‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.03s

SELECT * FROM user WHERE area=‘泰国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.07s

SELECT * FROM user WHERE area=‘台湾省台湾大宽频‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.50s

SELECT * FROM user WHERE area=‘美国弗吉尼亚州‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.02s

SELECT * FROM user WHERE area=‘德国奔驰汽车‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.05s

SELECT * FROM user WHERE area=‘台湾省中华电信‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.06s

SELECT * FROM user WHERE area=‘韩国‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.02s

SELECT * FROM user WHERE area=‘拉美地区‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.02s

SELECT * FROM user WHERE area=‘美国纽约(Prudential)‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.04s

SELECT * FROM user WHERE area=‘印度尼西亚‘ AND sex=0 ORDER BY lastlogin DESC limit 30; 0.06s

综上所述:1.建立索引不一定可以加快查询效率如sex这样的给反复次数特别多的列添加索引如sex这样的会减少查询效率,详细的原因有待查找

2.给反复次数比較少的列添加索引还是可以大幅度提高效率

3.给where和orderby之后的字段加入索引才会加快查询效率

4.为每个列单独建立索引,不能将索引的效率最大化,应该使用索引合并策略,即依据查询条件。建立联合索引

5.联合索引的顺序问题:将选择性高的索引放到前面

6.依据资料建立索引意味着索引依照最左列进行排序,然后事第二列。以此类推。如(lastlogin ,area)就会依照lastlogin进行排序,然后才是area

7.依据这次的这个查询条件来说最好的索引是:ALTER TABLE userADD INDEX index_last_login_area (last_login,area)。

在公司能有个机会。查看资料和实践索引真的非常不错哈!

推荐书籍:高性能mysql(第三版)

PDF版本号的:http://pan.baidu.com/s/1sjJIyRV

时间: 2024-08-08 00:57:20

一百万数据索引实例測试--mysql的相关文章

Coreseek:第二步建索引及測试

1,建索引非常easy.一行代码 g:/service/coreseek/bin/indexer -c g:/service/coreseek/etc/csft_mysql.conf   person 前面是你bin文件夹下的indexer程序 后面-c指后面会跟配置文件 然后是配置文件地址 最好都用绝对地址 在后面是索引 也能够用--all 即对配置文件的全部部分都做索引 然后写成一个批处理文件.这样比較方便. 2.如今就能够在命令行下做測试了. 測试英文:g:\service\coresee

Bandwidth内存带宽測试工具

本博文为原创,遵循CC3.0协议,转载请注明出处:http://blog.csdn.net/lux_veritas/article/details/24766015 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------

小程聊微服务-增艺眼中的自己主动化測试

假设说"生活不仅仅有眼前的苟且,还有诗和远方"的话,那么自己主动化測试可以说是非常多測试人员心中的"诗和远方". "诗和远方"OR"禁果" 測试自己主动化,须要持续改进.但因为其本身是一种过于激动人心的想法:用程序去測试程序--解放了測试人员的生产力.节省大量的人力成本.这就有点"禁果"的意思了. 一个常见的行动模式是:在实施自己主动化測试时,设定一些量化指标,比如依据业务.接口.模块设置的覆盖率. 技术团

TestNg的工厂測试引用@DataProvider数据源----灵活使用工厂測试

之前说过@Factory更适合于同一类型的參数变化性的測试,那么假设參数值没有特定的规律时,我们能够採用@Factory和@DataProvider相结合的方式进行測试 注意要点:请注意測试方法将被一共运行的次数.由于@Factory本身就属于循环測试的类型.@DataProvider也是属于測试总体循环的类型 Java code: /** * * <p> * Title: TestngFactoryDataProvider * </p> * * <p> * 配置文件:

MySQL单表百万数据记录分页性能优化

原文地址:http://www.cnblogs.com/lyroge/p/3837886.html MySQL单表百万数据记录分页性能优化 背景: 自己的一个网站,由于单表的数据记录高达了一百万条,造成数据访问很慢,Google分析的后台经常报告超时,尤其是页码大的页面更是慢的不行. 测试环境: 先让我们熟悉下基本的sql语句,来查看下我们将要测试表的基本信息 use infomation_schemaSELECT * FROM TABLES WHERE TABLE_SCHEMA = 'dbna

基于MySQL元数据的Hive的安装和简单測试

引言: Hive是一种强大的数据仓库查询语言,类似SQL,本文将介绍怎样搭建Hive的开发測试环境. 1. 什么是Hive? hive是基于Hadoop的一个数据仓库工具,能够将结构化的数据文件映射为一张数据库表,并提供简单的sql查询功能,能够将sql语句转换为MapReduce任务进行执行. 其长处是学习成本低,能够通过类SQL语句高速实现简单的MapReduce统计.不必开发专门的MapReduce应用,十分适合数据仓库的统计分析. 2.  依照Hive的准备条件 2.1  Hadoop集

Android自己主动化測试之Monkeyrunner用法及实例

眼下android SDK里自带的现成的測试工具有monkey 和 monkeyrunner两个.大家别看这俩兄弟名字相像,但事实上是完全然全不同的两个工具,应用在不同的測试领域.总的来说,monkey主要应用在压力和可靠性測试上,执行该命令能够随机地向目标程序发送各种模拟键盘事件流,而且能够自定义发送的次数,以此观察被測应用程序的稳定性和可靠性,应用起来也比較简单,记住那几个命令即可了.而monkeyrunner呢,相比之下会强大一些,它主要可应用于功能測试,回归測试,而且能够自定义測试扩展,

MYSQL BLOB 字段大小以及个数的限制測试。

測试结论mysql版本号 5.1    表类型: innodb, row_format=compact (这是默认的行格式)    插入超过10个blob, blob的数据量非常小(<768字节), 插入成功.    插入超过10个blob, blob的数据量非常大(>768字节), 插入失败:报 Got error 139 from storage engine.     注意,假设mysqlserver版本号是5.1, innodb_file_format选项不存在, 也就无从谈起Barr

JPA測试实例

依赖架包 实体 import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; import javax.persistence.Table; import javax.persistence.Transien