SQL优化一则:灵活运用字段的选择性

??

概念:

优化策略:字段选择性

  • 选择性较低索引 可能带来的性能问题
  • 索引选择性=索引列唯一值/表记录数;
  • 选择性越高索引检索价值越高,消耗系统资源越少;选择性越低索引检索价值越低,消耗系统资源越多;
  • 查询条件含有多个字段时,不要在选择性很低字段上创建索引
  • 可通过创建组合索引来增强低字段选择性和避免选择性很低字段创建索引带来副作用;
  • 尽量减少possible_keys,正确索引会提高sql查询速度,过多索引会增加优化器选择索引的代价,不要滥用索引;

现象:

昨天数据库一个SQL在读库上占用很高的CPU,CPU一直在100%,CPU wait达到100多,通EM的SQL监视 可以看到,有一条SQl在多次执行时一直没有完成,有的运行20多分钟还在执行。

SQL如下:

Select DistinctFppolicyin0_.Id                 AsId53_,
                                     Fppolicyin0_.Carrier_Code       As Carrier2_53_,
                                     Fppolicyin0_.Policy_Id          As Policy3_53_,
                                     Fppolicyin0_.Carrier_Policyid   As Carrier4_53_,
                                     Fppolicyin0_.Agent_Id           As Agent5_53_,
                                     Fppolicyin0_.Airline            As Airline53_,
                                     Fppolicyin0_.Airline_Type       As Airline7_53_,
                                     Fppolicyin0_.Apply_Flightnos    As Apply8_53_,
                                     Fppolicyin0_.Exc_Flightnos      As Exc9_53_,
                                     Fppolicyin0_.Class_Codes        As Class10_53_,
                                     Fppolicyin0_.Shortstay_Time     As Shortstay11_53_,
                                     Fppolicyin0_.Useweek            As Useweek53_,
                                     Fppolicyin0_.Tkt_Type           As Tkt13_53_,
                                     Fppolicyin0_.Psg_Type           As Psg14_53_,
                                     Fppolicyin0_.Is_Special         As Is15_53_,
                                     Fppolicyin0_.Is_Commend         As Is16_53_,
                                     Fppolicyin0_.Limit_Code         As Limit17_53_,
                                     Fppolicyin0_.Pre_Outtkt_Day     As Pre18_53_,
                                     Fppolicyin0_.Last_Outtkt_Day    As Last19_53_,
                                     Fppolicyin0_.Flight_Price       As Flight20_53_,
                                     Fppolicyin0_.Rebate             As Rebate53_,
                                     Fppolicyin0_.Flight_Price_Chd   As Flight22_53_,
                                     Fppolicyin0_.Rebate_Chd         As Rebate23_53_,
                                     Fppolicyin0_.Flight_Price_Um    As Flight24_53_,
                                     Fppolicyin0_.Rebate_Um          As Rebate25_53_,
                                     Fppolicyin0_.Flight_Price_Inf   As Flight26_53_,
                                     Fppolicyin0_.Rebate_Inf         As Rebate27_53_,
                                     Fppolicyin0_.Out_Tktcity        As Out28_53_,
                                     Fppolicyin0_.Ei                 As Ei53_,
                                     Fppolicyin0_.Signticket         As Signticket53_,
                                     Fppolicyin0_.Endorsement        As Endorse31_53_,
                                     Fppolicyin0_.Refundmemo         As Refundmemo53_,
                                     Fppolicyin0_.Enei               As Enei53_,
                                     Fppolicyin0_.Enendorsement      As Enendor34_53_,
                                     Fppolicyin0_.Enrefundmemo       As Enrefun35_53_,
                                     Fppolicyin0_.Tkt_Startdate      AsTkt36_53_,
                                     Fppolicyin0_.Tkt_Enddate        As Tkt37_53_,
                                     Fppolicyin0_.Tkt_Useweek        As Tkt38_53_,
                                     Fppolicyin0_.Fp_Startdate       As Fp39_53_,
                                     Fppolicyin0_.Fp_Enddate         As Fp40_53_,
                                     Fppolicyin0_.Fp_Useweek         As Fp41_53_,
                                     Fppolicyin0_.Exc_Startdate      As Exc42_53_,
                                     Fppolicyin0_.Exc_Enddate        As Exc43_53_,
                                     Fppolicyin0_.Is_Open            As Is44_53_,
                                     Fppolicyin0_.Ret_Startdates     As Ret45_53_,
                                     Fppolicyin0_.Ret_Enddates       As Ret46_53_,
                                     Fppolicyin0_.Mf_Remark          As Mf47_53_,
                                     Fppolicyin0_.Remark             As Remark53_,
                                     Fppolicyin0_.Create_Date        As Create49_53_,
                                     Fppolicyin0_.Longstay_Time      As Longstay50_53_,
                                     Fppolicyin0_.Levcity            As Levcity53_,
                                     Fppolicyin0_.Arvcity            As Arvcity53_,
                                     Fppolicyin0_.Levdrome           As Levdrome53_,
                                     Fppolicyin0_.Arvdrome           As Arvdrome53_,
                                     Fppolicyin0_.Is_To_Pata         As Is55_53_,
                                     Fppolicyin0_.Commend_Rmk        As Commend56_53_,
                                     Fppolicyin0_.Isgroup            As Isgroup53_,
                                     Fppolicyin0_.Policy_Source      As Policy58_53_,
                                     Fppolicyin0_.Tkt_Price          As Tkt59_53_,
                                     Fppolicyin0_.Other_Segment      As Other60_53_,
                                     Fppolicyin0_.Spe_Display_Name   As Spe61_53_,
                                     Fppolicyin0_.Fp_Startdate_Back  As Fp62_53_,
                                     Fppolicyin0_.Fp_Enddate_Back    As Fp63_53_,
                                     Fppolicyin0_.Exc_Startdate_BackAs Exc64_53_,
                                     Fppolicyin0_.Exc_Enddate_Back   As Exc65_53_,
                                     Fppolicyin0_.Fp_Starttime       As Fp66_53_,
                                     Fppolicyin0_.Fp_Endtime         As Fp67_53_,
                                     Fppolicyin0_.Fp_Starttime_Back  As Fp68_53_,
                                     Fppolicyin0_.Fp_Endtime_Back    As Fp69_53_
     From Fp_Policyinfo_Dt   Fppolicyin0_,
               Code_Airways       Tkcodeairw1_,
               Fp_Policy_Issue_Dt Fppolicyis2_,
               Code_Airdrome      Tkcodeaird3_,
               Code_Airdrome      Tkcodeaird4_,
               Fp_Policy_Issue_Dt Fppolicyis5_
 Where Fppolicyin0_.Policy_Id =Fppolicyis5_.Id
      And 1 = 1
      And Fppolicyis5_.Status = ‘1‘
      And Fppolicyin0_.Carrier_Code =Tkcodeairw1_.Id
      And Fppolicyin0_.Policy_Id =Fppolicyis2_.Id
      And (Fppolicyin0_.Levdrome =Tkcodeaird3_.Airdromeid Or
               Fppolicyin0_.Levdrome = ‘*‘)
      And (Fppolicyin0_.Arvdrome =Tkcodeaird4_.Airdromeid Or
               Fppolicyin0_.Arvdrome = ‘*‘)
      And ((Fppolicyin0_.Levcity = ‘PEK‘ Or --变量1
               Fppolicyin0_.Levcity =
               (Select Tkcodeaird6_.Cityid
                            FromCode_Airdrome Tkcodeaird6_
                         WhereTkcodeaird6_.Airdromeid = ‘PEK‘)) And --变量2
               (Fppolicyin0_.Arvcity = ‘SHA‘ Or --变量3
               Fppolicyin0_.Arvcity =
               (Select Tkcodeaird7_.Cityid
                            FromCode_Airdrome Tkcodeaird7_
                        Where Tkcodeaird7_.Airdromeid = ‘SHA‘)) Or --变量4
               Fppolicyin0_.Airline = ‘*-*‘)
      And (Fppolicyin0_.Policy_Source IsNull Or
               Fppolicyin0_.Policy_Source <> ‘P‘)
      And (Fppolicyin0_.Tkt_Startdate IsNull Or
               Fppolicyin0_.Tkt_Startdate <= ‘2016-03-17‘) --变量5
      And (Fppolicyin0_.Tkt_Enddate IsNull Or
               Fppolicyin0_.Tkt_Enddate >= ‘2016-03-17‘) --变量6

分析解决:

通过和开发人员沟通,此SQL是航班查询的功能。

查看SQL的执行计划,发现cost值非常大:

查看SQL中相关的的5个表,其中Fp_Policyinfo_Dt 、FP_POLICY_ISSUE_DT表数据量最大,记录数达900多万行,应该是没使用正确的索引导致的SQL查询慢。

首先考虑是否有索引碎片,尝试重建几个比较大的索引无效

接着尝试新建某些索引也无效

再次研究发现Fp_Policy_Issue_Dt表中Status字段值(1)在SQL中是写死的,于是查看Status字段在表中的分布

--查看字段值分布

SQL> Select status,Count(1) FromFP_POLICY_ISSUE_DT group by cube(status);

STATUS      COUNT(1)

---------- ----------

9663424 --总记录数

0                1200

1              586632

2               13730

3             9061862

--查看status值为1的占总记录数的比例

SQL> select round(586632/9663424,2) fromdual;

ROUND(586632/9663424,2)

-----------------------

0.06

SQL> select round(586632/9663424,2) fromdual;

ROUND(586632/9663424,2)

-----------------------

0.06

可以看到STATUS=3的为绝大多数,status=1只占6%,所以在status上字段创建索引,在SQL查询status=1时,可以提高SQL索引的选择性,使SQL查询效率会更高。

--下面我们尝试创建索引

SQL> reate index ind_IDX_ FP_POLICY_ISSUE_STATUSon FP_POLICY_ISSUE_DT (status) online;

--再查看执行计划

可以看到SQl使用了我们新建的索引IDX_ FP_POLICY_ISSUE_STATUS,SQL的执行时间也由原来的30分钟以上,优化后6秒就能出来了。

总结:

在我们使用索引的选择性建立索引时,要充分了所查字段值的分布情况,在SQL查询中要根据字段值实际分布建立合适的索引,灵活使用索引的选择性,提高SQL的查询性能。

时间: 2024-07-29 14:05:54

SQL优化一则:灵活运用字段的选择性的相关文章

oracle sql优化

第一掌 避免对列的操作 任何对列的操作都可能导致全表扫描,这里所谓的操作包括数据库函数.计算表达式等等,查询时要尽可能将操作移至等式的右边,甚至去掉函数. 例1:下列SQL条件语句中的列都建有恰当的索引,但30万行数据情况下执行速度却非常慢: select * from record where  substrb(CardNo,1,4)='5378'(13秒) select * from record where  amount/30< 1000(11秒) select * from recor

Oracle sql优化必知——表的访问

<访问数据的方法> 访问表中的数据有两种:1.直接访问表   2.先访问索引,再回表 1.直接访问表的两种方法: ①.全表扫描 全表扫描是指Oracle在访问目标表的数据时,会从该表所占用的第一个区(extent)的第一个块(block)开始扫描,一直扫描到该表的高水位线,这段范围内的所有数据库都必须读到,当然如果目标sql的where中指定的过滤条件,最后只返回满足条件的数据即可:(有时候全表扫描的效率还是非常高的,但是随着表的数据增多 资源消耗也会在逐步增加) ②.rowid扫描 rowi

基于oracle的sql优化方法论

Oracle数据库里SQL优化的终极目标就是要缩短目标SQL语句的执行时间.要达到上述目的,我们通常只有如下三种方法可以选择: 1.降低目标SQL语句的资源消耗: 2.并行执行目标SQL语句: 3.平衡系统的资源消耗. "方法1:降低目标SQL语句的资源消耗"以缩短执行时间,这是最常用的SQL优化方法.这种方法的核心是要么通过在不更改业务逻辑的情况下改写SQL来降低目标SQL语句的资源消耗,要么不改SQL但通过调整执行计划或相关表的数据来降低目标SQL语句的资源消耗. 方法2:并行执行

mysql慢sql优化

影响sql查询慢的因素 1.没有索引或则无效索引导致的全表扫描.2.表的数据量和关联的表数量. 致索引无效的情况 1.表关联查询时,字段类型或长度不一致.如:varchar(10)和varchar(20)2.查询参数的数据类型与索引字段类型不一致.如int = '1313'3.in () 索引字段in查询时,通常是有效的.但是当in中指定的数据太多,优化器认为全表扫描更快时,也不会使用索引.4.其他如:like.索引上使用函数等使索引失效. sql的执行顺序 from>join>on>w

Sql优化器究竟帮你做了哪些工作?

关系型数据库的一大优势之一,用户无需关心数据的访问方式,因为这些优化器都帮我们处理好了,但sql查询优化的时候,我不得不要对此进行关注,因为这牵扯到查询性能问题. 有经验的程序员都会对一些sql优化了如指掌,比如我们常说的最左匹配原则,非BT谓词规避等等,那么优化器是如何确定这些的?以及为何一定要最左匹配,最左匹配的原理是什么,你是否有深入了解? 这一篇我们就通过一些实例来剖析优化器做了哪些工作,以方便我们更好的优化SQL查询. 本篇你可以知道: sql的访问路径是什么 优化器如何确定最优访问路

SQL优化技巧

我们开发的大部分软件,其基本业务流程都是:采集数据→将数据存储到数据库中→根据业务需求查询相应数据→对数据进行处理→传给前台展示.对整个流程进行分析,可以发现软件大部分的操作时间消耗都花在了数据库相关的IO操作上.所以对我们的SQL语句进行优化,可以提高软件的响应性能,带来更好的用户体验. 在开始介绍SQL优化技巧之前,先推介一款数据库管理神器Navicat,官网:https://www.navicat.com.cn/whatisnavicat Navicat是一套快速.可靠和全面的数据库管理工

数据库SQL优化大总结之百万级数据库优化方案

网上关于SQL优化的教程很多,但是比较杂乱.近日有空整理了一下,写出来跟大家分享一下,其中有错误和不足的地方,还请大家纠正补充. 这篇文章我花费了大量的时间查找资料.修改.排版,希望大家阅读之后,感觉好的话推荐给更多的人,让更多的人看到.纠正以及补充. 1.对查询进行优化,要尽量避免全表扫描,首先应考虑在 where 及 order by 涉及的列上建立索引. 2.应尽量避免在 where 子句中对字段进行 null 值判断,否则将导致引擎放弃使用索引而进行全表扫描,如: select id f

Sql优化和体系结构

01.SQL优化与体系结构 本章目标: 1.了解sql优化基本技巧 2.掌握使用索引提高查询效率 3.了解对表进行分区操作 4.了解常见数据库对象 1.sql优化技巧 1)一般优化技巧: 不要用*代替所有列名 删除所有数据用truncate代替delete 用not exists 代替 not in 用exists 代替 in 用exists代替distinct 注:后三点在11g之前有用,11g之后本身进行了优化 第5条的实例如下:查询出出现在教师表里的不同的部门编号 select disti

SQL优化经验

SQL 优化经验总结34条 我们要做到不但会写SQL,还要做到写出性能优良的SQL,以下为笔者学习.摘录.并汇总部分资料与大家分享! (1) 选择最有效率的表名顺序(只在基于规则的优化器中有效): ORACLE 的解析器按照从右到左的顺序处理FROM子句中的表名,FROM子句中写在最后的表(基础表 driving table)将被最先处理,在FROM子句中包含多个表的情况下,你必须选择记录条数最少的表作为基础表.如果有3个以上的表连接查询, 那就需要选择交叉表(intersection tabl