MySQL 优化之 ICP (index condition pushdown:索引条件下推)

ICP技术是在MySQL5.6中引入的一种索引优化技术。它能减少在使用 二级索引 过滤where条件时的回表次数 和 减少MySQL server层和引擎层的交互次数。在索引组织表中，使用二级索引进行回表的代价相比堆表中是要高一些的。相关文档地址：http://dev.mysql.com/doc/refman/5.6/en/index-condition-pushdown-optimization.html

Index Condition Pushdown optimization is used for the range, ref, eq_ref, and ref_or_null access methods when there is a need to access full table rows. This strategy can be used for InnoDB and MyISAM tables. (Note that index condition pushdown is not supported with partitioned tables in MySQL 5.6; this issue is resolved in MySQL 5.7.) For InnoDB tables, however, ICP is used only for secondary indexes. The goal of ICP is to reduce the number of full-record reads and thereby reduce IO operations. For InnoDB clustered indexes(值主键索引), the complete record is already read into the InnoDB buffer. Using ICP in this case does not reduce IO.

要想深入理解 ICP 技术，必须先理解数据库是如何处理 where 中的条件的。具体可以参考何登成博士的文章：http://hedengcheng.com/?p=577

对 where 中过滤条件的处理，根据索引使用情况分成了三种：index key, index filter, table filter

1. index key

用于确定SQL查询在索引中的连续范围(起始范围+结束范围)的查询条件，被称之为Index Key。由于一个范围，至少包含一个起始与一个终止，因此Index Key也被拆分为Index First Key和Index Last Key，分别用于定位索引查找的起始，以及索引查询的终止条件。

2. index filter

在使用 index key 确定了起始范围和介绍范围之后，在此范围之内，还有一些记录不符合where 条件，如果这些条件可以使用索引进行过滤，那么就是 index filter。

3. table filter

where 中的条件不能使用索引进行处理的，只能访问table，进行条件过滤了。

如何确定 index key, index filter, table filter，可以参考何博士的文章。

在 MySQL5.6 之前，并不区分Index Filter与Table Filter，统统将Index First Key与Index Last Key范围内的索引记录，回表读取完整记录，然后返回给MySQL Server层进行过滤。而在MySQL 5.6之后，Index Filter与Table Filter分离，Index Filter下降到InnoDB的索引层面进行过滤，减少了回表与返回MySQL Server层的记录交互开销，提高了SQL的执行效率。

所以所谓的 ICP 技术，其实就是 index filter 技术而已。只不过因为MySQL的架构原因，分成了server层和引擎层，才有所谓的“下推”的说法。所以ICP其实就是实现了index filter技术，将原来的在server层进行的table filter中可以进行index filter的部分，在引擎层面使用index filter进行处理，不再需要回表进行table filter。

4. ICP 技术启用前后比较

To see how this optimization works, consider first how an index scan proceeds when Index Condition Pushdown is not used:

1. Get the next row, first by reading the index tuple, and then by using the index tuple to locate and read the full table row.
2. Test the part of the WHERE condition that applies to this table. Accept or reject the row based on the test result.

When Index Condition Pushdown is used, the scan proceeds like this instead:

1. Get the next row‘s index tuple (but not the full table row).
2. Test the part of the WHERE condition that applies to this table and can be checked using only index columns. If the condition is not satisfied, proceed to the index tuple for the next row.
3. If the condition is satisfied, use the index tuple to locate and read the full table row.
4. Test the remaining part of the WHERE condition that applies to this table. Accept or reject the row based on the test result.

When Index Condition Pushdown is used, the Extra column in EXPLAIN output shows Using index condition. It will not show Index only because that does not apply when full table rows must be read.

5. ICP 例子

官方文档给出了一个例子：

Suppose that we have a table containing information about people and their addresses and that the table has an index defined as INDEX (zipcode, lastname, firstname). If we know a person‘s zipcode value but are not sure about the last name, we can search like this:

SELECT * FROM people WHERE zipcode=‘95054‘ AND lastname LIKE ‘%etrunia%‘ AND address LIKE ‘%Main Street%‘;

MySQL can use the index to scan through people with zipcode=‘95054‘. The second part (lastname LIKE ‘%etrunia%‘) cannot be used to limit the number of rows that must be scanned, so without Index Condition Pushdown, this query must retrieve full table rows for all the people who have zipcode=‘95054‘.

With Index Condition Pushdown, MySQL will check the lastname LIKE ‘%etrunia%‘ part before reading the full table row. This avoids reading full rows corresponding to all index tuples that do not match the lastname condition.

Index Condition Pushdown is enabled by default; it can be controlled with the optimizer_switch system variable by setting the index_condition_pushdown flag. See Section 8.9.2, “Controlling Switchable Optimizations”.

上面例子中的 lastername like ‘%etrunia%‘ 和 address like ‘%Main Street%‘ 本来是无法使用复合索引 index(zipcode, lastername, firstname) 进行过滤的，但是因为有了ICP技术，所以他们可以在 index filter 阶段使用索引进行过滤，而不需要回表进行 table filter.

例子2：

role_goods 表上有组合索引 index(roleId,status,number)，下面的select语句，因为 “索引最左前缀原则”，只能使用到 组合索引的 roleId 部分，但是因为 ICP 技术的存在，现在 number 条件过滤也可以在 index filter 阶段完成了，无需像以前一样需要进行 table filer 了：

mysql> explain select * from role_goods where roleId=100000001 and number=1;
+----+-------------+------------+------+---------------+----------+---------+-------+------+-----------------------+
| id | select_type | table      | type | possible_keys | key      | key_len | ref   | rows | Extra                 |
+----+-------------+------------+------+---------------+----------+---------+-------+------+-----------------------+
|  1 | SIMPLE      | role_goods | ref  | roleId_2      | roleId_2 | 9       | const |   14 | Using index condition |
+----+-------------+------------+------+---------------+----------+---------+-------+------+-----------------------+
1 row in set (0.01 sec)

可以看到 key_len = 9, 因为 roleId 是big int 类型，所以 key_len = 8 + 1 = 9; 所以在 index key 阶段中，并没有使用到 组合索引 index(roleId,status,number) 中的 number 字段(因为中间有一个status字段没有出现在where 条件中)，但是 “Using index condition” 却说明使用到了ICP技术，显然是 number =1 条件过滤使用到了ICP技术。

参考：

http://hedengcheng.com/?p=577

http://dev.mysql.com/doc/refman/5.6/en/index-condition-pushdown-optimization.html