MySQL 5.5: InnoDB Change Buffering

To speed up bulk loading of data, InnoDB implements an insert
buffer, a special index in the InnoDB system tablespace that buffers
modifications to secondary indexes when the leaf pages are not in the buffer
pool. Batched merges from the insert buffer to the index pages result in less
random access patterns than when updating the pages directly. This speeds up the
operation on hard disks.

In MySQL 5.5, the insert buffer has been extended to a change
buffer, which covers all modifications of secondary index leaf pages. This
will improve the performance of bulk deletes and updates, transaction rollback
and the purging of deleted records (reducing the “purge lag”).

To assess the benefits of the extended buffering, you may want to run
benchmarks with the
settingsinnodb_change_buffering=all, innodb_change_buffering=inserts,
and innodb_change_buffering=none. Users of solid-state storage,
where random reads are about as fast as sequential reads, might benefit from
disabling the buffering altogether.

Read on to learn how the change buffering works.

Operations on Secondary Indexes

InnoDB can perform three kinds of modifications on secondary index records.
If the affected index page is not in the buffer pool, the modifications can be
buffered in the change buffer. When an index lookup or scan needs a page that is
not in the buffer pool, the page will be read from the tablespace and any
buffered changes will be merged to it.

The following operations can modify secondary index pages:

Insert

Inserting a record; supported in all versions of InnoDB

Delete-mark

Marking a record for deletion

Purge

Removing a deleted record that is no longer accessible by active
transactions

Before MySQL 5.5, UPDATE, DELETE and
purge operations were performed directly on the index pages, resulting in
random-access I/O. In MySQL 5.5, all of these operations can be buffered.

Implications of InnoDB Multiversioning

In InnoDB, there are two types of indexes: the clustered index B-tree, where
the records are stored in the PRIMARY KEYorder, and secondary
index B-trees, which identify rows by primary key. InnoDB multiversion
concurrency control (MVCC) treats these indexes differently.

Records in the clustered index can be updated in place, and their hidden
system columns DB_TRX_ID, DB_ROLL_PTRpoint
to undo log entries from which earlier versions can be reconstructed. InnoDB
secondary index records do not contain any system columns, and their data is
never updated in place. An UPDATE of an indexed column
requires the
operations Delete-mark(old), Insert(new)
and eventually Purge(old) in the secondary index.
An UPDATE of a PRIMARY KEY results
in Delete-mark, Insert and
eventually Purge in all indexes.

When a secondary index record has been marked for deletion or when the page
has been updated by a newer transaction, InnoDB will look up the clustered index
record. In the clustered index, it suffices to check
the DB_TRX_IDand only retrieve the correct version from the
undo log when the record was modified after the reading transaction started.

To Buffer or not to Buffer

When a page is in the buffer pool, it will always be updated directly. When a
page is loaded to the buffer pool, any buffered changes will be merged to it, so
that users never see unmerged changes.

Because change buffering works on individual leaf pages, we cannot buffer
changes that would result into page splits or merges, but must perform such
changes on the B-tree pages directly.

The insert buffer bitmap keeps track on the available
space on pages and prevents overflows when buffering inserts. Delete-marking
records can always be buffered, because the flag will be updated in place.
Purging a delete-marked record could result in an empty page, something that we
do not allow. We determine the non-emptiness of a page from previously buffered
operations on the same page. If there are no previously buffered operations, the
purge will have to load the index page to the buffer pool.

InnoDB refuses to buffer an operation when the on-disk change buffer tree
would grow bigger than ? of the in-memory buffer pool
(innodb_buffer_pool_size). This might be a good rule-of-thumb, but some
setups could benefit from the ability of setting the change buffer size
independently of the buffer pool size.

Conclusion

The InnoDB change buffer is a persistent data structure and a complex
mechanism that comes into play when the workload does not fit in the buffer
pool. Because it trades random I/O with a larger amount of sequential I/O, it
speeds up operation on hard disks, where random access is much slower than
sequential access.

On solid-state storage, there is not much difference between sequential and
random access times. Change buffering may still be useful if writes to
solid-state storage are expensive, either in terms of speed or the consumption
of limited program/erase cycles. Change buffering could reduce the write load on
user tablespaces and cause more writes to the system tablespace (which contains
the insert buffer) and the redo log. These should be placed on a hard disk.

change buffering

The general term for the
features involving the change
buffer, consisting of insert
buffering, delete
buffering, and purge
buffering. Index changes resulting from SQL statements, which
could normally involve random I/O operations, are held back and performed
periodically by a background thread. This sequence of operations can
write the disk blocks for a series of index values more efficiently than if each
value were written to disk immediately. Controlled by the innodb_change_bufferingand innodb_change_buffer_max_size configuration
options.

insert buffering

The technique of storing
secondary index changes due to INSERT operations in the insert buffer rather than writing them
immediately, so that the physical writes can be performed to minimize random
I/O. It is one of the types of change
buffering; the others are delete
buffering and purge
buffering.

Insert buffering is not used if
the secondary index is unique,
because the uniqueness of new values cannot be verified before the new entries
are written out. Other kinds of change buffering do work for unique
indexes.

unique index

An index on a column or set
of columns that have a unique
constraint. Because the index is known not to contain any
duplicate values, certain kinds of lookups and count operations are more
efficient than in the normal kind of index. Most of the lookups against this
type of index are simply to determine if a certain value exists or not. The
number of values in the index is the same as the number of rows in the table, or
at least the number of rows with non-null values for the associated columns.

The insert
buffering optimization does not apply to unique indexes. As
a workaround, you can temporarily
set unique_checks=0 while doing a bulk
data load into an InnoDB table.

delete buffering

The technique of storing
index changes due to DELETE operations in
the insert buffer rather
than writing them immediately, so that the physical writes can be performed to
minimize random I/O. (Because delete operations are a two-step process, this
operation buffers the write that normally marks an index record for deletion.)
It is one of the types of change
buffering; the others are insert
buffering and purge
buffering.

purge buffering

The technique of storing
index changes due to DELETE operations in
the insert buffer rather
than writing them immediately, so that the physical writes can be performed to
minimize random I/O. (Because delete operations are a two-step process, this
operation buffers the write that normally purges an index record that was
previously marked for deletion.) It is one of the types of change
buffering; the others are insert
buffering. and delete
buffering

When INSERT, UPDATE, and DELETE operations are done to a table, often the
values of indexed columns (particularly the values of secondary keys) are not in
sorted order, requiring substantial I/O to bring secondary indexes up to date.
InnoDB has an insert
buffer that caches changes to secondary index entries when the
relevantpage is
not in the buffer
pool, thus avoiding I/O operations by not reading in the page
from the disk. The buffered changes are merged when the page is loaded to the
buffer pool, and the updated page is later flushed to disk using the normal
mechanism. The InnoDB main thread
merges buffered changes when the server is nearly idle, and during a slow
shutdown.

Because it can result in fewer disk reads and writes, this feature is most
valuable for workloads that are I/O-bound, for example applications with a high
volume of DML operations such as bulk inserts.

However, the insert buffer occupies a part of the buffer pool, reducing the
memory available to cache data pages. If the working set almost fits in the
buffer pool, or if your tables have relatively few secondary indexes, it may be
useful to disable insert buffering. If the working set entirely fits
in the buffer pool, insert buffering does not impose any extra overhead, because
it only applies to pages that are not in the buffer pool.

innodb_change_buffering

Whether InnoDB performs change buffering, an
optimization that delays write operations to secondary indexes so that the I/O
operations can be performed sequentially. The permitted values are inserts (buffer insert operations), deletes (buffer delete operations; strictly
speaking, the writes that mark index records for later deletion during a purge
operation), changes (buffer insert and
delete-marking operations), purges (buffer purge operations, the writes when
deleted index entries are finally garbage-collected), all (buffer insert, delete-marking, and purge
operations) and none (do not buffer any
operations). The default is all.

innodb_change_buffer_max_size

Maximum size for the InnoDB change
buffer, as a percentage of the total size of the buffer
pool. You might increase this value for a MySQL server with heavy insert,
update, and delete activity, or decrease it for a MySQL server with unchanging
data used for reporting.

参考：

https://blogs.oracle.com/mysqlinnodb/entry/mysql_5_5_innodb_change

http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_change_buffering

http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_insert_buffer

http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_unique_index

http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_delete_buffering

http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_purge_buffering

http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_change_buffer_max_size

http://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_change_buffering

http://dev.mysql.com/doc/innodb/1.1/en/innodb-performance-change_buffering.html#