Innodb额外内存池的分配策略以及性能

                                           Innodb额外内存池的分配策略以及性能
                                    作者：明天会更好
                                    QQ:715169549
备注：未经同意，严禁转载，谢谢合作。

//内存池结构体
/** Data structure for a memory pool. The space is allocated using the buddy
algorithm, where free list i contains areas of size 2 to power i. */
struct mem_pool_t{
    byte*        buf;        /*!< memory pool */
    ulint        size;        /*!< memory common pool size */
    ulint        reserved;    /*!< amount of currently allocated
                    memory */
    ib_mutex_t        mutex;        /*!< mutex protecting this struct */
    UT_LIST_BASE_NODE_T(mem_area_t)
            free_list[64];    /*!< lists of free memory areas: an
                    area is put to the list whose number
                    is the 2-logarithm of the area size */
};
//记录内存池中的块的使用情况的结构体
/** Memory area header */
struct mem_area_t{
    ulint        size_and_free;    /*!< memory area size is obtained by
                    anding with ~MEM_AREA_FREE; area in
                    a free list if ANDing with
                    MEM_AREA_FREE results in nonzero */
    UT_LIST_NODE_T(mem_area_t)
            free_list;    /*!< free list node */
};

//额外内存池分配的函数
void srv_general_init(void)
/*==================*/
{
    ut_mem_init();
    /* Reset the system variables in the recovery module. */
    recv_sys_var_init();
    os_sync_init();
    sync_init();
    //    srv_mem_pool_size = (ulint) innobase_additional_mem_pool_size;
    //  此处执行了额外内存池分配的大小为srv_mem_pool_size
    mem_init(srv_mem_pool_size);
    que_init();
    row_mysql_init();
}

//内存初始化函数
void mem_init(
/*=====*/
    ulint    size)    /*!< in: common pool size in bytes */
{
#ifdef UNIV_MEM_DEBUG

    ulint    i;

    /* Initialize the hash table */
    ut_a(FALSE == mem_hash_initialized);

    mutex_create(mem_hash_mutex_key, &mem_hash_mutex, SYNC_MEM_HASH);

    for (i = 0; i < MEM_HASH_SIZE; i++) {
        UT_LIST_INIT(*mem_hash_get_nth_cell(i));
    }

    UT_LIST_INIT(mem_all_list_base);

    mem_hash_initialized = TRUE;
#endif

    //如果srv_use_sys_malloc的值是由innodb_use_sys_malloc初始化的。
    //    innodb_use_sys_malloc=0表示使用额外内存池一次性分配，以后的释放和申请都是在这个内存池进行的，而不需要额外
    //进行malloc申请
    if (UNIV_LIKELY(srv_use_sys_malloc)) {
        /* When innodb_use_sys_malloc is set, the
        mem_comm_pool won‘t be used for any allocations.  We
        create a dummy mem_comm_pool, because some statistics
        and debugging code relies on it being initialized. */
        size = 1;//如果innodb_use_sys_malloc=1，则在整个mysql运行期间额外内存池的内存池结构体大小(mem_pool_t->size)都是1个字节。
    }
    mem_comm_pool = mem_pool_create(size);
}

srv_use_sys_malloc：
  这个参数是控制innodb 额外内存分配方式；默认是on(1),表示内存申请采用malloc，内存的使用申请情况大小记录在mem_area_t中，额外内存池结构体中的size仅仅是一个字节大小；如果为off(0),则表示使用innodb内存池进行分配。
注意，这里仅仅是针对额外内存池的内存配(UNIV_INTERN mem_pool_t*    mem_comm_pool    = NULL).当srv_use_sys_malloc=0的时候，在mysql启动后，一次性申请srv_mem_pool_size个字节的内存放到mem_pool_t中，并初始化每个使用情况的结构体(mem_area_t)。

如果srv_use_sys_malloc=1,表示内存申请是使用malloc的方式。

mem_heap_create_block_func(
/*=======================*/
    mem_heap_t*    heap,    /*!< in: memory heap or NULL if first block
                should be created */
    ulint        n,    /*!< in: number of bytes needed for user data */
#ifdef UNIV_DEBUG
    const char*    file_name,/*!< in: file name where created */
    ulint        line,    /*!< in: line where created */
#endif /* UNIV_DEBUG */
    ulint        type)    /*!< in: type of heap: MEM_HEAP_DYNAMIC or
                MEM_HEAP_BUFFER */
说明：
   mem_heap_create_block_func中调用mem_area_alloc(&len, mem_comm_pool));这个函数主要是记录mem_pool_t中的free_list信息(申请了多大内存)
此时mem_comm_pool->size都是为1，仅仅变化的是free_list中的信息。

void* mem_area_alloc(
/*===========*/
    ulint*        psize,    /*!< in: requested size in bytes; for optimum
                space usage, the size should be a power of 2
                minus MEM_AREA_EXTRA_SIZE;
                out: allocated size in bytes (greater than
                or equal to the requested size) */
    mem_pool_t*    pool)    /*!< in: memory pool */
   说明：
     mem_area_alloc函数返回return(ut_malloc(size))，使用malloc方式申请额外内存池服务。

//官方说明：
14.13.3 Configuring the Memory Allocator for InnoDB

When InnoDB was developed, the memory allocators supplied with operating systems and run-time libraries were often lacking in performance and scalability. At that time, there were no memory allocator libraries tuned for multi-core CPUs. Therefore, InnoDB implemented its own memory allocator in the mem subsystem. This allocator is guarded by a single mutex, which may become a bottleneck. InnoDB also implements a wrapper interface around the system allocator (malloc and free) that is likewise guarded by a single mutex.

Today, as multi-core systems have become more widely available, and as operating systems have matured, significant improvements have been made in the memory allocators provided with operating systems. New memory allocators perform better and are more scalable than they were in the past. The leading high-performance memory allocators include Hoard, libumem, mtmalloc, ptmalloc, tbbmalloc, and TCMalloc. Most workloads, especially those where memory is frequently allocated and released (such as multi-table joins), benefit from using a more highly tuned memory allocator as opposed to the internal, InnoDB-specific memory allocator.

You can control whether InnoDB uses its own memory allocator or an allocator of the operating system, by setting the value of the system configuration parameter innodb_use_sys_malloc in the MySQL option file (my.cnf or my.ini). If set to ON or 1 (the default), InnoDB uses the malloc and free functions of the underlying system rather than manage memory pools itself. This parameter is not dynamic, and takes effect only when the system is started. To continue to use the InnoDB memory allocator, set innodb_use_sys_malloc to 0.

When the InnoDB memory allocator is disabled, InnoDB ignores the value of the parameter innodb_additional_mem_pool_size. The InnoDB memory allocator uses an additional memory pool for satisfying allocation requests without having to fall back to the system memory allocator. When the InnoDB memory allocator is disabled, all such allocation requests are fulfilled by the system memory allocator.

On Unix-like systems that use dynamic linking, replacing the memory allocator may be as easy as making the environment variable LD_PRELOAD or LD_LIBRARY_PATH point to the dynamic library that implements the allocator. On other systems, some relinking may be necessary. Please refer to the documentation of the memory allocator library of your choice.

Since InnoDB cannot track all memory use when the system memory allocator is used (innodb_use_sys_malloc is ON), the section “BUFFER POOL AND MEMORY” in the output of the SHOW ENGINE INNODB STATUS command only includes the buffer pool statistics in the “Total memory allocated”. Any memory allocated using the mem subsystem or using ut_malloc is excluded.

Note
innodb_use_sys_malloc and innodb_additional_mem_pool_size are deprecated in MySQL 5.6.3 and will be removed in a future release.