jemalloc管理块(arena、bin)

arena是jemalloc的总的管理块,一个进程中可以有多个arena,arena的最大个可以通过静态变量narenas_auto,。

可通过静态数组arenas获取进程中所有arena的指针:

(gdb) p  narenas_auto
$359 = 2
(gdb) p *[email protected]
$360 = {0x7f93e02200, 0x7f93f12280}

可知,目前进程中arena的最大个数是2,它们的指针分别为0x7f93e02200,0x7f93f12280。

arena的声明如下:

typedef struct arena_s arena_t;

struct arena_s {
    /* This arena‘s index within the arenas array. */
    unsigned        ind;

    /*
     * Number of threads currently assigned to this arena.  This field is
     * synchronized via atomic operations.
     */
    unsigned        nthreads;

    /*
     * There are three classes of arena operations from a locking
     * perspective:
     * 1) Thread assignment (modifies nthreads) is synchronized via atomics.
     * 2) Bin-related operations are protected by bin locks.
     * 3) Chunk- and run-related operations are protected by this mutex.
     */
    malloc_mutex_t        lock;

    arena_stats_t        stats;
    /*
     * List of tcaches for extant threads associated with this arena.
     * Stats from these are merged incrementally, and at exit if
     * opt_stats_print is enabled.
     */
    ql_head(tcache_t)    tcache_ql;

    uint64_t        prof_accumbytes;

    /*
     * PRNG state for cache index randomization of large allocation base
     * pointers.
     */
    uint64_t        offset_state;

    dss_prec_t        dss_prec;

    /*
     * In order to avoid rapid chunk allocation/deallocation when an arena
     * oscillates right on the cusp of needing a new chunk, cache the most
     * recently freed chunk.  The spare is left in the arena‘s chunk trees
     * until it is deleted.
     *
     * There is one spare chunk per arena, rather than one spare total, in
     * order to avoid interactions between multiple threads that could make
     * a single spare inadequate.
     */
    arena_chunk_t        *spare;

    /* Minimum ratio (log base 2) of nactive:ndirty. */
    ssize_t            lg_dirty_mult;

    /* True if a thread is currently executing arena_purge_to_limit(). */
    bool            purging;

    /* Number of pages in active runs and huge regions. */
    size_t            nactive;

    /*
     * Current count of pages within unused runs that are potentially
     * dirty, and for which madvise(... MADV_DONTNEED) has not been called.
     * By tracking this, we can institute a limit on how much dirty unused
     * memory is mapped for each arena.
     */
    size_t            ndirty;

    /*
     * Unused dirty memory this arena manages.  Dirty memory is conceptually
     * tracked as an arbitrarily interleaved LRU of dirty runs and cached
     * chunks, but the list linkage is actually semi-duplicated in order to
     * avoid extra arena_chunk_map_misc_t space overhead.
     *
     *   LRU-----------------------------------------------------------MRU
     *
     *        /-- arena ---     *        |            |
     *        |            |
     *        |------------|                             /- chunk -     *   ...->|chunks_cache|<--------------------------->|  /----\ |<--...
     *        |------------|                             |  |node| |
     *        |            |                             |  |    | |
     *        |            |    /- run -\    /- run -\   |  |    | |
     *        |            |    |       |    |       |   |  |    | |
     *        |            |    |       |    |       |   |  |    | |
     *        |------------|    |-------|    |-------|   |  |----| |
     *   ...->|runs_dirty  |<-->|rd     |<-->|rd     |<---->|rd  |<----...
     *        |------------|    |-------|    |-------|   |  |----| |
     *        |            |    |       |    |       |   |  |    | |
     *        |            |    |       |    |       |   |  \----/ |
     *        |            |    \-------/    \-------/   |         |
     *        |            |                             |         |
     *        |            |                             |         |
     *        \------------/                             \---------/
     */
    arena_runs_dirty_link_t    runs_dirty;
    extent_node_t        chunks_cache;

    /*
     * Approximate time in seconds from the creation of a set of unused
     * dirty pages until an equivalent set of unused dirty pages is purged
     * and/or reused.
     */
    ssize_t            decay_time;
    /* decay_time / SMOOTHSTEP_NSTEPS. */
    nstime_t        decay_interval;
    /*
     * Time at which the current decay interval logically started.  We do
     * not actually advance to a new epoch until sometime after it starts
     * because of scheduling and computation delays, and it is even possible
     * to completely skip epochs.  In all cases, during epoch advancement we
     * merge all relevant activity into the most recently recorded epoch.
     */
    nstime_t        decay_epoch;
    /* decay_deadline randomness generator. */
    uint64_t        decay_jitter_state;
    /*
     * Deadline for current epoch.  This is the sum of decay_interval and
     * per epoch jitter which is a uniform random variable in
     * [0..decay_interval).  Epochs always advance by precise multiples of
     * decay_interval, but we randomize the deadline to reduce the
     * likelihood of arenas purging in lockstep.
     */
    nstime_t        decay_deadline;
    /*
     * Number of dirty pages at beginning of current epoch.  During epoch
     * advancement we use the delta between decay_ndirty and ndirty to
     * determine how many dirty pages, if any, were generated, and record
     * the result in decay_backlog.
     */
    size_t            decay_ndirty;
    /*
     * Memoized result of arena_decay_backlog_npages_limit() corresponding
     * to the current contents of decay_backlog, i.e. the limit on how many
     * pages are allowed to exist for the decay epochs.
     */
    size_t            decay_backlog_npages_limit;
    /*
     * Trailing log of how many unused dirty pages were generated during
     * each of the past SMOOTHSTEP_NSTEPS decay epochs, where the last
     * element is the most recent epoch.  Corresponding epoch times are
     * relative to decay_epoch.
     */
    size_t            decay_backlog[SMOOTHSTEP_NSTEPS];

    /* Extant huge allocations. */
    ql_head(extent_node_t)    huge;
    /* Synchronizes all huge allocation/update/deallocation. */
    malloc_mutex_t        huge_mtx;

    /*
     * Trees of chunks that were previously allocated (trees differ only in
     * node ordering).  These are used when allocating chunks, in an attempt
     * to re-use address space.  Depending on function, different tree
     * orderings are needed, which is why there are two trees with the same
     * contents.
     */
    extent_tree_t        chunks_szad_cached;
    extent_tree_t        chunks_ad_cached;
    extent_tree_t        chunks_szad_retained;
    extent_tree_t        chunks_ad_retained;

    malloc_mutex_t        chunks_mtx;
    /* Cache of nodes that were allocated via base_alloc(). */
    ql_head(extent_node_t)    node_cache;
    malloc_mutex_t        node_cache_mtx;

    /* User-configurable chunk hook functions. */
    chunk_hooks_t        chunk_hooks;

    /* bins is used to store trees of free regions. */
    arena_bin_t        bins[NBINS];

    /*
     * Quantized address-ordered trees of this arena‘s available runs.  The
     * trees are used for first-best-fit run allocation.
     */
    arena_run_tree_t    runs_avail[1]; /* Dynamically sized. */
};

其他成员暂时不关注,这里我们先讨论bins这个arena_bin_t数组,数组大小是36,对应36种大小的region和run。

binind表示bins中的偏移,每一个binind对应一个固定大小的region。其对应关系为:

usize = index2size(binind);

size_tindex2size(szind_t index)
{
return (index2size_lookup(index));
}

size_t index2size_lookup(szind_t index)
{
    size_t ret = (size_t)index2size_tab[index];
return (ret);
}

arena_bin_t的声明如下:

typedef struct arena_bin_s arena_bin_t;

struct arena_bin_s {
    /*
     * All operations on runcur, runs, and stats require that lock be
     * locked.  Run allocation/deallocation are protected by the arena lock,
     * which may be acquired while holding one or more bin locks, but not
     * vise versa.
     */
    malloc_mutex_t        lock;

    /*
     * Current run being used to service allocations of this bin‘s size
     * class.
     */
    arena_run_t        *runcur;

    /*
     * Tree of non-full runs.  This tree is used when looking for an
     * existing run when runcur is no longer usable.  We choose the
     * non-full run that is lowest in memory; this policy tends to keep
     * objects packed well, and it can also help reduce the number of
     * almost-empty chunks.
     */
    arena_run_tree_t    runs;

    /* Bin statistics. */
    malloc_bin_stats_t    stats;
};

runcur是当前可用于分配的run,

runs是个红黑树,链接当前arena中,所有可用的相同大小region对应的run,

如果runcur已满,就会从runs里找可用的run。

stats是当前bin对应的run和region的状态信息。

我们看一下实际运行过程中,bins中的index为5的一个arena_bin_t:

(gdb) p (*je_arenas[0])->bins[5]
$365 = {
  lock = {
    lock = {
      __private = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
    }
  },
  runcur = 0x7f68408ad0,
  runs = {
    rbt_root = 0x7f78e06e38
  },
  stats = {
    nmalloc = 236529,
    ndalloc = 229379,
    nrequests = 1181919,
    curregs = 7150,
    nfills = 60225,
    nflushes = 42510,
    nruns = 64,
    reruns = 5402,
    curruns = 31
  }
}
时间: 2024-10-27 06:20:56

jemalloc管理块(arena、bin)的相关文章

Linux用户管理之使用/bin/false和/usr/sbin/nologin拒绝用户登录及其功能分析(转)

/bin/nologin,/bin/false的意思是禁止某个用户登录. 比较常用的用法: #添加一个不能登录的用户 useradd -d /usr/local/apache -g apache -s /bin/false apache 要拒绝系统用户登录,可以将其shell设置为/usr/sbin/nologin或者/bin/false usermod -s | --shell /usr/sbin/nologin username 或者 usermod -s | -shell /bin/fal

codis+redis 集群搭建管理

Codis 是一个分布式 Redis 解决方案, 对于上层的应用来说, 连接到 Codis Proxy 和连接原生的 Redis Server 没有明显的区别 (不支持的命令列表), 上层应用可以像使用单机的 Redis 一样使用, Codis 底层会处理请求的转发, 不停机的数据迁移等工作, 所有后边的一切事情, 对于前面的客户端来说是透明的, 可以简单的认为后边连接的是一个内存无限大的 Redis 服务. Codis 由四部分组成: Codis Proxy (codis-proxy) Cod

Go内存管理

1.前言 编写过C语言程序的肯定知道通过malloc()方法动态申请内存,其中内存分配器使用的是glibc提供的ptmalloc2.除了glibc,业界比较出名的内存分配器有Google的tcmalloc和Facebook的jemalloc. 二者在避免内存碎片和性能上均比glic有比较大的优势,在多线程环境中效果更明显. Golang中也实现了内存分配器,原理与tcmalloc类似,简单的说就是维护一块大的全局内存,每个线程(Golang中为P)维护一块小的私有内存,私有内存不足再从全局申请.

STM32-内存管理

#ifndef __MALLOC_H #define __MALLOC_H #include "stm32f10x.h" ////////////////////////////////////////////////////////////////////////////////// //本程序只供学习使用,未经作者许可,不得用于其它任何用途 //ALIENTEK战舰STM32开发板V3 //内存管理 驱动代码 //正点原子@ALIENTEK //技术论坛:www.openedv.c

[精通Objective-C]块(block)

[精通Objective-C]块(block) 参考书籍:<精通Objective-C>[美] Keith Lee 目录 精通Objective-C块block 目录 块的语法 块的词汇范围 块的内存管理 块的使用 使用块为数组排序 使用块的并行编程方式 块的语法 块是一个实现的闭包,一个允许访问其常规范围之外变量的函数.此外,一个Objective-C块实际上就是一个对象,它是NSObject类的子类,拥有NSObject类的相关属性. 块的声明: int (^oneParamBlock)(

虚拟机的管理

系统虚拟机管理 1.安装 #!/bin/bash                                                ##命令运行环境的指定virt-install \                                             ##安装虚拟机--name $1 \                                                ##虚拟机名称指定,$1表示脚本后的第一串字符--ram 1000 \       

Nova: 虚机的块设备总结 [Nova Instance Block Device]

和物理机一样,虚拟机包括几个重要的部分:CPU.内存.磁盘设备.网络设备等.本文将简要总结虚机磁盘设备有关知识. 1. Nova boot CLI 中有关虚机块设备的几个参数 nova boot CLI 的完整参数如下: usage: nova boot [--flavor <flavor>] [--image <image>] //boot from image with id [--image-with <key=value>] //image metadata p

python实现主机批量管理

在日常的运维工作中批量对主机的是很常见的,市面上也有许多主机批量管理的软件,但有时候这些软件并不能完全的满足我们的需求.python中刚好提供了关于主机批量管理的模块,今天就让我们来看看如何利用python实现主机批量管理 python提供主机批量管理的模块主要有三个paramiko.fabric与pexpect,今天我们主要说的是paramiko模块,paramiko模块是第三方模块 安装:pip install paramiko或者yum install python-paramiko如果都

用脚本实现虚拟机的安装和管理

用脚本安装虚拟机,并管理虚拟机的简单运行开启等操作 安装虚拟机 #! bin/bash     ###定义文件为bin目录下运行的bash vir-install \    ###开始安装虚拟机 --name $1 \      ###命名虚拟机 --ram 1024 \     ###设定内存大小 --file /var/lib/libvirt/images/$1.img\  ###设定硬盘文件 --file-size 8 \ ##硬盘分配大小 --cdrom /root/Desktop/rh