Memcached缓存瓶颈分析

Memcached缓存瓶颈分析

  • 获取Memcached的统计信息

    Shell:

    # echo "stats" | nc 127.0.0.1 11211
    

    PHP:

    $mc = new Memcached();
    $mc->addServer(‘127.0.0.1‘,11211);
    $stats = $mc->getStats();
    
  • Memcached缓存瓶颈分析的一些指标

Posted in MemcachedPerformance analysis & tuning.

Nginx配置优化说明

Posted on 2013 年 5 月 22 日 by Jose

Nginx配置优化说明

  • Nginx配置可优化项

    # worker_processes的数目应该不多于服务器的CPU的核数目,
    # 每个CPU核超过1个worker不会使得Nginx性能有什么提升
    worker_processes 24;
    
    # 给Nginx使用的文件描述符的数目.这个需要一起设置系统内核参数‘ulimit -n 200000‘
    # 或修改‘/etc/security/limits.conf‘, 参考《高负载Linux调优》
    worker_rlimit_nofile 200000;
    
    # 设定每个worker进程最多可以服务多少客户端,
    # Max clients = worker_connections * worker_processes
    # "Max clients"其实也受到系统可使用的socket连接(socket connections)的限制,大约是64K
    worker_connections 4000;
    
    # 对于Linux系统来说选择epoll,那样每个线程可以服务更多的客户端
    use epoll;
    
    # 在Nginx获得有新连接的通知之后,接受尽可能多的连接
    # 需要注意的是:如果worker_connections设置太低的话,这样可能会造成拥堵
    multi_accept on;
    
    # 缓存打开的文件描述符(open FDs),经常被访问的文件的信息.
    # 适当设置下面这些值可以极大提高每秒的请求数.
    # 实际设置的值不一定跟下面的一样,请根据具体机器来调优.下面只是测试环境中的一个设置例子.
    open_file_cache max=200000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;
    
    # 缓冲日志写入可以加速I/O,或是有可能的话甚至可以禁止,看具体的生产环境要求.
    #access_log /var/log/nginx/access.log main buffer=16k;
    access_log off;
    
    # 使用sendfile从Linux内核中的一个文件描述符(FD)拷贝数据到另一个文件描述符比read() + write()更有效率,
    # 因为read() + write()的方式需要在用户空间(user space)来回传输数据
    sendfile on;
    
    # 使用tcp_nopush会使Nginx尝试使用一个数据包发送它的HTTP响应头(HTTP response head),而不是使用部分帧
    # (partial frames).无论是对调用sendfile之前追加头部(prepending headers)或是对是吞吐量优化来说,
    # 都是非常有用的.
    tcp_nopush on;
    
    # 不要缓冲数据发送(data-sends)(禁止Nagle算法).
    # 设置为on非常适合于实时频繁发送小数据.(比如图片,pdf,ppt文件之类不是小数据的,应该设置为off)
    tcp_nodelay on;
    
    # keep-alive连接的超时时间.服务器在这个超时时间之后会关闭这个连接.
    keepalive_timeout 30;
    
    # 可通过keep-alive连接的客户端请求数.下面的值仅是测试环境使用,具体设置请根据实际机器来调.
    keepalive_requests 100000;
    
    # 在客户端停止响应之后,允许服务器关闭连接,释放socket关联的内存
    reset_timedout_connection on;
    
    # 设置客户端读请求的超时时间,默认是60s
    client_body_timeout 10;
    
    # 设置客户端的响应超时时间.如果客户端停止读取数据,在这么多时间之后就释放过期的客户端连接,默认是60s
    send_timeout 2;
    
    # 压缩设置.减少网络传输数据量.
    gzip on;
    gzip_min_length 10240;
    gzip_proxied expired no-cache no-store private auth;
    gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml;
    gzip_disable "MSIE [1-6]\.";
    

    这些nginx配置项所处的位置,请继续往下看.

  • Nginx配置例子

    下面的配置例子仅为说明上面提到的优化项在nginx.conf中的位置,具体的优化值请根据实际机器来调.

    # cat nginx.conf
    user www www;
    worker_processes 2;
    worker_cpu_affinity 0001 0010;
    worker_rlimit_nofile 100000;
    pid logs/nginx.pid;
    error_log logs/error.log crit;
    
    events {
        # Maximum clients = worker_processes * worker_connections
        worker_connections 4096;
        multi_accept on;
        use epoll;
    }
    #google_perftools_profiles log/profile;
    
    http {
        # Basic Settings
        sendfile on;
        open_file_cache max=100000 inactive=20s;
        open_file_cache_valid 30s;
        open_file_cache_min_uses 2;
        open_file_cache_errors on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 20;
        #keepalive_requests 100000;
        reset_timedout_connection on;
    
        client_max_body_size 15m;
        client_body_timeout 60;
        client_header_timeout 60;
        client_body_buffer_size  128K;
        client_header_buffer_size 1k;
        large_client_header_buffers 4 32k;
        send_timeout 60;
        types_hash_max_size 2048;
        server_tokens off;
    
        include mime.types;
        default_type text/html;
        charset utf-8;    
    
        # Log Format
        log_format main ‘$remote_addr - $remote_user [$time_local] ‘
                ‘"$request" $status $body_bytes_sent "$http_referer" ‘
                ‘"$http_user_agent" "$http_x_forwarded_for"‘;
    
        # Gzip Settings
        gzip on;
        gzip_static on;
        gzip_disable "MSIE [1-6]\.";
        gzip_vary on;
        gzip_proxied any;
        # gzip_proxied expired no-cache no-store private auth;
        gzip_comp_level 2;
        gzip_min_length 5000;
        gzip_http_version 1.1;
        gzip_buffers 16 8k;
        gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/json application/xml  application/rss+xml font/truetype application/x-font-ttf font/opentype application/vnd.ms-fontobject image/svg+xml;
        ignore_invalid_headers on;
    
        #Virtual Host configs
        include conf.d/*;
        include sites-enabled/*.conf;
    }
    

相关阅读

  1. 《优化Nginx服务器内核参数》
  2. 《高负载Linux调优》
  3. 《高负载PHP-FPM调优》

Posted in Performance analysis & tuning.

MySQL配置my.cnf调优项详解

Posted on 2013 年 5 月 19 日 by Jose

MySQL配置调优项详解

以下是一份机器内存:64GB RAM,最大连接数为2000,MySQL使用InnoDB为主的配置说明,
某些项的最优值请根据实际生产需要来调.

[[email protected] conf]# cat my.cnf
### MySQL config 5.0/5.1/5.5
### RAM: 64GB RAM dedicated server
### Connections: 2000

[mysqld_safe]
nice = -15

[client]
socket		          = /var/run/mysqld/mysqld.sock
default-character-set = utf8

[mysqld]
#############################
### Charset and Collation ###
#############################
character-set-server  = utf8
collation-server      = utf8_general_ci

############################
### Basic Settings       ###
############################
user                  = mysql
pid-file              = /var/run/mysqld/mysqld.pid
port		      = 3306
socket		      = /var/run/mysqld/mysqld.sock
basedir               = /usr/local/mysql
datadir               = /db/data01
tmpdir                = /tmp
#tmpdir               = /db/tmp01:/db/tmp02:/db/tmp03 #Recommend using RAMDISK for tmpdir

default-storage-engine = InnoDB

skip-external-locking
skip-name-resolve

## Table and TMP settings
max_heap_table_size             = 1G    #recommend same size as tmp_table_size
tmp_table_size                  = 1G    #recommend 1G min

## Default Table Settings
#sql_mode             = NO_AUTO_CREATE_USER

##############################
### Error Logs & Slow logs ###
##############################

## Log Errors
log_error             = /db/logs01/mysql-error.err
log_warnings # default: 1, Print out warnings such as Aborted connection... to the error log.

## Log general queries
#general_log           = 1
#general_log_file      = /db/logs01/mysql-gen.log
#log-output            = file

## Log slow queries
#slow-query-log
#slow_query_log_file    = /db/logs01/mysql-slow.log
#log_queries_not_using_indexes

## It‘s worth noting that query execution time does not include the time taken to acquire
## table locks. If a query regularly runs slowly because of a high level of locking, it
## will not be logged.The value can be specified to a resolution of microseconds.
## Default : 10 (s)
#long_query_time = 10

## Optionally, you can also restrict the slow query log to those queries that cause
## more than a certain number of rows to be examined.
## This feature was introduced in MySQL 5.5.
#min_examined_row_limit = 500

###########################
### Connections         ###
###########################

## The number of outstanding connection requests MySQL can have. This comes into play
## when the main MySQL thread gets very many connection requests in a very shot time.
## It then takes some time (although very little) for the main thread
## to check the connection and start a new thread.  You need to increase this
## only if you expect a large number of connections in a short period of time.
## This value is the size of the listen queue for incoming TCP/IP connections.
## Your operating system has its own limit on the size of this queue
## Check your OS documentation for the maximum value for this variable.
## back_log cannot be set higher than your operating system limit.
back_log = 300

## The maximum permitted number of simultaneous client connections.
## Without considering MyISAM buffers, each connection uses about  192KB of memory.
## You need to make sure that-(max_connections * 192KB) + innodb_buffer_pool_size is
## less than your total amount of system memory, otherwise MySQL could start swapping.
## 2000 connections will use ~400MB of memory.
## Set max_connections as high as the theoretical maximum amount of connections
## that your application can ever have.
## max_connections = pm.max_children * number of application servers
## Default: 151
max_connections=2000

## If more than this many successive connection requests from a host are interrupted
## without a successful connection, the server blocks that host from further connections.
## You can unblock blocked hosts by flushing the host cache. To do so, issue a FLUSH HOSTS
## statement or execute a mysqladmin flush-hosts command. If a connection is established
## successfully within fewer than max_connect_errors attempts after a previous connection
## was interrupted, the error count for the host is cleared to zero.
## However, once a host is blocked, flushing the host cache is the only way to unblock it.
max_connect_errors = 100

## The number of seconds that the mysqld server waits for a connect packet
## before responding with Bad handshake.
## default: 5 (<= 5.1.22), 10 (>= 5.1.23)
connect_timeout    = 30

## The packet message buffer is initialized to net_buffer_length bytes, but can grow up
## to max_allowed_packet bytes when needed.
## This value by default is small, to catch large (possibly incorrect) packets.
## You must increase this value if you are using large BLOB columns or long strings.
## It should be as big as the largest BLOB you want to use.
## The protocol limit for max_allowed_packet is 1GB. The value should be a multiple of 1024;
## nonmultiples are rounded down to the nearest multiple
##
max_allowed_packet = 32M   #max size of incoming data to allow

############################
### Table Cache Settings ###
############################

## This controls the maximum number of open tables the cache can hold for all threads.
## You can check whether you need to increase the table cache by checking the Opened_tables
## status variable.If the value is very large or increases rapidly and
## you do not use FLUSH TABLES often, then you should increase this value.
## Or compared with the number of currently open tables (server status ‘Open_tables‘)
## The table_open_cache and max_connections system variables affect the maximum number of
## files the server keeps open.
## Range: 64~ 524288, default: 400
table_open_cache = 2048

## The number of table definitions (from .frm files) that can be stored in the
## definition cache. If you use a large number of tables, you can create a large
## table definition cache to speed up opening of tables. The table definition cache
## takes less space and does not use file descriptors, unlike the normal table cache.
## However, table_definition_cache doesn‘t offer as great a performance increase
## as the standard table_cache. The recommended way to utilize it is as a secondary
## caching mechanism when the table cache becomes full.
## Range: 400~524288, Default: 400
table_definition_cache = 400

########################
### File Descriptors ###
########################

## The number of files that the operating system permits mysqld to open. If you find MySQL
## complaining about Too Many Open Files, raising this value should be your first avenue.
## The value is 0 on systems where MySQL cannot change the number of open files.
open_files_limit = 16384

######################
### Thread Cache ###
######################

## How many threads the server should cache for reuse. When a client disconnects,
## the client‘s threads are put in the cache. if there are fewer than thread_cache_size
## threads there. By examining the difference between the Connections and Threads_created
## status variables,  you can see how efficient the thread cache is.
## The cache miss ratio = Threads_created/Connections.
## default : 0 (effectively disabling this feature), recommend 5% of max_connections
thread_cache_size = 100

## This variable is specific to Solaris systems.
## Range: 1~512, Default: 10, recommend 2x CPU cores
#thread_concurrency = 16

######################
### Query Cache    ###
######################

## Disable the query cache. Both of these must be set as 0 due to a bug in MySQL.
## The query cache adds a global lock and performs poorly with a non-trivial write-load.
query_cache_size=0
query_cache_type=0

#query_cache_size                = 64M   #global buffer
#query_cache_limit               = 512K  #max query result size to put in cache

###################################
### Per-Thread Buffers          ###
###################################

## The sort buffer is allocated on a per-client basis for any query that needs to perform
## a sort operation (that is, ORDER BY and GROUP BY operations).sort_buffer_size is not
## specific to any storage engine and applies in a general manner for optimization.
## If you see many ‘Sort_merge_passes‘ per second in the global server status output,
## you can consider increasing this value.
## On Linux, MySQL uses mmap() rather than malloc() for allocating sort buffer sizes larger than
## 256 KB, and this is somewhat slower.So, ideally you should keep the sort buffer
## at 256 KB or less. There is a similar threshold at 2 MB. If you do require a value
## higher than 256 KB, you should also aim to keep it under 2 MB.
## The maximum permissible setting for sort_buffer_size is 4GB.
## Values larger than 4GB are permitted for 64-bit platforms.
sort_buffer_size = 2M

## The read buffer is used for queries that perform sequential scans of tables.
## Each thread that does a sequential scan for a MyISAM table allocates a buffer
## of this size (in bytes) for each table it scans.
## If you do many sequential scans,  you might want to increase this value.
## default: 128K, change in increments of 4K, Maximum allowed: 2G
read_buffer_size = 2M

## The read_rnd cache is the counterpart to read_buffer used when reading sorted rows
## (rather than sequential rows).
## When reading rows from a MyISAM table in sorted order following a key-sorting
## operation, the rows are read through this buffer to avoid disk seeks.
## Setting the variable to a large value can improve ORDER BY performance by a lot.
## However, this is a buffer allocated for each client, so you should
## not set the global variable to a large value. Instead, change this session variable
## only from within those clients that need to run large queries
## default: 256K, Maximum allowed: 2G
read_rnd_buffer_size = 2M

## The minimum size of the buffer that is used for plain index scans, range index scans,
## and joins that do not use indexes and thus perform full table scans.
## In most cases, better column indexing produces a greater performance increase than
## raising this buffer. Care should be taken not to make the join buffer too big
## default: 128K, Maximum allowed: 4G
join_buffer_size = 2M

## Stack size for each thread.The default of 192KB (256KB for 64-bit systems) is large
## enough for normal operation. If the thread stack size is too small, it limits the
## complexity of the SQL statements that the server can handle, the recursion depth
## of stored procedures, and other memory-consuming actions.
## default: 32bit: 192K, 64bit: 256K
thread_stack = 512k

#############################
### MySQL Master Settings ###
#############################

## This mandatory variable is a unique number for the server
## within the current MySQL topology.
server_id = 1

## auto_increment_* setting For multi-master topology
#replicate-same-server-id = 0
#auto_increment_increment= 2
#auto_increment_offset   = 1

## Replication Semi-Synchronous 5.5.x only, requires dynamic plugin loading ability
#rpl_semi_sync_master_enabled   = 1 #enable = 1, disable = 0
#rpl_semi_sync_master_timeout   = 1000 #in milliseconds , master only setting

## This value controls how the master will wait for a timeout from one or more slaves
## before reverting to asynchronous replication.
#rpl_semi_sync_master_wait_no_slave = 1 # Default: ON

## This defines the level of debugging logging. The allowed values are
## 1 (general level logging), 16 (detailed level logging),
## 32 (network wait logging), and 64 (function level logging).
#rpl_semi_sync_master_trace_level = 1

## This enables the binary log and is mandatory for replication on the master host.
## This variable also defines the basename of the binary log files.
log_bin       = /db/logs01/mysql-bin
log_bin_index = /db/logs01/mysql-index

## This variable controls the type of binary logging.
## STATEMENT (the default): logs the actual SQL statement to the binary log
## ROW : log changed data blocks to the binary log
## MIXED: will choose the most applicable method for the given statement necessary
## to ensure data consistency
binlog_format = MIXED

## This cache is used to hold changes that are to be written to the binary log
## during a transaction. Increasing this value for very large transactions can
## possibly increase performance. The Binlog_cache_use and Binlog_cache_disk_use
## status variables can be useful for tuning the size of this variable.
## Default:32k,
binlog_cache_size = 10M

## Beginning with MySQL 5.5.9, this variable specifies the size of the cache
## for the binary log to hold non-transactional statements during transactions
## on a per client basis. There may be a benefit total increasing this value
## using large non-transactional statements. In MySQL 5.5.3 through 5.5.8, the size
## for both caches is set using binlog_cache_size. This means that, in these MySQL
## versions, the total memory used for these caches is double the value set for
## binlog_cache_size. The Binlog_stmt_cache_use and Binlog_stmt_cache_disk_use
## status variables can be useful for tuning the size of this variable.
## Default: 32k
binlog_stmt_cache_size = 10M

## This is the maximum size of the binary log file before a new file is created.
## The FLUSH BINARY LOGS command will also dynamically close the current
## binary log and create a new file.
## Range: 4k~1G, Default: 1G
max_binlog_size = 256M

## This variable defines the number of days binary log files are retained.
## Files older than the number of days are removed (similar in operation
## to a PURGE MASTER LOGS command) when a new binary log file is created.
expire_logs_days = 30

## These variables on the master host limit which statements are logged to
## the binary log based on the specified database name, preceded by a USE qualifier.
## Use of binlog_do_db and binlog_ingnore_db can make a binary log unusable
## in a point in time recovery of a full primary database.  These options are also
## incomplete, as they require all SQL to be preceded by an applicable USE, and do not
## handle cross-schema joins as you would expect.
#binlog-do-db = book3
#binlog-ignore-db=mysql

############################
### MySQL Slave Settings ###
############################

## By default, when a slave server starts, an implied SLAVE START occurs.
## With this variable specified, the slave is not automatically started and
## must be performed manually with START SLAVE.
#skip_slave_start

## The relay logs hold replicated database changes retrieved from the
## master binary log and written with the I/O thread
relay_log = /db/logs01/mysql-relay-bin

## This variable defines the name of the relay log index that holds the names
## of all the relay logs available. The default filename is the relay_log variable
## value with the extension .index
relay-log-index= /db/logs01/mysql-relay-index

## auto_increment_* in multi-master replication
#replicate-same-server-id       =
#auto-increment-increment       =
#auto-increment-offset          = 

## When defined and binary logging is enabled on a slave, all replicated changes
## from the SQL thread are also written to the slave server binary log. This option
## is used to chain multiple nodes together through replication. For example, if you
## have three servers (A, B, and C) and want to connect them in a chain you would
## use log_slave_updates on B. B would replicate from A, and C from B, forming a chain,
## (A -> B -> C).
## Binary logging must be enabled on the slave for this variable to have any effect.
## Default: FALSE
log-slave-updates

## This variable defines that the slave will not accept DML or DDL statements other
## than those applied by the replication slave SQL thread. The exception is a user
## with SUPER privilege will override this setting.
## Default: False
#read_only

## This variable controls how the relay log files are purged. The default of 1
## specifies that the relay log files are removed when they are no longer needed
## for applying replication events. A value of 0 retains the log files.
## Default: 1
#relay_log_purge = 1

## These variables control how frequently a file sync is performed on the
## respective relay log and relay log info file. The number represents the name of
## executed SQL statements to apply before action.
## The default is 0; the safest durability setting is 1
sync_relay_log = 1
sync_relay_log_info = 1

## These variables are used to filter which recorded master binary log statements
## are applied on the slave. The replicate_do_db and replicate_ingnore_db can
## cause errors, as they require all SQL to be preceded by an applicable USE and
## do not handle cross-schema joins as you would expect.
#replicate-do-db                =
#replicate-ignore-db            =
#replicate-do-table             =
#relicate-ignore-table          =
#replicate-rewrite-db           =
#replicate-wild-do-table        = b.%
#replicate-wild-ignore-table    = a.%

## Replication error codes can be skipped automatically when specified with
## this variable. It is rarely a good idea to specify a value for slave_skip_errors,
## because there is no accountability of the occurrences of these silent errors,
## which will generally lead to data drift and/or loss of data integrity.
## Error 1062 means Duplicate entry
## Error 1060 means Duplicate column name
## Error 1050 means Table already exists
## Error 1051 means Unknown table
#slave-skip-errors = 1062,1050,1060,1052
#slave-skip-errors=1062,1053,1146,1051,1050

## Semisynchronous Replication settings on Slave (MySQL 5.5+)
## When this option set to ON, semisynchronous replication on the slave is possible
#rpl_semi_sync_slave_enabled = 1

## This defines the level of debugging logging. The allowed values are 1, 16, 32, and 64
#rpl_semi_sync_slave_trace_level = 16

#######################
### MyISAM Settings ###
#######################

## This is MyISAM key cache: a memory buffer used to hold frequently accessed
## index (key) blocks. The key cache is used only for buffering indexes, and
## MySQL still needs memory for the tables‘s contents. For a dedicated MySQL
## server, the general rule of thumb is to allocate 25%~50% of the total memory
## to the key cache. If the key cache is too large, the system may run out of
## physical memory and start swapping.
## Key read miss ratio = Key_reads / Key_read_requests,
## the ratio should normally be less than 0.01.
##
## Key write miss ration = Key_writes/Key_write_requests,
## the ratio is usually near 1 if you are using mostly updates and deletes,
## but might be much smaller if you tend to do updates that affect many rows
## at the same time or if you are using the DELAY_KEY_WRITE table option.
## The fraction of the key buffer in use can be determined using
## key_buffer_size in conjunction with the Key_blocks_unused status variable
## and the buffer block size, which is available from the key_cache_block_size
## system variable: 1 - ((Key_blocks_unused * key_cache_block_size) / key_buffer_size)
## Default: 8388608 (~8M), Max Limit: 4G for each key cache
key_buffer_size = 30M

## Multiple Key Caches
## 1. through mysql command line.
## mysql > SET GLOBAL hot_cache.key_buffer_size = 1024*1024* 16;
## mysql > select @@GLOBAL.hot_cache.key_buffer_size, @@GLOBAL.hot_cache.key_cache_block_size;
## mysql > CACHE INDEX t1, t2 IN hot_cache;
## mysql > LOAD INDEX INTO CACHE t1, t2 IGNORE_LEAVES;
##
## 2. through conf file
## in .my.cnf:
#mycache1.key_buffer_size = 512M
#mycache2.key_buffer_size = 1G
#init_file=/var/lib/mysql/custom.sql
## custom.sql contents as follows:
##CACHE INDEX categories, comments IN mycache1
##CACHE INDEX userprofiles IN mycache2

## The size of the buffer that is allocated when preloading indexes.
## Range: 1k~1G, Default: 32k
#preload_buffer_size = 32k 

## The size in bytes of blocks in the key cache.
## Your motivation for changing these is to match the block size used by Linux
## for disk I/O (not to be confused with the filesystem‘s block size).
## On x86 Linux, use a value of 4 KB.
## show the disk I/O block size:  perl -e ‘$a=(stat ".")[11]; print $a‘
## Range: 512~16384 (16k), Default: 1024 (1k)
#key_cache_block_size = 4k

## This specifies the percentage of the key cache to allocate to the warm list.
## The default value, 100, effectively causes MIS(Midpoint Insertion Strategy)
## to be disabled (because the hot list will be of zero size).When lowering this value,
## remember that the warm list will be used more than the hot list, and the sizes
## of each should reflect this. Broadly speaking, a division limit of approximately
## 60 percent to 90 percent should be right in most cases.
#key_cache_division_limit = 100

## This controls how long an unused entry should stay in the hot list
## before being moved back into the warm list.
## The default is 300 seconds, and permitted values range from 100 seconds upward
#key_cache_age_threshold=300

## Enabled by default, the concurrent_insert option enables INSERT statements
## to be executed on a table at the same time as SELECT queries are reading from it.
## This partly alleviates the table-level locking. (Although it does nothing
## to help the table-level locking that occurs on UPDATE and DELETE queries.)
## 0: turns this option off
##
## 1 (the default) : enables concurrent inserts only when there are no deleted
## rows in the middle of the table.
##
## 2: enables concurrent inserts even if such deleted rows exist. If any SELECT
## queries are running, the data will be inserted at the end of the table. Otherwise,
## it will be inserted in the gap left by the deleted row (the default behavior).
## Default : 1
concurrent_insert  = 2

## If set to 1, all INSERT, UPDATE, DELETE, and LOCK TABLE WRITE statements wait until there is
## no pending SELECT or LOCK TABLE READ on the affected table.
## This affects only storage engines that use only table-level locking (such as MyISAM, MEMORY,
## and MERGE).If you choose not to enable this option globally, it can still be set on a per-query
## basis using the following syntax:
## update low_priority into ...
## Default : 0
# low_priority_updates = 0

## OFF : DELAY_KEY_WRITE is ignored.
## ON (the default): MySQL honors any DELAY_KEY_WRITE option specified in CREATE TABLE statements.
## ALL : All new opened tables are treated as if they were created with the DELAY_KEY_WRITE
## option enabled.
## When a MyISAM index/key is updated, the default behavior is to write the changes back to disk.
## For frequently modified tables,this can result in a lot of disk writes.By enabling
## delay_key_write (either on a per table basis,or globally by specifying a value of ALL),
## changes to the indexes are buffered in memory, and not written out to disk until the table is closed. ## This greatly speeds up index updates. The downside is that if MySQL is not shut down cleanly,
## there is a good chance of index corruption (because the buffer will not have been flushed to disk).
## You can fix this with myisamchk, but for big tables, it will take a while to run.
## Or if you use this feature, you should add automatic checking of all MyISAM tables by
## starting the server with the --myisam-recover option (for example, --myisam-recover=BACKUP,FORCE).
## If you enable external locking with --external-locking, there is no protection
## against index corruption for tables that use delayed key writes.
## delay_key_write = 1

## The maximum size of the temporary file that MySQL is permitted to use while re-creating
## a MyISAM index (during REPAIR TABLE, ALTER TABLE, or LOAD DATA INFILE).
## If the file size would be larger than this value, the index is created
## using the key cache instead, which is slower. The value is given in bytes.
## The default value is 2GB. If MyISAM index files exceed this size and disk space is available, increasing the value may help performance.
## The space must be available in the file system containing the directory where the original
## index file is located
#myisam_max_sort_file_size = 10G. 

## The size of the buffer that is allocated when sorting MyISAM indexes during a REPAIR TABLE
## or when creating indexes with CREATE INDEX or ALTER TABLE
## Default:8388608 (8M), Max Limit: 4G+
#myisam_sort_buffer_size = 128M

## MyISAM uses a special tree-like cache to make bulk inserts faster for INSERT ... SELECT,
## INSERT ... VALUES (...), (...), ..., and LOAD DATA INFILE when adding data to nonempty tables.
## This variable limits the size of the cache tree in bytes per thread.
## Setting it to 0 disables this optimization (This is a limit per thread.)
## Default: 8M
#bulk_insert_buffer_size = 8M

## Set the mode for automatic recovery of crashed MyISAM tables
myisam-recover = BACKUP,FORCE

## Thread quantity when running repairs.
## If this value is greater than 1, MyISAM table indexes are created in parallel (each index in its own thread) during the Repair by sorting process.
## Default: 1
#myisam_repair_threads = 4

#######################
### InnoDB Settings ###
#######################

######################################
### InnoDB Table-space and logging ###
######################################

## The common part of the directory path for all InnoDB data files in the shared tablespace.
## This setting does not affect the location of per-file tablespaces when innodb_file_per_table is enabled
innodb_data_home_dir = /usr/local/mysql/data

## The paths to individual data files and their sizes.
innodb_data_file_path=ibdata1:128M;ibdata2:10M:autoextend

## The directory path to the InnoDB redo log files
innodb_log_group_home_dir = /usr/local/mysql/data

## The number of log files in the log group, default: 2
## combined size of all logs <4GB. <16G_RAM = 2, >16G_RAM = 3
innodb_log_files_in_group = 2

## The size in bytes of each log file in a log group. The combined size of log files
## must be less than 4GB.
## Bigger log file size = less I/O used for writes, longer recovery time during a failure.
## Maximum = innodb_buffer_pool_size / innodb_log_files_in_group
## Calculation: Check mysql status ‘Innodb_os_log_written‘ to calculate how many MB of data per minute are getting written to the file,
## and setting the value to large enough to hold one hour‘s worth of writes.
## Default:5M, on small buffer pool settings (under 4G), advise the same value as innodb_buffer_pool_size
## 64G_RAM+ = 368, 24G_RAM+ = 256, 8G_RAM+ = 128, 2G_RAM+ = 64
innodb_log_file_size = 368M

## If innodb_file_per_table is disabled (the default), InnoDB creates tables in the system tablespace.
## If innodb_file_per_table is enabled, InnoDB creates each new table using its own .ibd file
## for storing data and indexes, rather than in the system tablespace.
## It only impacts new tables and will not affect old "monolithic file" style InnoDB tables
## default: On (>= 5.5.0, <= 5.5.6), Off (>= 5.5.7)
innodb_file_per_table = 1

## InnoDB has two different versions: Antelope (the older version) and Barracuda (the newest).
## This applies only for tables that have their own tablespace, so for it to have an effect, innodb_file_per_table must be enabled.
## Tell InnoDB that we always want to use the Barracuda.
## Default:Barracuda (>= 5.5.0, <= 5.5.6), Antelope(>=5.5.7)
innodb_file_format=barracuda

## This specifies the maximum number of .ibd files that MySQL can keep open at one time.The file descriptors
## used for .ibd files are for InnoDB tables only. They are independent of those specified by the --open-files-limit
## server option, and do not affect the operation of the table cache.
## What is the right setting? check: sudo lsof | grep -c "\.ibd$" (myisam you should count the *.MYD)
## Minimum: 10, default: 300
innodb_open_files = 16384

############################
### InnoDB I/O resources ###
############################

## Tells the operating system that MySQL will be doing its own caching and that it should skip using the file system cache.
## Prevents double caching of the data (once inside of MySQL and again by the operating system.)
## Default: fdatasync
innodb_flush_method=O_DIRECT

## Set this to 1 on your master server for safest, ACID compliant operation (sync after every transaction, high I/O).
## Set this to 2 on your slave, which can cause up to a second of data loss after an operating system crash, but frees up I/O
## because it only fsyncs data to disk once per second.
## default: 1
innodb_flush_log_at_trx_commit=1

## If the value of this variable is greater than 0, the MySQL server synchronizes its binary log to disk
## after every sync_binlog writes to the binary log. There is one write to the binary log per statement if autocommit is enabled,
## and one write per transaction otherwise. A value of 1 is the safest choice, because in the event of a crash
## you lose at most one statement or transaction from the binary log.However, it is also the slowest choice.
## Default: 0 (no synchronizing to disk). Set this to 1 on your master server.
sync_binlog=1

## Background Threads
## InnoDB uses background threads to prefetch and store data. The default is 4 threads, but should really be 4 * # of CPU cores.
## Each thread can handle up to 256 pending requests, and a maximum of 64 threads may be configured.
## Before rushing to increase these limits, remember that they will not increase bandwidth on individual disks; although they will
## potentially help to alleviate bottlenecks in high-load environments, where data is spread across multiple disks.
## Then Pending reads and Pending writes columns of the InnoDB monitor‘s output can help you judge if the thread concurrency would benefit
## from being raised. Recalling that each thread can queue up to 256 requests, if you see more than 256 x [number of threads] pending reads or writes,
## this will clearly cause a bottleneck, and a gradual increase in the number of read/write threads would be beneficial.
## Range: 1 ~ 64, Default: 4
innodb_read_io_threads=32
innodb_write_io_threads=32

## innodb_io_capacity enables you to specify the number of I/O operations per second that the disk subsystem can handle.
## This should be set as the maximum amount of IOPS that your system has. It sets a max cap on how much I/O that InnoDB can use.
## IOPS available from Disk:
## Drive Type              Value
## Enterprise SSD          50,000
## Single Consumer SSD     10,000
## 4 Drives in a RAID-10   5000
## Single 7200RPM Drive    200
##
## Range: 100~2**64-1, Default: 200
innodb_io_capacity = 5000

## Enables InnoDB support for two-phase commit in XA transactions, causing an extra disk flush for transaction preparation.
## This setting is the default. The XA mechanism is used internally and is essential for any server that has its binary log
## turned on and is accepting changes to its data from more than one thread. If you turn it off, transactions can be written
## to the binary log in a different order from the one in which the live database is committing them. This can produce different
## data when the binary log is replayed in disaster recovery or on a replication slave
## Recommend 0 on read-only slave, disable xa to negate extra disk flush
innodb_support_xa = 1

## By default, InnoDB stores all data twice, first to the doublewrite buffer, and then to the actual data files.
## For benchmarks or cases when top performance is needed rather than concern for data integrity or possible failures,
## doublewrite can be turn off.
#skip-innodb-doublewrite

## The number of background threads devoted to the InnoDB purge operation.
## 0 (the default): the purge operation is performed as part of the master thread.
## 1 : Running the purge operation in its own thread can reduce internal contention within InnoDB, improving scalability.
## In theory, a separate thread should improve performance. But in many situations, it simply shifts
## the bottleneck from one place (queue in the thread) to another (disk contention).
##innodb_purge_threads = 0

## read-ahead Requests
## In InnoDB, memory pages are grouped in extents, where an extent consists of 64 consecutive pages. If more than a certain number of pages
## from an extent exists in the buffer cache, MySQL preloads the remaining pages in the extent.
## This variable controls the number of sequential pages in an extent that must be accessed (and be in the buffer cache) to trigger a read-ahead
## for the remaining pages. When the last page of a sequence falls at the end of an extent, InnoDB will also read in the whole of the next extent.
## Monitor via : mysql> show status like ‘%ahead%‘;
## or checking "Pages read ahead, evicted without access, Random read ahead" in BUFFER POOL AND MEMORY via "mysql> show engine innodb status \G"
## Range: 0 ~64, default: 56
#innodb_read_ahead_threshold = 56

## The main thread in InnoDB tries to write pages from the buffer pool so that the percentage of dirty (not yet written) pages will not exceed this value
## Range: 0~99, Default:75
#innodb_max_dirty_pages_pct = 90

## Adaptive Flushing
## With adaptive flushing, InnoDB attempts to calculate the rate at which flushing needs to occur, based on the number of dirty pages and
## the rate at which they have historically been flushed. This allows the master thread to perform flushing based on workload at a much more constant rate,
## eliminating I/O spikes in disk usage. Adjusting the flush rate dynamically is intended to avoid bursts of I/O activity.
## Default: ON
#innodb_adaptive_flushing = 1

## The maximum delay between polls for a spin lock.
## The os_waits column of the SHOW ENGINE INNODB MUTEX output shows the number of times that InnoDB failed to acquire a lock through polling,
## and fell back on the operating system‘s thread sleeping. Rapidly increasing values here (remember that you‘re usually interested in the rate of increase,
## rather than the absolute figure) could signify that the mutex is causing a bottleneck, and it may be worth experimenting with
## raising innodb_spin_wait_delay in the hope that less threads need to be sent to sleep.
## Range:0~4294967295(2**32-1), Default: 6
#innodb_spin_wait_delay = 6

###############################
### InnoDB Memory resources ###
###############################

## The size in bytes of the buffer that InnoDB uses to write to the log files on disk.
## If you have big transactions, making the log buffer larger saves disk I/O
## default:8M, General recomendations range: 8M~256M
innodb_log_buffer_size = 128M

## The size in bytes of the memory buffer InnoDB uses to cache data and indexes of its tables.
## The larger you set this value, the less disk I/O is needed to access data in tables. On a dedicated database server,
## you may set this to up to 90% of the machine physical memory size
## When the size of the buffer pool is greater than 1GB, setting innodb_buffer_pool_instances to a value
## greater than 1 can improve the scalability on a busy server.
## 64GB -> 57GB, 32GB -> 28GB, 16GB -> 14GB, 8GB -> 7GB.
## On a read-heavy workload, if you use iostat and see that you have a very high utilization or service time,
## you can usually add more memory (and increase innodb_buffer_pool_size) to improve performance.
## On a write-heavy workload (i.e., MySQL Master), it‘s far less important.
## Default: 128MB
##
innodb_buffer_pool_size = 57000M

## InnoDB uses a modified LRU for the buffer pool, based on an MIS. With the InnoDB buffer pool,
## the default division is for the cold list (containing less frequently accessed items) to occupy 37 percent
## of the pool size, with the hot list (frequently accessed items) taking the remaining space.
## For applications that occasionally access large tables, it often make sense to reduce innodb_old_blocks_pct,
## to prevent this less commonly accessed data from being cached so heavily. Conversely, for small,frequently accessed tables,
## raising innodb_old_blocks_pct increases the likelihood that this data will be kept in memory for future use.
## Default: 37 (3/8 of the pool), Range: 5~95
#innodb_old_blocks_pct = 37

## As with other MIS algorithms, new pages are inserted at the top of the cold list, making them prime candidates for promotion to the hot list.
## innodb_old_blocks_time specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access
## before it can be moved to the new sublist. The default value is 0: A block inserted into the old sublist moves immediately to the new sublist
## the first time it is accessed, no matter how soon after insertion the access occurs.If the value is greater than 0, blocks remain
## in the old sublist until an access occurs at least that many ms after the first access.
## Monitor BUFFER POOL AND MEMORY via ‘show engine innodb status‘.
## ‘youngs (not-youngs)/s‘ shows the rate (in seconds) at which pages in the cold list have or have not been promoted to the hot list.
## A low number of youngs/s shows that few pages are promoted to the hot list. In an application that regularly accesses the same data,
## this would suggest that innodb_old_blocks_time be lowered. Conversely, a high number of youngs/s on applications that perform
## frequent large scans would suggest that innodb_old_blocks_time be raised.
#innodb_old_blocks_time = 0

## The size in bytes of a memory pool InnoDB uses to store data dictionary information and other internal data structures.
## The more tables you have in your application, the more memory you need to allocate here. If InnoDB runs out of memory in this pool,
## it starts to allocate memory from the operating system and writes warning messages to the MySQL error log.
## Default: 8M, Typical setting: 16M ~ 128M
innodb_additional_mem_pool_size = 20M

## Using Multiple Buffer Pools,  New as of MySQL 5.5
## On busy systems with large buffer pools, there will typically be many threads accessing data simultaneously
## from the buffer pool, and this contention can be a bottleneck. Since MySQL 5.5, InnoDB enables multiple buffer pools to be created.
## Each is managed independently and maintains its own LRU and mutual exclusion (mutex).
## The innodb_buffer_pool_instances configuration option is used to control this and takes a value between 1 (the default) and 64.
## Because the use of multiple pools is intended only for high-end systems, this option has no effect when innodb_buffer_pool_size is lower than 1 GB.
## The main benefit of changing this from the default value is to increase concurrency when using larger buffer pools
## that have a high rate of data being changed. MySQL recommends setting this to a value such that each buffer pool instance
## remains at a minimum size of 1 GB or more.
## Range: 1~64, Default: 1
innodb_buffer_pool_instances = 4

## Whether InnoDB performs change buffering, an optimization that delays write operations to secondary indexes
## so that the I/O operations can be performed sequentially. The permitted values are:
## none : do not buffer any operations
## inserts: Caches insert operations only
## deletes: Caches delete operations; strictly speaking, the writes that mark index records for later deletion during a purge operation
## changes: Caches both inserts and deletes
## purges: Caches purges only, the writes when deleted index entries are finally garbage-collected
## all: buffer insert, delete-marking, and purge operations(physical deletion). This is the default value.
#innodb_change_buffering = all

## Adaptive hashing is a feature of InnoDB designed to improve performance on machines with large amounts of physical memory.
## This value controlls whether the InnoDB adaptive hash index is enabled or disabled. The adaptive hash index feature is useful for some workloads,
## and not for others; conduct benchmarks with it both enabled and disabled, using realistic workloads
## Default: ON
#innodb_adaptive_hash_index = 1

## Whether InnoDB uses the operating system memory allocator (ON) or its own (OFF).
## The default value is ON.
#innodb_use_sys_malloc = 1

##################################
### InnoDB Concurrency settings ###
##################################

## This limits the number of threads that InnoDB can perform concurrently at a given time. Once the number of threads reaches this limit,
## additional threads are placed into a wait state within a FIFO queue for execution. Threads waiting for locks are not counted
## in the number of concurrently executing threads. Setting it to 0 means
## that it‘s infinite and is a good value for Percona 5.5.
## For non-Percona setups, a recommended value is 2 times the number of CPUs plus the number of disks.
## Range: 0~1000, Default: 0
#innodb_thread_concurrency = 0

## The number of threads that can commit at the same time. A value of 0 (the default) permits
## any number of transactions to commit simultaneously
## Default: 0
#innodb_commit_concurrency = 0

##################################
### InnoDB Timeout settings ###
##################################

## The timeout in seconds an InnoDB transaction waits for a row lock before giving up.
## When a lock wait timeout occurs, the current statement is rolled back (not the entire transaction).
## To have the entire transaction roll back, start the server with the --innodb_rollback_on_timeout option
## You might decrease this value for highly interactive applications or OLTP systems,
## to display user feedback quickly or put the update into a queue for processing later.
## You might increase this value for long-running back-end operations, such as a transform step
## in a data warehouse that waits for other large insert or update operations to finish.
## innodb_lock_wait_timeout applies to InnoDB row locks only. The lock wait timeout value does
## not apply to deadlocks, because InnoDB detects them immediately and rolls back one of
## the deadlocked transactions.
## Default: 50
innodb_lock_wait_timeout = 50

## In MySQL 5.5, InnoDB rolls back only the last statement on a transaction timeout by default.
## If this configuration option is enabled, a transaction timeout causes InnoDB to abort and
## roll back the entire transaction.
## Setting to 1 can avoid error 1052 in MySQL Slave.
## Default: 0
#innodb_rollback_on_timeout = 1

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[myisamchk]
key_buffer_size = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

Posted in Performance analysis & tuningPerformance Optimization.

高负载PHP-FPM调优

Posted on 2013 年 5 月 19 日 by Jose

高负载PHP调优

针对PHP的Linux调优

  • 调整文件描述符限制
    # ulimit -n 1000000
    # vi /etc/security/limits.conf
    
    # Setting Shell Limits for File Descriptors
    *  soft nofile 1000000
    *  hard nofile 1000000
    
  • 禁止PHP代码文件所在分区的文件系统访问时间更新
    # vi /etc/fstab
    

    比如PHP代码所在分区:

    /dev/sdb1   ext4    errors=remount-ro  0  1
    

    修改为:

    /dev/sdb1   ext4    noatime,nodiratime,errors=remount-ro  0  1
    
  • 将临时文件存储到tmpfs

    在需要处理大量的小图片(比如:用户头像)上传的时候,使用内存文件系统tmpfs来存储,可以减少一些I/O的开销.但是如果用户是上传非常大的文件(比如:视频)的话,就不适合使用tmpfs.

    # vi /etc/fstab
    
    tmpfs                   /tmp                    tmpfs   defaults,nosuid,noatime 0 0
    

PHP配置调优

  • php.ini配置调优
    # vi php-app.ini
    
    [PHP]
    engine = On
    expose_php = Off
    
    max_execution_time = 5
    memory_limit = 256M
    error_reporting = E_ALL & ~E_DEPRECATED
    display_errors = Off
    display_startup_errors = Off
    html_errors = Off
    default_socket_timeout = 5
    
    file_uploads = On
    upload_tmp_dir = /tmp/php
    upload_max_filesize = 50M
    post_max_size = 50M
    max_file_uploads = 20
    
    date.timezone = ‘Asia/Shanghai‘
    

    注意这里设置max_execution_time只有5秒.对于一个快的web应用来说,我们真的不想要web应用里有任何长时间运行的web请求,一个web请求持续超过5秒通常意味着有些东西出了问题.我们的目标是页面响应在300-500ms(微秒)之内.

  • PHP-FPM配置调优
    # vi php-fpm.conf
    
    [my_app]
    ;FastCGI/PHP-FPM使用UNIX Sockets
    listen = /data/my_app/tmp/php.sock
    listen.backlog = 300
    user = www
    group = www
    pm = dynamic
    ;估算pm.max_children = (MAX_MEMORY - 500MB) / 20MB
    pm.max_children = 100
    ;推荐为最大的pm.max_children的%10
    pm.start_servers = 10
    pm.min_spare_servers = 5
    pm.max_spare_servers = 15
    pm.max_requests = 1000
    pm.status_path = /php_status
    
    request_terminate_timeout = 0
    request_slowlog_timeout = 0
    slowlog = /data/my_app/logs/slow.log
    

Posted in Performance analysis & tuning.

高负载Linux调优

Posted on 2013 年 5 月 19 日 by Jose

高负载Linux调优

  • 调整内核参数

    调整内核参数:

    # vi /etc/sysctl.conf
    
    # tells the Kernel it‘s ok if services bind to non-existant IP ADDR
    # net.ipv4.ip_nonlocal_bind = 1
    
    # defines the size of the kernel queue for accepting new connections, Defaults to 128
    net.core.somaxconn = 40000
    
    # Maximum number of remembered connection requests, Defaults to 1024
    net.ipv4.tcp_max_syn_backlog = 40000
    
    # Tuning TCP Ports, increase the range to allow for more connections.
    # The number of avaliable ports limits the number of simultanious open connections.
    net.ipv4.ip_local_port_range = 1024 65535
    
    # Tell kernel to reuse faster the connection in TIME_WAIT
    net.ipv4.tcp_tw_reuse = 1
    net.ipv4.tcp_tw_recycle = 1
    
    # Resetting the File Descriptors, the maximum number of open files allowed per process
    fs.file-max = 1000000
    

    使设置生效:

    # sysctl -p
    
  • 调整最大打开文件描述符限制
    调整文件描述符限制:
    # ulimit -n 1000000
    # vi /etc/security/limits.conf
    
    # Setting Shell Limits for File Descriptors
    *  soft nofile 1000000
    *  hard nofile 1000000
    

    使用PAM模块限制资源:

    # vi /etc/pam.d/login
    
    session  required  pam_limits.so
    

Posted in Performance analysis & tuning.

时间: 2024-10-10 04:53:54

Memcached缓存瓶颈分析的相关文章

Linux转发性能评估与优化(转发瓶颈分析与解决方案)

线速问题 很多人对这个线速概念存在误解.认为所谓线速能力就是路由器/交换机就像一根网线一样.而这,是不可能的.应该考虑到的一个概念就是延迟.数据包进入路由器或者交换机,存在一个核心延迟操作,这就是选路,对于路由器而言,就是路由查找,对于交换机而言,就是查询MAC/端口映射表,这个延迟是无法避开的,这个操作需要大量的计算机资源,所以不管是路由器还是交换机,数据包在内部是不可能像在线缆上那样近光速传输的.类比一下你经过十字街头的时候,是不是要左顾右盼呢? 那么,设备的线速能力怎么衡量呢?如果一个数据

Linux转发性能评估与优化-转发瓶颈分析与解决方式(补遗)

补遗 关于网络接收的软中断负载均衡,已经有了成熟的方案,可是该方案并不特别适合数据包转发,它对server的小包处理非常好.这就是RPS.我针对RPS做了一个patch.提升了其转发效率. 下面是我转载的我自己的原文. 线速问题 非常多人对这个线速概念存在误解.觉得所谓线速能力就是路由器/交换机就像一根网线一样.而这.是不可能的.应该考虑到的一个概念就是延迟. 数据包进入路由器或者交换机.存在一个核心延迟操作,这就是选路,对于路由器而言.就是路由查找,对于交换机而言,就是查询MAC/port映射

Linux转发性能评估与优化-转发瓶颈分析与解决方案(补遗)

补遗 关于网络接收的软中断负载均衡,已经有了成熟的方案,但是该方案并不特别适合数据包转发,它对服务器的小包处理非常好,这就是RPS.我针对RPS做了一个patch,提升了其转发效率. 下面是我转载的我自己的原文. 线速问题 很多人对这个线速概念存在误解.认为所谓线速能力就是路由器/交换机就像一根网线一样.而这,是不可能的.应该考虑到的一个概念就是延迟.数据包进入路由器或者交换机,存在一个核心延迟操作,这就是选路,对于路由器而言,就是路由查找,对于交换机而言,就是查询MAC/端口映射表,这个延迟是

系统瓶颈分析

系统瓶颈分析 一.系统瓶颈分析示例 例1: 交易的响应时间如果很长,远远超过系统性能需求,表示消耗CPU的数据库操作,例如排序.执行aggregate functions(例如sam.min.max.count)等较多,可以考虑是否有索引以及索引建立的是否合理:尽量使用简单的表联接:水平分割大表格等方法来降低该值. 例2: 分段排除错误.测试工具可以模拟不同的虚拟用户来单独访问web服务器.应用服务器和数据库服务器,这样就可以在web端测出的响应时间减去以上各个分段测出的时间就可以知道瓶颈在哪并

linux系统瓶颈分析(精)

linux系统瓶颈分析(精) (2013-09-17 14:22:00)   分类: linux服务器瓶颈分析 1.0 性能监控介绍 性能优化就是找到系统处理中的瓶颈以及去除这些的过程,多数管理员相信看一些相关的"cook book"就 可以实现性能优化,通常通过对内核的一些配置是可以简单的解决问题,但并不适合每个环境,性能优化其实 是对OS 各子系统达到一种平衡的定义,这些子系统包括了: CPU Memory IO Network 这些子系统之间关系是相互彼此依赖的,任何一个高负载都

Memcached源码分析之从SET命令开始说起

作者:Calix 如果直接把memcached的源码从main函数开始说,恐怕会有点头大,所以这里以一句经典的“SET”命令简单地开个头,算是回忆一下memcached的作用,后面的结构篇中关于命令解析部分主要也是围绕着SET命令展开分析,相信把一句SET命令背后做的事情都搞清楚,那么memcached大部分源码都了解得七七八八了. 那么,回忆一下,set命令做了个什么事情? 无非就是把一个value set到某个key上面,保存在内存当中. 再细化一下: 1)memcached是一个缓存服务器

memcached缓存基本概念

memcached缓存基本概念 Memcached是一套分布式内存对象缓存系统. 用于在动态应用系统中缓存数据库的数据,减少数据库的访问压力,达到提升网站系统性能的目的:memcached在企业应用场景中一般是用来作为数据库的cache服务使用,memcached是通过预分配指定的内存空间来存取数据的,因此它比数据库直接操作磁盘要快很多,可以提供比直接读取数据库更好的性能. 另外,memcached也经常作为集群架构节点应用服务器之间的session数据共享的存储. 1.Memcached服务在

memcached 缓存数据库应用实践

1.1 数据库对比 缓存: 将数据存储到内存中,只有当磁盘胜任不了的时候,才会启用缓存   缺点:断电数据丢失(双电),用缓存存储数据的目的只是为了应付大并发的业务. 数据库: mysql(关系型数据库,能够保证数据一致性,保证数据不丢失,当因为功能太多,导致性能不高) ===数据参考 缓存数据库:  memcache redis(非关系型数据库,性能极高,但不保证数据完整性) === 业务的数据提供者           memcachedb 会将内存的数据写入到磁盘中   redis 主要工

MySQL瓶颈分析与优化

简介 通过sysbench的oltp_read_write测试来模拟业务压力.以此来给指定的硬件环境配置一份比较合理的MySQL配置文件. 环境介绍 硬件配置 软件环境 优化层级与指导思想 优化层级 MySQL数据库优化可以在多个不同的层级进行,常见的有: SQL优化 参数优化 架构优化 本文重点关注:参数优化 指导思想 日志先行 -- 一个事务能否成功提交的关键是日志是否成功落盘,与数据没有太大的关系:也就是说对写的优化可以表述为各方面的资源向写操作倾斜. 瓶颈分析 -- 通过show glo