0、Bug描述
ats-x.x.x 使用 Read While Writer + Open Read Retry Timeout 实现合并回源 和 icp回源会产生冲突，导致合并回源效果不好
在ats-6.2中有一个 collapsed_forwarding 插件，将这个插件编译好部署在 ats-x.x.x 中，之前还担心会不兼容，效果还好；
1、编译方式
collapsed_forwarding 文件夹下只有：
1 [[email protected]10.110.26.73 collapsed_forwarding]# pwd 2 /path/trafficserver-6.2.1/plugins/experimental/collapsed_forwarding 3 [[email protected]10.110.26.73 collapsed_forwarding]# ll 4 total 52 5 -rw-rw-r-- 1 1000 1000 10941 Dec 17 02:14 collapsed_forwarding.cc 6 -rw-rw-r-- 1 1000 1000 996 Dec 17 02:14 Makefile.am 7 -rw-rw-r-- 1 1000 1000 31281 Dec 17 02:14 Makefile.in 8 -rw-rw-r-- 1 1000 1000 3023 Dec 17 02:14 README 9 [[email protected]10.110.26.73 collapsed_forwarding]#
直接make编译会报错，于是我采用ats自带的编译工具 tsxs 进行编译：
/path/ats/bin/tsxs -I ../../../proxy/api -L ../../../lib -ldl -o collapsed_forwarding.so collapsed_forwarding.cc- 将编译好的
collapsed_forwarding.so拷贝到/path/ats/libexec/trafficserver/目录下
2、需要改的配置
collapsed_forwarding是基于open_write_fail_action 和 Read While Writer的，所以需要在在主配置文件中开启相应的功能；- 因为这个插件是
remap mode所以需要在remap.config 中配置使用； - 关于background fill，详见
- 这个插件需要2个参数，我测试的时候使用了：
delay=50 + retries=10， 后续可根据测试效果进一步修改； - 没有在 plugin.config 中启用；
remap.config
1 regex_map ... 2 ... 3 @plugin=/path/ats/libexec/trafficserver/collapsed_forwarding.so @pparam=--delay=50 @pparam=--retries=10
record.config
1 # active the collapsed_forwarding plugin 2 CONFIG proxy.config.http.cache.open_write_fail_action INT 1 3 CONFIG proxy.config.cache.enable_read_while_writer INT 1 4 CONFIG proxy.config.http.redirection_enabled INT 1 5 CONFIG proxy.config.http.number_of_redirections INT 10 6 CONFIG proxy.config.http.redirect_use_orig_cache_key INT 1 7 CONFIG proxy.config.http.background_fill_active_timeout INT 0 8 CONFIG proxy.config.http.background_fill_completed_threshold FLOAT 0.000000
3、原理解释
1 //////////////////////////////////////////////////////////////////////////////// 2 // collapsed_forwarding:: 3 // 4 // ATS plugin to allow collapsed forwarding of concurrent requests for the same 5 // object. This plugin is based on open_write_fail_action feature, which detects 6 // cache open write failure on a cache miss and returns a 502 error along with a 7 // special @-header indicating the reason for 502 error. The plugin acts on the 8 // error by using an internal redirect follow back to itself, essentially blocking 9 // the request until a response arrives, at which point, relies on read-while-writer 10 // feature to start downloading the object to all waiting clients. The following 11 // config parameters are assumed to be set for this plugin to work: 12 //////////////////////////////////////////////////////////////////////////////////// 13 // proxy.config.http.cache.open_write_fail_action 1 ///////////////////////// 14 // proxy.config.cache.enable_read_while_writer 1 ///////////////////////// 15 // proxy.config.http.redirection_enabled 1 ///////////////////////// 16 // proxy.config.http.number_of_redirections 10 ///////////////////////// 17 // proxy.config.http.redirect_use_orig_cache_key 1 ///////////////////////// 18 // proxy.config.http.background_fill_active_timeout 0 ///////////////////////// 19 // proxy.config.http.background_fill_completed_threshold 0 ///////////////////////// 20 //////////////////////////////////////////////////////////////////////////////////// 21 // Additionally, given that collapsed forwarding works based on cache write 22 // lock failure detection, the plugin requires cache to be enabled and ready. 23 // On a restart, Traffic Server typically takes a few seconds to initialize 24 // the cache depending on the cache size and number of dirents. While the 25 // cache is not ready yet, collapsed forwarding can not detect the write lock 26 // contention and so can not work. The setting proxy.config.http.wait_for_cache 27 // may be enabled which allows blocking incoming connections from being 28 // accepted until cache is ready. 29 //////////////////////////////////////////////////////////////////////////////////// 30 // This plugin currently supports only per-remap mode activation. 31 ////////////////////////////////////////////////////////////////////////////////////
collapsed_forwardingopen_write_fail_actionread-while-writer
Open Write Fail Action In addition to the open read retry settings TS supports a new setting proxy.config.http.cache.open_write_fail_action that allows to further reduce multiple concurrent requests hitting the origin for the same object by either returning a stale copy, in case of hit-stale or an error in case of cache miss for all but one of the requests.
4、测试效果
1 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 2 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 3 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 4 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 5 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 6 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 7 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 8 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 9 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 10 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 11 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 12 10.110.26.73 HTTP/1.1 303 TCP_MISS_REDIRECT:DIRECT 13 10.110.26.73 HTTP/1.1 200 TCP_MISS:PARENT_HIT 14 10.110.26.73 HTTP/1.1 200 TCP_HIT:NONE 15 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 16 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 17 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 18 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 19 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 20 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 21 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 22 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 23 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 24 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE 25 10.110.26.73 HTTP/1.1 200 TCP_MEM_HIT:NONE
也测试了 icp 回源，也是OK的！
5、参考文献
时间： 2024-10-10 06:41:55