分布式文件系统FastDFS介绍和配置过程

http://ylw6006.blog.51cto.com/470441/948729/

由于网站使用nfs共享方式保存用户上传的图片，附件等资料，然后通过apache下载的方式供用户访问，在网站架构初期，使用这种简单的方式实现了静态资源的读写分离，但随着网站数据量的增加，图片服务器渐渐成为整个网站的短板，缘次催生了使用fastfds的想法，故而先进行一番简单的测试！在开始之前还是先来看看fastfds的介绍信息：


FastDFS is an open source high performance distributed file system (DFS). It‘s major functions include: file storing, file syncing and file accessing, and design for high capacity and load balancing.

FastDFS is an open source high performance distributed file system. It‘s major functions include: file storing, file syncing and file accessing (file uploading and file downloading), and it can resolve the high capacity and load balancing problem. FastDFS should meet the requirement of the website whose service based on files such as photo sharing site and vidio sharing site.

FastDFS has two roles: tracker and storage. The tracker takes charge of scheduling and load balancing for file access. The storage store files and it‘s function is file management including: file storing, file syncing, providing file access interface. It also manage the meta data which are attributes representing as key value pair of the file. For example: width=1024, the key is "width" and the value is "1024".

The tracker and storage contain one or more servers. The servers in the tracker or storage cluster can be added to or removed from the cluster by any time without affecting the online services. The servers in the tracker cluster are peer to peer.

The storarge servers organizing by the file volume/group to obtain high capacity. The storage system contains one or more volumes whose files are independent among these volumes. The capacity of the whole storage system equals to the sum of all volumes‘ capacity. A file volume contains one or more storage servers whose files are same among these servers. The servers in a file volume backup each other, and all these servers are load balancing. When adding a storage server to a volume, files already existing in this volume are replicated to this new server automatically, and when this replication done, system will switch this server online to providing storage services. When the whole storage capacity is insufficiency, you can add one or more volumes to expand the storage capacity. To do this, you need to add one or more storage servers.

The identification of a file is composed of two parts: the volume name and the file name.

大意为：
fastdfs是一个开源的，高性能的的分布式文件系统，他主要的功能包括：文件存储，同步和访问，设计基于高可用和负载均衡，fastfd非常适用于基于文件服务的站点，例如图片分享和视频分享网站

fastfds有两个角色：跟踪服务和存储服务，跟踪服务控制，调度文件以负载均衡的方式访问；存储服务包括：文件存储，文件同步，提供文件访问接口，同时以key value的方式管理文件的元数据

跟踪和存储服务可以由1台或者多台服务器组成，同时可以动态的添加，删除跟踪和存储服务而不会对在线的服务产生影响，在集群中，tracker服务是对等的

存储系统由一个或多个卷组成，卷与卷之间的文件是相互独立的，所有卷的文件容量累加就是整个存储系统中的文件容量。一个卷可以由一台或多台存储服务器组成，一个卷下的存储服务器中的文件都是相同的，卷中的多台存储服务器起到了冗余备份和负载均衡的作用。在卷中增加服务器时，同步已有的文件由系统自动完成，同步完成后，系统自动将新增服务器切换到线上提供服务。当存储空间不足或即将耗尽时，可以动态添加卷。只需要增加一台或多台服务器，并将它们配置为一个新的卷，这样就扩大了存储系统的容量。

下面几张图可以清楚的说明fastfds的架构和文件上传和下载流程等：

下面将介绍下fastdfs在rhel上的部署过程
tracker服务器：192.168.123.110/24
storage服务器：192.168.123.20/24

一：编译安装

  1. [[email protected] ~]# wget http://cloud.github.com/downloads/libevent/libevent/libevent-2.0.14
  2. -stable.tar.gz
  3. [[email protected] ~]# tar -zxvpf libevent-2.0.14-stable.tar.gz
  4. [[email protected] ~]# cd libevent-2.0.14-stable
  5. [[email protected] libevent-2.0.14-stable]# ./configure --prefix=/usr/local/libevent-2.0.14 &&
  6. make && make install
  7. [[email protected] ~]# wget http://fastdfs.googlecode.com/files/FastDFS_v3.02.tar.gz
  8. [[email protected] ~]# tar -zxvf FastDFS_v3.02.tar.gz
  9. [[email protected] ~]# cd FastDFS
  10. [[email protected] FastDFS]# grep -A 4 ‘/usr/local/FastDFS‘ make.sh
  11. TARGET_PREFIX=/usr/local/FastDFS
  12. TARGET_CONF_PATH=/etc/fdfs
  13. WITH_HTTPD=1
  14. WITH_LINUX_SERVICE=1
  15. [[email protected] FastDFS]# ./make.sh C_INCLUDE_PATH=/usr/local/libevent-2.0.14/include
  16. LIBRARY_PATH=/usr/local/libevent-2.0.14/lib
  17. [[email protected] FastDFS]# ./make.sh install
  18. [[email protected] FastDFS]# ls /etc/fdfs/
  19. client.conf  http.conf  mime.types  storage.conf  tracker.conf

二：tracker配置文件

  1. [[email protected] ~]# grep -v ‘^#‘ /etc/fdfs/tracker.conf |grep -v ‘^$‘
  2. disabled=false
  3. bind_addr=192.168.123.110
  4. port=22122
  5. connect_timeout=30
  6. network_timeout=60
  7. base_path=/home/data/fastdfs
  8. max_connections=256
  9. work_threads=4
  10. store_lookup=2
  11. store_group=group2
  12. store_server=0
  13. store_path=0
  14. download_server=0
  15. reserved_storage_space = 4GB
  16. log_level=info
  17. run_by_group=
  18. run_by_user=
  19. allow_hosts=*
  20. sync_log_buff_interval = 10
  21. check_active_interval = 120
  22. thread_stack_size = 64KB
  23. storage_ip_changed_auto_adjust = true
  24. storage_sync_file_max_delay = 86400
  25. storage_sync_file_max_time = 300
  26. use_trunk_file = false
  27. slot_min_size = 256
  28. slot_max_size = 16MB
  29. trunk_file_size = 64MB
  30. http.disabled=false
  31. http.server_port=8080
  32. http.check_alive_interval=30
  33. http.check_alive_type=tcp
  34. http.check_alive_uri=/status.html
  35. http.need_find_content_type=true
  36. [[email protected] ~]# grep -v ‘^#‘ /etc/fdfs/http.conf  |grep -v ‘^$‘
  37. http.default_content_type = application/octet-stream
  38. http.mime_types_filename=/etc/fdfs/mime.types
  39. http.anti_steal.check_token=false
  40. http.anti_steal.token_ttl=900
  41. http.anti_steal.secret_key=FastDFS1234567890
  42. http.anti_steal.token_check_fail=/home/data/fastdfs/conf/anti-steal.jpg

三：启动tracker服务，需要注意tracker.conf文件最后一行为#include httpd.conf

  1. [[email protected] ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
  2. /usr/local/FastDFS/bin/fdfs_trackerd: error while loading shared libraries: libevent-
  3. 2.0.so.5: cannot open shared object file: No such file or directory
  4. [[email protected] ~]# echo ‘/usr/local/libevent-2.0.14/include/‘ >> /etc/ld.so.conf
  5. [[email protected] ~]# echo ‘/usr/local/libevent-2.0.14/lib/‘ >> /etc/ld.so.conf
  6. [[email protected] ~]# ldconfig
  7. [[email protected] ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
  8. [2012-07-04 17:52:25] ERROR - file: tracker_func.c, line: 160, "/home/data/fastdfs"
  9. can‘t be accessed, error info: No such file or directory
  10. [[email protected] ~]# mkdir -p /home/data/fastdfs
  11. [[email protected] ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
  12. [[email protected] ~]# echo $?
  13. 22
  14. [[email protected] ~]# cat  /home/data/fastdfs/logs/trackerd.log
  15. [2012-07-04 17:52:50] ERROR - file: ../common/fdfs_http_shared.c, line: 128, param
  16. "http.mime_types_filename" not exist or is empty
  17. [[email protected] ~]# tail -1 /etc/fdfs/tracker.conf
  18. #include http.conf
  19. [[email protected] ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
  20. [[email protected] ~]# echo $?
  21. 0
  22. [[email protected] ~]# ps -ef |grep track
  23. root      3535     1  0 15:47 ?        00:00:00 /usr/local/FastDFS/bin/fdfs_trackerd
  24. /etc/fdfs/tracker.conf
  25. [[email protected] ~]# netstat -ntpl |grep fdfs
  26. tcp        0      0 192.168.123.110:22122       0.0.0.0:*                   LISTEN
  27. 3535/fdfs_trackerd
  28. tcp        0      0 192.168.123.110:8080        0.0.0.0:*                   LISTEN
  29. 3535/fdfs_trackerd

四：storage配置文件

  1. [[email protected] ~]# grep -v ‘^#‘ /etc/fdfs/storage.conf |grep -v ‘^$‘
  2. disabled=false
  3. group_name=group1
  4. bind_addr=192.168.123.20
  5. client_bind=true
  6. port=23000
  7. connect_timeout=30
  8. network_timeout=60
  9. heart_beat_interval=30
  10. stat_report_interval=60
  11. base_path=/home/data/fastdfs
  12. max_connections=256
  13. buff_size = 256KB
  14. work_threads=4
  15. disk_rw_separated = true
  16. disk_rw_direct = false
  17. disk_reader_threads = 1
  18. disk_writer_threads = 1
  19. sync_wait_msec=50
  20. sync_interval=0
  21. sync_start_time=00:00
  22. sync_end_time=23:59
  23. write_mark_file_freq=500
  24. store_path_count=1
  25. store_path0=/home/data/fastdfs
  26. subdir_count_per_path=256
  27. tracker_server=192.168.123.110:22122
  28. log_level=info
  29. run_by_group=
  30. run_by_user=
  31. allow_hosts=*
  32. file_distribute_path_mode=0
  33. file_distribute_rotate_count=100
  34. fsync_after_written_bytes=0
  35. sync_log_buff_interval=10
  36. sync_binlog_buff_interval=10
  37. sync_stat_file_interval=300
  38. thread_stack_size=512KB
  39. upload_priority=10
  40. if_alias_prefix=
  41. check_file_duplicate=0
  42. key_namespace=FastDFS
  43. keep_alive=0
  44. http.disabled=false
  45. httphttp.domain_name=
  46. http.server_port=8888
  47. http.trunk_size=256KB
  48. http.need_find_content_type=true
  49. [[email protected] ~]# grep -v ‘^#‘ /etc/fdfs/client.conf  |grep -v ‘^$‘
  50. connect_timeout=30
  51. network_timeout=60
  52. base_path=/home/data/fastdfs
  53. tracker_server=192.168.123.110:22122
  54. log_level=info
  55. http.tracker_server_port=8080
  56. [[email protected] ~]#  grep -v ‘^#‘ /etc/fdfs/http.conf |grep -v ‘^$‘
  57. http.default_content_type = application/octet-stream
  58. http.mime_types_filename=mime.types
  59. http.anti_steal.check_token=false
  60. http.anti_steal.token_ttl=900
  61. http.anti_steal.secret_key=FastDFS1234567890
  62. http.anti_steal.token_check_fail=/home/data/fastdfs/conf/anti-steal.jpg

五：启动storage，需要注意storage.conf文件最后一行为#include httpd.conf

  1. [[email protected] ~]# mkdir -p /home/data/fastdfs
  2. [[email protected] ~]# echo ‘/usr/local/libevent-2.0.14/include/‘ >> /etc/ld.so.conf
  3. [[email protected] ~]# echo ‘/usr/local/libevent-2.0.14/lib/‘ >> /etc/ld.so.conf
  4. [[email protected] ~]# ldconfig
  5. [[email protected] ~]# tail -2 /etc/fdfs/storage.conf
  6. #use "#include" directive to include HTTP other settings
  7. #include http.conf
  8. [[email protected] ~]# /usr/local/FastFDS/bin/fdfs_storaged /etc/fdfs/storage.conf
  9. mkdir data path: 00 ...
  10. mkdir data path: 01 ...
  11. mkdir data path: 02 ...
  12. ———输出省略————
  13. data path: /home/data/fastdfs/data, mkdir sub dir done.
  14. [[email protected] ~]# ps -ef |grep fdfs
  15. root     14451     1  0 16:15 ?        00:00:00 /usr/local/FastFDS/bin/fdfs_storaged
  16. /etc/fdfs/storage.conf
  17. root     14468  8238  0 16:16 pts/1    00:00:00 grep fdfs
  18. [[email protected] ~]# netstat -ntpl |grep fdfs
  19. tcp        0      0 192.168.123.20:8888         0.0.0.0:*                   LISTEN
  20. 14451/fdfs_storaged
  21. tcp        0      0 192.168.123.20:23000        0.0.0.0:*                   LISTEN
  22. 14451/fdfs_storaged

五：测试文件上传


[[email protected] ~]# /usr/local/FastFDS/bin/fdfs_test /etc/fdfs/client.conf upload /etc/passwd
This is FastDFS client test program v3.02
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/ 
for more detail.

[2012-07-30 16:25:59] INFO - base_path=/home/data/fastdfs, connect_timeout=30,

network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0tracker_query_storage_store_list_without_group: 
server 1. group_name=group1, ip_addr=192.168.123.20, port=23000 group_name=group1, ip_addr=192.168.123.20, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783
source ip address: 192.168.123.20
file timestamp=2012-07-30 16:25:59
file size=1832
file crc32=295790097
file url: http://192.168.123.110:8080/group1/M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783_big
source ip address: 192.168.123.20
file timestamp=2012-07-30 16:25:59
file size=1832
file crc32=295790097
file url: http://192.168.123.110:8080/group1/M00/00/00/wKh7FFAWRRfcMOGtAAAHKBGhZhE8065783_big

[[email protected] ~]# /usr/local/FastFDS/bin/fdfs_test /etc/fdfs/client.conf upload 2.jpg 
This is FastDFS client test program v3.02
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/ 
for more detail.

[2012-07-30 16:29:22] INFO - base_path=/home/data/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0tracker_query_storage_store_list_without_group: 
server 1. group_name=group1, ip_addr=192.168.123.20, port=23000 group_name=group1, ip_addr=192.168.123.20, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472.jpg
source ip address: 192.168.123.20
file timestamp=2012-07-30 16:29:22
file size=71850
file crc32=3833797383
file url: http://192.168.123.110:8080/group1/M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472.jpg
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472_big.jpg
source ip address: 192.168.123.20
file timestamp=2012-07-30 16:29:22
file size=71850
file crc32=3833797383
file url: http://192.168.123.110:8080/group1/M00/00/00/wKh7FFAWReLzPs4sAAEYquSDGwc472_big.jpg

时间： 2024-10-04 18:29:55

分布式文件系统FastDFS介绍和配置过程的相关文章

FastDFS介绍和配置过程

由于网站使用nfs共享方式保存用户上传的图片,附件等资料,然后通过apache下载的方式供用户访问,在网站架构初期,使用这种简单的方式实现了静态资源的读写分离,但随着网站数据量的增加,图片服务器渐渐成为整个网站的短板,缘次催生了使用fastfds的想法,故而先进行一番简单的测试!在开始之前还是先来看看fastfds的介绍信息: FastDFS is an open source high performance distributed file system (DFS). It's major

Linux下分布式文件系统FastDFS安装与配置

1.fastdfs 分布式文件系统,对文件进行管理,主要功能包括文件存储.文件同步.文件访问(文件的上传和下载)等, 解决了大容量存储和负载均衡的问题. 2.fastdfs的两个核心分别是跟踪器(tracker)和存储节点(storage),其中tracker主要的功能是完成调度工作和实现负载均衡, storage主要的功能是存储数据. 3.fastdfs文件分布系统中实现文件上传下载的原理: 上传: 1)Client(客户端)要通过Tracker server(跟踪服务器)将文件上传到Stor

记录：CentOS 7 安装配置分布式文件系统 FastDFS 5.1.1

CentOS 7 安装配置分布式文件系统 FastDFS 5.1.1 软件下载:http://download.csdn.net/download/qingchunwuxian1993/9897458 yum-y install net-tools.x86_64 前言 项目中用到文件服务器,有朋友推荐用FastDFS,所以就了解学习了一番,感觉确实颇为强大,在此再次感谢淘宝资深架构师余庆大神开源了如此优秀的轻量级分布式文件系统,本篇文章就记录一下FastDFS的最新版本5.1.1在CentOS7

轻量级分布式文件系统FastDFS使用安装说明手册（新手入门级）

轻量级分布式文件系统FastDFS使用安装说明手册(新手入门级) 实验室所在的课题组以研究云计算为主,但所有的研究都是在基于理论的凭空想像,缺少分布式环境的平台的实践,云计算神马的都是浮云了.因此,我想借助Hadoop把实验室的服务器搭建出一个基础的分布式实验环境,方便于以后进行实验测试神马的.无意中,看到了一款开源的轻量级分布式文件系统FastDFS,它是用纯C语言实现,详细信息可参见博客:分布式文件系统FastDFS架构剖析.通过收集资料,自己动手做了一些安装和测试使用,现在对前人的一些经验

高可用高性能分布式文件系统FastDFS进阶keepalived+nginx对多tracker进行高

在上一篇 分布式文件系统FastDFS如何做到高可用 中已经介绍了FastDFS的原理和怎么搭建一个简单的高可用的分布式文件系统及怎么访问. 高可用是实现了,但由于我们只设置了一个group,如果现在有5台服务器那将会出现5台只有一个group,每台服务器内的文件内容都相同(互备份)如下图,会造成资源浪费. 因此下面就5台服务器进行优化改造,进一步添加keepalived+nginx多tracker 架构,做到真正的高可用和高性能. FastDFS集群服务器分布 其中keepalived+ngi

【架构设计】分布式文件系统 FastDFS的原理和安装使用

本文地址 分享提纲: 1.概述 2. 原理 3. 安装 4. 使用 5. 参考文档 1. 概述 1.1)[常见文件系统] Google了一下,流行的开源分布式文件系统有很多,介绍如下: -- mogileFS:Key-Value型元文件系统,不支持FUSE,应用程序访问它时需要API,主要用在web领域处理海量小图片,效率相比mooseFS高很多. -- fastDFS:国人 余庆老师(GitHub)在mogileFS的基础上进行改进的key-value型文件系统,同样不支持FUSE,提供比mo

分布式文件系统FastDFS动态扩容

当用户量越来越大,则集群中某个group总会到达其极限,这时就得扩展集群的容量了. FastDFS的扩容分为对group纵向扩容和横向扩容 纵向扩容 指在同一个group组中增加服务器,实现数据冗余,数据备份.同一个group中最大容量取决于最小的storage的存储容量.因此如果还想继续使用此group,则需要对此group对应的所有服务器挂载同样容量的磁盘,指定store_path1--,但这样做的话成本相当的高,不推荐. 横向扩容是通过集群实现,指新增一个group,增加整个FastDFS

分布式文件系统fastDFS

分布式文件系统fastDFS 1.什么是FastDFS FastDFS是一个开源的轻量级分布式文件系统.它解决了大数据量存储和负载均衡等问题.特别适合以中小文件(建议范围:4KB < file_size <500MB)为载体的在线服务,如相册网站.视频网站等等.在UC基于FastDFS开发向用户提供了:网盘,社区,广告和应用下载等业务的存储服务. 2.FastDFS架构和原理 FastDFS服务端有三个角色:跟踪服务器(tracker server).存储服务器(storage server)

Moosefs分布式文件系统集群讲解配置

1 管理服务器(master-server):负责各个数据存储服务器的管理,文件读写调度,文件空间回收以及恢复.多节点拷贝 2 元数据日志服务器(changelog-server): 负责备份master服务器的变化,(一般情况下可以和管理服务器放在一起)文件类型为changelog_ml.*.mfs,以便于在master server出问题的时候接替其进行工作 3数据存储服务器(chunk-server):负责连接管理服务器,听从管理服务器调度,提供存储空间,并为客户提供数据传输. 4客户端(