搭建caffe环境及训练模型

Mac OS搭建caffe环境

一、相关软件安装

1、        安装CUDA8

(1)   安装xcode及命令行

(2)   安装CUDA的dmg包

(3)   配置环境变量并source ~/.bash_profile

export CUDA_ROOT=/Developer/NVIDIA/CUDA-8.0

export PATH=$CUDA_ROOT/bin${PATH:+:${PATH}}

export DYLD_LIBRARY_PATH=$CUDA_ROOT/lib${DYLD_LIBRARY_PATH:+:${DYLD_LIBRARY_PATH}}

export LD_LIBRARY_PATH=$CUDA_ROOT/lib:$LD_LIBRARY_PATH

(4)   执行nvcc –V验证安装

2、        安装必要的包

brew install –vd snappy leveldb gflags glog szip lmdb

brew tap homebrew/science

brew install hdf5 opencv

3、        brew edit opencv并修改两行

-DPYTHON_LIBRARY=#{py_prefix}/lib/libpython2.7.dylib

-DPYTHON_INCLUDE_DIR=#{py_prefix}/include/python2.7

4、        安装protobuf和boost-python

brew install --build-from-source --with-python –vd protobuf

brew install --build-from-source -vd boost boost-python

5、下载NVIDIA Mac drivers

二、caffe安装

1、下载caffe

git clone https://github.com/BVLC/caffe.git

2、打开Makefile.config, 配置BLAS

BLAS_INCLUDE := /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/Headers

BLAS_LIB := /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A

去掉CPU_ONLY的注释

去掉USE_LEVELDB := 0的注释

修改PYTHON_INCLUDE的目录为python2.7的目录

三、编译caffe

make all

make test

make runtest

make pycaffe

make pytest

make distribute

会有一些warning, 需要安装什么包就安装什么包

1)“dot” not found in path问题需要安装graphviz,不用pip用brew安装

2) make pytest出现了如下问题:

注释掉/python/caffe/proto/caffe_pb2.py 所有 “syntax = proto2 “

3)需要进入python文件夹执行python才能import caffe

4)安装scikit-image库

5)pip不好使,就pip install –U pip

Centos 7安装Caffe

一、安装CUDA

wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda-repo-rhel7-8-0-local-ga2-8.0.61-1.x86_64-rpm

rpm -i cuda-repo-rhel7-8-0-local-ga2-8.0.61-1.x86_64-rpm

yum clean all

yum install cuda

设置CUDA环境变量:

export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:/usr/local/cuda-8.0/extras/CUPTI/lib64:$LD_LIBRARY_PATH(指定共享库,编译opencv、cudasample等)

export CUDA_HOME=/usr/local/cuda-8.0/(指定CUDA安装路径)

export PATH=/usr/local/cuda-8.0/bin:$PATH(在任意环境下调用cuda可执行文件)

更新运行库缓存:

ldconfig

检验CUDA是否安装成功:

nvcc --version

GPU支持:

安装Nvidia驱动:

根据GPU型号从相应网站下载驱动,例如使用NVIDIA Tesla M60,从NVIDIA网站选择对应的型号和操作系统,CUDA Toolkit版本,下载驱动文件,如NVIDIA-Linux-x86_64-375.66.run,运行驱动文件,根据提示安装:

sh  NVIDIA-Linux-x86_64-375.66.run

通过NVIDIA命令工具nvidia-smi检验GPU情况:

nvidia-smi

安装cuDNN:

从NVIDIA网站下载cuDNN 安装包,根据GPU及CUDA版本选择对应cuDNN版本,下载cuDNN v5.1 for CUDA8.0,解压拷贝到CUDA安装目录

cp include/* /usr/local/cuda/include

cp lib64/* /usr/local/cuda/lib64

二、yum安装依赖包

yum install protobuf-devel  leveldb-devel  snappy-devel  opencv-devel  boost-devel  hdf5-devel  gflags-devel  glog-devel  lmdb-devel  atlas-devel

n  cd  /usr/lib64/atlas

n  sudo ln -sv libsatlas.so.3.10 libcblas.so

n  sudo ln -sv libsatlas.so.3.10 libatlas.so

(如果有需要,可安装Openblas或者MKL作为atlas的备用)

二、非yum手动安装依赖包

安装 gflags:

git clone https://github.com/gflags/gflags

cd gflags

mkdir build && cd build

export CXXFLAGS="-fPIC" && cmake .. -DCMAKE_INSTALL_PREFIX=/data2/shixi_wuhao3/local -DBUILD_SHARED_LIBS=ON

make VERBOSE=1

sudo make install

安装 glog:

git clone https://github.com/google/glog

cd glog

./configure

make

automake --add-missing

sudo make install

安装 lmdb:

git clone https://github.com/LMDB/lmdb

cd lmdb/libraries/liblmdb

make

sudo make install

安装 hdf5:

wget https://support.hdfgroup.org/ftp/HDF5/current18/src/hdf5-1.8.18.tar.gz

tar -xvf hdf5-1.8.18.tar.gz

cd hdf5-1.8.18

./configure --prefix=/usr/local

make

sudo make install

安装protobuf

tar -xvf protobuf

cd protobuf

./configure --prefix=/usr/local/protobuf

make

make check

make install

安装boost

wget http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz

tar -zxvf boost_1_59_0.tar.gz

cd boost_1_59_0

./bootstrap.sh --prefix=path/to/install

./b2 install

这会把boost安装到path/to/install,若不指定,默认将安装到/usr/local目录,但需要权限。

安装opencv

leveldb要和snappy一起安装

三、下载caffe

git clone https://github.com/BVLC/caffe

四、安装Python依赖包(注意python2和python3的)

for req in $(cat caffe/python/requirements.txt); do pip install $req; done

五、编辑Makefile.config 文件,根据情况修改配置:

BLAS := atlas

BLAS_INCLUDE := /usr/include/atlas

BLAS_LIB := /usr/lib64/atlas

PYTHON_INCLUDE := /usr/local/include/python2.7 \

/usr/local/lib/python2.7/site-packages/numpy/core/include

PYTHON_LIB := /usr/lib64

USE_CUDNN := 1(使用cuDNN)

(去掉CPU_ONLY的注释——只用CPU)

(去掉USE_LEVELDB := 0的注释——使用LevelDB)

根据情况确定是否修改Makefile.cofig中的相关位置路径

## Refer to http://caffe.berkeleyvision.org/installation.html

# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).

# USE_CUDNN := 1

"CuDNN是NVIDIA专门针对Deep Learning框架设计的一套GPU计算加速库,用于实现高性能的并行计算,在有GPU并且安装CuDNN的情况下可以打开即将注释去掉。"

# CPU-only switch (uncomment to build without GPU support).

#CPU_ONLY := 1

"表示是否用GPU,如果只有CPU这里要打开"

# uncomment to disable IO dependencies and corresponding data layers

USE_OPENCV := 1

"因为要用到OpenCV库所以要打开,下面这两个选项表示是选择Caffe的数据管理第三方库,两者都不打开 Caffe默认用的是LMDB,这两者均是嵌入式数据库管理系统编程库。"

# USE_LEVELDB := 0

# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)

#   You should not set this flag if you will be reading LMDBs with any

#   possibility of simultaneous read and write

# ALLOW_LMDB_NOLOCK := 1

"当需要读取LMDB文件时可以取消注释,默认不打开。"

# Uncomment if you‘re using OpenCV 3

OPENCV_VERSION := 2.4.10 (我的2.4.5)

"用pkg-config --modversion opencv命令查看opencv版本"

# To customize your choice of compiler, uncomment and set the following.

# N.B. the default for Linux is g++ and the default for OSX is clang++

# CUSTOM_CXX := g++

"linux系统默认使用g++编译器,OSX则是clang++。"

# CUDA directory contains bin/ and lib/ directories that we need.

CUDA_DIR := /usr/local/cuda

"CUDA的安装目录"

# On Ubuntu 14.04, if cuda tools are installed via

# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:

# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.

# For CUDA < 6.0, comment the *_50 lines for compatibility.

CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \

-gencode arch=compute_20,code=sm_21 \

-gencode arch=compute_30,code=sm_30 \

-gencode arch=compute_35,code=sm_35 \

-gencode arch=compute_50,code=sm_50 \

-gencode arch=compute_50,code=compute_50

"这些参数需要根据GPU的计算能力

(http://blog.csdn.net/jiajunlee/article/details/52067962)来进行设置,6.0以下的版本不支持×_50的计算能力。"

# BLAS choice:

# atlas for ATLAS (default)

# mkl for MKL

# open for OpenBlas

BLAS := open

"如果用的是ATLAS计算库则赋值atlas,MKL计算库则用mkl赋值,OpenBlas则赋值open。"

# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.

# Leave commented to accept the defaults for your choice of BLAS

# (which should work)!

BLAS_INCLUDE := /usr/local/OpenBlas/include

BLAS_LIB := /usr/local/OpenBlas/lib

"blas库安装目录"

# Homebrew puts openblas in a directory that is not on the standard search path

# BLAS_INCLUDE := $(shell brew --prefix openblas)/include

# BLAS_LIB := $(shell brew --prefix openblas)/lib

"如果不是安装在标准路径则要指明"

# This is required only if you will compile the matlab interface.

# MATLAB directory should contain the mex binary in /bin.

# MATLAB_DIR := /usr/local

# MATLAB_DIR := /Applications/MATLAB_R2012b.app

"matlab安装库的目录"

# NOTE: this is required only if you will compile the python interface.

# We need to be able to find Python.h and numpy/arrayobject.h.

PYTHON_INCLUDE := /usr/include/python2.7 \

/usr/lib/python2.7/dist-packages/numpy/core/include

"python安装目录"

# Anaconda Python distribution is quite popular. Include path:

# Verify anaconda location, sometimes it‘s in root.

# ANACONDA_HOME := $(HOME)/anaconda

# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \

# $(ANACONDA_HOME)/include/python2.7 \

# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# Uncomment to use Python 3 (default is Python 2)

# PYTHON_LIBRARIES := boost_python3 python3.5m

# PYTHON_INCLUDE := /usr/include/python3.5m \

#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.

PYTHON_LIB := /usr/lib

"python库位置"

# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)

# PYTHON_INCLUDE += $(dir $(shell python -c ‘import numpy.core; print(numpy.core.__file__)‘))/include

# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)

WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include

LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies

# INCLUDE_DIRS += $(shell brew --prefix)/include

# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.

# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)

# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`

BUILD_DIR := build

DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171

# DEBUG := 1

# The ID of the GPU that ‘make runtest‘ will use to run unit tests.

TEST_GPUID := 0

"所用的GPU的ID编号"

# enable pretty build (comment to see full commands)

Q ?= @

六、编译及测试:

make all

make test

make runtest

测试成功,结果如下:

编译caffe的python环境及测试:

make pycaffe

make pytest

这样进入python交互行import caffe成功即可

http://caffe.berkeleyvision.org/installation.html

pip安装future库

安装最新版的cudnn(美团有链接,可能要注册)

cmake要设置CPU参数OFF, cmake -DUSE_CUDA=OFF ..

git要加参数recursive

make install时要加sudo

在site-package目录下编写.pth来使得caffe在任何目录下都可以import

时间: 2024-08-10 19:09:32

搭建caffe环境及训练模型的相关文章

Mac OSX (EI Capitan)搭建Caffe环境并配置python接口

Caffe是一个清晰而高效的深度学习框架,其作者是博士毕业于UC Berkeley的贾扬清.Caffe是纯粹的C++/CUDA架构,支持命令行.Python和MATLAB接口:可以在CPU和GPU直接无缝切换.我在MacbookPro(无NVIDIA显卡)上大费周章地配置了Caffe的环境,并花了许多时间配置其python接口. 一.下载Caffe github上的下载地址:https://github.com/BVLC/caffe进入到下载后的路径,并复制 Makefile.config.ex

手把手教你搭建caffe及手写数字识别(全程命令提示、纯小白教程)

手把手教你搭建caffe及手写数字识别 作者:七月在线课程助教团队,骁哲.小蔡.李伟.July时间:二零一六年十一月九日交流:深度学习实战交流Q群 472899334,有问题可以加此群共同交流.另探究实验背后原理,请参看此课程:11月深度学习班. 一.前言 在前面的教程中,我们搭建了tensorflow.torch,教程发布后,大家的问题少了非常多.但另一大框架caffe的问题则也不少,加之caffe也是11月深度学习班要讲的三大框架之一,因此,我们再把caffe的搭建完整走一遍,手把手且全程命

搭建lnmp环境,部署php动态网站

搭建LNMP 前言:"N"代表Nginx与apache的作用一样,都是为了搭建网站服务器,由俄罗斯人lgor sysoev开发,其特点是占有内存少,并发能力强,单台物理服务器可支持3万-5万个并发请求,中国使用nginx网站用户有:百度.京东.新浪.网易.腾讯.淘宝等. 通过下面的实验搭建LNMP环境,部署天空影城的php动态网站 本实验在虚拟机中运行,使用Redhat6.5系统部署! 一.安装及运行 1.搭建yum仓库,安装支持软件 nginx的配置及运行需要pcre,zlib等软件

搭建 LNMP 环境

搭建 LAMP 环境 一.环境介绍 二.必装依赖软件 三.编译安装 Nginx-1.6.3 四.二进制安装 MYSQL-5.5.33 五.编译安装 PHP-5.3.27 六.检测 nginx与mysql nginx与php 是否相连 七.安装 wordpress (分别基于IP,域名访问) 一.环境介绍 搭建这个环境,所有服务都安装在一台虚拟机中,在此介绍一下我的虚拟机 CentOS-6.7-x86_64    保证虚拟机可以正常上网 IP:192.168.6.36 hostname:yuci

ubuntu16.04搭建opencv3环境

为了搭建opencv3,看了各种教程,装了卸,卸了装,多次失败后,在几篇文档的借鉴下,终于是成功搭建了环境. 首先,更新ubuntu上的一些库: sudo apt-get update sudo apt-get upgrade 其次,需要安装一些搭建opencv3的库(要在opencv3搭建前安装好) 具体需要的库如下: sudo apt-get install cmakesudo apt-get install build-essential libgtk2.0-dev libavcodec-

?搭建LAMP环境及快速部署双网站并实现基于域名的虚拟主机

本节所讲内容: 实战:搭建LAMP环境及快速部署双网站并实现基于域名的虚拟主机 LAMP架构:??? Linux+Apache+Mysql+PHP Linux+Apache+Mysql/MariaDB+Perl/PHP/Python一组常用来搭建动态网站或者服务器的开源软件,共同组成了一个强大的Web应用程序平台. 一.安装需要的软件包 [[email protected] ~]# yum install httpd mysql-server mysql php php-mysql  -y ht

GNS3结合VMWORE搭建虚拟化环境的好处

GNS3结合VMWORE搭建虚拟化环境的好处 在对虚拟化进行学习和测试时,光使用VMWORE就能搭建虚拟化的测试环境.那为何还要使用GNS3模拟器呢!使用GNS3结合VMWORE的好处如下图所示: 如上图所示,假设不使用GNS3模拟的交换机时,EXSI主机之间是不能通信的,因为EXSI主机连到了不同的虚拟网络.如果要让它们之间能通信的话,使用GNS3就是一种很好的办法.总之GNS3结合VMWORE搭建虚拟化环境的好处有如下两点: GNS3能够互连不同的虚拟网络,增加实验灵活性. GNS3使得环境

使用eclipse+tomcat搭建本地环境

项目开发工具很多,这里简单介绍下使用eclipse+tomcat如何搭建本地环境. 安装开发工具如下: 1. jdk的安装参考 下载地址:http://pan.baidu.com/s/1sj9rVYX 安装参考地址:http://www.cnblogs.com/pxue/archive/2011/05/10/2042530.html 如果cmd中javac运行不了,尝试将环境变量中的path的%JAVA_HOME%路径设置为绝对路径 关键点:(设置三个环境变量) JAVA_HOME      

Netbeans搭建Android环境

原文:Netbeans搭建Android环境 Netbeans环境的搭建主要依赖于NBAndroid插件,项目地址: http://www.nbandroid.org/p/installation.html 开发机器:Windows 7 IDE:Netbeans 7.3 第一步:安装插件 启动Netbeans,在工具——插件——设置面板点击“添加”,如下图所示: 输入一个名称,URL填写:http://nbandroid.org/release72/updates/updates.xml 如果你