最近编译Spark的时候,遇到一个问题:
执行SPARK_HADOOP_VERSION=0.20.2-cdh3u5 SPARK_HIVE=true sbt/sbt assembly后,报错:
error: Protocol https not supported or disabled in libcurl while accessing https://github.com/apache/spark.git/info/refs
fatal: HTTP request failed
是在git clone的时候出了问题,怀疑是curl不支持https,在我检查后发现,curl是支持https的。
那就把问题定位到了,git,git在编译的时候估计libcurl是不支持https的。
手工修复的成本太大,修复了很久,这是搜到了一篇stackoverflow讨论此issues的文章:http://stackoverflow.com/questions/19015282/how-do-i-enable-https-support-in-libcurl
评论里有人提到用brew doctor,这个只有Mac OS才有这个软件。不过Linux平台也有移植版linuxbrew
brew是一个类似apt-get, yum 的安装工具包。支持的特性如下:
- Can install software to a home directory and so does not require sudo
- Install software not packaged by the native distribution
- Install up-to-date versions of software when the native distribution is old
- Use the same package manager to manage both your Mac and Linux machines
- Linuxbrew 允许将包安装到用户的 HOME 目录,这样的话,就不再 需要执行 sudo;
- 对于在 Linux 包管理器中缺少的包,可以通过 Linuxbrew 来搞定;
- 如果 Linux 包管理器中所带包的版本过旧,那么利用 Linuxbrew 能够安装该包的最新版本。
一、安装linuxbrew:
安装就从git clone下来,然后配置环境变量如下:
git clone https://github.com/Homebrew/linuxbrew.git ~/.linuxbrew export PATH="$HOME/.linuxbrew/bin:$PATH" export LD_LIBRARY_PATH="$HOME/.linuxbrew/lib:$LD_LIBRARY_PATH"
二、重新安装git:
删除了Git,重新用linuxbrew来安装git:
[[email protected] shengli]# brew install git --with-brewed-curl ==> Installing dependencies for git: pkg-config, makedepend, openssl, curl ==> Installing git dependency: pkg-config ==> Downloading http://pkgconfig.freedesktop.org/releases/pkg-config-0.28.tar.gz ......
问题解决。
参考: http://www.linuxeden.com/html/news/20140312/149399.html
https://github.com/Homebrew/linuxbrew
原创文章:转载请注明出自:http://blog.csdn.net/oopsoom/article/details/39231091