Pig环境安装
pig的安装非常简单，将pig-0.14.0.tar.gz解压到合适的目录下
tar -zxvf pig-0.14.0.tar.gz
修改环境变量：
#pig export PIG_HOME=/usr/local/cloud/pig-0.11.1/pig-0.11.1 export PATH=.:$PIG_HOME/bin:$PATH export PIG_CLASSPATH=$HADOOP_HOME/etc/hadoop #为了让pig识别你的hadoop版本，如果只想用pig的本地模式，不用配置这项。
使环境变量修改生效：
source /etc/profile
本地模式启动方式pig -x local
[[email protected] pig-0.11.1]# pig -x local 2014-11-24 07:50:19,622 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53 2014-11-24 07:50:19,622 [main] INFO org.apache.pig.Main - Logging error messages to: /usr/local/cloud/pig-0.11.1/pig-0.11.1/logs/pig_1416844219621.log 2014-11-24 07:50:19,663 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /root/.pigbootup not found 2014-11-24 07:50:19,901 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:/// 2014-11-24 07:50:19,903 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used 2014-11-24 07:50:19,907 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS 2014-11-24 07:50:19,907 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2014-11-24 07:50:20,188 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum 2014-11-24 07:50:20,190 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
修改pig默认的日志存放目录
pig解压后的目录下pig-0.14.0/conf/pig.properties中，修改为合适的目录：
pig.logfile=/usr/local/cloud/pig-0.11.1/pig-0.11.1/logs
pig的mapreduce启动方式(必须启动hadoop)：
[[email protected] pig-0.11.1]# pig 2014-11-24 07:57:16,370 [main] INFO org.apache.pig.Main - Apache Pig version <span style="font-family: Arial, Helvetica, sans-serif;">0.11.1</span><span style="font-family: Arial, Helvetica, sans-serif;">(r1459641) compiled Mar 22 2013, 02:13:53</span> 2014-11-24 07:57:16,370 [main] INFO org.apache.pig.Main - Logging error messages to: /usr/local/cloud/pig-0.11.1/pig-0.11.1/logs/pig_1416844636369.log 2014-11-24 07:57:16,410 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /root/.pigbootup not found 2014-11-24 07:57:16,681 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://192.168.1.240:9000 2014-11-24 07:57:16,684 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used 2014-11-24 07:57:16,685 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS 2014-11-24 07:57:17,634 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
测试是否安装成功：
grunt> ls hdfs:/// hdfs://192.168.1.240:9000/source <dir> hdfs://192.168.1.240:9000/testdata <dir> hdfs://192.168.1.240:9000/tmp <dir> hdfs://192.168.1.240:9000/user <dir> hdfs://192.168.1.240:9000/usr <dir>
pig常见命令：
grunt> cd /user/root/output
grunt> ls hdfs://192.168.1.240:9000/user/root/output/_policy<r 1> 194 hdfs://192.168.1.240:9000/user/root/output/clusteredPoints <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-0 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-1 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-10-final <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-2 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-3 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-4 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-5 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-6 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-7 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-8 <dir> hdfs://192.168.1.240:9000/user/root/output/clusters-9 <dir> hdfs://192.168.1.240:9000/user/root/output/data <dir> hdfs://192.168.1.240:9000/user/root/output/random-seeds <dir>
grunt> pig -help
2014-11-24 08:04:11,969 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Encountered " <IDENTIFIER> "pig "" at line 1, column 1.
Was expecting one of:
<EOF>
"cat" ...
"clear" ...
"fs" ...
"sh" ...
"cd" ...
"cp" ...
"copyFromLocal" ...
"copyToLocal" ...
"dump" ...
"describe" ...
"aliases" ...
"explain" ...
"help" ...
"history" ...
"kill" ...
"ls" ...
"mv" ...
"mkdir" ...
"pwd" ...
"quit" ...
"register" ...
"rm" ...
"rmf" ...
"set" ...
"illustrate" ...
"run" ...
"exec" ...
"scriptDone" ...
"" ...
"" ...
<EOL> ...
";" ...
时间： 2024-10-11 17:22:16