1.1.1 下载sqoop1：
sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
1.1.2 解压并查看目录：
[[email protected] ~]$ tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz --解压 [[email protected] ~]$ cd sqoop-1.4.7.bin__hadoop-2.6.0 [[email protected] sqoop-1.4.7.bin__hadoop-2.6.0]$ ls -ll -查看目录 total 2020 drwxr-xr-x. 2 hadoop hadoop 4096 Dec 19 2017 bin -rw-rw-r--. 1 hadoop hadoop 55089 Dec 19 2017 build.xml -rw-rw-r--. 1 hadoop hadoop 47426 Dec 19 2017 CHANGELOG.txt -rw-rw-r--. 1 hadoop hadoop 9880 Dec 19 2017 COMPILING.txt drwxr-xr-x. 2 hadoop hadoop 150 Dec 19 2017 conf drwxr-xr-x. 5 hadoop hadoop 169 Dec 19 2017 docs drwxr-xr-x. 2 hadoop hadoop 96 Dec 19 2017 ivy -rw-rw-r--. 1 hadoop hadoop 11163 Dec 19 2017 ivy.xml drwxr-xr-x. 2 hadoop hadoop 4096 Dec 19 2017 lib -rw-rw-r--. 1 hadoop hadoop 15419 Dec 19 2017 LICENSE.txt -rw-rw-r--. 1 hadoop hadoop 505 Dec 19 2017 NOTICE.txt -rw-rw-r--. 1 hadoop hadoop 18772 Dec 19 2017 pom-old.xml -rw-rw-r--. 1 hadoop hadoop 1096 Dec 19 2017 README.txt -rw-rw-r--. 1 hadoop hadoop 1108073 Dec 19 2017 sqoop-1.4.7.jar -rw-rw-r--. 1 hadoop hadoop 6554 Dec 19 2017 sqoop-patch-review.py -rw-rw-r--. 1 hadoop hadoop 765184 Dec 19 2017 sqoop-test-1.4.7.jar drwxr-xr-x. 7 hadoop hadoop 73 Dec 19 2017 src drwxr-xr-x. 4 hadoop hadoop 114 Dec 19 2017 testdata
1.2 配置sqoop——mysql连接器：
下载mysql-connector-java-8.0.16.jar，并将其拷贝至sqoop安装目录下的lib文件夹里
1.3 配置sqoop环境变量：
[[email protected] sqoop-1.4.7.bin__hadoop-2.6.0]$ cd conf [[email protected] conf]$ ls -ll total 28 -rw-rw-r--. 1 hadoop hadoop 3895 Dec 19 2017 oraoop-site-template.xml -rw-rw-r--. 1 hadoop hadoop 1404 Dec 19 2017 sqoop-env-template.cmd -rwxr-xr-x. 1 hadoop hadoop 1345 Dec 19 2017 sqoop-env-template.sh -rw-rw-r--. 1 hadoop hadoop 6044 Dec 19 2017 sqoop-site-template.xml -rw-rw-r--. 1 hadoop hadoop 6044 Dec 19 2017 sqoop-site.xml
1.3.1 拷贝复制sqoop-env.sh样本，并添加hadoop、hbase、hive、zookeeper的安装目录（注：没有的就不添加）
[[email protected] conf]$ cp sqoop-env-template.sh sqoop-env.sh [[email protected] conf]$ gedit sqoop-env.sh 修改的内容： #Set path to where bin/hadoop is available export HADOOP_COMMON_HOME=/home/hadoop/hadoop-3.2.0 #Set path to where hadoop-*-core.jar is available export HADOOP_MAPRED_HOME=/home/hadoop/hadoop-3.2.0 #set the path to where bin/hbase is available export HBASE_HOME=/home/hadoop/hbase-2.2.1 #Set the path to where bin/hive is available export HIVE_HOME=/home/hadoop/apache-hive-3.1.2-bin #Set the path for where zookeper config dir is export ZOOCFGDIR=/home/hadoop/apache-zookeeper-3.5.5
1.3.2 配置linux环境变量
[[email protected] conf]$ gedit ~/.bash_profile [[email protected] conf]$ source ~/.bash_profile 新添加： #sqoop export SQOOP_HOME=/home/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0 export PATH=$PATH:$SQOOP_HOME/bin
1.4 验证sqoop是否安装成功
[[email protected] sqoop-1.4.7.bin__hadoop-2.6.0]$ bin/sqoop help --执行该命令，看到如下信息就表示成功 Warning: /home/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /home/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /home/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty 2019-09-29 23:38:28,571 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 usage: sqoop COMMAND [ARGS]Available commands: codegen Generate code to interact with database records create-hive-table Import a table definition into Hive eval Evaluate a SQL statement and display the results export Export an HDFS directory to a database table help List available commands import Import a table from a database to HDFS import-all-tables Import tables from a database to HDFS import-mainframe Import datasets from a mainframe server to HDFS job Work with saved jobs list-databases List available databases on a server list-tables List available tables in a database merge Merge results of incremental imports metastore Run a standalone Sqoop metastore version Display version information See ‘sqoop help COMMAND‘ for information on a specific command.[[email protected] sqoop-1.4.7.bin__hadoop-2.6.0]$
1.5 测试sqoop与mysql的连接
sqoop list-tables --username User --password ‘User_123456‘ --connect jdbc:mysql://localhost:3306/hahive /**方式1**/
sqoop list-tables --username User -P --connect jdbc:mysql://localhost:3306/hahive /**方式2**/
Enter password:
如果能执行以上操作，可以看见结果，说明sqoop能连接mysql。
原文地址：https://www.cnblogs.com/CQ-LQJ/p/11617097.html
时间： 2024-07-31 11:33:48