sqoop:000> show connector
+------------------------+---------+------------------------------------------------------------+----------------------+
| Name | Version | Class | Supported Directions |
+------------------------+---------+------------------------------------------------------------+----------------------+
| generic-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
| kite-connector | 1.99.7 | org.apache.sqoop.connector.kite.KiteConnector | FROM/TO |
| oracle-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO |
| ftp-connector | 1.99.7 | org.apache.sqoop.connector.ftp.FtpConnector | TO |
| hdfs-connector | 1.99.7 | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
| kafka-connector | 1.99.7 | org.apache.sqoop.connector.kafka.KafkaConnector | TO |
| sftp-connector | 1.99.7 | org.apache.sqoop.connector.sftp.SftpConnector | TO |
+------------------------+---------+------------------------------------------------------------+----------------------+
sqoop:000> create link -c generic-jdbc-connector
Creating link for connector with name generic-jdbc-connector
Please fill following values to create new link object
Name: mysql
Database connection
Driver class: com.mysql.jdbc.Driver
Connection String: jdbc:mysql://localhost:3306/lte
Username: root
Password: ****
Fetch Size: 100
Connection Properties:
There are currently 0 values in the map:
entry#
SQL Dialect
Identifier enclose:
Wed Sep 06 23:38:48 CST 2017 WARN: Establishing SSL connection without server‘s identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn‘t set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to ‘false‘. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
New link was successfully created with validation status OK and name mysql
sqoop:000>
sqoop:000>
sqoop:000> create link -c hdfs-connector
Creating link for connector with name hdfs-connector
Please fill following values to create new link object
Name: hdfs
HDFS cluster
URI: hdfs://localhost:9000
Conf directory: /Users/admaster/workspace/hadoop-2.8.1/etc/hadoop
Additional configs::
There are currently 0 values in the map:
entry#
New link was successfully created with validation status OK and name hdfs
sqoop:000> create job -f mysql -t hdfs
Creating job for links with from name mysql and to name hdfs
Please fill following values to create new job object
Name: city
Database source
Schema name: lte
Table name: city
SQL statement:
Column names:
There are currently 0 values in the list:
element#
Partition column: id
Partition column nullable:
Boundary query:
Incremental read
Check column:
Last value:
Target configuration
Override null value:
Null value:
File format:
0 : TEXT_FILE
1 : SEQUENCE_FILE
2 : PARQUET_FILE
Choose: 0
Compression codec:
0 : NONE
1 : DEFAULT
2 : DEFLATE
3 : GZIP
4 : BZIP2
5 : LZO
6 : LZ4
7 : SNAPPY
8 : CUSTOM
Choose:
Custom codec:
Output directory: /user/sqoop
Append mode:
Throttling resources
Extractors: 2
Loaders: 1
Classpath configuration
Extra mapper jars:
There are currently 0 values in the list:
element#
New job was successfully created with validation status OK and name city
sqoop:000>
sqoop:000>
sqoop:000> show job
+----+------+--------------------------------+-----------------------+---------+
| Id | Name | From Connector | To Connector | Enabled |
+----+------+--------------------------------+-----------------------+---------+
| 1 | city | mysql (generic-jdbc-connector) | hdfs (hdfs-connector) | true |
+----+------+--------------------------------+-----------------------+---------+
sqoop:000> start job -n city