cd /usr/local
下载logstash 6.4.3版本
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.4.3.tar.gz
解压
tar -zxvf logstash-6.4.3.tar.gz
下载input和output插件
cd logstash-6.4.3 bin/logstash-plugin install logstash-input-jdbc bin/logstash-plugin install logstash-output-elasticsearch
新建logstash.conf
vi /usr/local/logstash-6.4.3/config/logstash.conf
内容输入案例
input { jdbc { jdbc_driver_library => "/usr/local/sql/mysql-connector-java-5.1.46.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_connection_string => "jdbc:mysql://localhost:3306/test" jdbc_user => "root" jdbc_password => "root" schedule => "* * * * *" statement => "SELECT * FROM user WHERE update_time >= :sql_last_value" use_column_value => true tracking_column_type => "timestamp" tracking_column => "update_time" last_run_metadata_path => "syncpoint_user" } } output { elasticsearch { # ES的IP地址及端口 hosts => ["172.31.73.140:9200","172.31.73.140:9201"] # 索引名称 可自定义 index => "user" # 需要关联的数据库中有有一个id字段,对应类型中的id document_id => "%{user_id}" document_type => "user" } stdout { # JSON格式输出 codec => json_lines } }
注意:这里边需要先安装下mysql-connector-java-5.1.46.jar包,存放到以上指定位置
配置pipelines.yml
vi /usr/local/logstash-6.4.3/config/pipelines.yml
在文件底部添加
- pipeline.id: user path.config: "/usr/local/logstash-6.4.3/config/logstash.conf"
启动ogstash
cd /usr/local/logstash-6.4.3 ./bin/logstash -f
原文地址:https://www.cnblogs.com/wbl001/p/11625814.html
时间: 2024-10-10 10:54:57