这里我演示的是在windows下的操作
首先下载logstash-5.6.1,直接去官网自行下载
1.需要创建如下jdbc.conf和myes.sql两个文件
input { stdin { } jdbc { jdbc_driver_library => "D:\jdbcconfig\sqljdbc4-4.0.jar" jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc_connection_string => "jdbc:sqlserver://127.0.0.1:1433;DatabaseName=abtest" jdbc_user => "sa" jdbc_password => "123456" # schedule => 分 时 天 月 年 # schedule => * 22 * * * //will execute at 22:00 every day schedule => "* * * * *" jdbc_paging_enabled => true jdbc_page_size => 1000 clean_run => false use_column_value => true #设置查询条件的字段 tracking_column => lastedittime record_last_run => true last_run_metadata_path => "D:\jdbcconfig\data\station_parameter.txt" #设置列名小写 lowercase_column_names => true statement_filepath => "/jdbcconfig/myes.sql" #索引的类型 type => "test" } } filter { json { source => "message" remove_field => ["message"] } } output { elasticsearch { hosts => ["172.16.57.233:9200"] action => "index" #索引 index => "test" document_id => "%{productid}" } stdout { #codec => json_lines #设置输出的格式 codec => line { format => "productid: %{[productid]} isdeleted: %{[isdeleted]} lastedittime: %{[lastedittime]} " } } }一定要注意配置的路径是否正确,否者执行会报错
myes.sql是里面你需要执行查询的SQL语句
SELECT *
FROM [dbo].test
首先要启动elasticsearch,
然后在logstash的bin目录下执行如下命令:logstash -f jdbcconfig/jdbc.conf
数据就开始同步了。。。。
时间: 2024-09-30 16:28:37