logstash如何从redis拉出日志并写入es？
input {
redis {
batch_count => 1
host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"
type => "syslog"
data_type => "list"
key => "logstash:syslog-log"
}
redis {
batch_count => 1
host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"
type => "log4j-json-web"
data_type => "list"
key => "logstash:xxxxxweb-web-log"
}
redis {
batch_count => 1
host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"
type => "log4j-json-tomcat"
data_type => "list"
key => "logstash:xxxxxweb-tomcat-log"
}
}
output{
if [type] == "syslog" {
elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]
index => "dev-system-logs-%{+YYYY.MM.dd}"
document_type => "system-logs" }
}
if [type] == "log4j-json-web" {
elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]
index => "dev-xxxxxweb-master-logs-%{+YYYY.MM.dd}"
document_type => "%{class}" }
}
if [type] == "log4j-json-tomcat" {
elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]
index => "dev-xxxxxweb-master-logs-%{+YYYY.MM.dd}"
document_type => "tomcat_logs" }
}
}
以上配置中还涉及到了rsyslog日志的传输，借鉴了另一位大神的配置，他的博客地址：http://dl528888.blog.51cto.com
总结
本文中最后其实少了kibana效果展示，是因为展示图还在完善中，在后面的记述中，会展示一些kibana呈现的效果。