在使用elk 传输记录 java 日志时，如下
一个java的报错
在elk中会按每一行 产生多条记录，不方便查阅
这里修改配置文件 使用 multiline 插件 即可实现多行合一的 输出模式
修改配置文件
# vi /etc/logstash/conf.d/logstash.conf
input {
file {
path => "/w_logs/error.log.2018-06-05"
type => "test"
}
}
filter {
multiline {
pattern => "^\d{4}-\d{1,2}-\d{1,2}\s\d{1,2}:\d{1,2}:\d{1,2}"
negate => true
what => "previous"
}
grok {
match => [ "message", "%{NOTSPACE:day} %{NOTSPACE:datetime} %{NOTSPACE:level} %{GREEDYDATA:msginfo} " ]
}
}
output {
if [type] == "test" {
elasticsearch {
hosts => ["10.10.15.95:9200"]
index => "12.83-test"
}
}
}
修改完 重启logstash
报错：
[ERROR] 2018-07-13 15:37:59.834 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] registry - Tried to load a plugin‘s code, but failed. {:exception=>#<LoadError: no such file to load -- logstash/filters/multiline>, :path=>"logstash/filters/multiline", :type=>"filter", :name=>"multiline"}
[ERROR] 2018-07-13 15:37:59.838 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::PluginLoadingError", :message=>"Couldn‘t find any filter plugin named ‘multiline‘. Are you sure this is correct? Trying to load the multiline filter plugin resulted in this error: no such file to load -- logstash/filters/multiline", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:192:in `lookup_pipeline_plugin‘", "/usr/share/logstash/logstash-core/lib/logstash/plugin.rb:140:in `lookup‘", "/usr/share/logstash/logstash-core/lib/logs
提示缺少 插件 filters/multiline
我们看看logstash都安装了哪些插件
# /usr/share/logstash/bin/logstash-plugin list logstash-codec-cef logstash-codec-collectd logstash-codec-dots logstash-codec-edn logstash-codec-edn_lines logstash-codec-es_bulk logstash-codec-fluent logstash-codec-graphite logstash-codec-json logstash-codec-json_lines logstash-codec-line logstash-codec-msgpack logstash-codec-multiline logstash-codec-netflow logstash-codec-plain logstash-codec-rubydebug logstash-filter-aggregate logstash-filter-anonymize logstash-filter-cidr logstash-filter-clone logstash-filter-csv logstash-filter-date logstash-filter-de_dot logstash-filter-dissect logstash-filter-dns logstash-filter-drop logstash-filter-elasticsearch logstash-filter-fingerprint logstash-filter-geoip logstash-filter-grok logstash-filter-jdbc_static logstash-filter-jdbc_streaming logstash-filter-json logstash-filter-kv logstash-filter-metrics logstash-filter-mutate logstash-filter-ruby logstash-filter-sleep logstash-filter-split logstash-filter-syslog_pri logstash-filter-throttle logstash-filter-translate logstash-filter-truncate logstash-filter-urldecode logstash-filter-useragent logstash-filter-xml logstash-input-beats logstash-input-dead_letter_queue logstash-input-elasticsearch logstash-input-exec logstash-input-file logstash-input-ganglia logstash-input-gelf logstash-input-generator logstash-input-graphite logstash-input-heartbeat logstash-input-http logstash-input-http_poller logstash-input-imap logstash-input-jdbc logstash-input-kafka logstash-input-pipe logstash-input-rabbitmq logstash-input-redis logstash-input-s3 logstash-input-snmptrap logstash-input-sqs logstash-input-stdin logstash-input-syslog logstash-input-tcp logstash-input-twitter logstash-input-udp logstash-input-unix logstash-output-cloudwatch logstash-output-csv logstash-output-elasticsearch logstash-output-email logstash-output-file logstash-output-graphite logstash-output-http logstash-output-kafka logstash-output-lumberjack logstash-output-nagios logstash-output-null logstash-output-pagerduty logstash-output-pipe logstash-output-rabbitmq logstash-output-redis logstash-output-s3 logstash-output-sns logstash-output-sqs logstash-output-stdout logstash-output-tcp logstash-output-udp logstash-output-webhdfs logstash-patterns-core
有一个logstash-codec-multiline
并没有我们需要的 logstash-filter-multiline
我们来安装这个插件，先看一下 logstash-plugin 的用法
Usage:
bin/logstash-plugin [OPTIONS] SUBCOMMAND [ARG] ...
Parameters:
SUBCOMMAND subcommand
[ARG] ... subcommand arguments
Subcommands:
list List all installed Logstash plugins
install Install a Logstash plugin
remove Remove a Logstash plugin
update Update a plugin
pack Package currently installed plugins, Deprecated: Please use prepare-offline-pack instead
unpack Unpack packaged plugins, Deprecated: Please use prepare-offline-pack instead
generate Create the foundation for a new plugin
uninstall Uninstall a plugin. Deprecated: Please use remove instead
prepare-offline-pack Create an archive of specified plugins to use for offline installation
Options:
-h, --help print help
安装插件是 # logstash-plugin install logstash-filter-multiline
# logstash-plugin install logstash-filter-multiline Validating logstash-filter-multiline Installing logstash-filter-multiline Installation successfu
原文地址：https://www.cnblogs.com/centos2017/p/9306471.html
时间： 2024-10-10 12:13:47