logstash输出到influxdb

用了这个logstash扩展

https://github.com/PeterPaulH/logstash-influxdb/blob/master/src/influxdb.rb

把这个文件放到 logstash-1.4.2/lib/logstash/outputs

看一下午logstash的文档，终于解决了自己的需求

用python描述就是这样的

开发要求统计日志中各种类型的数量

while True:
    line = f.readline()
    try:
        if ‘"type":"text","receiver_id"‘ in line:
            type = ‘directmessage‘
        elif ‘"subtype":"unfollow"‘ in line:
            type = ‘unfollow‘
        elif ‘"subtype":"follow"‘ in line:
            type = ‘follow‘
        elif ‘"subtype":"status"‘ in line:
            type = ‘weibo‘
        elif ‘"subtype":"comment"‘ in line:
            type = ‘comment‘
        else:
            type = None

        if type:
            data = [
                {"name":"pingpong_processor",
                 "columns" : ["type"],
                 "points" : [[type]]

                }
            ]

logstash配置文件如下

input {
  stdin {}
}

filter {
  if ‘"type":"text"‘ in [message] {
      mutate {
        add_field => { "type" => "directmessage" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else if ‘"subtype":"unfollow"‘ in [message] {
      mutate {
        add_field => { "type" => "unfollow" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else if ‘"subtype":"follow"‘ in [message] {
      mutate {
        add_field => { "type" => "follow" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else if ‘"subtype":"status"‘ in [message] {
      mutate {
        add_field => { "type" => "weibo" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else if ‘"subtype":"comment"‘ in [message] {
      mutate {
        add_field => { "type" => "comment" }
        remove_field => [ "message", "search" , "@version" ]
      }
  } else {
      drop {}
  }

}

output {
  influxdb {
    host => "10.75.28.180"
    port => 4444
    name => ["pingpong_processor"]
    columns => ["type", "host"]
    points => [
        "%{type}", "c",
        "%{host}", "c"
    ]

  }

  stdout {}
}

别忘记把influxdb的配置也修改下，因为默认upd协议是没有打开的

# Configure the udp api
  [input_plugins.udp]
  enabled = true
  port = 4444
  database = "pingpong_processor"

我用的influxdb版本是 influxdb-0.8.2-1.x86_64,用/etc/init.d/influxdb这个启动报错，无奈手动启动的

/usr/bin/influxdb -pidfile /tmp/influxdb.pid -config config.toml

看一下udp端口是否启动了

netstat -anup|grep influxdb
udp        0      0 :::4444                     :::*                                    27512/influxdb

完工