ELK获取用户真实IP

原理：在filebeat这台服务器上的nginx中获取到客户端真实IP($clientRealIp)，    然后在访问日志中添加"$clientRealIp"字段。
1. 通过map获取到用户真实IP，并调整日志格式，增加$clientRealIp段
http {
        map $http_x_forwarded_for  $clientRealIp {
        ""      $remote_addr;
        ~^(?P<firstAddr>[0-9\.]+),?.*$  $firstAddr;
    }
    
    log_format  main  ‘$remote_addr - $remote_user [$time_local] "$request" ‘
                      ‘$status $body_bytes_sent "$http_referer" ‘
                      ‘"$http_user_agent" "$clientRealIp" ‘
                      ‘$upstream_addr $upstream_response_time $upstream_status‘;
    }        
    
2. 在logstash的filter里，对日志进行过滤。
    2.1 自定义日志过滤匹配规则
    vi /data/elk/logstash/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-4.1.1/patterns/grok-patterns
    #NGINXPHONEACCESS
    NGINXPHONEACCESS %{IPORHOST:remote_addr} - %{USER:remote_user} \[%{HTTPDATE:time_local}\] "%{WORD:request_method} %{URIPATHPARAM:request_path} HTTP/%{NUMBER:http_version}" %{INT:http_status} %{INT:body_bytes_sent} "%{DATA:http_referer}" "%{DATA:http_user_agent}" "%{DATA:user_real_ip}" %{IPPORT:upstream_addr_port} %{NUMBER:upstream_response_time} %{INT:upstream_status}
    2.2 在filter中过滤
        if[type]== "phone-proxy-nginx-access" {
                grok {  
                        match => { "message" => "%{NGINXPHONEACCESS}" }
                }

mutate {
                        remove_field => ["message"]
                }

mutate {
                        convert => ["upstream_response_time", "float"]
                         convert => ["body_bytes_sent", "integer"]
                }

geoip {
                        source => "user_real_ip"
                        target => "geoip"
                        database => "/data/elk/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.2.1-java/vendor/GeoLite2-City.mmdb"
                        add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
                        add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
                }

mutate {
                        convert => [ "[geoip][coordinates]", "float"]
                }

}

2.3 经过上面的过滤后，将会在日志中添加geoip信息，例如这样：
    {
      "@version" => "1",
    "@timestamp" => "2015-01-01T22:15:13.000Z",
          "host" => "iMac-de-Consulthys.local",
          "path" => "/home/kibana/Documents/external_noise.log",
          "type" => "external_noise",
     "timestamp" => "Jan 1 23:15:13",
        "action" => "drop",
     "logsource" => "%LOGSOURCE%",
     "interface" => ">eth1",
          "rule" => "7",
      "rule_uid" => "{C1336766-9489-4049-9817-50584D83A245}",
           "src" => "218.8.245.123",
           "dst" => "%DSTIP%",
         "proto" => "tcp",
       "product" => "VPN-1&FireWall-1",
       "service" => "2967",
        "s_port" => "6000",
         "geoip" => {
                      "ip" => "218.8.245.123",
           "country_code2" => "CN",
           "country_code3" => "CHN",
            "country_name" => "China",
          "continent_code" => "AS",
             "region_name" => "08",
               "city_name" => "Harbin",
                "latitude" => 45.75,
               "longitude" => 126.64999999999998,
                "timezone" => "Asia/Harbin",
        "real_region_name" => "Heilongjiang",
                "location" => [
            [0] 126.64999999999998,
            [1] 45.75
        ],
             "coordinates" => [
            [0] 126.64999999999998,
             [1] 45.75
        ]
    }
    }

时间： 2024-10-09 12:24:18

ELK获取用户真实IP的相关文章

java 获取用户真实ip

/** * 获取用户真实ip * @param request * @return */ public static String getIpAddr(HttpServletRequest request){ String ip = request.getHeader("x-forwarded-for"); if ((ip == null) || (ip.length() == 0) || ("unknown".equalsIgnoreCase(ip))) { ip

前端Nginx，后端Apache获取用户真实IP地址

Nginx作为前端,Apache作为后端的情况下,Apache只能获取到Nginx前端的内网ip地址(10.10.0.*),而无法获取到用户的真实ip地址,在这种情况下,后端是Apache如何获取用户真实IP地址? nginx 关键配置 proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

Java中使用HttpRequest获取用户真实IP地址

1 import javax.servlet.http.HttpServletRequest; 2 3 /** 4 * 自定义访问对象工具类 5 * 6 * 获取对象的IP地址等信息 7 * @author X-rapido 8 * 9 */ 10 public class CusAccessObjectUtil { 11 12 /** 13 * 获取用户真实IP地址,不使用request.getRemoteAddr();的原因是有可能用户使用了代理软件方式避免真实IP地址, 14 * 16 *

php获取用户真实ip地址与地理位置

echo getcposition(getIP()); //获取用户真实ip function getIP() { if (isset($_SERVER)) { if (isset($_SERVER[HTTP_X_FORWARDED_FOR])) { $realip = $_SERVER[HTTP_X_FORWARDED_FOR]; } elseif (isset($_SERVER[HTTP_CLIENT_IP])) { $realip = $_SERVER[HTTP_CLIENT_IP]; }

获取用户真实IP：（模拟：客户端--F5--nginx--tomcat 后端获取用户真实IP）

模拟:客户端--F5--nginx--tomcat 后端获取用户真实IP 192.168.109.137 :nginx01(充当第一层代理==F5)192.168.109.138 :nginx02(二层代理,业务转发)192.168.109.139 :tomcat (后端业务层) 192.168.109.1 :客户端IP ----------------------------------------------------------------------------------------

CDN下nginx获取用户真实IP地址

随着nginx的迅速崛起,越来越多公司将apache更换成nginx. 同时也越来越多人使用nginx作为负载均衡, 并且代理前面可能还加上了CDN加速,但是随之也遇到一个问题:nginx如何获取用户的真实IP地址,如果后端是apache,请跳转到,如果是后端真实服务器是nginx,那么继续往下看. 实例环境: 用户IP 120.22.11.11 CDN前端 61.22.22.22 CDN中转 121.207.33.33 公司NGINX前端代理 192.168.50.121(外网121.207.

PHP获取用户真实IP地址

PHP获取客户端真实IP地址方法 在PHP获取客户端IP中常使用 $_SERVER["REMOTE_ADDR"] . (1) 但如果客户端是使用代理服务器来访问,那取到的是代理服务器的 IP 地址,而不是真正的客户端 IP 地址.要想透过代理服务器取得客户端的真实 IP 地址,就要使用 $_SERVER["HTTP_X_FORWARDED_FOR"] 来读取. (2) 但只有客户端使用“透明代理”的情况下,$_SERVER["HTTP_X_FORWARDE

php获取用户真实IP和防刷机制

一. 如何获取用户IP地址 public static function getClientIp() { if (getenv('HTTP_CLIENT_IP')) { $ip = getenv('HTTP_CLIENT_IP'); } if (getenv('HTTP_X_REAL_IP')) { $ip = getenv('HTTP_X_REAL_IP'); } elseif (getenv('HTTP_X_FORWARDED_FOR')) { $ip = getenv('HTTP_X_FO

docker内的服务无法获取用户真实IP

背景:MySQL数据库和Redis运行在宿主机上(Linux),server运行在docker内,web运行在Nginx内(Nginx运行在docker内),获取的用户IP为10.0.0.10类似的docker内部IP 需求:获取真实IP 方法: 一.修改Nginx配置文件,docker容器内[/etc/nginx/conf.d/default.conf] server { listen 80; server_name localhost; #charset koi8-r; #access_lo