springboot集成elk 三:springboot + Elasticsearch Rest-Client

注:  该集成方式,对Elasticsearch无版本限制,但是需要自行封装请求,解析结果等。

<dependency>
   <groupId>org.elasticsearch.client</groupId>
   <artifactId>elasticsearch-rest-client</artifactId>
   <version>7.3.0</version>
</dependency>

<dependency>
   <groupId>com.alibaba</groupId>
  <artifactId>fastjson</artifactId>
  <version>1.2.47</version>
</dependency>

客户端配置RestConfig

@Configuration

public class RestConfig {

    @Bean

    public RestClient getClient() throws KeyStoreException, NoSuchAlgorithmException, KeyManagementException {

        // 如果有多个从节点可以持续在内部new多个HttpHost,参数1是ip,参数2是HTTP端口,参数3是通信协议

        RestClientBuilder clientBuilder = RestClient.builder(new HttpHost("134.175.237.32", 9200, "http"));

        // 添加其他配置,返回来的还是RestClientBuilder对象,这些配置都是可选的

        //【1. 设置请求头】 设置请求头,每个请求都会带上这个请求头

        //Header[] defaultHeaders = {new BasicHeader("header", "value")};

        //clientBuilder.setDefaultHeaders(defaultHeaders);

        Header[] defaultHeaders = {new BasicHeader("charset", "utf-8"),

                new BasicHeader("content-type", "application/json")};

        clientBuilder.setDefaultHeaders(defaultHeaders);

        //【2. 设置超时时间】 设置超时时间,多次尝试同一请求时应该遵守的超时。默认值为30秒,与默认套接字超时相同。若自定义套接字超时,则应相应地调整最大重试超时

        //clientBuilder.setMaxRetryTimeoutMillis(60000);

        //【3. 设置节点失败监听器】设置监听器,每次节点失败都可以监听到,可以作额外处理

        /*clientBuilder.setFailureListener(new RestClient.FailureListener() {

            @Override

            public void onFailure(Node node) {

                super.onFailure(node);

                System.out.println(node.getName() + "==节点失败了");

            }

        });*/

        /*【4. 设置节点选择器】 配置节点选择器,客户端以循环方式将每个请求发送到每一个配置的节点上,

        发送请求的节点,用于过滤客户端,将请求发送到这些客户端节点,默认向每个配置节点发送,

        这个配置通常是用户在启用嗅探时向专用主节点发送请求(即只有专用的主节点应该被HTTP请求命中)

        */

       // clientBuilder.setNodeSelector(NodeSelector.SKIP_DEDICATED_MASTERS);

        // 进行详细的配置

       /* clientBuilder.setNodeSelector(new NodeSelector() {

            // 设置分配感知节点选择器,允许选择本地机架中的节点(如果有),否则转到任何机架中的任何其他节点。

            @Override

            public void select(Iterable<Node> nodes) {

                boolean foundOne = false;

                for (Node node: nodes) {

                    String rackId = node.getAttributes().get("rack_id").get(0);

                    if ("rack_one".equals(rackId)) {

                        foundOne = true;

                        break;

                    }

                }

                if (foundOne) {

                    Iterator<Node> nodesIt = nodes.iterator();

                    while (nodesIt.hasNext()) {

                        Node node = nodesIt.next();

                        String rackId = node.getAttributes().get("rack_id").get(0);

                        if ("rack_one".equals(rackId) == false) {

                            nodesIt.remove();

                        }

                    }

                }

            }

        });*/

       /* 【5. 配置HTTP异步请求ES的线程数】

       配置异步请求的线程数量,Apache Http Async Client默认启动一个调度程序线程,以及由连接管理器使用的许多工作线程

        (与本地检测到的处理器数量一样多,取决于Runtime.getRuntime().availableProcessors()返回的数量)。线程数可以修改如下,

        这里是修改为1个线程,即默认情况

        */

        /*clientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {

            @Override

            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpAsyncClientBuilder) {

                return httpAsyncClientBuilder.setDefaultIOReactorConfig(

                        IOReactorConfig.custom().setIoThreadCount(1).build()

                );

            }

        });*/

        /*【6. 配置连接超时和套接字超时】

        配置请求超时,将连接超时(默认为1秒)和套接字超时(默认为30秒)增加,

        这里配置完应该相应地调整最大重试超时(默认为30秒),即上面的setMaxRetryTimeoutMillis,一般于最大的那个值一致即60000

        */

       /* clientBuilder.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {

            @Override

            public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {

                // 连接5秒超时,套接字连接60s超时

                return requestConfigBuilder.setConnectTimeout(5000).setSocketTimeout(60000);

            }

        });*/

       /*【7. 配置ES安全认证】

        如果ES设置了密码,那这里也提供了一个基本的认证机制,下面设置了ES需要基本身份验证的默认凭据提供程序

            */

        /*final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();

        credentialsProvider.setCredentials(AuthScope.ANY,

                new UsernamePasswordCredentials("user", "password"));

        clientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {

            @Override

            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {

                return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);

            }

        });*/

        /*

        上面采用异步机制实现抢先认证,这个功能也可以禁用,这意味着每个请求都将在没有授权标头的情况下发送,然后查看它是否被接受,

        并且在收到HTTP 401响应后,它再使用基本认证头重新发送完全相同的请求,这个可能是基于安全、性能的考虑

         */

        /*clientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {

            @Override

            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {

                // 禁用抢先认证的方式

                httpClientBuilder.disableAuthCaching();

                return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);

            }

        });*/

        /* 【8. 配置通信加密】

        配置通信加密,有多种方式:setSSLContext、setSSLSessionStrategy和setConnectionManager(它们的重要性逐渐递增)

         */

        /*KeyStore truststore = KeyStore.getInstance("jks");

        try (InputStream is = Files.newInputStream(keyStorePath)) {

            truststore.load(is, keyStorePass.toCharArray());

        }

        SSLContextBuilder sslBuilder = SSLContexts.custom().loadTrustMaterial(truststore, null);

        final SSLContext sslContext = sslBuilder.build();

        clientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {

            @Override

            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {

                return httpClientBuilder.setSSLContext(sslContext);

            }

        });*/

        // 最后配置好的clientBuilder再build一下即可得到真正的Client

        return clientBuilder.build();

    }

}

实体类:

public class Book {

    private String id;

    private String name;

    private String author;

    public String getAuthor() {

        return author;

    }

    public void setAuthor(String author) {

        this.author = author;

    }

    public String getId() {

        return id;

    }

    public void setId(String id) {

        this.id = id;

    }

    public String getName() {

        return name;

    }

    public void setName(String name) {

        this.name = name;

    }

    @Override

    public String toString() {

        return "Book{" +

                "id=" + id +

                ", name=‘" + name + ‘\‘‘ +

                ‘}‘;

    }

}

Rest  Controller:

@RestController

@RequestMapping("/book")

public class BookController {

    @Autowired

    private RestClient client;

    /**新增

     * 添加ES对象, Book的ID就是ES中存储的document的ID,所以最好不要为空,自定义生成的ID太浮夸

     *

     * @return ResponseEntity

     * @throws IOException

     */

    @PostMapping("/add")

    public ResponseEntity<String> add(Book book) throws IOException {

        // 构造HTTP请求,第一个参数是请求方法,第二个参数是服务器的端点,host默认是http://localhost:9200,

        // endpoint直接指定为index/type的形式

        Request request = new Request("POST", new StringBuilder("/test_db/book/").

                append(book.getId()).toString());

        // 设置其他一些参数比如美化json

        request.addParameter("pretty", "true");

        String bookString=JSON.toJSONString(book);

        System.out.println(bookString);

        // 设置请求体并指定ContentType,如果不指定默认为APPLICATION_JSON

        request.setEntity(new NStringEntity(bookString, ContentType.APPLICATION_JSON));

        // 发送HTTP请求

        Response response = client.performRequest(request);

        // 获取响应体, id: AWXvzZYWXWr3RnGSLyhH

        String responseBody = EntityUtils.toString(response.getEntity());

        return new ResponseEntity<>(responseBody, HttpStatus.OK);

    }

    /**

     * 根据id获取ES对象

     *

     * @param id

     * @return

     * @throws IOException

     */

    @GetMapping("/getBookById/{id}")

    public ResponseEntity<String> getBookById(@PathVariable("id") String id) {

        Request request = new Request("GET", new StringBuilder("/test_db/book/").

                append(id).toString());

        // 添加json返回优化

        request.addParameter("pretty", "true");

        Response response = null;

        String responseBody = null;

        try {

            // 执行HHTP请求

            response = client.performRequest(request);

            responseBody = EntityUtils.toString(response.getEntity());

        } catch (IOException e) {

            return new ResponseEntity<>("can not found the book by your id", HttpStatus.NOT_FOUND);

        }

        return new ResponseEntity<>(responseBody, HttpStatus.OK);

    }

    /**

     * 根据id更新Book

     *

     * @param id

     * @param book

     * @return

     */

    @PostMapping("/updateBook/{id}")

    public ResponseEntity<String> updateBook(@PathVariable("id") String id, Book book) throws IOException {

        // 构造HTTP请求

        Request request = new Request("POST", new StringBuilder("/test_db/book/").

                append(id).append("/_update").toString());

        request.addParameter("pretty", "true");

        // 将数据丢进去,这里一定要外包一层“doc”,否则内部不能识别

        JSONObject jsonObject = new JSONObject();

        jsonObject.put("doc", book);

        request.setEntity(new NStringEntity(jsonObject.toString(), ContentType.APPLICATION_JSON));

        // 执行HTTP请求

        Response response = client.performRequest(request);

        // 获取返回的内容

        String responseBody = EntityUtils.toString(response.getEntity());

        return new ResponseEntity<>(responseBody, HttpStatus.OK);

    }

    /**

     * 使用脚本更新Book

     * @param id

     * @param

     * @return

     * @throws IOException

     */

    @PostMapping("/update2/{id}")

    public ResponseEntity<String> updateBook2(@PathVariable("id") String id, @RequestParam("name") String name) throws IOException {

        // 构造HTTP请求

        Request request = new Request("POST", new StringBuilder("/test_db/book/").

                append(id).append("/_update").toString());

        request.addParameter("pretty", "true");

        JSONObject jsonObject = new JSONObject();

        // 创建脚本语言,如果是字符变量,必须加单引号

        StringBuilder op1 = new StringBuilder("ctx._source.name=").append("‘" + name + "‘");

        jsonObject.put("script", op1);

        request.setEntity(new NStringEntity(jsonObject.toString(), ContentType.APPLICATION_JSON));

        // 执行HTTP请求

        Response response = client.performRequest(request);

        // 获取返回的内容

        String responseBody = EntityUtils.toString(response.getEntity());

        return new ResponseEntity<>(responseBody, HttpStatus.OK);

    }

    @PostMapping("/deleteById/{id}")

    public ResponseEntity<String> deleteById(@PathVariable("id") String id) throws IOException {

        Request request = new Request("DELETE", new StringBuilder("/test_db/book/").append(id).toString());

        request.addParameter("pretty", "true");

        // 执行HTTP请求

        Response response = client.performRequest(request);

        // 获取结果

        String responseBody = EntityUtils.toString(response.getEntity());

        return new ResponseEntity<>(responseBody, HttpStatus.OK);

    }

    /**

     * 获取ES对象列表

     *

     * @return

     * @throws IOException

     */

    @GetMapping("/getBookList")

    public ResponseEntity<Object> getBookList(@RequestParam("author") String author) {

        // 构造HTTP请求

        Request request = new Request("POST", new StringBuilder("/test_db/book/_search").toString());

        // 添加json返回优化

        request.addParameter("pretty", "true");

        StringBuilder requestBody = new StringBuilder();

        requestBody.append("{\"size\":10,\"query\":{\"match\":{\"author\":\""+author+"\"}},\"from\": 0,\"_source\": [\"id\", \"name\", \"author\"]}");

        //JSONObject jsonObject = new JSONObject();

       // jsonObject.put("doc", requestBody.toString());

        request.setEntity(new NStringEntity(requestBody.toString(), ContentType.APPLICATION_JSON));

        Response response = null;

        String responseBody = null;

        Object result=null;

        try{

            // 执行HTTP请求

            response = client.performRequest(request);

            // 获取返回的内容

            responseBody = EntityUtils.toString(response.getEntity());

            Map jsonObject = JSON.parseObject(responseBody);

            result =jsonObject.get("hits");

            System.out.println(result);

        } catch (IOException e) {

            e.printStackTrace();

            return new ResponseEntity<>("can not found the book by your id", HttpStatus.NOT_FOUND);

        }

        return new ResponseEntity<>(result, HttpStatus.OK);

    }

springboot-elasticsearch-rest-client.zi

p

原文地址:https://www.cnblogs.com/brant/p/11680165.html

时间: 2024-10-02 20:28:03

springboot集成elk 三:springboot + Elasticsearch Rest-Client的相关文章

springboot集成elk 一: springboot + Elasticsearch

1.ELK介绍 1> Elasticsearch是实时全文搜索和分析引擎, 提供搜集.分析.存储数据三大功能: 是一套开放REST和JAVA API等结构提供高效搜索功能,可扩展的分布式系统. 2> Logstash是一个用来搜集.分析.过滤日志的工具. 它支持几乎任何类型的日志,包括系统日志.错误日志和自定义应用程序日志. 它可以从许多来源接收日志,这些来源包括 syslog.消息传递(例如 RabbitMQ)和JMX,它能够以多种方式输出数据,包括电子邮件.websockets和Elast

springboot集成elk 四:springboot + Elasticsearch+Jest

依赖 <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>io.searchbox</groupId> <artifactId>

springboot集成elk 二:springboot + elk 采集日志

到logstash-2.0.0\bin下新建文件 logstash.conf input { tcp { mode => "server" host => "0.0.0.0" port => 4560 codec => json_lines } } output { elasticsearch { hosts => "localhost:9200" index => "springboot-logst

springboot整合logback集成elk实现日志的汇总、分析、统计和检索功能

在Spring Boot当中,默认使用logback进行log操作.logback支持将日志数据通过提供IP地址.端口号,以Socket的方式远程发送.在Spring Boot中,通常使用logback-spring.xml来进行logback配置. 首先.创建一个elk的springboot项目,然后先对logback进行配置,配置各项的详细说明可以去看http://aub.iteye.com/blog/1101222,说的很详细.也多参考一下别人关于日志的描述https://www.cnbl

springboot 集成elasticsearch

In this article, we will discuss about “How to create a Spring Boot + Spring Data + Elasticsearch Example”. Tools used in this article : Spring Boot 1.5.1.RELEASE Spring Boot Starter Data Elasticsearch 1.5.1.RELEASE Spring Data Elasticsearch 2.10.REL

第三章 SpringBoot 集成框架(二)

1.SpringBoot 默认能直接访问的静态资源目录: /static    public  /resources  /META-INF/resources 2.SpringBoot 集成 JSP :需要添加 JSP 的依赖 ,JSP 页面使用 JSTL 标签 . 3.SpringBoot 框架集成 JSP 时,打包的格式是war          打包:右键项目 --> Run as --> Maven build...    pom.xml中的打包方式:<packaging>

SpringBoot集成Zipkin实现分布式全链路监控

目录 Zipkin 简介 Springboot 集成 Zipkin 安装启动 zipkin 版本说明 项目结构 工程端口分配 引入 Maven 依赖 配置文件.收集器的设置 编写 Controller 发送请求进行测试 Springboot 启动类 运行分析 核心概念 Zipkin 简介 Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency proble

SpringBoot集成MyBatis的分页插件PageHelper

俗话说:好??不吃回头草,但是在这里我建议不管你是好马还是不好马,都来吃吃,带你复习一下分页插件PageHelper. 昨天给各位总结了本人学习springboot整合mybatis第一阶段的一些学习心得和源码,主要就算是敲了一下SpringBoot的门儿,希望能给各位的入门带给一点儿捷径,今天给各位温习一下MyBatis的分页插件PageHelper和SpringBoot的集成,它的使用也非常简单,开发更为高效.因为PageHelper插件是属于MyBatis框架的,所以相信很多哥们儿都已经用

springboot集成swagger2构建RESTful API文档

在开发过程中,有时候我们需要不停的测试接口,自测,或者交由测试测试接口,我们需要构建一个文档,都是单独写,太麻烦了,现在使用springboot集成swagger2来构建RESTful API文档,可以在访问接口上,直接添加注释 先介绍一下开发环境: jdk版本是1.8 springboot的版本是1.4.1 开发工具为 intellij idea 我们先引入swagger2的jar包,pom文件引入依赖如下: <dependency> <groupId>io.springfox&