java优雅的使用elasticsearch

本文给出一种优雅的拼装elasticsearch查询的方式,可能会使得使用elasticsearch的方式变得优雅起来,使得代码结构很清晰易读。

建立elasticsearch连接部分请参看另一篇博客:http://www.cnblogs.com/kangoroo/p/7127003.html

1、ESDao -- 类似DB中的dao层

封装了增删改查ES的各种方法

package com.dqa.sentinel.client.es;

import com.xiaoju.dqa.sentinel.client.es.entity.ESDocument;
import com.xiaoju.dqa.sentinel.client.es.entity.SearchDocument;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.index.query.*;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.get.MultiGetItemResponse;
import org.elasticsearch.action.get.MultiGetRequestBuilder;
import org.elasticsearch.action.get.MultiGetResponse;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchRequestBuilder;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.nio.charset.Charset;
import java.util.Collection;
import java.util.List;
import java.util.Map;

import static org.elasticsearch.index.query.QueryBuilders.termQuery;
import static org.elasticsearch.index.query.QueryBuilders.termsQuery;

@Component
public class ESDao {

    @Autowired
    private ESClient esClient;

    public void createCache(ESDocument doc) throws Exception {
        IndexResponse response = esCache.getClient().prepareIndex(doc.getIndexName(), doc.getTypeName(), doc.getId())
                .setSource(doc.getSource().getBytes(Charset.forName("UTF-8"))).setRefresh(true).get();
        if (!response.isCreated()) {
            throw new Exception();
        }
    }

    public Map<String, Object> get(ESDocument doc) throws Exception {
        try {
            GetResponse response = esClient.getClient().prepareGet(doc.getIndexName(), doc.getTypeName(), doc.getId()).get();
            return response.getSourceAsMap();
        } catch (Exception ex) {
            throw new Exception();
        }
    }

    public void create(ESDocument doc) throws Exception {
        IndexResponse response = esClient.getClient().prepareIndex(doc.getIndexName(), doc.getTypeName(), doc.getId())
                .setSource(doc.getSource().getBytes(Charset.forName("UTF-8"))).setRefresh(true).get();
        if (!response.isCreated()) {
            throw new Exception();
        }
    }

    public void mutiGet(List<ESDocument> docList) throws Exception {
        MultiGetRequestBuilder multiGetRequestBuilder = esClient.getClient().prepareMultiGet();

        for (ESDocument doc : docList) {
            multiGetRequestBuilder.add(doc.getIndexName(), doc.getTypeName(), doc.getId());
        }
        MultiGetResponse multiGetItemResponses = multiGetRequestBuilder.get();

        for (MultiGetItemResponse itemResponse : multiGetItemResponses) {
            GetResponse response = itemResponse.getResponse();
            if (response.isExists()) {
                String json = response.getSourceAsString();
                System.out.println(json);
            }
        }
    }

    public void update(ESDocument doc) throws Exception {
        try {
            UpdateRequest request = new UpdateRequest().index(doc.getIndexName())
                    .type(doc.getTypeName()).id(doc.getId())
                    .doc(doc.getSource().getBytes(Charset.forName("UTF-8"))).refresh(true);
            UpdateResponse response = esClient.getClient().update(request).get();
            long docVersion = response.getVersion();
            if (!response.isCreated()) {
                throw new Exception();
            }
        } catch (Exception e) {
            throw new Exception();
        }
    }

    public void deleteById(ESDocument doc) throws Exception{
        try {
            DeleteResponse response = esClient.getClient().prepareDelete(doc.getIndexName(),
                    doc.getTypeName(), doc.getId())
                    .execute()
                    .actionGet();
            long docVersion = response.getVersion();
            if (!response.isFound()) {
                throw new Exception();
            }
        } catch (Exception e) {
            throw new Exception();
        }

    }

    public SearchHit[] searchClient(SearchDocument searchDocument) {
        BoolQueryBuilder query = QueryBuilders.boolQuery();
        // term
        for (Map<String, String> termMap : searchDocument.getTermList()) {
            TermQueryBuilder termQueryBuilder = termQuery(termMap.get("field"), termMap.get("value"));
            query.must(termQueryBuilder);
        }
        // terms
        for (Map<String, Object> termsMap : searchDocument.getTermsList()) {
            if (termsMap.size() != 0) {
                TermsQueryBuilder termsQueryBuilder = termsQuery(termsMap.get("field").toString(), (Collection<?>) termsMap.get("terms"));
                query.must(termsQueryBuilder);
            }
        }
        // range
        for (Map rangeMap : searchDocument.getRangeList()) {
            RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery(rangeMap.get("field").toString()).from(rangeMap.get("from")).to(rangeMap.get("to")).includeLower(true).includeUpper(false);
            query.must(rangeQueryBuilder);
        }
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(query);
        SearchRequestBuilder searchRequestBuilder = esClient.getClient()
                .prepareSearch(searchDocument.getIndexName()).setTypes(searchDocument.getTypeName());
        SearchResponse response = searchRequestBuilder
                .setQuery(query)
                .setFrom(0).setSize(10000)
                .setExplain(true).execute().actionGet();
        SearchHits hits = response.getHits();
        return hits.getHits();
    }

2、查询字段的传入我们是通过Document类完成的

有三个Document,BaseDocument,ESDocument,SearchDocument,后两个继承前一个。

BaseDocument:

public class BaseDocument {
    private String indexName;
    private String typeName;

    public String getIndexName() {
        return this.indexName;
    }
    public void setIndexName(String indexName) {
        this.indexName = indexName;
    }
    public String getTypeName() {
        return this.typeName;
    }
    public void setTypeName(String typeName) {
        this.typeName = typeName;
    }

}

ESDocument:

public class ESDocument extends BaseDocument {
    private String id;
    private String source;
    public String getId() {
        return id;
    }
    public void setId(String id) {
        this.id = id;
    }
    public String getSource() {
        return source;
    }
    public void setSource(String source) {
        this.source = source;
    }

}

SearchDocument

public class SearchDocument extends BaseDocument {
    private List<Map<String, String>> termList;
    private List<Map<String, String>> rangeList;
    private List<Map<String, Object>> termsList;
    public SearchDocument() {
        super();
        termList = new ArrayList<>();
        rangeList = new ArrayList<>();
        termsList = new ArrayList<>();
    }
    public List<Map<String, String>> getTermList() {
        return this.termList;
    }
    public void setTermList(List<Map<String, String>> termList) {
        this.termList = termList;
    }
    public List<Map<String, String>> getRangeList() {
        return this.rangeList;
    }
    public void setRangeList(List<Map<String, String>> rangeList) {
        this.rangeList = rangeList;
    }

    public void setTermsList(List<Map<String, Object>> termsList) {
        this.termsList = termsList;
    }
    public List<Map<String, Object>> getTermsList() {
        return termsList;
    }
}

3、使用的时候,拼装Document,并通过ESDao查询即可

例如:

     SearchDocument searchDocument = new SearchDocument();
        // term查询, field:topicName
        Map<String, String> topicMap = new HashMap<>();
        topicMap.put("field", "topicName");
        topicMap.put("value", topicName.toLowerCase());
        List<Map<String, String>> termList = new ArrayList<>();
        termList.add(topicMap);

        // range查询, field:timestamp
        Map<String, String> timeRange =  new HashMap<>();
        timeRange.put("field", "timestamp");
        timeRange.put("from", from);
        timeRange.put("to", to);
        List<Map<String, String>> rangeList = new ArrayList<>();
        rangeList.add(timeRange);

        searchDocument.setTermList(termList);
        searchDocument.setRangeList(rangeList);
        searchDocument.setIndexName("audit");
        searchDocument.setTypeName("kafka");
        SearchHit[] searchHits = esDao.searchCache(searchDocument);
        String hourStatus = "0";
        for (SearchHit hit : searchHits) {
            Map<String, Object> eachMap = hit.getSource();
    }
时间: 2024-10-07 04:50:39

java优雅的使用elasticsearch的相关文章

JAVA优雅停机的实现

最近在项目中需要写一个数据转换引擎服务,每过5分钟同步一次数据.具体实现是启动engine server后会初始化一个ScheduledExecutorService和一个ThreadPoolExecutor线程池.schduel executor每过5分钟将dataTransformList中每一个tranform加入到线程池中运行.每一个数据转化器负责转换一组数据库数据.在执行过程中存在服务重启并且此时tranform正在转换数据并且数据没有全部操作完,此时希望正在执行的work能正常完成作

Java搜索引擎选择: Elasticsearch与Solr(转)

Elasticsearch简介 Elasticsearch是一个实时的分布式搜索和分析引擎.它可以帮助你用前所未有的速度去处理大规模数据. 它可以用于全文搜索,结构化搜索以及分析,当然你也可以将这三者进行组合. Elasticsearch是一个建立在全文搜索引擎 Apache Lucene™ 基础上的搜索引擎,可以说Lucene是当今最先进,最高效的全功能开源搜索引擎框架. 但是Lucene只是一个框架,要充分利用它的功能,需要使用JAVA,并且在程序中集成Lucene.需要很多的学习了解,才能

JAVA - 优雅的记录日志(log4j实战篇) (转)

写在前面 项目开发中,记录错误日志有以下好处: 方便调试 便于发现系统运行过程中的错误 存储业务数据,便于后期分析 在java中,记录日志有很多种方式: 自己实现 自己写类,将日志数据,以io操作方式,写数据到文本文件.数据库中. 使用log4j log4j可以将日志输出到console窗口.文本文件.数据库等,功能强大! 使用slfj slfj也是一个很强大的功能,slfj旨在一统天下,提供了logging.jar 和 log4j的接口,可以通过slfj来调用log4j,也可以调用jdk的lo

elasticsearch java操作 api

默认进行了elasticsearch安装和ik安装, 超时配置, 分页压力配置等 添加maven依赖 <dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>2.4.0</version> </dependency> <dependency> <group

Elasticsearch Java Api--DeleteByQuery

一.安装插件 要删除某个索引的一个type下的所有文档,相当于关系型数据库中的清空表操作.查阅了一些资料可以通过Delete-by-Query插件删除,首先使用插件管理器安装Delete-by-Query插件: sudo bin/plugin install delete-by-query 离线方式与head类似,解压放置elasticsearch/plugins/下即可. 集群有多个节点的情况下,每个节点都需要安装并重启. 如果想要移除插件,可以执行以下命令: sudo bin/plugin

elasticsearch 搭建-集群-java应用

搭建 首先下载最新的elasticsearch安装版本: elasticsearch下载. 下载最新的elasticsearch 0.90.1版本.下载完成后.解压缩在安装目录. 在cmd命令行进入安装目录,再进入 bin目录,运行elasticsearch.bat命令: 启动成功后.在浏览器中输入:http://localhost:9200/ 1 2 3 4 5 6 7 8 9 10 { "ok" : true, "status" : 200, "nam

elasticsearch系列七:ES Java客户端-Elasticsearch Java client(ES Client 简介、Java REST Client、Java Client、Spring Data Elasticsearch)

一.ES Client 简介 1. ES是一个服务,采用C/S结构 2. 回顾 ES的架构 3. ES支持的客户端连接方式 3.1 REST API ,端口 9200 这种连接方式对应于架构图中的RESTful style API这一层,这种客户端的连接方式是RESTful风格的,使用http的方式进行连接 3.2 Transport 连接 端口 9300 这种连接方式对应于架构图中的Transport这一层,这种客户端连接方式是直接连接ES的节点,使用TCP的方式进行连接 4. ES提供了多种

启动elk中elasticsearch服务报错which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)

解决办法: vi /etc/sysconfig/elasticsearch JAVA_HOME=/usr/local/java sudo systemctl restart elasticsearch.service sudo systemctl status elasticsearch.service 原文地址:https://www.cnblogs.com/minseo/p/10451172.html

Elasticsearch java api操作(二)(Java High Level Rest Client)

一.说明: 一.Elasticsearch提供了两个JAVA REST Client版本: 1.java low level rest client: 低级别的rest客户端,通过http与集群交互,用户需自己编组请求JSON串,及解析响应JSON串.兼容所有Elasticsearch版本. 特点:maven引入 使用介绍: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-low.h