elastic search使用总结

1. elasticsearch安装

官方下载地址：https://www.elastic.co/downloads/elasticsearch

解压文件 elasticsearch-2.4.0.zip

修改配置文件

elasticsearch-2.4.0 cat config/elasticsearch.yml |grep -v "#"
cluster.name: rainbow
network.host: 127.0.0.1
http.port: 9200

配置说明

cluster.name表示es集群的名称，可以自定义一个自己需要的集群名称
http.port 表示对外提供http服务时的http端口。
network.host 表示本地监听绑定的ip地址，此处为测试环境，直接使用本机的ip地址 127.0.0.1.

启动说明

elasticsearch-2.4.0 nohup ./bin/elasticsearch &

启动后显示信息

es启动监听两个端口，9300和9200
9300端口是使用tcp客户端连接使用的端口；
9200端口是通过http协议连接es使用的端口；

2. 使用http方式增加和查询数据

增加数据（http PUT）：

? curl -XPUT localhost:9200/user_idx/type_tags/12 -d\ ‘{"name" : "Mr.YF", "tags" : ["Go","Java","Lua","C++","Tcl","..."]}‘

{"_index":"user_idx","_type":"type_tags","_id":"12","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true}

查询数据（http GET）

3. 使用elasticsearch客户端编写java代码访问es

编写es客户端提供者类，对es连接做简单的封装

SearchClientProvider.java

package org.wyf.elasticsearch.search;

/**
 * Created by wyf on 16/9/25.
 */
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.wyf.common.property.XmlProperties;

import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;

import static java.lang.System.out;

public class SearchClientProvider {
    private static final Logger LOGGER = LoggerFactory.getLogger(SearchClientProvider.class);

    private TransportClient client = null;
    private volatile boolean inited = false;

    public TransportClient get() {
        return this.client;
    }

    @PreDestroy
    public synchronized void close() {
        if (this.client != null) {
            this.client.close();
        }
    }

    @PostConstruct
    public synchronized void init() {
        if (!inited) {
            try {
                Map<String, String> config = XmlProperties.loadFromXml("properties/elasticsearch.xml");
                if (config == null) {
                    out.println("load xml err");
                    return;
                }
                Iterator<Map.Entry<String, String>> iterator = config.entrySet().iterator();
                Map<String, String> settingConfig = new HashMap<String, String>();
                while (iterator.hasNext()) {
                    Map.Entry<String, String> next = iterator.next();
                    if (!next.getKey().equals("transport.addresses")) {
                        settingConfig.put(next.getKey(), next.getValue());
                    }
                }
                Settings settings = Settings.builder().put(settingConfig).build();
                TransportClient client = TransportClient.builder().settings(settings).build();
                this.client = client;

                String[] addresses = config.get("transport.addresses").split(",");
                for (String address : addresses) {
                    String[] hostAndPort = address.split(":");
                    client.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(hostAndPort[0]), Integer.valueOf(hostAndPort[1])));

                }
                this.inited = true;
            } catch (UnknownHostException e) {
                LOGGER.error(String.format("init search client err:=>msg:[%s]", e.getMessage()), e);
                if (client != null) {
                    this.client.close();
                }
            }
        }
    }
}

编写elasticsearch的配置文件加载类

XmlProperties.java

package org.wyf.common.property;

import org.slf4j.LoggerFactory;
import org.slf4j.Logger;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.*;

public class XmlProperties {
    private static final Logger LOGGER = LoggerFactory.getLogger(XmlProperties.class);

    private XmlProperties() {
    }

    public static Map<String, String> loadFromXml(String xmlPropertiesPath) {
        try {
            Object in = XmlProperties.class.getClassLoader().getResourceAsStream(xmlPropertiesPath);
            if(in != null) {
                LOGGER.info("Found the xml properties [{}] in class path,use it", xmlPropertiesPath);
                Map e1 = loadFromXml((InputStream)in);
                return e1;
            }
            Map<String, String> resMap = null;

            File e = new File(xmlPropertiesPath);
            if(!e.isFile()) {
                return resMap;
            }

            LOGGER.info("Found the xml properties [{}] in file path,use it", xmlPropertiesPath);
            in = new FileInputStream(new File(xmlPropertiesPath));
            resMap = loadFromXml((InputStream)in);
            ((InputStream) in).close();
            return resMap;
        } catch (Exception var7) {
            LOGGER.error("Load xml properties [" + xmlPropertiesPath + "] error.", var7);
        }
        return null;
    }

    public static Map<String, String> loadFromXml(InputStream in) throws IOException {
        Properties properties = new Properties();
        properties.loadFromXML(in);
        HashMap map = new HashMap();
        Set entries = properties.entrySet();
        Iterator iter = entries.iterator();

        while(iter.hasNext()) {
            Map.Entry entry = (Map.Entry)iter.next();
            map.put((String)entry.getKey(), (String)entry.getValue());
        }

        return map;
    }
}

编写elasticsearch简单的操作类，包含成员函数 save update get 和delete

SearchImpl.java

package org.wyf.elasticsearch;

/**
 * Created by wyf on 16/9/25.
 */
import org.elasticsearch.action.delete.DeleteRequestBuilder;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequestBuilder;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequestBuilder;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.index.engine.VersionConflictEngineException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.wyf.elasticsearch.search.SearchClientProvider;

import javax.annotation.PostConstruct;
import java.util.Map;

public class SearchImpl implements ISearch {

    private static final Logger LOGGER = LoggerFactory.getLogger(SearchImpl.class);

    private SearchClientProvider searchClientProvider;
    public  SearchImpl() {
        searchClientProvider = new SearchClientProvider();
        searchClientProvider.init();
    }

    public Map<String, Object> save(String index, String type, String id, Map<String, Object> data) {
        TransportClient client = searchClientProvider.get();
        IndexRequestBuilder builder = client.prepareIndex(index, type, id);
        IndexResponse response = builder
                .setSource(data)
                .execute()
                .actionGet();
        LOGGER.info("save index:=>index:{}, type:{}, id:{}, data:{}, rsp:{}", index, type, id, data, response);
        return data;
    }

    public int update(String index, String type, String id, Map<String, Object> data) {
        int i = 2;
        do {
            try {
                if (_innerUpdate(index, type, id, data)) return 1;
            } catch (VersionConflictEngineException e) {
                LOGGER.warn(String.format("update index:=>index:%s, type:%s, id:%s, data:%s, rsp:%s",
                        index, type, id, data, e.getMessage()), e);
            }
        } while ((i--) > 0);
        return _innerUpdate(index, type, id, data) ? 1 : 0;
    }

    public int delete(String index, String type, String id) {
        TransportClient client = searchClientProvider.get();
        DeleteRequestBuilder builder = client.prepareDelete(index, type, id);
        DeleteResponse response = builder.execute().actionGet();
        LOGGER.info("delete index:=>index:{}, type:{}, id:{}, rsp:{}",
                index, type, id, response);
        return response.isFound() ? 1 : 0;
    }

    public Map<String, Object> get(String index, String type, String id) {
        TransportClient client = searchClientProvider.get();
        GetRequestBuilder builder = client.prepareGet(index, type, id);
        GetResponse response = builder.execute().actionGet();
        return response.isExists() ? response.getSource() : null;
    }

    private boolean _innerUpdate(String index, String type, String id, Map<String, Object> data) {
        TransportClient client = searchClientProvider.get();
        GetRequestBuilder getRequestBuilder = client.prepareGet(index, type, id);
        GetResponse getResponse = getRequestBuilder.execute().actionGet();
        if (getResponse.isExists()) {
            final long version = getResponse.getVersion();
            final Map<String, Object> source = getResponse.getSource();
            source.putAll(data);

            IndexRequestBuilder builder = client.prepareIndex(index, type, id);
            IndexResponse response = builder
                    .setVersion(version)
                    .setSource(source)
                    .execute()
                    .actionGet();
            LOGGER.info("update index:=>index:{}, type:{}, id:{}, data:{}, rsp:{}",
                    index, type, id, data, response);
            return true;
        }
        throw new RuntimeException(String.format("can not get document:=>index:%s, type:%s, id:%s ", index, type, id));
    }
}

编写单元测试类

ElasticSearchTest.java

package org.wyf;

import org.junit.Before;
import org.junit.Test;
import org.wyf.elasticsearch.ISearch;
import org.wyf.elasticsearch.SearchImpl;

import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.util.Set;

/**
 * Created by wyf on 16/9/25.
 */
public class ElasticsearchTest {
    ISearch search;

    @Test
    public void get() {
        Map<String,Object> map = search.get("a", "type1", "123");
        Set<String> set = map.keySet();
        for (Iterator<String> it = set.iterator(); it.hasNext();) {
            String key = it.next();
            System.out.println(key + ":" + map.get(key));
        }
    }

    @Test
    public void save() {
        Map<String, Object> values = new HashMap<String, Object>();
        values.put("k1", "v1");
        values.put("k2", "v2");

        Map<String,Object> map = search.save("a", "type1", "123", values);
        Set<String> set = map.keySet();
        for (Iterator<String> it = set.iterator(); it.hasNext();) {
            String key = it.next();
            System.out.println(key + ":" + map.get(key));
        }

    }

    @Before
    public void before() {
        search = new SearchImpl();
    }
}

elasticsearch的配置文件

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
    <comment>elasticsearch的配置</comment>
    <entry key="client.transport.sniff">true</entry>
    <entry key="cluster.name">rainbow</entry>
    <entry key="transport.addresses">127.0.0.1:9300</entry>
</properties>

测试结果：

测试save函数结果

http接口查询结果页面

遇到的问题

最开始调试时返回如下错误

NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{127.0.0.1}{127.0.0.1:9200}]
]
	at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:290)
	at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:207)
	at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55)
	at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:283)
	at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:347)
	at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:85)
	at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:59)
	at org.wyf.elasticsearch.SearchImpl.save(SearchImpl.java:36)
	at org.wyf.ElasticsearchTest.save(ElasticsearchTest.java:35)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:253)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

解决办法是检查es集群的名称和端口是否正确

由于端口配置错误，导致如上问题

正确的端口是9300，正如最开始说的那样，使用tcp方式访问es时，使用端口9300，使用http方式访问es使用端口9200.

代码git地址：

https://github.com/gityf/java_demo/tree/master/demo/src/main/java/org/wyf/elasticsearch

Done.

时间： 2024-11-04 14:59:12

elastic search使用总结的相关文章

elastic search查询命令集合

Technorati 标签: elastic search,query,commands 基本查询:最简单的查询方式 query:{"term":{"title":"crime"}} terms:目标列包含指定词 match:会把输入的"短语"进行分解成词 bool match:match分解成多个词,并且是and的关系,默认是or的关系 match phrase:分词,但是要指定分词之间词语的个数 match phrase

elastic search 学习笔记

Elastic search在数据分析的应用中相当于一个数据库的搜索引擎. 跟MySQL类似,它有自己的查询语言,只不过不是关系型数据库,属于NoSQL. 可以根据索引从分布式服务器文件系统中快速存取数据.mysql的数据库和表存放在某台服务器上,使用的时候连接到服务器对数据库进行操作,数据使用unicode编码的mysql raw data格式:而elasticsearch通过http使用json数据进行数据索引.

Elastic Search 相关度计算

为了搞清楚elastic search背后是如何计算文档搜索时候的相关度,我决定自己做实验去探索 这篇博客讲得还不错 http://blog.csdn.net/dm_vincent/article/details/42099063 而博客本身也只是翻译了官方文档 https://www.elastic.co/guide/en/elasticsearch/guide/current/scoring-theory.html 我准备验证一下 在进行文档的搜索时,应用了以下几个基础算法的组合.名字听起来

ELASTIC SEARCH 安装

elastic search 2017年3月18日 安装&使用 环境 表 1 环境信息 Centos cat /etc/issue CentOS release 6.8 (Final) cat /proc/version Linux version 2.6.32-431.el6.x86_64 ([email protected]) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Fri Nov 22 03:15:09 UT

[elastic search][redis] 初试 ElasticSearch / redis

现有项目组,工作需要. http://www.cnblogs.com/xing901022/p/4704319.html Elastic Search权威指南(中文版) https://es.xiaoleilu.com/010_Intro/00_README.html https://www.elastic.co/guide/cn/elasticsearch/guide/current/index.html https://github.com/elastic/elasticsearch Ela

Elastic Search如何快速检索

Elastic Search如何快速检索 http://www.infoq.com/cn/articles/database-timestamp-02 Elasticsearch是通过Lucene的倒排索引技术实现比关系型数据库更快的过滤.特别是它对多条件的过滤支持非常好,比如年龄在18和30之间,性别为女性这样的组合查询.倒排索引很多地方都有介绍,但是其比关系型数据库的b-tree索引快在哪里?到底为什么快呢? 笼统的来说,b-tree索引是为写入优化的索引结构.当我们不需要支持快速的更新的时

Elastic Search搜索引擎在SpringBoot中的实践

实验环境 ES版本:5.3.0 spring bt版本:1.5.9 首先当然需要安装好elastic search环境,最好再安装上可视化插件 elasticsearch-head来便于我们直观地查看数据. 当然这部分可以参考本人的帖子:<centos7上elastic search安装填坑记>https://www.jianshu.com/p/04f4d7b4a1d3 我的ES安装在http://113.209.119.170:9200/这个地址(该地址需要配到springboot项目中去)

elastic search文档详解

在elastic search中文档(document)类似于关系型数据库里的记录(record),类型(type)类似于表(table),索引(index)类似于库(database). 文档一定有三个元数据 _index 文档在哪存放 _type 文档表示的对象类别 _id 文档唯一标识 _index可以理解为数据库,在elastic search中通常是将格式相同的数据存在同一_index下,_type即将取消,以后不同type建议分为不同_index,_id类似于数据库里的主键,你自己要

深入分析Elastic Search的写入过程

摘要 之前写过一篇ElasticSearch初识之吐槽,不知觉竟然过去了两年了.哎,时光催人老啊.最近又用到了ES,想找找过去的总结文档,居然只有一篇,搞了半年的ES,遇到那么多的问题,产出只有这么点,真是说不过去啊.只好又重新捡起ES,发现ES槽点依然很多,不兼容的更新太多了,各个版本之间的差异不小,感觉ES就是偏理论算法的人设计出来的,而不是工程学家写的.非常像公司里面,算法工程师吐槽后端应用开发算法能力弱,后端应用开发吐槽算法工程师工程能力太差.作为一个应用开发对ES差不多就是这种感觉.不

Elastic search performance testing finding

Setup1 Node cluster on my local laptop: 8core, Xms=8G, Xmx=8G Indexing performance (Single index):10 million payments, each one about 5KB, with batch size = 10000. Each batch takes roughly 2.5 s → 4 s, total time to index 10 million payment is around