ElasticSearch简介

a.ElasticSearch是一个基于Lucene开发的搜索服务器，具有分布式多用户的能力，ElasticSearch是用Java开发的开源项目(Apache许可条款)，基于Restful Web接口，能够达到实时搜索、稳定、可靠、快速、高性能、安装使用方便，同时它的横向扩展能力非常强，不需要重启服务。

b.ElasticSearch是一个非常好用的实时分布式搜索和分析引擎，可以帮助我们快速的处理大规模数据，也可以用于全文检索，结构化搜索以及分析等。c.目前很多网站都在使用ElasticSearch进行全文检索，例如：GitHub、StackOverflow、Wiki等。d.ElasticSearch式建立在全文检索引擎Lucene基础上的，而Lucene是最先进、高效的开元搜索引擎框架，,它可以使你的引用程序添加索引和搜索能力，但是Lucene只是一个框架，要充分利用它的功能，我们需要很高的学习成本，而ElasticSearch使用Lucene作为内部引擎，在其基础上封装了功能强大的Restful API，让开发人员可以在不需要了解背后复杂的逻辑，即可实现比较高效的搜索。

• ElasticSearch安装
  Elasticsearch是基于java写的，所以它的运行环境中需要java的支持。

  [root@localhost 40]# java -version-bash: java: command not found[root@localhost 40]# wget http://download.oracle.com/otn-pub/java/jdk/8u161-b12/2f38c3b165be4555a1fa6e98c45e0808/jdk-8u161-linux-x64.tar.gz[root@localhost 40]# tar xvf jdk-8u161-linux-x64.gz[root@localhost 40]# vim /etc/profileexport JAVA_HOME=/opt/40/jdk1.8.0_161export JAVA_BIN=/opt/40/jdk1.8.0_161/binexport PATH=$PATH:$JAVA_HOME/binexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarexport JAVA_HOME JAVA_BIN PATH CLASSPATH[root@localhost 40]# source /etc/profile[root@localhost 40]# java -versionjava version "1.8.0_161"Java(TM) SE Runtime Environment (build 1.8.0_161-b12)Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)[root@localhost 40]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.0.tar.gz[root@localhost 40]# tar zxvf elasticsearch-6.2.0.tar.gz

* ElasticSearch启动

[root@localhost 40]# cd elasticsearch-6.2.0/bin/

[2018-02-05T08:29:10,244][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as rootat org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-6.2.0.jar:6.2.0]at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.2.0.jar:6.2.0]at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.2.0.jar:6.2.0]at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.2.0.jar:6.2.0]at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.2.0.jar:6.2.0]at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.2.0.jar:6.2.0]at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.2.0.jar:6.2.0]Caused by: java.lang.RuntimeException: can not run elasticsearch as rootat org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:105) ~[elasticsearch-6.2.0.jar:6.2.0]at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) ~[elasticsearch-6.2.0.jar:6.2.0]at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.0.jar:6.2.0]at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.0.jar:6.2.0]... 6 more[root@localhost bin]#

当使用root账户启动ES会出现以上错误信息，这是因为处于系统安装考虑的设置，由于Elasticsearch可以接收用户输入的脚本并且执行，为了系统安全考虑，不允许root账号启动，所以建议给Elasticsearch单独创建一个用户来运行Elasticsearch。

[root@localhost bin]# groupadd justin

[root@localhost bin]# useradd justin -g justin -p 51cto[root@localhost bin]# chown -R justin:justin /opt/40/elasticsearch-6.2.0/[root@localhost bin]# su - justin[justin@localhost ~]$ cd /opt/40/elasticsearch-6.2.0/bin/[justin@localhost bin]$ cp ../config/elasticsearch.yml{,.default}[justin@localhost bin]$ vim ../config/elasticsearch.yml54 #55 network.host: 0.0.0.056 #57 # Set a custom port for HTTP:58 #59 http.port: 920060 #[justin@localhost bin]$ ./elasticsearch -d[justin@localhost bin]$ ps -ef|grep -i elasjustin 13348 1 38 08:34 pts/2 00:00:06 /opt/40/jdk1.8.0_161/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch.xljvX8Ss -XX:+HeapDumpOnOutOfMemoryError -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Des.path.home=/opt/40/elasticsearch-6.2.0 -Des.path.conf=/opt/40/elasticsearch-6.2.0/config -cp /opt/40/elasticsearch-6.2.0/lib/* org.elasticsearch.bootstrap.Elasticsearch -djustin 13366 13282 0 08:34 pts/2 00:00:00 grep --color=auto -i elas[justin@localhost bin]$

发现启动一会就自动关闭了。启动时候不要后台启动，这样可以打印出日志，方便找问题，等稳定了在后台启动

[justin@localhost bin]$ ./elasticsearch

[2018-02-05T11:39:47,616][INFO ][o.e.b.BootstrapChecks ] [IAWTuJo] bound or publishing to a non-loopback address, enforcing bootstrap checksERROR: [3] bootstrap checks failed[1]: max file descriptors [65535] for elasticsearch process is too low, increase to at least [65536][2]: max number of threads [3856] for user [justin] is too low, increase to at least [4096][3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144][2018-02-05T11:39:47,643][INFO ][o.e.n.Node ] [IAWTuJo] stopping ...[2018-02-05T11:39:47,687][INFO ][o.e.n.Node ] [IAWTuJo] stopped[2018-02-05T11:39:47,687][INFO ][o.e.n.Node ] [IAWTuJo] closing ...[2018-02-05T11:39:47,723][INFO ][o.e.n.Node ] [IAWTuJo] closed[justin@localhost bin]$

切换到root用户，做如下修改

[root@localhost ~]# vim /etc/sysctl.conf

vm.max_map_count = 1048576[root@localhost ~]# sysctl -p[root@localhost ~]# vim /etc/security/limits.conf

• hard nofile 65536
• soft nofile 65536
  [root@localhost ~]# vim /etc/security/limits.d/20-nproc.conf
• soft nproc 65535
  root soft nproc unlimited
  [root@localhost ~]#

  再次启动

  [root@localhost ~]# su - justin

  [ES@localhost ~]$ cd /opt/elasticsearch-6.2.0/bin/
  ERROR: [1] bootstrap checks failed
  [1]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
  [ES@localhost bin]$

Centos6不支持SecComp，而ES5.2.0后的版本默认bootstrap.system_call_filter为true，在elasticsearch.yml中增加bootstrap.system_call_filter为false，注意要在Memory下面: #bootstrap.memory_lock: truebootstrap.system_call_filter: false看到以下界面说明启动成功![](http://i2.51cto.com/images/blog/201802/09/a7f945f39f7381f3096338d3b8805d67.png?x-oss-process=image/watermark,size_16,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_100,g_se,x_10,y_10,shadow_90,type_ZmFuZ3poZW5naGVpdGk=)-----Elasticsearch Head是集群管理、数据可视化、增删改查、查询语句可视化工具，它的安装方式有两种：命令安装、包安装安装。也可以从下面地址下载https://github.com/mobz/elasticsearch-head、https://github.com/mobz/elasticsearch-head/archive/master.zip

[root@localhost opt]# git clone git://github.com/mobz/elasticsearch-head.git

Initialized empty Git repository in /opt/elasticsearch-head/.git/remote: Counting objects: 4224, done.remote: Total 4224 (delta 0), reused 0 (delta 0), pack-reused 4224Receiving objects: 100% (4224/4224), 2.16 MiB | 46 KiB/s, done.Resolving deltas: 100% (2329/2329), done.[root@localhost opt]# wget [root@localhost opt]# vim /etc/profileexport PATH=$PATH::/opt/node-v8.9.4-linux-x64/bin[root@localhost opt]# source !$[root@localhost opt]# cd elasticsearch-head/[root@localhost elasticsearch-head]# npm installnpm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expressionnpm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules/fsevents):npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE

npm ERR! errno 1npm ERR! phantomjs-prebuilt@2.1.16 install: node install.jsnpm ERR! Exit status 1npm ERR! npm ERR! Failed at the phantomjs-prebuilt@2.1.16 install script.npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:

npm ERR! /root/.npm/_logs/2018-02-09T07_03_39_766Z-debug.log[root@localhost elasticsearch-head]# npm install phantomjs-prebuilt@2.1.14 --ignore-scripts[root@localhost elasticsearch-head]# npm install[root@localhost elasticsearch-head]# npm run start

或者通过grunt来启动

[root@localhost elasticsearch-head]# npm install -g grunt-cli

[root@localhost elasticsearch-head]# grunt server

修改elasticsearch的参数            增加新的参数，这样head插件可以访问es

[ES@localhost bin]$ pwd

/opt/elasticsearch-6.2.0/bin[ES@localhost bin]$ vim ../config/elasticsearch.ymlhttp.cors.enabled: truehttp.cors.allow-origin: "*" #前引号前，后引号后有空格[ES@localhost bin]$

然后通过http://10.10.2.83:9100/访问或者直接用命令安装

[root@localhost opt]# cd /opt/elasticsearch-6.2.1/bin

[root@localhost bin]# ./plugin install mobz/elasticsearch-head

然后可以通过http://127.0.0.1:9200/_plugin/head/访问修改head插件源码            修改服务器监听地址:Gruntfile.js

[root@localhost elasticsearch-head]# vim Gruntfile.js

connect: { server: { options: { hostname: '0.0.0.0', #增加hostname属性port: 9100,base: '.',keepalive: true}}}

修改连接地址：_site/app.js

[root@localhost elasticsearch-head]# vim _site/app.js

app.App = ui.AbstractWidget.extend({ defaults: { base_uri: null},init: function(parent) { this._super();this.prefs = services.Preferences.instance();this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || ""; #把localhost修改成ES的服务器地址if( this.base_uri.charAt( this.base_uri.length - 1 ) !== "/" ) { // XHR request fails if the URL is not ending with a "/"this.base_uri += "/";}if( this.config.auth_user ) { var credentials = window.btoa( this.config.auth_user + ":" + this.config.auth_password );$.ajaxSetup({ headers: { "Authorization": "Basic " + credentials}});}[root@localhost elasticsearch-head]# nohup grunt server &

-----x-pack安全模块(security机制)只要安装好elasticsearch后，输入正确的ip地址和端口，并有head插件就可以看到索引库，所以如果别人知道IP和和端口，那么数据就完全裸奔在别人面前。安装x-pack后用户名和密码默认为elastic和changeme。官方文档参考地址:https://www.elastic.co/guide/en/x-pack/5.4/logstash.html1、修改Elasticsearch配置文件

[root@localhost bin]# pwd

/opt/elasticsearch-6.2.0/bin[root@localhost bin]# ./elasticsearch-plugin install x-packfor descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y

Continue with installation? [y/N]y

Elasticsearch keystore is required by plugin [x-pack-security], creating...-> Installed x-pack with: x-pack-security,x-pack-logstash,x-pack-deprecation,x-pack-watcher,x-pack-upgrade,x-pack-core,x-pack-ml,x-pack-monitoring,x-pack-graph[root@localhost bin]# [root@localhost elasticsearch-head]# vim /opt/elasticsearch-6.2.0/config/elasticsearch.yml http.cors.allow-headers: "Authorization" #前引号前，后引号后有空格[ES@localhost bin]$ ./elasticsearch -d

2、页面访问http://10.10.2.83:9100/?auth_user=elastic&auth_password=changemex-pack也可以创建一个权限组，然后创建用户，给这个用户分配权限。1.设置权限组http://10.10.2.83:9200/_xpack/security/role/logstash_writer/{ "cluster": ["manage_index_templates","monitor"], "indices": [    {     "names": [ "logstash-*","indexdb*" ],     "privileges":["write","delete","create_index"]    }  ]}2.给用户授权,这里的logstash_internal为用户名。http://10.10.2.83:9200/_xpack/security/user/logstash_internal{ "password" : "changeme", "roles" : [ "logstash_writer"], "full_name" : "Internal Logstash User"}如果不需要该插件，也可以通过非常简单的方式删除

[root@localhost bin]# pwd

/opt/elasticsearch-6.2.0/bin[root@localhost bin]# ./elasticsearch-plugin remove x-pack-> removing [x-pack]...-> preserving plugin config files [/opt/elasticsearch-6.2.0/config/x-pack] in case of upgrade; use --purge if not needed[root@localhost bin]#

---

• 内存配置修改

配置 Elasticsearch 堆大小时需要通过 $ES_HEAP_SIZE 环境变量应用两个规则：

不要超过可用 RAM 的 50%Lucene 能很好利用文件系统的缓存，它是通过系统内核管理的。如果没有足够的文件系统缓存空间，性能会受到影响。 此外，专用于堆的内存越多意味着其他所有使用 doc values 的字段内存越少。 不要超过 32 GB如果堆大小小于 32 GB，JVM 可以利用指针压缩，这可以大大降低内存的使用：每个指针 4 字节而不是 8 字节。

[justin@es-2-10-0005 bin]$ vim ../config/jvm.options#Xms represents the initial size of total heap space#Xmx represents the maximum size of total heap space-Xms16g-Xmx16g#################################################################Expert settings

OK