搭建ELK日志系统（一）

朱雀 2020-10-19 11:53 1103阅读 0赞

先安装elasticsearch

1、确保jdk已经安装，这里省略

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.0-linux-x86_64.tar.gz
    wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.0-linux-x86_64.tar.gz.sha512
    shasum -a 512 -c elasticsearch-7.5.0-linux-x86_64.tar.gz.sha512 
    tar -xzf elasticsearch-7.5.0-linux-x86_64.tar.gz
    cd elasticsearch-7.5.0/

先进入到除了root以外的目录，找一个你喜欢的目录（一般是放软件安装包的），按照这个步骤一步步的来安装，第三步的时候我遇到了 -bash: shasum: command not found 这个问题，通过执行 yum install perl-Digest-SHA 解决。

安装成功之后，启动es，进到es的安装目录，执行 ./bin/elasticsearch，报错：

future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/jdk1.8.0_171/jre] does not meet this requirement
    Exception in thread "main" java.lang.RuntimeException: starting java failed with [1]
    output:
    #
    # There is insufficient memory for the Java Runtime Environment to continue.
    # Native memory allocation (mmap) failed to map 986513408 bytes for committing reserved memory.
    # An error report file with more information is saved as:
    # logs/hs_err_pid21335.log
    error:
    Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)
    	at org.elasticsearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:118)
    	at org.elasticsearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:86)
    	at org.elasticsearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:59)
    	at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:92)

无法分配内存，cat  conf/jvm.options 里看到的 -Xms1g -Xmx1g，我把他们都改为256m，然后再启动，换了个报错信息：

future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/jdk1.8.0_171/jre] does not meet this requirement
    [2019-12-06T11:07:54,035][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [iZuf62rnwcgp54evv9uoo0Z] uncaught exception in thread [main]
    org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
    	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.5.0.jar:7.5.0]
    	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.5.0.jar:7.5.0]
    	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.5.0.jar:7.5.0]
    	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.5.0.jar:7.5.0]
    	at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.5.0.jar:7.5.0]
    	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.5.0.jar:7.5.0]
    	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.5.0.jar:7.5.0]
    Caused by: java.lang.RuntimeException: can not run elasticsearch as root
    	at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:105) ~[elasticsearch-7.5.0.jar:7.5.0]
    	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) ~[elasticsearch-7.5.0.jar:7.5.0]
    	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.5.0.jar:7.5.0]
    	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.5.0.jar:7.5.0]
    	... 6 more

这个是因为es默认不能用root用户启动，那下面我们去创建一个新的用户，和一个新的用户组

新增用户：

useradd elasticsearch  
passwd elasticsearch

新增用户组：

groupadd elasticsearch

然后我 su elasticsearch 切换用户，然后去执行 ./bin/elasticsearch 报错：

Error: Could not find or load main class org.elasticsearch.tools.java_version_checker.JavaVersionChecker

还记得我上面最开始的时候 第一个加红色的地方吗？一开始我是在root里安装es的，然后启动就会报这个错，所以我上面说不要在root目录里安装；然后我root用户执行

mv elasticsearch-7.5.0 /home/elasticsearch/ 命令把es包移到了home目录下，再启动es，报错：

future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/jdk1.8.0_171/jre] does not meet this requirement
    Exception in thread "main" java.nio.file.AccessDeniedException: /home/elasticsearch/elasticsearch-7.5.0/config/jvm.options
    	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
    	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    	at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
    	at java.nio.file.Files.newByteChannel(Files.java:361)
    	at java.nio.file.Files.newByteChannel(Files.java:407)
    	at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
    	at java.nio.file.Files.newInputStream(Files.java:152)
    	at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:62)

这个是因为没权限，我上面新增用户和用户组之后，没有给用户授权，root用户执行

chown -R elasticsearch:elasticsearch  /home/elasticsearch/ 即可，再次用elasticsearch用户启动es，成功。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70][]

新开一个窗口，执行curl http://localhost:9200/ 测试一下，出现下面json格式的信息，说明启动成功了

{
      "name" : "iZuf62rnwcgp54evv9uoo0Z",
      "cluster_name" : "elasticsearch",
      "cluster_uuid" : "liYwoOgGQrirka3bSYrtiw",
      "version" : {
        "number" : "7.5.0",
        "build_flavor" : "default",
        "build_type" : "tar",
        "build_hash" : "e9ccaed468e2fac2275a3761849cbee64b39519f",
        "build_date" : "2019-11-26T01:06:52.518245Z",
        "build_snapshot" : false,
        "lucene_version" : "8.3.0",
        "minimum_wire_compatibility_version" : "6.8.0",
        "minimum_index_compatibility_version" : "6.0.0-beta1"
      },
      "tagline" : "You Know, for Search"
    }

至此，ES安装并启动成功，

启动：一般是后台 启动  ./elasticsearch -p /tmp/elasticsearch-pid -d

停止：jps | grep Elasticsearch 或者 cat /tmp/elasticsearch-pid && echo 找到PID，然后 kill -SIGTERM  PID。

但是你有没有发现，这里只能用http://localhost:9200/访问，用服务器内网和外网都不能访问，如果我们想在浏览器里用外网访问该怎么办呢？

vim /home/elasticsearch/elasticsearch-7.5.0/config/elasticsearch.yml
    
    Network模块里
    
    network.host: 0.0.0.0
    network.publish_host: 你自己的服务器外网地址

重启，然后在阿里云安全组里打开9200这个端口，浏览器访问：http://外网:9200 即可访问

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 1][]

期间遇到的问题：可参考这个，我遇到的问题和这一样。。。

然后还需要安装es的可视化插件es-head

git clone git://github.com/mobz/elasticsearch-head.git  (没有git命令的话 yum install npm)
    cd elasticsearch-head
    npm install   (没有npm命令的话 yum install npm)
    npm run start

我是在npm install这一步出现错误的，错误信息如下：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 2][]

可以看到应该是没有bzip2命令，安装 yum install bzip2

再次执行npm install，成功

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 3][]

然后启动：npm run start

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 4][]

启动成功，测试一下 curl http://localhost:9100

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 5][]

是成功的，然后我们把阿里云安全组里加入 9100端口，浏览器里使用外网访问：

![20191209170347581.png][]

外网可以访问，但是右边集群健康值:是未连接的，我们重启下 es，没有报错，但是浏览器里我还是连接不上我的es，

后来百度查到可能是 跨域的问题，我在elasticsearch.yml里新增下面两条命令

#allow origin
    http.cors.enabled: true
    http.cors.allow-origin: "*"

重启es，打开浏览器ES控制面板，成功连接

![20191209173117123.png][]

至此为止，ES和ES的可视化插件就已经安装好了。

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70]: /images/1603108383037.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 1]: /images/1603108362937.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 2]: /images/1603108345489.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 3]: /images/1603108326228.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 4]: /images/1603108307518.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzMTAxNjc1_size_16_color_FFFFFF_t_70 5]: /images/1603108292704.png
[20191209170347581.png]: /images/1603108271765.png
[20191209173117123.png]: /images/1603108210276.png