Elasticsearch

👉 This writeup is part of “Elasticsearch Deploy Docs” series. Important Elasticsearch Configuration 📄 Official Docs Mainly 3 configuration files elasticsearch.yml - Elasticsearch config jvm.options - Elasticsearch JVM settings config log4j2.properties - Elasticsearch logging config Environment Variables export the ES_PATH_CONF etc/default/elasticsearch (Sourced environment variables from. Recommended) Settings 📄 Official Docs Before going to production, it is recommended go through be below elasticsearch configs. Refer sample_config directory for configuration Configuration Description Configuration Reference Path settings Log and data config Refer here Cluster name Cluster name Refer here Node name Node name Refer here Network host IP address that elasticsearch bind on Refer here Discovery settings Cluster discovery and initial master config Refer here Heap size JVM heap memory configuration Recommended heap size should be half of system memory. Make sure min and max heap memory same value. Refer here Heap dump path Heap dump location path config Default config is sufficient. Refer here GC logging Garbage collection logging configuration Default config is sufficient. Refer here Temp directory Configure private temporary directory that Elasticsearch uses is excluded from periodic cleanup Important System Configuration 📄 Offical Docs ...

👉 This writeup is part of “Elasticsearch Deploy Docs” series Install 📄 Office Docs Hardware Requirement 📄 Offical Docs Resource Minimum Recommended Memory 16 GB 64 GB CPU 8 Cores 16 Disk Depends Depends JDK Installation Pick JVM compatibility version with elasticsearch from here Install OpenJDK from here Download and install JDK 11 (Another guide here) $ apt-get install openjdk-11-jdk -y $ java -version openjdk version "11.0.6" 2020-01-14 OpenJDK Runtime Environment (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1) OpenJDK 64-Bit Server VM (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1, mixed mode, sharing) Elasticsearch Installation Download latest elasticsearch from here (As of today the latest version is 7.6.2) Recommended to download/install package via .dep or PPA which postscripts creates user, groups and adds under systemd Install via apt-get from here ...

👉 This writeup is part of “Elasticsearch Deploy Docs” series Rolling Upgrade Elasticsearch 📄 Official Docs ⚠️ A rolling upgrade allows an Elasticsearch cluster to be upgraded one node at a time so upgrading does not interrupt service As of now, the current latest version of elasticsearch is v7.7.1. Below procedure is for rolling upgrade from 7.6.2=>7.7.1. 1. Divide the cluster into 2 groups Example node names carbon-x a. Non master-eligible nodes carbon-2 carbon-3 b. Master-eligible nodes carbon-1 Upgrade order (Important!) ...

👉 This writeup is part of “Elasticsearch Deploy Docs” series Elastalert 📄 Official docs 📂 Official project Repo Elastalert is developed by Yelp written in python, queries docs in elasticsearch and send alerts depends on the rules. Since Elastalert is not part of Elasticsearch plugin, we can install it where ever we want to. Installation ❗ Refer elasticsearch-deploy-notes/elastalert for example config $ sudo apt-get install python3-pip $ sudo pip3 install elastalert $ sudo pip3 install -U PyYAML $ mkdir -p /opt/elastalert/rules ## Copy alert rules yaml files and config file to /opt/elastalert and /opt/elastalert/rules accordingly from this repo Recommended to create index in elasticsearch for elastalert to store metadata $ elastalert-create-index Elastic Version: 7.7.0 Reading Elastic 6 index mappings: Reading index mapping 'es_mappings/6/silence.json' Reading index mapping 'es_mappings/6/elastalert_status.json' Reading index mapping 'es_mappings/6/elastalert.json' Reading index mapping 'es_mappings/6/past_elastalert.json' Reading index mapping 'es_mappings/6/elastalert_error.json' New index elastalert_status created Done! Test rules in case if it is needed $ elastalert-test-rule --config /opt/elastalert/config.yaml /opt/elastalert/rules/heartbeat_checks.yml Postfix Gmail SMTP In oder to use Gmail as SMTP, you need to enable 2-Factor authentication and generate app password ...