Yes. If your system is behind a firewall and cannot connect to the Internet to download anything, please contact Sematext to get the packages that have all dependencies included in them.
The config script has an option to provide the IP address from the config script:
bash ./config.py init --ip ADDR
The config script has an option to remove a node from the cluster, but not for the primary node:
bash ./config.py remove --id
The ID of the node can be found by running the status command:
bash ./config.py status
The cluster will not work without a primary node.To get the cluster up and running some manual steps will have to be executed.We plan to automate this in the very near future to to make things easy in these rare cases.
At the moment, the cluster name is hardcoded to sematext-es.
For sending emails, a local SMTP server is used. To use a different server changes need to be done in /opt/sematext/conf/common/common.onpremises.properties
properties email.host=email_server_addr email.port=email_server_port email.user=email_username email.password=email_passowrd email.from=Some Name
email.smtp.auth=true / false email.smtp.starttls.enable=true / false email.smtp.ssl.enable=true / false email.smtp.socketFactory.class=javax.net.ssl.SSLSocketFactory / javax.net.SocketFactory
We recommend to use a load balancer that can also handle the HTTPS and SSL offloading.
When running out of space because of Elasticsearch, please add more Data nodes. Data will be distributed between all nodes. Please keep in mind that this does not happen immediately and plan in advance. Do not add more than one node at the same time. Depending how busy the cluster is, it can take some time to stabilize. You can use SPM to check when the cluster has stabilized. Check that cluster stateis Green before and after adding new nodes.
When running out of space because of Kafka, please add more Master nodes. Data will be distributed between all nodes. Please keep in mind that this does not happen immediately and plan in advance.Do not add more than one node at the same time. Depending how busy the cluster is, it can take some time to stabilize. You can use SPM to check when the cluster has stabilized.There is also a setting that controls how long logs are kept before being discarded in /opt/sematext/kafka/config. The default value is:
When using “http”://localhost” instead of
The I/O scheduler should be set to NOOPat runtime:
bash echo noop | sudo tee /sys/block/xvdh/queue/scheduler (xvdh is the SSD drive)persistent: add elevator=noop to the "kernel" like of /etc/grub.conf
The volume of per-server metrics or logs, and log event sizes can vary a lot, so it is nearly impossible to give good estimates. However, SPM monitors itself to help with capacity planning. Disk usage tell you how much disk is being used and how much is left. CPU usage tells you whether CPU is being maxed out or not. To expand, you can either add more VMs/nodes, or give the existing VMs more CPU or disk.
Kibana indices are backed up daily in
bash /opt/sematext/script/es-index.py --location /opt/sematext/data/kibana/
In case you are having issues with Sematext Apps, you can create diagnostics package on affected nodes by running:
bash sudo /opt/sematext/script/diag.sh
The resulting package will contain all relevant info needed for our investigation.