share share share 

Logagent Configuration Parameters for Containers

The Logagent container can be configured through the following environment variables:

• REGION: Sematext Cloud region US or EU (default: US). The receiver URL will be set to EU/US default values. When using REGION, you don't need to set LOGS_RECEIVER_URL (see below).

• LOGS_RECEIVER_URL: The URL of your Elasticsearch Endpoint (defaults to Sematext Cloud US https://logsene-receiver.sematext.com).

  • For Sematext Europe use https://logsene-receiver.eu.sematext.com.
  • For Elasticsearch https://elasticserch-server-name:9200.

• LOGS_TOKEN: The index to ship logs to. For Sematext use the Logs App Token.

• LOGAGENT_ARGS: Additional command line arguments for Logagent

  LOGAGENT_ARGS="-n httpd"

  to specify a log source name or

  LOGAGENT_ARGS="-u 514"

  to act as syslog server. Please refer to Logagent command line arguments in the Logagent Documentation

• LOG_GLOB: Semicolon-separated list of file globs

  LOG_GLOB=/mylogs/**/*.log;/var/log/**/*.log
  

  Mount your server log files into the container using a Docker volume e.g.

  -v /var/log:/mylogs
  

  This feature is developed with the Glob module. Here's a guide on how to use Glob patterns.

• -v /var/run/docker.sock:/var/run/docker.sock - Collect container logs by mounting the docker socket (mandatory)

Configuration Parameters

Mandatory Parameters

Parameter / Environment variable	Description
LOGS_TOKEN	Logs Token enables logging to Sematext Cloud, see logging specific parameters for filter options and Log Routing section to route logs from different containers to separate Logs Apps (indices)
-v /var/run/docker.sock	Path to the docker socket

Optional Parameters

Parameter / Environment variable	Description
REGION	Sematext Cloud region US or EU (default: US). The receiver URL will be set to EU/US default values. When using REGION, you don't need to set LOGS_RECEIVER_URL (see below).
LOG_GLOB	Semicolon-separated list of file globs (e.g. /var/log//*.log;/mylogs//*.log) to collect log files from the host, assuming the log files are mounted to /mylogs using Docker -v /var/logs:/mylogs
LOGAGENT_ARGS	Additional command line arguments for Logagent (e.g. LOGAGENT_ARGS="-n httpd" to specify a log source name or LOGAGENT_ARGS="-u 514" to act as syslog server)
HTTPS_PROXY	URL for a proxy server (behind firewalls)
LOGS_RECEIVER_URL	URL for bulk inserts into Sematext Cloud. Required for Sematext Enterprise (local IP:PORT) or Sematext Cloud Europe: https://logsene-receiver.eu.sematext.com
LOGS_RECEIVER_URLS	Specify multiple receiver URLs for bulk inserts into Sematext Cloud. Required for Sematext Enterprise (local IP:PORT) or Sematext Cloud Europe: https://logsene-receiver.eu.sematext.com
JOURNALD_UPLOAD_PORT	Port number for the collection of journald logs, forwarded by systemd-journal-upload.service. Equals to Logagent argument --journald PORT.
SYSTEMD_UNIT_FILTER	A regular expression to filter journald logs by systemd unit name, e.g. "ssh.service|docker.service". The default value is ".*".
CONFIG_FILE	Path to the configuration file, containing environment variables key=value. Default value: /run/secrets/logagent. Create a secret with docker secret create logagent ./logagent.cfg. Start Logagent with `docker service create --mode global --secret logagent --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock sematext/logagent
LA_CONFIG	Point to the location of a logagent config file. E.g. LA_CONFIG=/etc/sematext/logagent.conf
LA_CONFIG_OVERRIDE	Ignores env vars for LOGS_TOKEN, REGION, and LOGS_RECEIVER_URL. E.g. LA_CONFIG_OVERRIDE=true
--privileged	The parameter might be helpful when Logagent could not start because of limited permission to connect and write to the Docker socket /var/run/docker.sock. The privileged mode is a potential security risk, we recommend to enable the appropriate security. Please read about Docker security: https://docs.docker.com/engine/security/security/

Docker Logs Parameters

Parameter / Environment variable	Description
TAGGING_LABELS	A list of docker label names or environment variable names to tag container logs. Supporting wildcards. Default value: com.docker.*,io.kubernetes.*,annotation.io.*
IGNORE_LOGS_PATTERN	Filter logs with a regular expression. IGNORE_LOGS_PATTERN=healthcheck|ping This will match log lines that contain "healthcheck" or "ping" in the message, and drop them. Only add the regular expression without forward slashes.
LOGSENE_ENABLED_DEFAULT	Enables log collection for containers having no explicit label/environment variable LOGSENE_ENABLED set. Default value: true. See section Log Routing.

Whitelist Containers for Logging

Parameter / Environment variable	Description
MATCH_BY_NAME	Regular expression to whitelist container names. MATCH_BY_NAME=.*nginx.* This will match any container name that contains "nginx". Only add the regular expression without forward slashes.
MATCH_BY_IMAGE	Regular expression to whitelist image names. MATCH_BY_IMAGE=.*nginx.* This will match any image that contains "nginx". Only add the regular expression without forward slashes.

Blacklist Containers

Parameter / Environment variable	Description
SKIP_BY_NAME	Regular expression to blacklist container names for logging. SKIP_BY_NAME=.*nginx.* This will match any container name that contains "nginx". Only add the regular expression without forward slashes.
SKIP_BY_IMAGE	Regular expression to blacklist image names for logging. SKIP_BY_IMAGE=.*nginx.* This will match any image that contains "nginx". Only add the regular expression without forward slashes.

Set Log Patterns

Logagent supports various log formats defined in patterns.yml file. The Log Parser Patterns can be customized by proving your YAML file.

Parameter / Environment variable	Description
PATTERNS_URL	Load pattern.yml via HTTP e.g. PATTERNS_URL=https://myserver/patterns.yml
LOGAGENT_PATTERNS	Pass patterns.yml via env. variable e.g. LOGAGENT_PATTERNS="$(cat ./patters.yml)"
LOGAGENT_PATTERNS_BASE64	Set to "true" if the LOGAGENT_PATTERNS patterns file you are passing in via env. variable is base64 encoded e.g LOGAGENT_PATTERNS_BASE64="$(cat ./patterns.yml | base64)" . Useful if your patterns file is not getting set properly due to shell interpretation or otherwise.
PATTERN_MATCHING_ENABLED	Activate logagent-js parser, default value is true. To disable the log parser set the value to false. This could increase the throughput of log processing for nodes with a very high log volume.
-v /patterns.yml:/etc/logagent/patterns.yml	Mount a patterns file to customize patterns for log parsing

Other Options

Parameter / Environment variable	Description
-v /tmp:/log-buffer	Directory to store logs, in a case of a network or service outage. Docker Agent deletes these files after successful transmission.
GEOIP_ENABLED	Enables GeoIP lookups in the log parser, default value: "false"
GEOIP_FIELD	The field to perform geo IP lookup e.g. GEOIP_FIELD="client_ip"
MAXMIND_LICENSE_KEY	Your MaxMind license key
MAXMIND_DB_DIR	Directory for the Geo-IP lite database, must end with /. Storing the DB in a volume could save downloads for updates after restarts. Using /tmp/ (ramdisk) could speed up Geo-IP lookups (requires add. ~30 MB main memory).
REMOVE_FIELDS	Removes fields from parsed/enriched logs. E.g. REMOVE_FIELDS=password,creditCardNo