Skip to content
share

Logagent Configuration Parameters for Containers

The Logagent container can be configured through the following environment variables:

  • REGION: Sematext Cloud region US or EU (default: US). The receiver URL will be set to EU/US default values. When using REGION, you don't need to set LOGS_RECEIVER_URL (see below).
  • LOGS_RECEIVER_URL: The URL of your Elasticsearch Endpoint (defaults to Sematext Cloud US https://logsene-receiver.sematext.com).

    • For Sematext Europe use https://logsene-receiver.eu.sematext.com.
    • For Elasticsearch https://elasticserch-server-name:9200.
  • LOGS_TOKEN: The index to ship logs to. For Sematext use the Logs App Token.

  • LOGAGENT_ARGS: Additional command line arguments for Logagent

    LOGAGENT_ARGS="-n httpd"
    to specify a log source name or
    LOGAGENT_ARGS="-u 514"
    to act as syslog server. Please refer to Logagent command line arguments in the Logagent Documentation

  • LOG_GLOB: Semicolon-separated list of file globs

    LOG_GLOB=/mylogs/**/*.log;/var/log/**/*.log
    

    Mount your server log files into the container using a Docker volume e.g.

    -v /var/log:/mylogs
    

    This feature is developed with the Glob module. Here's a guide on how to use Glob patterns.

  • -v /var/run/docker.sock:/var/run/docker.sock - Collect container logs by mounting the docker socket (mandatory)

Configuration Parameters

Mandatory Parameters

Parameter / Environment variable Description
LOGS_TOKEN Logs Token enables logging to Sematext Cloud, see logging specific parameters for filter options and Log Routing section to route logs from different containers to separate Logs Apps (indices)
-v /var/run/docker.sock Path to the docker socket

Optional Parameters

Parameter / Environment variable Description
REGION Sematext Cloud region US or EU (default: US). The receiver URL will be set to EU/US default values. When using REGION, you don't need to set LOGS_RECEIVER_URL (see below).
LOG_GLOB Semicolon-separated list of file globs (e.g. /var/log//*.log;/mylogs//*.log) to collect log files from the host, assuming the log files are mounted to /mylogs using Docker -v /var/logs:/mylogs
LOGAGENT_ARGS Additional command line arguments for Logagent (e.g. LOGAGENT_ARGS="-n httpd" to specify a log source name or LOGAGENT_ARGS="-u 514" to act as syslog server)
HTTPS_PROXY URL for a proxy server (behind firewalls)
LOGS_RECEIVER_URL URL for bulk inserts into Sematext Cloud. Required for Sematext Enterprise (local IP:PORT) or Sematext Cloud Europe: https://logsene-receiver.eu.sematext.com
LOGS_RECEIVER_URLS Specify multiple receiver URLs for bulk inserts into Sematext Cloud. Required for Sematext Enterprise (local IP:PORT) or Sematext Cloud Europe: https://logsene-receiver.eu.sematext.com
JOURNALD_UPLOAD_PORT Port number for the collection of journald logs, forwarded by systemd-journal-upload.service. Equals to Logagent argument --journald PORT.
SYSTEMD_UNIT_FILTER A regular expression to filter journald logs by systemd unit name, e.g. "ssh.service|docker.service". The default value is ".*".
CONFIG_FILE Path to the configuration file, containing environment variables key=value. Default value: /run/secrets/logagent. Create a secret with docker secret create logagent ./logagent.cfg. Start Logagent with `docker service create --mode global --secret logagent --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock sematext/logagent
LA_CONFIG Point to the location of a logagent config file. E.g.
LA_CONFIG=/etc/sematext/logagent.conf
LA_CONFIG_OVERRIDE Ignores env vars for LOGS_TOKEN, REGION, and LOGS_RECEIVER_URL. E.g.
LA_CONFIG_OVERRIDE=true
--privileged The parameter might be helpful when Logagent could not start because of limited permission to connect and write to the Docker socket /var/run/docker.sock. The privileged mode is a potential security risk, we recommend to enable the appropriate security. Please read about Docker security: https://docs.docker.com/engine/security/security/

Docker Logs Parameters

Parameter / Environment variable Description
TAGGING_LABELS A list of docker label names or environment variable names to tag container logs. Supporting wildcards. Default value:
com.docker.*,io.kubernetes.*,annotation.io.*
IGNORE_LOGS_PATTERN Filter logs with a regular expression.
IGNORE_LOGS_PATTERN=healthcheck|ping
This will match log lines that contain "healthcheck" or "ping" in the message, and drop them. Only add the regular expression without forward slashes.
LOGSENE_ENABLED_DEFAULT Enables log collection for containers having no explicit label/environment variable LOGSENE_ENABLED set. Default value: true. See section Log Routing.

Whitelist Containers for Logging

Parameter / Environment variable Description
MATCH_BY_NAME Regular expression to whitelist container names.
MATCH_BY_NAME=.*nginx.*
This will match any container name that contains "nginx". Only add the regular expression without forward slashes.
MATCH_BY_IMAGE Regular expression to whitelist image names.
MATCH_BY_IMAGE=.*nginx.*
This will match any image that contains "nginx". Only add the regular expression without forward slashes.

Blacklist Containers

Parameter / Environment variable Description
SKIP_BY_NAME Regular expression to blacklist container names for logging.
SKIP_BY_NAME=.*nginx.*
This will match any container name that contains "nginx". Only add the regular expression without forward slashes.
SKIP_BY_IMAGE Regular expression to blacklist image names for logging.
SKIP_BY_IMAGE=.*nginx.*
This will match any image that contains "nginx". Only add the regular expression without forward slashes.

Set Log Patterns

Logagent supports various log formats defined in patterns.yml file. The Log Parser Patterns can be customized by proving your YAML file.

Parameter / Environment variable Description
PATTERNS_URL Load pattern.yml via HTTP e.g.
PATTERNS_URL=https://myserver/patterns.yml
LOGAGENT_PATTERNS Pass patterns.yml via env. variable e.g.
LOGAGENT_PATTERNS="$(cat ./patters.yml)"
LOGAGENT_PATTERNS_BASE64 Set to "true" if the LOGAGENT_PATTERNS patterns file you are passing in via env. variable is base64 encoded e.g
LOGAGENT_PATTERNS_BASE64="$(cat ./patterns.yml | base64)"
. Useful if your patterns file is not getting set properly due to shell interpretation or otherwise.
PATTERN_MATCHING_ENABLED Activate logagent-js parser, default value is true. To disable the log parser set the value to false. This could increase the throughput of log processing for nodes with a very high log volume.
-v /patterns.yml:/etc/logagent/patterns.yml Mount a patterns file to customize patterns for log parsing

Other Options

Parameter / Environment variable Description
-v /tmp:/log-buffer Directory to store logs, in a case of a network or service outage. Docker Agent deletes these files after successful transmission.
GEOIP_ENABLED Enables GeoIP lookups in the log parser, default value: "false"
GEOIP_FIELD The field to perform geo IP lookup e.g.
GEOIP_FIELD="client_ip"
MAXMIND_LICENSE_KEY Your MaxMind license key
MAXMIND_DB_DIR Directory for the Geo-IP lite database, must end with /. Storing the DB in a volume could save downloads for updates after restarts. Using /tmp/ (ramdisk) could speed up Geo-IP lookups (requires add. ~30 MB main memory).
REMOVE_FIELDS Removes fields from parsed/enriched logs. E.g.
REMOVE_FIELDS=password,creditCardNo