Elastic Container Service (ECS) Logs Integration
By configuring AWS Firelens for AWS ECS on AWS Fargate/EC2, you can forward all your container logs to Sematext and get insight into your whole AWS Elastic Container Service (ECS) cluster in one place!
ECS Logs Quick Start¶
The ECS Logs integration collects logs from ECS Tasks and Services running in:
- EC2 Container Instances
- Fargate
To collect logs from EKS, see the Kubernetes integration.
First of all create an AWS ECS Logs App.
AWS ECS on AWS Fargate/EC2 With FireLens¶
You can forward logs from containers running in AWS ECS on AWS Fargate/EC2 to Sematext with the help of FireLens
- AWS FireLens -
awsfirelens
With Firelens you can route logs to another AWS service, like Firehose, or use Fluentd or Fluent Bit. AWS provides the image for Fluentd / Fluent Bit. You need to configure the output module to use http and send the logs to Sematext.
1. Enable FireLens¶
In the ECS Task Definition, check a checkbox called Enable FireLens integration. Choose Fluent Bit and AWS will populate the image name for you.
AWS will add an additional container called log_router
to the list of containers in your Task Definition.
2. Configure AWS Metadata for your Log Driver¶
Make sure you have this section in the log_router
container configuration.
"firelensConfiguration": { "type": "fluentbit", "options": { "enable-ecs-log-metadata": "true" } }
3. Configure the FireLens Log Driver¶
Next, in the same Task Definition but for your own container (not the log_router
), you configure the logConfiguration
like this, where 9c63d337-xxxx-xxxx-xxxx-abcc87342d47
is your LOGS_TOKEN
:
"logConfiguration": { "logDriver": "awsfirelens", "options": { "Format": "json", "Header_tag": "sourceName", "compress": "gzip", "Port": "443", "Host": "logs-ecs-receiver.sematext.com", "TLS": "on", "URI": "/9c63d337-xxxx-xxxx-xxxx-abcc87342d47", "Match": "*", "Name": "http" } }
Note: If you are using the EU region of Sematext you should set the Host like this:
"Host": "logs-ecs-receiver.eu.sematext.com"
Optionally, you can manually set the sourceName
of your logs. Instead of "Header_tag": "sourceName"
you can set "Header": "sourceName <SOURCE>"
if you want to parse certain types of logs. For example, "Header": "sourceName nginx"
would parse Nginx logs. Here's an example:
"logConfiguration": { "logDriver": "awsfirelens", "options": { "Format": "json", "Header": "sourceName nginx", "compress": "gzip", "Port": "443", "Host": "logs-ecs-receiver.sematext.com", "TLS": "on", "URI": "/9c63d337-xxxx-xxxx-xxxx-abcc87342d47", "Match": "*", "Name": "http" } }