At the end of November, we’ll be migrating the Sematext Logs backend from Elasticsearch to OpenSearch

S3 Archiving & Longer Retention for Logs

May 5, 2017

Table of contents

Need to have your recent logs searchable for your DevOps team(s) to troubleshoot issues, but also need to archive old logs for compliance reasons?  

Many organizations need that and Sematext Cloud makes that simple.  You can certainly simply pick a longer data retention option along with your plan and daily log volume.  Doing that makes all your logs searchable for a long time.  If, however, you simply need to store your older logs, so that you can retrieve them later on, but do not need to search them, you can use S3 Archiving option provided by Sematext Cloud.  

All you need to provide are Access Key ID, your Secret Access Key, and the name of the S3 bucket where you want your logs to be archived.

Sematext S3 Archiving

Logs are archived continuously in your bucket, organized by time:

sematext_[app-token-start]/[year]/[month]/[day]/[hour].  

Compression is optional, but if your log volume is high, we recommend you enable it to save on S3 costs.  Sematext will need s3:PutObject credentials on your bucket.  s3:GetObject and s3:DeleteObject are needed only temporarily, when setting up archiving to S3, and can be revoked right after that. See Sematext Logs S3 Archiving for more info.



 

Java Logging Basics: Concepts, Tools, and Best Practices

Imagine you're a detective trying to solve a crime, but...

Best Web Transaction Monitoring Tools in 2024

Websites are no longer static pages.  They’re dynamic, transaction-heavy ecosystems...

17 Linux Log Files You Must Be Monitoring

Imagine waking up to a critical system failure that has...