Sematext

Pygmalios helps companies monitor how customers and staff interact in real-time. Our retail analytics platforms tracks sales, display conversions, customers and staff behavior to deliver better service, targeted sales, faster check-outs and the optimal amount of staffing for a given time and location.

What were your team’s devops challenges and pain points prior to using Sematext?

We have decided to use the architecture based on microservices for our analytics platform. We have a complex system of Apache Spark, Apache Kafka, Cassandra and InfluxDB databases, Node.js backend and JavaScript frontend applications where every service has its own single responsibility which makes them easy to scale. We run them mostly in Docker containers apart from Spark and Cassandra. So keeping logging and monitoring in the same place and integration with custom metrics is crucial.

What were your requirements and considerations for monitoring and / or logging solutions?

As we use a microservices architecture we have a lot of applications across multiple servers so we need to orchestrate them. We also have many physical sensors outside in the retail stores which are our data sources. In the end, there are a lot of things we have to think about including correlations between them:

• Server monitoring - basic metrics for the hardware layer such as memory, CPU and network.
• Docker monitoring - in the software layer we want to know whether our applications inside Docker containers are running properly.
• Kafka, Spark and Cassandra monitoring - these are our core services, so their monitoring is a must.
• Sensors monitoring - sensors are deployed outside in the retail stores. We have to monitor them as well and use custom metrics.
• Notifications - we want to be alerted whenever anything breaks.
• Centralized logging - store all logs in one place, combine them with hardware usage and then analyze anomalies.

What made you choose Sematext?

Sematext had almost everything what we needed including a great support team.

What problems did our monitoring and logging solution solve for you?

Monitoring and logging for our application stack except for device monitoring.

Please share some insight into your implementation process and transition to Sematext.

We use Docker containers for our applications so integration was very easy - we just needed to run a new Sematext Docker container. Also integrating other applications was easy as Sematext supports them.
We have started with Sematext very early so we didn’t have to use historical logs or metrics.

What was the outcome and which product feature(s) do you find most helpful?

Combining server monitoring with Docker container monitoring and logging.

Anything else you would like to add?

It would be great to have support for IoT device monitoring, but this is very custom request and we have built our own platform for this.