At the end of November, we’ll be migrating the Sematext Logs backend from Elasticsearch to OpenSearch

How to Periodically Extract Webpage Performance Metrics from Browser API

September 7, 2023

Table of contents

To ensure a good end user experience, smart businesses periodically gather performance data from their websites. They measure the responsiveness and speed of their services to ensure fast and reliable websites. Having a responsive and fast website improves companies’ conversion rates, keeps their reputation intact, and helps increase traffic and revenue. Website monitoring applications help determine whether the website achieves the desired response times and uptimes. You can extract performance metrics from your browser APIs, chart them, and create alert rules to take immediate action when your website’s performance drops below the desired values.

Synthetic monitoring tools let you do these things automatically. For more information about Synthetic monitoring applications and different use cases for extracting numbers and metrics from web pages and APIs, see how to extract numerical data from a Web Page for Dashboarding and Alerting.

How to Extract Performance Metrics Using Sematext

With Sematext Synthetics you can actively monitor APIs, websites, and user journeys/click flows from multiple locations around the globe. With Browser Monitors you can write scripts that simulate user journey across multiple pages. You can call websites and APIs at a certain interval, monitor availability, average API response time, extract metrics from responses or web page content, chart them, and define alert rules.

In this example, we will extract the heap usage and script duration metric. We will do that by writing a user journey script that opens Youtube.com and fetches these two metrics from the browser itself. Note that here we will be getting the numerical value from the browser via one of its APIs, but we could also be extracting numerical values from the content of a web page itself or from the response of some REST API that returns JSON that we could be fetching instead of Youtube.com. Next, we’ll chart them with Sematext Dashboards and define threshold alert rules. We will also create an anomaly alert that will notify us when an extracted performance metric deviates from the baseline.

Extracting Metrics via User Journey Scripts

First, you need to create a Synthetics App.

Then click to create a Browser Monitor to write a journey script that extracts heap usage and duration for a user session.

You can choose different locations you want your script to run from, interval and the device type to simulate the user journey, but if what you want to extract doesn’t depend on where the request came from, you should probably pick only one location to keep your costs down.

See best practices tips to make the best use of synthetic monitoring and keep your costs minimal.

Then add the user journey script that extracts the desired metrics.

After your monitor starts running, you will see the general performance metrics such as response time, availability and failed runs over time by looking at the Overview page of your Monitor.

Visualizing Custom Metrics

With Sematext Dashboards, you can visualize the heap usage and script duration metrics you’ve extracted.

Navigate to Dashboards and select a visualization type.

In this example, we’ll visualize the extracted heap size metric with a TimeSeries chart.

Receiving Periodic Email Reports

You can always add more visualization to your Dashboard and use Scheduled Reports to periodically send the Dashboard via email to yourself or your teammates.

Creating Alert Rules

Dashboarding extracted metrics creates alert rules to be notified when the extracted metric exceeds the maximum threshold or deviates from the baseline and defines the priority for the alerts.

With Sematext Notification Hooks, you can choose from various platforms to receive alert notifications. Also, third-party integrations with various incident management platforms give you the ability to automatically create incidents, assign them to the responsible team, and define priorities to address issues or act on detected changes in the monitored APIs and websites.

Sematext All-In-One Platform Advantages

Sematext Cloud is an all-in-one platform, meaning on top of monitoring your endpoints, and websites, you can track your resource usage, metrics, logs of services, and hosts that are running these endpoints. Seeing all this in a single solution helps you correlate, and detect interdependencies between components when you work with distributed applications.

Imagine you detect an issue in your application’s endpoint. To drill down to the root cause you might want to check the metrics or logs of the service that is running that application.

When you start receiving alerts from your endpoints based on conditions you’ve set, you can check the resource usage metrics of the server that is hosting your application. This will let you check if the problem is caused by an unexpected increase in network traffic, for example. You can check the CPU or RAM usage within the same time period during the traffic growth, see if it is just a temporary spike or if it has been going on for a while, and decide to scale up your systems.

If you’re interested, we’ve also created a use case specifically on How to Extract Numerical Data from a Web Page for Dashboarding and Alerting.

Summary

In conclusion, Sematext Cloud provides a powerful way of extracting metrics from webpages, API endpoints, or browser APIs, charting them, and setting up alerting rules. You can track changes in another website or API or your own. When you monitor your own websites and APIs, and combine these with metrics and logs of services that are hosting these applications, you gain full stack visibility into your whole infrastructure.

Java Logging Basics: Concepts, Tools, and Best Practices

Imagine you're a detective trying to solve a crime, but...

Best Web Transaction Monitoring Tools in 2024

Websites are no longer static pages.  They’re dynamic, transaction-heavy ecosystems...

17 Linux Log Files You Must Be Monitoring

Imagine waking up to a critical system failure that has...