Skip to content
share

Express.js Monitoring Integration

This is the Express.js monitoring and logging agent for Sematext, written entirely in Node.js without CPU and memory overhead. It's easy to install and require in your source code.

Sematext Express.js Agent Quick Start

This lightweight, open-source Express.js monitoring agent collects Node.js process and performance metrics and sends them to Sematext. It is available as an npm package that can be added to Node.js source code like any other npm module.

First you install the npm module.

# Terminal
npm i sematext-agent-express

Configure Environment

Make sure to configure your ENVIRONMENT variables before adding sematext-agent-express. You can do this either by exporting the variables to your environment or by using dotenv. We suggest you use dotenv.

Export env vars

If you are using the US region of Sematext Cloud:

export REGION=US
export MONITORING_TOKEN=<YOUR_MONITORING_TOKEN>
export LOGS_TOKEN=<YOUR_LOGS_TOKEN>
export INFRA_TOKEN=<YOUR_INFRA_TOKEN>

Note: The US region is set by default, so you do not need to add an environment variable at all.

If you are using the EU region of Sematext Cloud:

export REGION=EU
export MONITORING_TOKEN=<YOUR_MONITORING_TOKEN>
export LOGS_TOKEN=<YOUR_LOGS_TOKEN>
export INFRA_TOKEN=<YOUR_INFRA_TOKEN>

Use dotenv

npm i dotenv

Create a .env file in the root of your project.

Add this code if you are using the US region of Sematext Cloud:

REGION=US
MONITORING_TOKEN=<YOUR_MONITORING_TOKEN>
LOGS_TOKEN=<YOUR_LOGS_TOKEN>
INFRA_TOKEN=<YOUR_INFRA_TOKEN>

Note: The US region is set by default, so you do not need to add an environment variable at all.

Add this code if you are using the EU region of Sematext Cloud:

REGION=EU
MONITORING_TOKEN=<YOUR_MONITORING_TOKEN>
LOGS_TOKEN=<YOUR_LOGS_TOKEN>
INFRA_TOKEN=<YOUR_INFRA_TOKEN>

Configure Agent

Make sure to load the environment variables at the top of your JavaScript entry point file. Then require sematext-agent-express.

The Agent has 3 parts:

  • stMonitor - Monitors metrics and sends to Sematext Monitoring
  • stLogger - A logger based on winston, that will send logs directly to Sematext Logs
  • stHttpLoggerMiddleware - Express.js middleware function that will send all HTTP endpoint logs to Sematext Logs

Usage

// Load env vars
require('dotenv').config()

// require all agents
const {
  stMonitor,
  stLogger,
  stHttpLoggerMiddleware
} = require('sematext-agent-express')

// Start monitoring metrics
stMonitor.start()

// ...

// At the top of your routes add the stHttpLoggerMiddleware to send HTTP logs to Sematext
const express = require('express')
const app = express()
app.use(stHttpLoggerMiddleware)

// ...

// Use the stLogger to send all types of logs directly to Sematext
app.get('/api', (req, res, next) => {
  stLogger.info('Hello World.')
  stLogger.error('Some error.')
  res.status(200).send('Hello World.')
})

You can use all parts of the Agent of use them separately. It's all up to you.

The Sematext Express.js Agent will start collecting dozens of key metrics and logs right away, and start showing you the performance and health of your Express.js applications immediately.

Collected Express.js Metrics

The Sematext Express.js Agent collects the following metrics.

Operating System

  • CPU usage
  • CPU load
  • Memory usage

Process Memory Usage

  • Released memory between garbage collection cycles
  • Process heap size
  • Process heap usage

Process Count

  • Number of master processes
  • Number of child processes

Process CPU Usage

  • CPU usage per process
  • CPU usage per PID

Process RSS Usage

  • RSS usage per process
  • RSS usage per PID

Process Uptime

  • Process Uptime per process
  • Process Uptime per PID

Process Thread Count

  • Number of threads per process
  • Number of threads per PID

Worker Processes (cluster module)

  • Worker count
  • Event loop latency per worker

Event Loop

  • Maximum event loop latency
  • Minimum event loop latency
  • Average event loop latency

Garbage Collection

  • Time consumed for garbage collection
  • Counters for full garbage collection cycles
  • Counters for incremental garbage collection cycles
  • Released memory after garbage collection

HTTP Server Stats

  • Request count
  • Request rate
  • Response time
  • Request/Response content-length
  • Error rates (total, 3xx, 4xx, 5xx)

Collected Express.js Logs

The Sematext Express.js Agent collects the following logs for every HTTP request.

Structured HTTP logs

Here's an example of what a structured HTTP log looks like:

timestamp message severity host ip method url status content length response time
2019-12-23T21:06:36.615Z your_message info/debug/error/warn your_hostname 127.0.0.1 GET/PUT/POST/DELETE /api/v1 2xx/3xx/4xx/5xx in bytes in ms

Displayed in JSON format:

{
  "@timestamp": "2019-12-23T21:06:36.615Z",
  "message": "HTTP LOG",
  "severity": "info",
  "host": "ip-172-31-29-198",
  "ip": "172.31.29.198",
  "method": "GET",
  "url": "/v1/users",
  "status": "200",
  "contentLength": "-",
  "responseTime": "18.403"
}

Using this data, you can create custom dashboards displaying detailed data about all HTTP requests hitting your Express.js application.

Running the Sematext Express.js Agent in Production

If you want to have a reliable application with high uptime, make sure to use the cluster module, with the addition of running the Node.js process with Systemd. This will ensure restarts when your application fails and make rolling updates much easier.

Use the cluster module to run Node.js

To make use of the full power of your server, you should run an instance of your Node.js application on each CPU core. The cluster module makes this easier than ever. Create another file called cluster.js.

// cluster.js

const cluster = require('cluster')
const numCPUs = require('os').cpus().length
const app = require('./app')
const port = process.env.PORT || 3000

const masterProcess = () => Array.from(Array(numCPUs)).map(cluster.fork)
const childProcess = () => app.listen(port)
if (cluster.isMaster) masterProcess()
else childProcess()
cluster.on('exit', (worker) => cluster.fork())

Now you can run your app with:

node cluster.js

The cluster will spin up a master process with a dedicated process ID and run numCPUs number of worker processes. They will be load balanced in a round-robin fashion from the master process.

This is not all, you should also make sure to run your Node.js application with Systemd to make it a system service and run automatically on startup and restart itself if it fails.

Set up Node.js with Systemd

The service files for the things that systemd controls all live under the directory path

/lib/systemd/system

Create a new file there:

sudo vim /lib/systemd/system/app.service

And add this piece of code:

# /lib/systemd/system/app.service

[Unit]
Description=app.js - running your Node.js app as a system service
Documentation=https://yourwebsite.com
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/bin/node /absolute/path/to/your/project/app.js
Restart=on-failure

[Install]
WantedBy=multi-user.target

To use Systemd to control the app you first need to reload the Daemon to register the new file.

sudo systemctl daemon-reload

Now launch your app with:

sudo systemctl start app

You've successfully launched your Node.js app using Systemd! If it doesn't work for some reason, make sure to check your paths in ExecStart are correct.

ExecStart=/usr/bin/node /absolute/path/to/your/project/app.js

These need to point to the node binary and the absolute path to your app.js file.

Use PM2 to run Node.js

You can also run your application with PM2 just like you would normally. Using the same setup as with a default Express.js server. Load the env vars and agent at the top of your source file.

// app.js

// Load env vars
require('dotenv').config({ path: '/absolute/path/to/your/project/.env' })

// require all agents
const {
  stMonitor,
  stLogger,
  stHttpLoggerMiddleware
} = require('sematext-agent-express')

// Start monitoring metrics
stMonitor.start()


// ...

// At the top of your routes add the stHttpLoggerMiddleware to send HTTP logs to Sematext
const express = require('express')
const app = express()
app.use(stHttpLoggerMiddleware)

// ...

Run the pm2 command to start your server.

pm2 start app.js -i max

The agent will detect you are running PM2 and start collecting metrics automatically.

Use Containers to run Node.js

You can also run your application in any container environment, like Docker, Docker Compose, Docker Swarm, or Kubernetes. You'll use the same setup as with a default Node.js server. But, instead of using dotenv, you'll add the environment variables when starting the container.

First of all, require the agent at the top of your init file:

// app.js

// require all agents
const {
  stMonitor,
  stLogger,
  stHttpLoggerMiddleware
} = require('sematext-agent-express')

// Start monitoring metrics
stMonitor.start()


// ...

// At the top of your routes add the stHttpLoggerMiddleware to send HTTP logs to Sematext
const express = require('express')
const app = express()
app.use(stHttpLoggerMiddleware)

// ...

Then, use this Dockerfile to make sure garbage collection metrics are enabled:

FROM alpine AS build
WORKDIR /usr/src/app
RUN apk add --no-cache --update \
  python3 \
  make \
  gcc \
  g++ \
  nodejs \
  nodejs-npm

COPY package.json package-lock.json ./
RUN npm install --production

#############

FROM alpine
WORKDIR /usr/src/app
RUN apk add --no-cache --update nodejs
COPY --from=build /usr/src/app/node_modules ./node_modules
COPY . .
EXPOSE <PORT>
CMD ["node", "app.js"]

Change the <PORT> you're exposing and if needed, change app.js to the entry point of your application.

Now build a Docker image from the Dockerfile above. Run this command in the same directory where you have the Dockerfile:

docker build -t <YOUR_IMAGE:TAG> .

Docker

Next, run the Docker image. Add your MONITORING_TOKEN. Change the <PORT> to the one you're exposing and <YOUR_IMAGE:TAG> to the name of the image you just built. Optionally, add additional flags if you need to.

docker run -d -p <PORT>:<PORT> \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e MONITORING_TOKEN=<MONITORING_TOKEN> \
  -e LOGS_TOKEN=<LOGS_TOKEN> \
  <YOUR_IMAGE:TAG>

Docker Compose

Alternatively, you can use Docker Compose. Use this docker-compose.yml file alongside the Dockerfile above to build and run your application. First add your <MONITORING_TOKEN>, then change the <PORT> to the one you're exposing and <YOUR_IMAGE:TAG> to the name of the image you just built.

version: '3'
services:
  your-nodejs-app:
    build:
      context: ./
      dockerfile: Dockerfile
    image: '<YOUR_IMAGE:TAG>'
    environment:
      - MONITORING_TOKEN=<MONITORING_TOKEN>
      - LOGS_TOKEN=<LOGS_TOKEN>
    restart: always
    volumes:
      - '/var/run/docker.sock:/var/run/docker.sock'
    ports:
      - '<PORT>:<PORT>'

Docker Swarm

The same approach works for Docker Swarm. First add your <MONITORING_TOKEN>, then change the <PORT> to the one you're exposing and <YOUR_IMAGE:TAG> to the name of the image you just built.

docker service create --name your-nodejs-app \
  -p <PORT>:<PORT> \
  --restart-condition any \
  --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
  -e MONITORING_TOKEN=<MONITORING_TOKEN> \
  -e LOGS_TOKEN=<LOGS_TOKEN> \
  <YOUR_IMAGE:TAG>

Alternatively, if you want to use a Docker Compose file with the docker stack command, add the snippet below to your docker-compose.yml file. First add your <MONITORING_TOKEN>, then change the <PORT> to the one you're exposing and <YOUR_IMAGE:TAG> to the name of the image you just built.

services:
  your-nodejs-app:
    image: '<YOUR_IMAGE:TAG>'
    environment:
      - MONITORING_TOKEN=<MONITORING_TOKEN>
      - LOGS_TOKEN=<LOGS_TOKEN>
    volumes:
      - '/var/run/docker.sock:/var/run/docker.sock'
    deploy:
      mode: replicated
      replicas: 1
      labels: [APP=APP]
      update_config:
        parallelism: 1
        delay: 5s
      restart_policy:
        condition: on-failure
        delay: 5s
    ports:
      - "<PORT>:<PORT>"

Kubernetes

To enable the agent in Kubernetes envs you first need to create a Cluster Role, Cluster Role Binding, and Service Account for the Agent to get permissions to access the Kubernetes API.

# Cluster Role bindings for Agent
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: spm-agent-nodejs
  labels:
    app: spm-agent-nodejs
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: spm-agent-nodejs
subjects:
- kind: ServiceAccount
  name: spm-agent-nodejs
  namespace: default
---
# Cluster Role for Agent
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: spm-agent-nodejs
  labels:
    app: spm-agent-nodejs
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - list
  - get
  - watch
---
# Service Account for Agent
apiVersion: v1
kind: ServiceAccount
metadata:
  name: spm-agent-nodejs
  labels:
    app: spm-agent-nodejs

Apply the cluster role.

kubectl create -f nodejs-cluster-role.yml

Next, run the Docker image in your Kubernetes Cluster as a Deployment. Add your <MONITORING_TOKEN> Change the <PORT> to the one you're exposing, and <YOUR_IMAGE:TAG> to the name of the image you just built. You can edit the replicas as you see fit. Make sure to add serviceAccountName: spm-agent-nodejs to enable the required permissions.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: your-nodejs-app-deployment
  labels:
    app: nodejs
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nodejs
  template:
    metadata:
      labels:
        app: nodejs
    spec:
      serviceAccountName: spm-agent-nodejs
      containers:
      - name: nodejs
        image: <YOUR_IMAGE:TAG>
        ports:
        - containerPort: <PORT>
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: MONITORING_TOKEN
          value: "<MONITORING_TOKEN>"
        - name: LOGS_TOKEN
          value: "<LOGS_TOKEN>"

Create the deployment.

kubectl create -f nodejs-deployment.yml

Next, expose your Deployment with a Service. Change the <PORT> to the one you're exposing, and set the clusterIP if needed.

apiVersion: v1
kind: Service
metadata:
  name: your-nodejs-app-service
spec:
  selector:
    app: nodejs
  ports:
    - protocol: TCP
      port: 80
      targetPort: <PORT>
  type: LoadBalancer
  # if you are using a cloud provider (AWS, GCP, etc...)
  # a LoadBalancer will be provisioned automatically.
  # Otherwise set the clusterIP below.
  # clusterIP: <YOUR_CLUSTER_IP>

Create the service.

kubectl create -f nodejs-service.yml

Integration

Metrics

Metric Name Key Agg Type Description
heap total nodejs.heap.size Avg Long
heap used nodejs.heap.used Avg Long
total released nodejs.gc.heap.diff Sum Double
total duration nodejs.gc.time Sum Double
full gc nodejs.gc.full Sum Long
inc gc nodejs.gc.inc Sum Long
memory rss nodejs.memory.rss Avg Long
process count process.count All Long
process cpu usage process.cpu.usage All Double
process rss usage process.rss All Double
process thread count process.thread.count All Long
process uptime process.uptime All Long
workers count nodejs.workers Avg Long
request count nodejs.requests Sum Long
error count nodejs.errors Sum Long
5xx count nodejs.errors.5xx Sum Long
4xx count nodejs.errors.4xx Sum Long
3xx count nodejs.errors.3xx Sum Long
total req. size nodejs.requests.size.total Sum Long
total res. size nodejs.response.size.total Sum Long
min response latency nodejs.responses.latency.min Min Long
max response latency nodejs.responses.latency.max Max Long
min latency nodejs.eventloop.latency.min Min Long
max latency nodejs.eventloop.latency.max Max Long

FAQ

How to Monitor OS and Infra metrics with the Node.js Integration?

We have deprecated the built-in Operating System monitor in the Node.js-based Agent and moved to using our Go-based Sematext Agent for Operating System and Infrastructure metrics. If you are using the spm-agent-nodejs >=4.0.0 or the sematext-agent-express >=2.0.0 you are required to install or upgrade the Sematext Agent to gather Operating System and Infrastructure metrics.

Can I install Express.js agent on Windows?

Yes. The native modules are automatically compiled during "npm install" (using node-gyp). On Windows the required build tools like python or C++ compilers are typically not installed by default. See https://github.com/TooTallNate/node-gyp for details about the required compiler and build tools.

How can I use Node.js agent behind Firewalls / Proxy servers?

By default data is transmitted via HTTPS. If no direct connection is possible, a proxy server can be used by setting the environment variable HTTPS_PROXY=https://your-proxy.

What should I do after upgrading to a new Node.js version?

If you switch the Node.jsversion the sematext-express-agent package will need to be installed again (due to the fact that included native modules may change from version to version). After the version change please run a fresh "npm install" if you added sematext-express-agent to the dependencies in your package.json or at the very least run "npm install sematext-express-agent".

How do I upgrade to the latest version of sematext-express-agent?

To use the latest version of sematext-express-agent we recommend you install/upgrade using:

npm install sematext-express-agent@latest

To add the dependency to your package.json simply use:

npm install sematext-express-agent@latest--save

Troubleshooting

If you are not seeing some or any Express.js metrics, you can create a "diagnostics dump" and contact us via live chat or email. To create the diagnostics dump just run the following command in the root directory of your project.

sudo node ./node_modules/spm-agent-nodejs/bin/spm-nodejs-diagnostics.js

This will create a ZIP file and show the Sematext Support email address to which the ZIP file should be sent.