Skip to content
share library_books

NodeJS Monitoring Integration

Sematext offers a simple Node.js monitoring agent, written entirely in Node.js without CPU and memory overhead. It's easy to install and require in your source code.

Sematext Node.js Monitoring Agent

This lightweight, open-source Node.js monitoring agent collects Node.js process and performance metrics and sends them to Sematext. It is available as an npm package that can be added to JavaScript source code like any other Node.js module.

First you install the npm module.

# Terminal
npm install spm-agent-nodejs

You need to add the SPM_TOKEN of your Sematext Monitoring App to your Node.js process environment with a module like dotenv, or directly before running the application.

# .env
SPM_TOKEN=<your-spm-token-goes-here>

Require it in your source code at the top if your source file.

// app.js
require('dotenv').config({ path: '/absolute/path/to/your/project/.env' }) // if you're using dotenv
require('spm-agent-nodejs')

Run your source file.

# Terminal
node app.js

Or without dotenv.

# Terminal
SPM_TOKEN=<your-spm-token-goes-here> node app.js

The Sematext Node.js monitoring agent will start collecting dozens of key metrics right away, and start showing you the performance and health of your Node.js applications immediately.

Collected Node.js Metrics

The Sematext Node.js monitoring agent collects the following metrics.

Operating System

  • CPU usage
  • CPU load
  • Memory usage

Process Memory Usage

  • Released memory between garbage collection cycles
  • Process heap size
  • Process heap usage

Worker Processes (cluster module)

  • Worker count
  • Event loop latency per worker

Event Loop

  • Maximum event loop latency
  • Minimum event loop latency
  • Average event loop latency

Garbage Collection

  • Time consumed for garbage collection
  • Counters for full garbage collection cycles
  • Counters for incremental garbage collection cycles
  • Released memory after garbage collection

HTTP Server Stats

  • Request count
  • Request rate
  • Response time
  • Request/Response content-length
  • Error rates (total, 3xx, 4xx, 5xx)

Use the cluster module to run Node.js

To make use of the full power of your server, you should run an instance of your Node.js application on each CPU core. The cluster module makes this easier than ever. Create another file called cluster.js.

// cluster.js

const cluster = require('cluster')
const numCPUs = require('os').cpus().length
const app = require('./app')
const port = process.env.PORT || 3000

const masterProcess = () => Array.from(Array(numCPUs)).map(cluster.fork)
const childProcess = () => app.listen(port)
if (cluster.isMaster) masterProcess()
else childProcess()
cluster.on('exit', (worker) => cluster.fork())

Now you can run your app with:

node cluster.js

The cluster will spin up a master process with a dedicated process ID and run numCPUs number of worker processes. They will be load balanced in a round-robin fashion from the master process.

This is not all, you should also make sure to run your Node.js application with Systemd to make it a system service and run automatically on startup and restart itself if it fails.

Set up Node.js with Systemd

The service files for the things that systemd controls all live under the directory path

/lib/systemd/system

Create a new file there:

sudo vim /lib/systemd/system/app.service

And add this piece of code:

# /lib/systemd/system/app.service

[Unit]
Description=app.js - running your Node.js app as a system service
Documentation=https://yourwebsite.com
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/bin/node /absolute/path/to/your/project/app.js
Restart=on-failure

[Install]
WantedBy=multi-user.target

To use Systemd to control the app you first need to reload the Daemon to register the new file.

sudo systemctl daemon-reload

Now launch your app with:

sudo systemctl start app

You've successfully launched your Node.js app using Systemd! If it doesn't work for some reason, make sure to check your paths in ExecStart are correct.

ExecStart=/usr/bin/node /absolute/path/to/your/project/app.js

These need to point to the node binary and the absolute path to your app.js file.

Troubleshooting

Generate diagnostics file for Sematext Support

If you are not seeing some or any Node.js metrics, you can create a "diagnostics dump" and contact us via chat or email. To create the diagnostics dump just run the following in your application directory:

node ./node_modules/spm-agent-nodejs/bin/spm-client-diagnostics.js

This will create a ZIP file and show the Sematext Support email address to which the ZIP file should be sent.

Integration

Metrics

Metric Name Key Agg Type Description
heap total nodejs.heap.size Avg Long
heap used nodejs.heap.used Avg Long
total released nodejs.gc.heap.diff Sum Double
total duration nodejs.gc.time Sum Double
full gc nodejs.gc.full Sum Long
inc gc nodejs.gc.inc Sum Long
memory rss nodejs.memory.rss Avg Long
workers count nodejs.workers Avg Long
request count nodejs.requests Sum Long
error count nodejs.errors Sum Long
5xx count nodejs.errors.5xx Sum Long
4xx count nodejs.errors.4xx Sum Long
3xx count nodejs.errors.3xx Sum Long
total req. size nodejs.requests.size.total Sum Long
total res. size nodejs.response.size.total Sum Long
min response latency nodejs.responses.latency.min Min Long
max response latency nodejs.responses.latency.max Max Long
min latency nodejs.eventloop.latency.min Min Long
max latency nodejs.eventloop.latency.max Max Long

Custom Metrics

To track custom metrics likethe number of concurrent users, the number of items placed in a shopping cart, or any other kind of business transaction or KPI we provide a Custom Metrics API and a Node.js client for it: spm-metrics-js

FAQ

Can I install Node.js agent on Windows?

Yes. The native modules are automatically compiled during "npm install" (using node-gyp). On Windows the required build tools like python or C++ compilers are typically not installed by default. See https://github.com/TooTallNate/node-gyp for details about the required compiler and build tools.

How can I configure Node.js agent for my app using PM2 process manager?

Install spm-agent-nodejs as a global module.

sudo npm i -g spm-agent-nodejs --unsafe-perm

Check the location (full path) for spm-agent-nodejs using:

sudo npm root -g

The result is typically /usr/local/lib/node_modules or /usr/lib/node_modules. Remember the path to use it in the interpreter_args step below in the PM2 configuration file. You will need to append /spm-agent-nodejs to this path to access the globally installed directory of the spm-agent-nodejs.

First generate the PM2 config file:

pm2 ecosystem

This command will create a file called ecosystem.config.js.

// ecosystem.config.js
module.exports = {
  apps : [{
    name: 'API',
    script: 'app.js',

    // Options reference: https://pm2.io/doc/en/runtime/reference/ecosystem-file/
    args: 'one two',
    instances: 1,
    autorestart: true,
    watch: false,
    max_memory_restart: '1G',
    env: {
      NODE_ENV: 'development'
    },
    env_production: {
      NODE_ENV: 'production'
    }
  }]
  // ...
};

Edit this file so it has an interpreter_args section, and SPM agent env variables.

// ecosystem.config.js
module.exports = {
  apps : [{
    name: 'API',
    script: 'app.js', // replace with your server file
    instances: 1, // does not work with more than 1 because cluster mode is not supported
    autorestart: true,
    watch: false,
    exec_mode: 'fork', // does not work in 'cluster'
    interpreter_args: '-r /usr/lib/node_modules/spm-agent-nodejs', // ADD THIS
    env: { // ADD ALL OF THIS TOO
      NODE_ENV: 'development',
      SPM_TOKEN: 'YOUR_SPM_TOKEN',
      spmagent_dbDir: './spmdb',
      spmagent_logger__dir: './spmlogs',
      spmagent_logger__silent: false,
      spmagent_logger__level: 'error'
    },
    env_production: {
      NODE_ENV: 'production',
      SPM_TOKEN: 'YOUR_SPM_TOKEN',
      spmagent_dbDir: './spmdb',
      spmagent_logger__dir: './spmlogs',
      spmagent_logger__silent: false,
      spmagent_logger__level: 'error'
    }
  }]
  // ...
};

Run PM2 with the config file:

pm2 start ecosystem.config.js

How can I use Node.js agent behind Firewalls / Proxy servers?

By default data is transmitted via HTTPS. If no direct connection is possible, a proxy server can be used by setting the environment variable HTTPS_PROXY=https://your-proxy.

What should I do after upgrading to a new Node.js version?

If you switch the Node.jsversion the spm-agent-nodejs package will need to be installed again (due to the fact that included native modules may change from version to version). After the version change please run a fresh "npm install" if you added spm-agent-nodejs to the dependencies in your package.json or at the very least run "npm install spm-agent-nodejs".

How do I upgrade to the latest version of spm-agent-nodejs?

To use the latest version of spm-agent-nodejs we recommend you install/upgrade using:

npm install spm-agent-nodejs@latest

To add the dependency to your package.json simply use:

npm install spm-agent-nodejs@latest--save