javascript

How Can You Master Log Management in Express.js With Morgan and Rotating File Streams?

Organized Chaos: Streamlining Express.js Logging with Morgan and Rotating-File-Stream

How Can You Master Log Management in Express.js With Morgan and Rotating File Streams?

Logging in a web application built with Express.js is super important for keeping tabs on what’s happening and fixing issues when they pop up. One great way to manage your logs is by combining morgan middleware and rotating-file-stream. This duo helps you store logs in rotating files so they stay organized and don’t grow endlessly.

First off, you’ll need to install both morgan and rotating-file-stream using npm. It’s as simple as running:

npm install morgan rotating-file-stream

Once you’ve got them installed, you can set up your Express app to use these modules.

Here’s a basic setup for logging requests into a file that rotates daily:

const express = require('express');
const morgan = require('morgan');
const rfs = require("rotating-file-stream");

const app = express();
const port = 3000;

// Create a rotating write stream
const accessLogStream = rfs.createStream("access.log", {
  interval: "1d", // rotate daily
  path: __dirname + "/log"
});

// Setup the logger
app.use(morgan("combined", { stream: accessLogStream }));

app.get('/', (req, res) => {
  res.send('hello, world!');
});

app.listen(port, () => {
  console.debug(`App listening on :${port}`);
});

In this example, rotating-file-stream creates a new log file every day, and morgan logs all requests in the Apache combined format to this rotating log stream.

If you want to customize your log format and add more details like the user’s IP or a unique request ID, you can do it this way:

const express = require('express');
const morgan = require('morgan');
const rfs = require("rotating-file-stream");
const uuid = require('node-uuid');

const app = express();
const port = 3000;

// Create a rotating write stream
const accessLogStream = rfs.createStream("access.log", {
  interval: "1d", // rotate daily
  path: __dirname + "/log"
});

// Define custom tokens
morgan.token('id', req => req.id);
morgan.token('ip', req => req.headers['x-forwarded-for'] || req.connection.remoteAddress);

// Assign a unique ID to each request
app.use((req, res, next) => {
  req.id = uuid.v4();
  next();
});

// Setup the logger with custom format
app.use(morgan(':id :ip ":method :url HTTP/:http-version" :status :res[content-length] ":referrer" ":user-agent"', { stream: accessLogStream }));

app.get('/', (req, res) => {
  res.send('hello, world!');
});

app.listen(port, () => {
  console.debug(`App listening on :${port}`);
});

In this setup, every log entry includes a unique request ID and the user’s IP address.

Sometimes you might need to rotate logs not just daily, but also based on size. This makes sure no single log file gets too big:

const express = require('express');
const morgan = require('morgan');
const rfs = require("rotating-file-stream");

const app = express();
const port = 3000;

// Create a rotating write stream based on size
const accessLogStream = rfs.createStream("access.log", {
  size: "10M", // rotate every 10MB written
  interval: "1d", // rotate daily
  compress: "gzip" // compress rotated files
});

// Setup the logger
app.use(morgan("combined", { stream: accessLogStream }));

app.get('/', (req, res) => {
  res.send('hello, world!');
});

app.listen(port, () => {
  console.debug(`App listening on :${port}`);
});

This way, logs will rotate daily and whenever the file size exceeds 10MB.

You can also make your logging setup more flexible using environment variables:

require('dotenv').config(); // load variables from .env file

const express = require('express');
const morgan = require('morgan');
const rfs = require("rotating-file-stream");

const app = express();
const port = 3000;

// Create a rotating write stream
const rfsStream = rfs.createStream(process.env.LOG_FILE || 'access.log', {
  size: process.env.LOG_SIZE || '10M',
  interval: process.env.LOG_INTERVAL || '1d',
  compress: 'gzip' // compress rotated files
});

// Setup the logger
const format = process.env.LOG_FORMAT || "dev";
app.use(morgan(format, { stream: rfsStream }));
app.use(morgan(format));

app.get('/', (req, res) => {
  res.send('hello, world!');
});

app.listen(port, () => {
  console.debug(`App listening on :${port}`);
});

This setup lets you control log file names, sizes, intervals, and formats using environment variables defined in a .env file.

Another handy feature is logging errors and successes separately. This way, you can easily track down issues without sifting through all other logs:

const express = require('express');
const morgan = require('morgan');
const rfs = require("rotating-file-stream");

const app = express();
const port = 3000;

// Create rotating write streams
const errorLogStream = rfs.createStream("error.log", {
  interval: "1d", // rotate daily
  path: __dirname + "/log"
});

const successLogStream = rfs.createStream("success.log", {
  interval: "1d", // rotate daily
  path: __dirname + "/log"
});

// Skip logic for error and success logs
const skipSuccess = (req, res) => res.statusCode < 400;
const skipError = (req, res) => res.statusCode >= 400;

// Setup the logger for errors
app.use(morgan("combined", { skip: skipSuccess, stream: errorLogStream }));

// Setup the logger for successes
app.use(morgan("combined", { skip: skipError, stream: successLogStream }));

app.get('/', (req, res) => {
  res.send('hello, world!');
});

app.listen(port, () => {
  console.debug(`App listening on :${port}`);
});

In this config, error responses (4xx and 5xx status codes) go into one file and successful responses into another.

At times, you might run into issues where log rotation doesn’t work as expected. A common fix for daily rotation issues is to ensure the file name generator is set up right:

const express = require('express');
const morgan = require('morgan');
const rfs = require("rotating-file-stream");
const path = require('path');
const { format } = require('date-fns');

const app = express();
const port = 3000;

function logFilename() {
  return `${format(new Date(), 'yyyy-MM-dd')}-access.log`;
}

const accessLogStream = rfs.createStream(logFilename(), {
  interval: "1d", // rotate daily
  path: path.resolve(__dirname, '..', 'log'),
});

// Setup the logger
app.use(morgan(':method :url :status :res[content-length] ":referrer" ":user-agent"', { stream: accessLogStream }));

app.get('/', (req, res) => {
  res.send('hello, world!');
});

app.listen(port, () => {
  console.debug(`App listening on :${port}`);
});

Here, the logFilename function makes sure the log file name includes the current date, which helps in daily rotation.

Combining morgan with rotating-file-stream provides a solid logging solution for your Express.js apps. You can customize your log formats, rotate logs by size or time, and even use environment variables to keep everything in check. This keeps your logs well-organized and easier to analyze, making debugging and monitoring a breeze.

Keywords: logging, Express.js, morgan middleware, rotating-file-stream, npm install, log rotation, daily logs, custom log format, environment variables, Express logging setup



Similar Posts
Blog Image
What's the Magic Tool to Make Debugging Express.js Apps a Breeze?

Navigating the Debugging Maze: Supercharge Your Express.js Workflow

Blog Image
How Can You Seamlessly Manage User Sessions with Express.js?

Mastering User Sessions in Express.js: Unleashing Endless Possibilities

Blog Image
Mastering JavaScript's Logical Operators: Write Cleaner, Smarter Code Today

JavaScript's logical assignment operators (??=, &&=, ||=) streamline code by handling null/undefined values, conditional updates, and default assignments. They enhance readability and efficiency in various scenarios, from React components to API data handling. While powerful, they require careful use to avoid unexpected behavior with falsy values and short-circuiting.

Blog Image
Why Should Serving Directory Listings Be a Headache with Express.js Magic?

Effortlessly Navigate Your Project with Express.js and Serve-Index Magic

Blog Image
Building a High-Performance HTTP/2 Server in Node.js: What You Need to Know

HTTP/2 boosts web performance with multiplexing, server push, and header compression. Node.js enables easy HTTP/2 server creation, optimizing speed through streaming, compression, and effective error handling.

Blog Image
What Secret Sauce Makes WebAssembly the Speedster of Web Development?

Unleashing the Speed Demon: How WebAssembly is Revolutionizing Web App Performance