Loading lesson path

Node.js•Perfomance & Scaling

Node.js Logging

Flash cards

Review the key moves

1/4

Core idea

What is the main idea behind Node.js Logging?

Lesson checks

Practice each idea before moving on

Short Mimo-style checks built from this lesson's code, terms, and sequence.

1Quick choice

Which statement best captures the main point of this lesson?

2Fill blank

Complete the missing token from the example code.

// ___ logging

3Order

Put the learning moves in the order that makes the concept easiest to apply.

Console Limitations

Basic Logging with Console

Why Logging Matters

Effective logging is essential for several reasons:

Debugging: Understand what's happening inside your application
Troubleshooting: Diagnose issues in production environments
Monitoring: Track application health and performance
Auditing: Record important events for compliance and security
Analytics: Gather data about application usage and behavior

Basic Logging with Console

Node.js provides built-in console methods for basic logging:

// Basic logging
console.log('Info message');
console.error('Error message');
console.warn('Warning message');
console.debug('Debug message');
// Log objects
const user = { id: 1, name: 'John', roles: ['admin', 'user'] };
console.log('User object:', user);
// Table output for arrays or objects
console.table([
 { name: 'John', age: 30, role: 'admin' },
 { name: 'Jane', age: 25, role: 'user' },
 { name: 'Bob', age: 40, role: 'guest' }
]);
// Timing operations
console.time('operation');
// Perform some operations...
for (let i = 0; i < 1000000; i++) {
 // Do something
}
console.timeEnd('operation'); // Outputs: operation: 4.269ms
// Grouping related logs
console.group('User Processing');
console.log('Loading user data...');
console.log('Validating user...');
console.log('Updating user profile...');
console.groupEnd();
// Stack trace
console.trace('Trace message');

Console Limitations

While the console is convenient, it has significant limitations for production use:

No built-in log levels for filtering
No log rotation or file management
No structured output formats like JSON
Limited integration with monitoring systems

Note

Console methods are synchronous when outputting to terminals/files and can impact performance if used frequently in production.

Structured Logging

Structured logging formats log messages as data objects (typically JSON) rather than plain text, making them easier to parse, search, and analyze.

Benefits of Structured Logging

Consistent format for machine readability
Better searchability and filtering
Simplified integration with log aggregation tools
Enhanced context with metadata

Example of a Structured Log Entry (JSON)

{
 "timestamp": "2023-11-28T15:24:39.123Z",
 "level": "error",
 "message": "Failed to connect to database",
 "service": "user-service",
 "context": {
 "requestId": "req-123-456",
 "userId": "user-789",
 "databaseHost": "db.example.com"
 },
 "error": {
 "name": "ConnectionError",
 "message": "Connection refused",
 "stack": "..."
 }
}

Winston

Winston is a versatile logging library with support for multiple transports (outputs):

Basic Winston Setup

const winston = require('winston');
// Create a logger
const logger = winston.createLogger({
 level: 'info',
 format: winston.format.json(),
 defaultMeta: { service: 'user-service' },
 transports: [
 // Write logs to a file
 new winston.transports.File({ filename: 'error.log', level: 'error' }),
 new winston.transports.File({ filename: 'combined.log' }),
 ],
});
// If not in production, also log to the console
if (process.env.NODE_ENV !== 'production') {
 logger.add(new winston.transports.Console({
 format: winston.format.simple(),
 }));
}
// Usage
logger.log('info', 'Hello distributed log files!');
logger.info('Hello again distributed logs');
logger.error('Something went wrong', { additionalInfo: 'error details' });

Custom Winston Formats

const winston = require('winston');
const { format } = winston;
const { combine, timestamp, label, printf } = format;
// Custom format
const myFormat = printf(({ level, message, label, timestamp }) => {
 return `${timestamp} [${label}] ${level}: ${message}`;
});
const logger = winston.createLogger({
 format: combine(
 label({ label: 'API Service' }),
 timestamp(),
 myFormat
),
transports: [
 new winston.transports.Console(),
 new winston.transports.File({ filename: 'combined.log' })
]
});
logger.info('Application started');

Pino

Pino is designed to be a low-overhead logger with optimal performance:

Basic Pino Setup

const pino = require('pino');
// Create a logger
const logger = pino({
 level: 'info',
 timestamp: pino.stdTimeFunctions.isoTime,
 base: { pid: process.pid, hostname: require('os').hostname() }
});
// Usage
logger.info('Application started');
logger.info({ user: 'john' }, 'User logged in');
logger.error({ err: new Error('Connection failed') }, 'Database connection error');

Pino with Express

const express = require('express');
const pino = require('pino');
const pinoHttp = require('pino-http');
const app = express();
const logger = pino();
const httpLogger = pinoHttp({ logger });
// Add request logging middleware
app.use(httpLogger);
app.get('/', (req, res) => {
 req.log.info('User accessed homepage');
 res.send('Hello World!');
});
app.get('/error', (req, res) => {
 req.log.error('Something went wrong');
 res.status(500).send('Error!');
});
app.listen(8080, () => {
 logger.info('Server started on port 8080');
});

Bunyan

Bunyan is a structured logging library with a CLI for viewing logs:

Basic Bunyan Setup

const bunyan = require('bunyan');
// Create a logger
const logger = bunyan.createLogger({
 name: 'myapp',
 streams: [
 {
 level: 'info',
 stream: process.stdout
 },
 {
 level: 'error',
 path: 'error.log'
 }
 ],
 serializers: bunyan.stdSerializers
});
// Usage
logger.info('Application started');
logger.info({ user: 'john' }, 'User logged in');
logger.error({ err: new Error('Connection failed') }, 'Database connection error');

Log Levels

Use appropriate log levels to categorize the importance and urgency of log messages:

error: Runtime errors, exceptions, and failures that require attention
warn: Warning conditions that don't stop the application but indicate potential issues
info: Informational messages about application events and milestones
debug: Detailed diagnostic information useful during development
trace: Very detailed debugging information (method entry/exit, variable values)

What to Log

DO LOG

Application startup/shutdown events
Authentication and authorization events
API requests and responses
Database operations and performance metrics
Errors and exceptions with context
Resource usage and performance metrics
Configuration changes

DON'T LOG

Passwords, tokens, API keys, or other credentials
Personally identifiable information (PII) without proper safeguards
Credit card numbers, social security numbers, or other sensitive data
Session IDs or cookies
Encryption keys

Contextual Logging

Include relevant context with each log entry to make troubleshooting easier:

const winston = require('winston');
// Create a base logger
const logger = winston.createLogger({
 level: 'info',
 format: winston.format.json(),
 transports: [new winston.transports.Console()]
});
// Create a child logger with request context
function createRequestLogger(req) {
 return logger.child({
 requestId: req.id,
 method: req.method,
 url: req.url,
 ip: req.ip,
 userId: req.user ? req.user.id : 'anonymous'
 });
}
// Usage in Express middleware
app.use((req, res, next) => {
 req.id = generateRequestId();
 req.logger = createRequestLogger(req);
 req.logger.info('Request received');
 const start = Date.now();
 res.on('finish', () => {
 const duration = Date.now() - start;
 req.logger.info({
 statusCode: res.statusCode,
 duration: duration
 }, 'Request completed');
 });
 next();
});
function generateRequestId() {
 return Date.now().toString(36) + Math.random().toString(36).substring(2);
}

Log Rotation

Prevent log files from growing too large by implementing log rotation:

Winston with rotating file transport

const winston = require('winston');
require('winston-daily-rotate-file');
const transport = new winston.transports.DailyRotateFile({
 filename: 'application-%DATE%.log',
 datePattern: 'YYYY-MM-DD',
 zippedArchive: true,
 maxSize: '20m',
 maxFiles: '14d'
});
const logger = winston.createLogger({
 level: 'info',
 format: winston.format.json(),
 transports: [
 transport,
 new winston.transports.Console()// Optional console transport
 ]
});
logger.info('Hello rotated logs');

Centralized Logging

For applications running across multiple servers or containers, centralize your logs for easier analysis:

Winston with Elasticsearch transport

const winston = require('winston');
require('winston-elasticsearch');
const esTransportOpts = {
 level: 'info',
 clientOpts: {
 node: 'http://localhost:9200'
 },
 indexPrefix: 'app-logs'
};
const logger = winston.createLogger({
 transports: [
 new winston.transports.Elasticsearch(esTransportOpts),
 new winston.transports.Console()// Optional console transport
 ]
});
logger.info('This log will go to Elasticsearch');

Popular Log Management Systems

ELK Stack (Elasticsearch, Logstash, Kibana): Comprehensive logging stack
Graylog : Centralized log management with a focus on security
Fluentd/Fluent Bit : Log collection and forwarding
Loki : Lightweight log aggregation system
Commercial options : Datadog, New Relic, Splunk, LogDNA, Loggly

Performance Considerations

Use asynchronous logging to avoid blocking the event loop
Buffer logs for better performance
Adjust log levels to reduce volume in production
Sample high-volume logs rather than logging every occurrence

Security Considerations

Sanitize sensitive data before logging
Protect log files with appropriate permissions
Use encryption when transmitting logs
Implement retention policies for log data
Verify compliance with relevant regulations (GDPR, HIPAA, etc.)

Node.js Monitoring & Observability

Loading lesson path

Node.js•Perfomance & Scaling

Node.js Logging

Flash cards

Review the key moves

1/4

Core idea

What is the main idea behind Node.js Logging?

Lesson checks

Practice each idea before moving on

Short Mimo-style checks built from this lesson's code, terms, and sequence.

1Quick choice

Which statement best captures the main point of this lesson?

2Fill blank

Complete the missing token from the example code.

// ___ logging

3Order

Put the learning moves in the order that makes the concept easiest to apply.

Console Limitations

Basic Logging with Console

Why Logging Matters

Effective logging is essential for several reasons:

Debugging: Understand what's happening inside your application
Troubleshooting: Diagnose issues in production environments
Monitoring: Track application health and performance
Auditing: Record important events for compliance and security
Analytics: Gather data about application usage and behavior

Basic Logging with Console

Node.js provides built-in console methods for basic logging:

// Basic logging
console.log('Info message');
console.error('Error message');
console.warn('Warning message');
console.debug('Debug message');
// Log objects
const user = { id: 1, name: 'John', roles: ['admin', 'user'] };
console.log('User object:', user);
// Table output for arrays or objects
console.table([
 { name: 'John', age: 30, role: 'admin' },
 { name: 'Jane', age: 25, role: 'user' },
 { name: 'Bob', age: 40, role: 'guest' }
]);
// Timing operations
console.time('operation');
// Perform some operations...
for (let i = 0; i < 1000000; i++) {
 // Do something
}
console.timeEnd('operation'); // Outputs: operation: 4.269ms
// Grouping related logs
console.group('User Processing');
console.log('Loading user data...');
console.log('Validating user...');
console.log('Updating user profile...');
console.groupEnd();
// Stack trace
console.trace('Trace message');

Console Limitations

While the console is convenient, it has significant limitations for production use:

No built-in log levels for filtering
No log rotation or file management
No structured output formats like JSON
Limited integration with monitoring systems

Note

Console methods are synchronous when outputting to terminals/files and can impact performance if used frequently in production.

Structured Logging

Structured logging formats log messages as data objects (typically JSON) rather than plain text, making them easier to parse, search, and analyze.

Benefits of Structured Logging

Consistent format for machine readability
Better searchability and filtering
Simplified integration with log aggregation tools
Enhanced context with metadata

Example of a Structured Log Entry (JSON)

{
 "timestamp": "2023-11-28T15:24:39.123Z",
 "level": "error",
 "message": "Failed to connect to database",
 "service": "user-service",
 "context": {
 "requestId": "req-123-456",
 "userId": "user-789",
 "databaseHost": "db.example.com"
 },
 "error": {
 "name": "ConnectionError",
 "message": "Connection refused",
 "stack": "..."
 }
}

Winston

Winston is a versatile logging library with support for multiple transports (outputs):

Basic Winston Setup

const winston = require('winston');
// Create a logger
const logger = winston.createLogger({
 level: 'info',
 format: winston.format.json(),
 defaultMeta: { service: 'user-service' },
 transports: [
 // Write logs to a file
 new winston.transports.File({ filename: 'error.log', level: 'error' }),
 new winston.transports.File({ filename: 'combined.log' }),
 ],
});
// If not in production, also log to the console
if (process.env.NODE_ENV !== 'production') {
 logger.add(new winston.transports.Console({
 format: winston.format.simple(),
 }));
}
// Usage
logger.log('info', 'Hello distributed log files!');
logger.info('Hello again distributed logs');
logger.error('Something went wrong', { additionalInfo: 'error details' });

Custom Winston Formats

const winston = require('winston');
const { format } = winston;
const { combine, timestamp, label, printf } = format;
// Custom format
const myFormat = printf(({ level, message, label, timestamp }) => {
 return `${timestamp} [${label}] ${level}: ${message}`;
});
const logger = winston.createLogger({
 format: combine(
 label({ label: 'API Service' }),
 timestamp(),
 myFormat
),
transports: [
 new winston.transports.Console(),
 new winston.transports.File({ filename: 'combined.log' })
]
});
logger.info('Application started');

Pino

Pino is designed to be a low-overhead logger with optimal performance:

Basic Pino Setup

const pino = require('pino');
// Create a logger
const logger = pino({
 level: 'info',
 timestamp: pino.stdTimeFunctions.isoTime,
 base: { pid: process.pid, hostname: require('os').hostname() }
});
// Usage
logger.info('Application started');
logger.info({ user: 'john' }, 'User logged in');
logger.error({ err: new Error('Connection failed') }, 'Database connection error');

Pino with Express

const express = require('express');
const pino = require('pino');
const pinoHttp = require('pino-http');
const app = express();
const logger = pino();
const httpLogger = pinoHttp({ logger });
// Add request logging middleware
app.use(httpLogger);
app.get('/', (req, res) => {
 req.log.info('User accessed homepage');
 res.send('Hello World!');
});
app.get('/error', (req, res) => {
 req.log.error('Something went wrong');
 res.status(500).send('Error!');
});
app.listen(8080, () => {
 logger.info('Server started on port 8080');
});

Bunyan

Bunyan is a structured logging library with a CLI for viewing logs:

Basic Bunyan Setup

const bunyan = require('bunyan');
// Create a logger
const logger = bunyan.createLogger({
 name: 'myapp',
 streams: [
 {
 level: 'info',
 stream: process.stdout
 },
 {
 level: 'error',
 path: 'error.log'
 }
 ],
 serializers: bunyan.stdSerializers
});
// Usage
logger.info('Application started');
logger.info({ user: 'john' }, 'User logged in');
logger.error({ err: new Error('Connection failed') }, 'Database connection error');

Log Levels

Use appropriate log levels to categorize the importance and urgency of log messages:

error: Runtime errors, exceptions, and failures that require attention
warn: Warning conditions that don't stop the application but indicate potential issues
info: Informational messages about application events and milestones
debug: Detailed diagnostic information useful during development
trace: Very detailed debugging information (method entry/exit, variable values)

What to Log

DO LOG

Application startup/shutdown events
Authentication and authorization events
API requests and responses
Database operations and performance metrics
Errors and exceptions with context
Resource usage and performance metrics
Configuration changes

DON'T LOG

Passwords, tokens, API keys, or other credentials
Personally identifiable information (PII) without proper safeguards
Credit card numbers, social security numbers, or other sensitive data
Session IDs or cookies
Encryption keys

Contextual Logging

Include relevant context with each log entry to make troubleshooting easier:

const winston = require('winston');
// Create a base logger
const logger = winston.createLogger({
 level: 'info',
 format: winston.format.json(),
 transports: [new winston.transports.Console()]
});
// Create a child logger with request context
function createRequestLogger(req) {
 return logger.child({
 requestId: req.id,
 method: req.method,
 url: req.url,
 ip: req.ip,
 userId: req.user ? req.user.id : 'anonymous'
 });
}
// Usage in Express middleware
app.use((req, res, next) => {
 req.id = generateRequestId();
 req.logger = createRequestLogger(req);
 req.logger.info('Request received');
 const start = Date.now();
 res.on('finish', () => {
 const duration = Date.now() - start;
 req.logger.info({
 statusCode: res.statusCode,
 duration: duration
 }, 'Request completed');
 });
 next();
});
function generateRequestId() {
 return Date.now().toString(36) + Math.random().toString(36).substring(2);
}

Log Rotation

Prevent log files from growing too large by implementing log rotation:

Winston with rotating file transport

const winston = require('winston');
require('winston-daily-rotate-file');
const transport = new winston.transports.DailyRotateFile({
 filename: 'application-%DATE%.log',
 datePattern: 'YYYY-MM-DD',
 zippedArchive: true,
 maxSize: '20m',
 maxFiles: '14d'
});
const logger = winston.createLogger({
 level: 'info',
 format: winston.format.json(),
 transports: [
 transport,
 new winston.transports.Console()// Optional console transport
 ]
});
logger.info('Hello rotated logs');

Centralized Logging

For applications running across multiple servers or containers, centralize your logs for easier analysis:

Winston with Elasticsearch transport

const winston = require('winston');
require('winston-elasticsearch');
const esTransportOpts = {
 level: 'info',
 clientOpts: {
 node: 'http://localhost:9200'
 },
 indexPrefix: 'app-logs'
};
const logger = winston.createLogger({
 transports: [
 new winston.transports.Elasticsearch(esTransportOpts),
 new winston.transports.Console()// Optional console transport
 ]
});
logger.info('This log will go to Elasticsearch');

Popular Log Management Systems

ELK Stack (Elasticsearch, Logstash, Kibana): Comprehensive logging stack
Graylog : Centralized log management with a focus on security
Fluentd/Fluent Bit : Log collection and forwarding
Loki : Lightweight log aggregation system
Commercial options : Datadog, New Relic, Splunk, LogDNA, Loggly

Performance Considerations

Use asynchronous logging to avoid blocking the event loop
Buffer logs for better performance
Adjust log levels to reduce volume in production
Sample high-volume logs rather than logging every occurrence

Security Considerations

Sanitize sensitive data before logging
Protect log files with appropriate permissions
Use encryption when transmitting logs
Implement retention policies for log data
Verify compliance with relevant regulations (GDPR, HIPAA, etc.)

Node.js Monitoring & Observability