Monitor Node.js services on Kubernetes

A Node.js process can consume little CPU while users wait behind a blocked event loop or a slow dependency. A Kubernetes Pod can be Ready while one endpoint returns errors. Monitoring must connect application outcomes, runtime behavior, and cluster state rather than treating any single health signal as the answer.

Start with the service objective, then instrument the paths that explain whether it is being met.

Measure user-visible outcomes

For an HTTP or agent-facing service, track:

• request arrival and completion rate;
• errors by stable class, not raw message;
• p50, p95, and p99 latency;
• timeouts and cancellations;
• queue depth and wait time;
• response or token throughput where relevant.

Break down only by bounded dimensions such as route template, method, status class, service version, and region. A raw URL, user ID, prompt, or error string can create unbounded metric cardinality and leak sensitive data.

Averages conceal the slowest requests and burst behavior. Tail latency and concurrent work often reveal saturation before CPU reaches a simple threshold.

Watch the event loop

Node.js handles many requests on one event loop. Synchronous CPU work, large JSON transformations, poorly bounded callbacks, or native extensions can delay every request sharing that process.

Node's perf_hooks module can monitor event-loop delay. This minimal sample logs a periodic summary:

import process from "node:process";
import { monitorEventLoopDelay } from "node:perf_hooks";

const delay = monitorEventLoopDelay({ resolution: 20 });
delay.enable();

const timer = setInterval(() => {
  const memory = process.memoryUsage();
  const nanosecondsToMilliseconds = 1_000_000;

  console.log(
    JSON.stringify({
      event: "node_runtime_sample",
      eventLoopDelayP99Ms:
        delay.percentile(99) / nanosecondsToMilliseconds,
      eventLoopDelayMaxMs: delay.max / nanosecondsToMilliseconds,
      heapUsedBytes: memory.heapUsed,
      rssBytes: memory.rss,
    }),
  );

  delay.reset();
}, 10_000);

timer.unref();

In a real service, export these values through the same metrics pipeline as request telemetry. Choose sampling intervals and alert thresholds from measured workload behavior. The monitor itself has overhead, so validate it under representative load.

Distinguish memory signals

Resident set size, V8 heap use, external memory, and container working set describe different parts of memory behavior. Track them together with garbage collection and request load.

A rising heap after comparable traffic and completed garbage collection can indicate retained objects. A rising resident set with stable heap may point to buffers, native modules, or allocator behavior. Kubernetes memory limits add another boundary: if the container is killed for exceeding its limit, the application may not have time to emit a final error.

Correlate application metrics with Pod restart count, termination reason, and node pressure. “The process disappeared” is not an application diagnosis.

Trace dependencies

Measure outbound calls to databases, queues, model endpoints, artifact stores, and identity services. Record destination service, operation, latency, status class, timeout, and retry count. Propagate a trace or correlation identifier across asynchronous boundaries where the transport supports it.

Retries must remain visible. A request that succeeds after three hidden attempts may satisfy the error-rate chart while consuming extra capacity and approaching its latency deadline.

Do not attach request bodies, prompts, tokens, or credentials to traces by default. Define explicit capture and redaction rules before enabling rich payload telemetry.

Align probes with behavior

Use startup, readiness, and liveness probes for different decisions:

• startup: did initialization complete?
• readiness: should this replica receive traffic?
• liveness: is the process stuck in a state a restart can repair?

An event-loop stall may justify a carefully designed liveness signal, but a downstream database outage usually should remove readiness or degrade a feature rather than restart every replica. See Kubernetes probes for ML services for the failure-mode approach.

Correlate with Kubernetes

Application dashboards should include deployment revision, image digest, Pod, node, zone, requested and limited resources, restarts, and readiness transitions. Compare service latency with CPU throttling, memory pressure, scheduling, network errors, and autoscaling activity.

Kubernetes resource metrics alone cannot explain event-loop or dependency behavior. Application metrics alone cannot explain a Pod evicted by the node. The correlation between them is the useful part.

Alert on impact and exhaustion

Page on user-visible impact or imminent inability to serve: sustained error-budget burn, tail-latency breach, queue growth without recovery, or a loss of ready capacity. Use warnings or investigation dashboards for lower-level anomalies that do not yet affect the objective.

Every alert should identify the affected service and environment, link to the relevant runbook, and include enough context to choose a first diagnostic step. Avoid one alert per Pod when a single deployment-level incident explains them all.

Polyaxon can run Node.js-based services and jobs as versioned containerized operations. Use run metadata and platform observability alongside application telemetry to connect a regression to the exact code, image, configuration, workload, and infrastructure that produced it.