Monitoring & Metrics

Overview

Tower exports OpenTelemetry metrics for observability and alerting in New Relic. Monitoring v2 introduced semantically correct error metrics that can both increase (new error) and decrease (error archived).

How it works

• iptower.tasks.errors.open (gauge) — count of active, non-archived errors. Goes up on a new error, down when one is archived. Replaces the old ever-increasing iptower.tasks.error.
• iptower.tasks.errors.archived (counter) — increments each time an error is archived (useful for statistics/reporting).
• iptower.monitoring.version: '2' resource attribute — filter in New Relic to target only towers running v2 metrics.

Archiving flows through a single atomic entry point: DataService.archiveError(uuid) sets error.archived, removes the task from errorTasks in memory, increments the counter, and persists. TaskService.taskArchive delegates to it entirely.

Key files

• src/monitoring.ts — metric definitions, stable ATTR_SERVICE_* conventions, iptower.monitoring.version attribute, resilient monitoring-params.json loading
• src/core/data/data.service.ts — errors.open / errors.archived metrics, archiveError()
• src/core/task/task.service.ts — taskArchive delegation

Technical decisions

• Deprecated SEMRESATTRS_SERVICE_* constants replaced with stable ATTR_SERVICE_NAME / ATTR_SERVICE_VERSION.
• monitoring-params.json loading is resilient: a missing file logs a warning instead of crashing the process.
• dk:otel:start npm script runs the production image with a tower-data volume and BCYIP_TARGET_ENV=DEV for local OTEL testing (port 9089).

Next steps

• Add latency/throughput histograms per task type