Skip to main content
Version: 3.1.0

Monitoring v2 — Error Metrics Refactoring

Version: 3.1.0 Date: 2026-06-12

Goal: Replace the always-increasing error gauge with semantically correct metrics that enable reliable New Relic alerting on new errors, and allow the gauge to decrease when errors are archived.

Solution: Introduced iptower.tasks.errors.open (gauge, decrements on archive) and iptower.tasks.errors.archived (counter). taskArchive now removes the task from errorTasks in memory. Added iptower.monitoring.version: '2' resource attribute to distinguish towers running v2 metrics from older deployments.

Features:

  • iptower.tasks.errors.open (gauge) — active (non-archived) error count; up on new error, down on archive. Replaces iptower.tasks.error
  • iptower.tasks.errors.archived (counter) — increments each time an error is archived
  • iptower.monitoring.version: '2' resource attribute on all metrics — filter in New Relic to target this version
  • Archiving an error removes it from errorTasks in memory and persists state in a single atomic archiveError call
  • Deprecated SEMRESATTRS_SERVICE_NAME / SEMRESATTRS_SERVICE_VERSION replaced with stable ATTR_SERVICE_NAME / ATTR_SERVICE_VERSION
  • monitoring-params.json loading is now resilient: missing file logs a warning instead of crashing
  • New npm script dk:otel:start for local OTEL testing (port 9089)
  • FtpTaskDetails loader has the same getTask fallback as GitTaskDetails for archived tasks

Implementation Details:

  • archiveError(uuid) in DataService is the single entry point: sets error.archived, removes from errorTasks, increments counter, persists
  • taskArchive in TaskService delegates entirely to dataService.archiveError(uuid)

Files Modified:

  • src/monitoring.ts - resilient params loading, ATTR_* stable conventions, iptower.monitoring.version resource attribute
  • src/core/data/data.service.ts - new metrics (errors.open, errors.archived), archiveError method
  • src/core/task/task.service.ts - taskArchive delegates to dataService.archiveError
  • src/core/task/task.service.spec.ts - mock archiveError in taskRetry test
  • ui/src/pages/dashboard/taskDetails/FtpTaskDetails.tsx - fallback to getTask for archived tasks
  • package.json - dk:otel:start script
  • README.md - OTEL section documenting metrics, local setup, alerting query