Skip to main content
Version: 3.1.0

Init Stream — Per-Source Progress, Status Bar & SSE Error Details

Version: 3.1.0 Date: 2026-06-12

Goal: Give operators real-time visibility into which repos and paths succeed or fail during source initialization, without waiting for the full report — and surface real underlying errors over the SSE boundary.

Solution: Refactored the SSE init stream to yield one chunk per project/path (instead of one per source provider), carrying sourceName, progress { current, total }, and the action entries. The UI renders a live, fixed-shape status badge per source. Per-source start markers make a source appear the moment the backend starts it, and serializeError() flattens non-serializable Error objects before emission.

Features:

  • Init report header shows a pill-shaped status bar (⏳ N pending → ✓ N success / ✗ N errors) per source row, aligned consistently, with reserved width (no layout shift)
  • Source rows appear as soon as the backend starts them (start markers), no blank gap
  • Reload clears the report immediately, then rebuilds progressively from the stream
  • Read config / Init config rows use the same pill style and status colors
  • Detailed entries grouped by repo/path with zebra striping, prominent repo chip, code-styled URLs (tooltip), and full error blocks (real FTP/HTTP error stack)

Implementation Details:

  • WebhookCapacityService.webhooksInitStream() and FtpCapacityService.ftpInitStream() — new async* generators, one yield per project/path, plus an initial empty chunk as a start marker
  • All five provider initSources() methods converted to async* generators (yield* delegation)
  • SourceService.initStream() forwards sourceName + progress per chunk and no longer filters empty chunks
  • app.controller.ts — res.flush() after each SSE write to defeat response buffering
  • serializeError() flattens errors to { message, stack, code } (Error message/stack are non-enumerable, so JSON.stringify turned them into {})
  • UI: Status.tsx tracks perSourceProgress + streamingSourceOrder; SourceGroupReport renders StatusBadge + shared ErrorBlock + grouping

Also includes: AWS S3 endpoint optional for standard S3 — backend Joi allows empty endpoint; frontend requires it only when the "S3 Compatible service" toggle is active.

Files Modified:

  • src/protocol/capacities/webhook-capacity/webhook-capacity.service.ts — webhooksInitStream(), start marker, serializeError()
  • src/protocol/capacities/ftp-capacity/ftp-capacity.service.ts — ftpInitStream(), start marker, serialize FTP errors
  • src/protocol/source/{gitlab,github,bitbucket,gitea,ftp}.service.ts — async generator initSources()
  • src/protocol/source/source.service.ts — initStream() return type + forwarding
  • src/app.controller.ts — flush SSE events
  • src/core/configuration/config.schema.ts — endpoint Joi.string().optional().allow('')
  • ui/src/pages/settings/Status.tsx — perSourceProgress state, clear on reload, source order
  • ui/src/pages/settings/report/InitReport.tsx — forward perSourceProgress, placeholder sources
  • ui/src/pages/settings/report/SourceGroupReport.tsx — StatusBar/StatusBadge, ErrorBlock, grouping, zebra
  • ui/src/pages/settings/report/UnitActionReport.tsx — StatusPill/ErrorBlock, no lingering spinner
  • ui/src/pages/settings/report/ValidationReport.tsx — status-colored Read config title
  • ui/src/pages/storages/StorageEditor.tsx — conditional endpoint validation
  • ui/src/protocol/tower.api.ts — onSource callback type
  • ui/src/protocol/tower.mock.ts — mock stream updated