Pipeline as First-Class Entity#

Problem#

The current domain model collapses “trigger” and “pipeline/workflow” into a single entity (RunRecord). A run triggered by a push can execute multiple workflow files (ci.yml, docs.yml), but all jobs land in a flat Vec<JobResult> with no workflow grouping.

Consequences:

Two jobs named build in different workflows collide.
Aggregate status hides which workflow failed.
Forge commit status is per-run, not per-workflow. Forgejo/GitHub post one check per workflow — MyCI posts one check for everything.
No workflow identity in the event stream.
Users cannot re-run a single workflow.
The runs list cannot show a workflow name column.

Current model#

One RunRecord per trigger event. All workflows merged.

{data_dir}/runs/{run_id}.json
{data_dir}/events/{run_id}.jsonl

RunRecord                          (1 per trigger)
+-- id, repo, ref_name, sha
+-- status                        (aggregate, treated as singular)
+-- started_at, finished_at
+-- repository_url
+-- jobs: Vec<JobResult>           (flat -- all workflows merged)
      +-- name, status, steps

EventKind:
  RunStarted { run_id, repo, ref, sha }
  JobQueued/Started/Finished { job }
  StepStarted/Finished { job, step }
  Log { job, step, stream, content }
  RunFinished { status }

Proposed model#

A trigger event (push webhook, CLI invocation) is the cause. Each matched workflow file becomes its own run — the entity with a lifecycle, status, commit check, and event stream.

TriggerEvent                           (1 per webhook / CLI invocation)
+-- id                                 (correlation ID)
+-- repo, ref_name, sha
+-- event: String                      ("push", "pull_request")
+-- received_at: DateTime

RunRecord                              (1 per matched workflow)
+-- id
+-- trigger_id: Option<String>         (links back to TriggerEvent)
+-- workflow: String                   (relative path)
+-- name: Option<String>              (YAML name: field, display label)
+-- repo, ref_name, sha               (denormalized from trigger)
+-- status: RunStatus
+-- started_at, finished_at
+-- repository_url
+-- jobs: Vec<JobResult>
      +-- (unchanged)

TriggerEvent is lightweight — it records “this happened” and links runs that share a cause. It has no lifecycle or status. RunRecord is the entity users interact with: it has a status, a commit check, a re-run action, and an event stream.

Runs from the same trigger share trigger_id. Grouping is a query (runs.filter(|r| r.trigger_id == id)), not nesting.

The trigger context (repo, ref, sha) is denormalized onto each run so runs are self-contained — no join needed to display a run.

Workflow identifier#

Use the relative path (.forgejo/workflows/docs.yml), not just the filename.

Guarantees uniqueness even if multiple workflow directories are supported in the future.
Matches GitHub/Forgejo commit status convention.
Unambiguous for debugging.
The YAML name: field is the human-readable display label. The path is the stable identifier.

The relative path is already computed in run_workflows (path.strip_prefix(repo_root)). discover_workflows_from_cache would need to prepend the directory prefix. The path is stored on each RunRecord in the workflow field.

Storage layout#

Group everything by trigger — the trigger is the causal boundary. Run record and event stream for each run are co-located.

{data_dir}/triggers/{trigger_id}/
  trigger.json                       (TriggerEvent metadata)
  {run_id}.json                      (RunRecord for ci.yml)
  {run_id}.events.jsonl              (event stream for ci.yml)
  {run_id}.json                      (RunRecord for docs.yml)
  {run_id}.events.jsonl              (event stream for docs.yml)

Why trigger-grouped:

Mirrors the domain: everything that exists was caused by a trigger. The filesystem reflects that relationship.
Co-location: run record and event stream live next to each other instead of in separate directory trees (runs/ and events/).
Natural access patterns: list all runs from a trigger = readdir. View a single run = read one file.
Cleanup is atomic: trimming old data removes one directory.
Per-run event streams: each run has its own JSONL file with independent lifecycle, writer, and broadcast channel.

Event stream#

Each run gets its own JSONL event file. The existing RunStarted and RunFinished events already bracket the stream — now scoped to one workflow execution. No new event types needed.

The old model (one run = multiple workflows) would have required new bracket events (PipelineStarted / PipelineFinished) to demarcate workflow boundaries within a shared stream, and a workflow filter on the SSE endpoint to select which workflow’s events to serve. With one run = one workflow, the existing event types and endpoints work unchanged.

The broadcast registry key remains run_id. Each SSE connection subscribes to exactly one run’s broadcast.

What this unlocks#

Per-workflow commit status: StatusReporter::report takes workflow context. One POST /statuses/{sha} per run.
Per-workflow re-run: Each run has its own identity. Re-run targets a specific workflow execution.
Workflow display name: name from YAML name: field. UI shows name if present, falls back to filename.
Parallel execution: Each run has its own event file, writer, broadcast. No shared mutable state.
Workflow-scoped job names: Jobs scoped within a single run. No collisions across workflows.
Workflow name in runs list: UI shows one row per run (= one workflow execution).

API#

GET /api/runs              -> Vec<RunRecord>
GET /api/runs/{id}         -> RunRecord
GET /api/runs/{id}/events  -> SSE

Each run is one workflow execution, so the API needs no workflow filter parameter — GET /api/runs/{id}/events returns exactly the events for that workflow. The old model would have needed ?workflow=.forgejo/workflows/docs.yml to select from a shared stream.

The UI runs list shows one row per run: workflow name (or filename), status, repo, branch, SHA. Runs from the same trigger share trigger_id for grouping.

Migration#

New fields on RunRecord (workflow, name, trigger_id) use #[serde(default)]. Old JSON files deserialize with these fields as None / empty — displayed as runs without workflow info. No data migration step needed.

The storage layout changes from flat directories to trigger-grouped. A v1→v2 migration moves existing files:

{data_dir}/runs/{run_id}.json
{data_dir}/events/{run_id}.jsonl

into:

{data_dir}/triggers/{run_id}/
  trigger.json              (synthesized, minimal)
  {run_id}.json             (moved from runs/)
  {run_id}.events.jsonl     (moved from events/)

For old records without a trigger, the run ID serves as the trigger ID (one run = one trigger). The migration synthesizes a minimal trigger.json from the run’s metadata.

Trade-offs#

Pros:

Correct domain model reflecting how workflows actually work.
Per-workflow status in UI and forge commit status.
Enables re-run, concurrency, filtering without further model changes.
Co-located storage (run + events in one directory).
Backward compatible via #[serde(default)] on new fields.
Event stream stays append-only, no new event types.

Cons:

Breaking change across the stack (domain, run store, worker, API, UI).
Storage layout migration needed.
Worker creates multiple records per trigger instead of one.

GitLab comparison#

GitLab supports only one workflow file (.gitlab-ci.yml) per repo — effectively one pipeline per trigger. The multi-workflow model (GitHub/Forgejo/MyCI) is more flexible: separate concerns into separate files, each with their own triggers and independent status. Making each workflow execution its own run is the natural consequence of supporting multiple workflows.

Pipeline as First-Class Entity

Contents