ci: the nightly cron is a second buildcache writer, and the default auto-cancel does not cover it #18

Closed
opened 2026-08-26 05:24:12 +00:00 by jlxq0 · 1 comment
Owner

Reporting state from the concurrency probe in jlxq0/mantis#32; the decision is yours.

.forgejo/workflows/ci.yml guards the buildcache export on refs/heads/main:

155:  if [[ "${GITHUB_REF}" == refs/heads/main ]]; then
156:    EXPORT_CACHE_ARG=(--export-cache "type=registry,ref=${CACHE_REF},mode=max")

That is one writer for push events, which is what #4 needed. It is not one writer overall, because two other triggers also produce GITHUB_REF of refs/heads/main:

9:   schedule:
10:     - cron: "17 3 * * *"
8:   workflow_dispatch:

Task 16519 is that cron on 2026-08-25: a docker job on main that exported sha256:e269c2ea…. workflow_dispatch has the same shape and was not in the original analysis.

What the probe changes

Measured on Forgejo 15.0.2+gitea-1.22.0, full evidence in jlxq0/mantis#32:

  • Forgejo already cancels a running invocation of the same workflow on the same ref on a new push, with no concurrency: block at all. A run was observed running, a push landed two seconds later, and it read cancelled six seconds after that.
  • The documented default covers on.push and on.pull_request (synchronize) only. A schedule or workflow_dispatch invocation neither cancels nor is cancelled. So the cron overlapping a merge build is the one case in this repository that the default does not close, and it is a real second writer.
  • Workflow-level concurrency: is honoured. Five runs across three refs serialised strictly one at a time for thirteen minutes on a runner with capacity: 4 that was demonstrably running other repositories' jobs at the same time.
  • Job-level concurrency: is silently ignored. The identical block moved under jobs. had no effect: cancel-in-progress: false did not prevent the default cancellation, the workflow parsed, and nothing warned. This is the failure mode #4's pull requests refused to risk, and refusing was correct.

The shape of the fix

Top-level, not under jobs::

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: false

Two things to weigh, which is why this is an issue rather than a pull request:

  1. It serialises the whole run, cargo job included, not just docker. On a shared runner already queueing for minutes, that lengthens the critical path for every trigger on the same ref.
  2. group: ci-${{ github.ref }} puts the cron and a main push in one group, which is the point, and leaves tag builds in their own group, which is fine now that a tag build only imports. Whether workflow_dispatch should share the group is a judgement about what you use it for.

The narrower alternative is to guard the export on the event as well as the ref, so only push exports and the cron imports only. That needs no concurrency support at all and costs the cron a cache refresh it may not need, since a cron run builds the same commit main already built.

Reporting state from the concurrency probe in `jlxq0/mantis#32`; the decision is yours. `.forgejo/workflows/ci.yml` guards the buildcache export on `refs/heads/main`: 155: if [[ "${GITHUB_REF}" == refs/heads/main ]]; then 156: EXPORT_CACHE_ARG=(--export-cache "type=registry,ref=${CACHE_REF},mode=max") That is one writer for `push` events, which is what `#4` needed. It is not one writer overall, because two other triggers also produce `GITHUB_REF` of `refs/heads/main`: 9: schedule: 10: - cron: "17 3 * * *" 8: workflow_dispatch: Task `16519` is that cron on 2026-08-25: a `docker` job on `main` that exported `sha256:e269c2ea…`. `workflow_dispatch` has the same shape and was not in the original analysis. ## What the probe changes Measured on Forgejo `15.0.2+gitea-1.22.0`, full evidence in `jlxq0/mantis#32`: - **Forgejo already cancels a running invocation of the same workflow on the same ref on a new push, with no `concurrency:` block at all.** A run was observed `running`, a push landed two seconds later, and it read `cancelled` six seconds after that. - **The documented default covers `on.push` and `on.pull_request` (synchronize) only.** A `schedule` or `workflow_dispatch` invocation neither cancels nor is cancelled. So the cron overlapping a merge build is the one case in this repository that the default does not close, and it is a real second writer. - **Workflow-level `concurrency:` is honoured.** Five runs across three refs serialised strictly one at a time for thirteen minutes on a runner with `capacity: 4` that was demonstrably running other repositories' jobs at the same time. - **Job-level `concurrency:` is silently ignored.** The identical block moved under `jobs.` had no effect: `cancel-in-progress: false` did not prevent the default cancellation, the workflow parsed, and nothing warned. This is the failure mode `#4`'s pull requests refused to risk, and refusing was correct. ## The shape of the fix Top-level, not under `jobs:`: concurrency: group: ci-${{ github.ref }} cancel-in-progress: false Two things to weigh, which is why this is an issue rather than a pull request: 1. It serialises the **whole run**, `cargo` job included, not just `docker`. On a shared runner already queueing for minutes, that lengthens the critical path for every trigger on the same ref. 2. `group: ci-${{ github.ref }}` puts the cron and a `main` push in one group, which is the point, and leaves tag builds in their own group, which is fine now that a tag build only imports. Whether `workflow_dispatch` should share the group is a judgement about what you use it for. The narrower alternative is to guard the export on the event as well as the ref, so only `push` exports and the cron imports only. That needs no concurrency support at all and costs the cron a cache refresh it may not need, since a cron run builds the same commit `main` already built.
jlxq0 closed this issue 2026-09-01 23:33:05 +00:00
Author
Owner

Fixed in #25, merged at d94a68c. Taking the narrower option: guard the export on the
event as well as the ref, rather than a top-level concurrency: block.

Why not concurrency:. Job-level is silently ignored on this Forgejo, so a
workflow-level block is one well-meant refactor away from being inert while still parsing.
It also serialises the whole run, cargo included, on a shared-capacity runner to solve
what one condition closes. And the file already makes this argument for tag builds: a cron
or dispatch run on main builds the commit main just built, so its export would be
byte-identical and buys nothing.

What passed

The condition, exercised over all five trigger cases rather than reasoned about:

ref                      event              old   new
refs/heads/main          push               yes   yes   <- control
refs/heads/main          schedule           yes   no
refs/heads/main          workflow_dispatch  yes   no
refs/tags/v0.2.2         push               no    no
refs/pull/25/head        pull_request       no    no

The first row is the control. Without it the four no rows are equally satisfied by a
guard that never exports at all.

The event vocabulary is this instance's own. /actions/tasks over 100 tasks reports
exactly push (117), pull_request (26), schedule (26) and workflow_dispatch (4), and
a schedule task carries head_branch=main, which is this issue's premise confirmed from
the API rather than from the workflow file.

What failed, and it was my acceptance rather than the fix

I planned to prove it against the buildcache ref's own digest: record D0 before,
merge (a push to main, which must export and move it), then dispatch (which must not).

D0 and D1 are identical, sha256:0592aacd0bbecb… both times, after a push to main
whose docker job reported success.

That is not a failure of the export. #25 changed .forgejo/workflows/ci.yml and
AGENTS.md, and the Dockerfile COPYs only fonts, scripts/fetch-fonts.sh, Cargo.toml,
Cargo.lock, src and templates. Neither changed file enters any layer, so every
layer is byte-identical and a mode=max export produces the same manifest and the same
digest. An unchanged digest is what a working export produces here.

So the instrument returns the same value whether the export ran or not, which is the
one thing an acceptance may not do. I designed it, ran it, and it could not have told me
the answer either way.

And it cannot be repaired, which is the part worth keeping. A cron or dispatch run
always builds the commit main has just built, so its export is always byte-identical to
the one already there. The property that makes the fix correct is the same property that
makes it unobservable in the registry.
The race this closes is a concurrent-write failure,
not a content difference.

What remains unmeasured

That the runner sets GITHUB_EVENT_NAME to those literals in the shell, as opposed to
github.event_name in YAML expressions, which this workflow already relies on and which is
known to work here. Job logs cannot be read on this instance (see #22), so it is not
directly observable.

Its failure mode is benign and eventually visible: if that variable were empty, the
condition would be false on every trigger, no run would export, and the cache would go stale
and builds slower. That is the safe direction, and it is the opposite of the concurrent-write
failure this issue is about.

Closing on the condition table and the event vocabulary, with the registry acceptance
recorded as attempted and inconclusive rather than quietly dropped.

Pitfall in AGENTS.md: GITHUB_REF does not identify a trigger, so a ref-only guard reads
as naming one writer while admitting three.

Fixed in #25, merged at `d94a68c`. Taking the narrower option: guard the export on the event as well as the ref, rather than a top-level `concurrency:` block. **Why not `concurrency:`.** Job-level is silently ignored on this Forgejo, so a workflow-level block is one well-meant refactor away from being inert while still parsing. It also serialises the whole run, `cargo` included, on a shared-capacity runner to solve what one condition closes. And the file already makes this argument for tag builds: a cron or dispatch run on `main` builds the commit `main` just built, so its export would be byte-identical and buys nothing. ## What passed **The condition, exercised over all five trigger cases** rather than reasoned about: ref event old new refs/heads/main push yes yes <- control refs/heads/main schedule yes no refs/heads/main workflow_dispatch yes no refs/tags/v0.2.2 push no no refs/pull/25/head pull_request no no The first row is the control. Without it the four `no` rows are equally satisfied by a guard that never exports at all. **The event vocabulary is this instance's own.** `/actions/tasks` over 100 tasks reports exactly `push` (117), `pull_request` (26), `schedule` (26) and `workflow_dispatch` (4), and a `schedule` task carries `head_branch=main`, which is this issue's premise confirmed from the API rather than from the workflow file. ## What failed, and it was my acceptance rather than the fix I planned to prove it against the buildcache ref's own digest: record `D0` before, merge (a push to `main`, which must export and move it), then dispatch (which must not). **`D0` and `D1` are identical**, `sha256:0592aacd0bbecb…` both times, after a push to `main` whose `docker` job reported success. That is not a failure of the export. `#25` changed `.forgejo/workflows/ci.yml` and `AGENTS.md`, and the Dockerfile COPYs only `fonts`, `scripts/fetch-fonts.sh`, `Cargo.toml`, `Cargo.lock`, `src` and `templates`. **Neither changed file enters any layer**, so every layer is byte-identical and a `mode=max` export produces the same manifest and the same digest. An unchanged digest is what a *working* export produces here. **So the instrument returns the same value whether the export ran or not**, which is the one thing an acceptance may not do. I designed it, ran it, and it could not have told me the answer either way. **And it cannot be repaired**, which is the part worth keeping. A cron or dispatch run always builds the commit `main` has just built, so its export is always byte-identical to the one already there. **The property that makes the fix correct is the same property that makes it unobservable in the registry.** The race this closes is a concurrent-write failure, not a content difference. ## What remains unmeasured That the runner sets `GITHUB_EVENT_NAME` to those literals in the *shell*, as opposed to `github.event_name` in YAML expressions, which this workflow already relies on and which is known to work here. Job logs cannot be read on this instance (see #22), so it is not directly observable. Its failure mode is benign and eventually visible: if that variable were empty, the condition would be false on every trigger, no run would export, and the cache would go stale and builds slower. That is the safe direction, and it is the opposite of the concurrent-write failure this issue is about. Closing on the condition table and the event vocabulary, with the registry acceptance recorded as attempted and inconclusive rather than quietly dropped. Pitfall in `AGENTS.md`: `GITHUB_REF` does not identify a trigger, so a ref-only guard reads as naming one writer while admitting three.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jlxq0/typst-mcp#18
No description provided.