jlxq0/typst-mcp
1
0
Fork 0
Code Issues 13 Pull requests Projects Releases Packages 1 Wiki Activity Actions

fix(ci): one writer to the buildcache ref #4

Merged
jlxq0 merged 1 commit from fix-buildcache-race into main 2026-08-25 03:48:18 +00:00
Conversation 1 Commits 1 Files changed 1 +10
jlxq0 commented 2026-08-25 02:52:11 +00:00
Owner
Copy link

Closes jlxq0/m365-mcp#4 for this repository.

The race

Merging a PR and pushing the release tag are two pushes, so two docker jobs
run concurrently. Both exported to the same unqualified :buildcache ref:

#21 exporting cache to registry
#21 ERROR: error writing layer blob: unknown
error: failed to solve: error writing layer blob: unknown

Observed on m365-mcp runs 49 and 50, failing within half a second of each
other — after the image had already been pushed successfully.

The fix, and why it is not either option in the issue

The tag build now imports the cache and does not export it.

That works because a tag build is the same commit as the main build that
preceded it. Reading the branch: the two paths differ only in TAGS and PUSH,
which are output settings rather than build inputs. So the two caches would be
byte-identical — exporting twice was pure duplication, and the duplication was
the race. One writer removes it by construction rather than by scheduling.

The issue offered two shapes and I took neither:

  • A concurrency: group serialises two builds that produce the same bytes,
    which is paying a queue to do work twice. Worse, Forgejo job-level
    concurrency is unverified here    : nothing in the fleet uses it, this instance
    is 15.0.2, and an unsupported key is ignored silently — a fix that looks
    applied and does nothing, which is the failure mode worth avoiding above all
    others tonight.
  • A per-ref cache gives the tag build a cold cache, which for a Rust image
    is the expensive path, and accumulates refs on a registry whose disk is
    currently the fleet's bottleneck.

carddav-mcp already did this

It guards its export on refs/heads/main and has exactly one writer. So this is
four copies converging on the one that was right, not a new invention — the same
shape as the divergence in oauth_redirect.rs, resolved in favour of the copy
that already worked.

Worth correcting one line of the issue: it says all five export to the same ref,
which is true, but only four of them export from both paths. carddav-mcp
needed no change.

Verification

Not verified by running, and it cannot be yet: the forge runner is out of
disk and every cargo job fails before reaching a gate. A green docker job
would not be evidence either — buildkit does not use that volume.

Correction: the assertion I first wrote was the wrong one

I originally said the check was "exactly one --export-cache site remains".
That does not distinguish fixed from broken, and typst-mcp is the proof:
before this change it had exactly one export site, guarded by
GITHUB_EVENT_NAME != pull_request — which fires on a main push and on a
tag push. One site, two writers. A grep for the count would have reported it
already fixed.

The property is exactly one writer, which is about the guard on the
export, not the number of sites. For three of the four repos those coincide;
for typst they do not.

repo export sites before guard before writers before
caldav-mcp 2 refs/tags/v* and refs/heads/main 2
jmap-mcp 2 refs/tags/v* and refs/heads/main 2
typst-mcp 1 != pull_request 2
m365-mcp 2 refs/tags/v* and the else 2
carddav-mcp 1 refs/heads/main 1 — exempt

After this change every one of the four exports only under
refs/heads/main, stated explicitly rather than implied by "not a tag", so a
trigger added on another branch later cannot quietly become a second writer.

What is checked

  • the workflow YAML parses;
  • the build script extracted from it passes bash -n;
  • the single --export-cache is inside a refs/heads/main guard — read,
    not counted.

Do not merge on a gate you cannot read. This wants one run after the runner
is back: a merge to main and a tag push landing together, both green.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SP7njJZ7ZtrMgULuXh8ddq

Closes jlxq0/m365-mcp#4 for this repository. ## The race Merging a PR and pushing the release tag are two pushes, so two `docker` jobs run concurrently. Both exported to the same unqualified `:buildcache` ref: ``` #21 exporting cache to registry #21 ERROR: error writing layer blob: unknown error: failed to solve: error writing layer blob: unknown ``` Observed on `m365-mcp` runs 49 and 50, failing within half a second of each other — **after** the image had already been pushed successfully. ## The fix, and why it is not either option in the issue **The tag build now imports the cache and does not export it.** That works because a tag build is *the same commit* as the `main` build that preceded it. Reading the branch: the two paths differ only in `TAGS` and `PUSH`, which are output settings rather than build inputs. So the two caches would be byte-identical — **exporting twice was pure duplication, and the duplication was the race.** One writer removes it by construction rather than by scheduling. The issue offered two shapes and I took neither: - **A `concurrency:` group** serialises two builds that produce the same bytes, which is paying a queue to do work twice. Worse, **Forgejo job-level concurrency is unverified here**: nothing in the fleet uses it, this instance is 15.0.2, and an unsupported key is *ignored silently* — a fix that looks applied and does nothing, which is the failure mode worth avoiding above all others tonight. - **A per-ref cache** gives the tag build a cold cache, which for a Rust image is the expensive path, and accumulates refs on a registry whose disk is currently the fleet's bottleneck. ## `carddav-mcp` already did this It guards its export on `refs/heads/main` and has exactly one writer. So this is four copies converging on the one that was right, not a new invention — the same shape as the divergence in `oauth_redirect.rs`, resolved in favour of the copy that already worked. Worth correcting one line of the issue: it says all five export to the same ref, which is true, but only **four** of them export from both paths. `carddav-mcp` needed no change. ## Verification **Not verified by running, and it cannot be yet:** the forge runner is out of disk and every `cargo` job fails before reaching a gate. A green `docker` job would not be evidence either — buildkit does not use that volume. ### Correction: the assertion I first wrote was the wrong one I originally said the check was *"exactly one `--export-cache` site remains"*. **That does not distinguish fixed from broken**, and `typst-mcp` is the proof: before this change it had exactly **one** export site, guarded by `GITHUB_EVENT_NAME != pull_request` — which fires on a `main` push *and* on a tag push. One site, two writers. A grep for the count would have reported it already fixed. The property is **exactly one writer**, which is about the *guard* on the export, not the number of sites. For three of the four repos those coincide; for typst they do not. | repo | export sites before | guard before | writers before | |---|---:|---|---:| | caldav-mcp | 2 | `refs/tags/v*` and `refs/heads/main` | 2 | | jmap-mcp | 2 | `refs/tags/v*` and `refs/heads/main` | 2 | | typst-mcp | **1** | `!= pull_request` | **2** | | m365-mcp | 2 | `refs/tags/v*` and the `else` | 2 | | *carddav-mcp* | *1* | *`refs/heads/main`* | *1 — exempt* | After this change every one of the four exports **only** under `refs/heads/main`, stated explicitly rather than implied by "not a tag", so a trigger added on another branch later cannot quietly become a second writer. ### What is checked - the workflow YAML parses; - the build script extracted from it passes `bash -n`; - **the single `--export-cache` is inside a `refs/heads/main` guard** — read, not counted. **Do not merge on a gate you cannot read.** This wants one run after the runner is back: a merge to `main` and a tag push landing together, both green. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01SP7njJZ7ZtrMgULuXh8ddq
jlxq0 force-pushed fix-buildcache-race from 45b682518f
Some checks failed
CI / cargo (pull_request) Failing after 2s
Details
CI / docker (pull_request) Has been skipped
Details
to bf1618e81a
All checks were successful
CI / cargo (pull_request) Successful in 1m59s
Details
CI / docker (pull_request) Successful in 21s
Details
2026-08-25 03:39:37 +00:00 Compare
jlxq0 merged commit 14b7ee2964 into main 2026-08-25 03:48:18 +00:00
jlxq0 deleted branch fix-buildcache-race 2026-08-25 03:48:18 +00:00
jlxq0 commented 2026-08-25 03:55:50 +00:00
Author
Owner
Copy link

Merged as 14b7ee29. Merge builds: cargo 16534, docker 16536, both green.

The Closes jlxq0/m365-mcp#4 in the body above did not fire. Cross-repository references do not close issues on Forgejo, and all four pull requests carried the same line, so the issue stayed open with nothing recording that the work was done. It is closed by hand now, and m365-mcp#4 holds the record for all four repositories: merge commits, what was verified, and two instruments that were proposed and withdrawn before anyone used them.

This repository is the one with a writer path still open. Its 17 3 * * * cron runs with GITHUB_REF of refs/heads/main, which is exactly the guard the export now sits behind, so the nightly run exports. Task 16519 is that cron on 2026-08-25, writing sha256:e269c2ea.... If it fires while a merge build is running, that is two writers again. Tracked in m365-mcp#7 with why a concurrency: group was not reached for. None of the other four repositories has a schedule.

Evidence for this repository specifically, stated plainly: there is none, and the change is unproven at runtime here. 16536 wrote no log, and the buildcache digest is unchanged at sha256:e269c2ea... because the build was a full cache hit re-exporting identical bytes, which is precisely the case every available instrument is blind to. Three of the four repositories have a confirmed export on the merge build; this one does not. The static check in the body above is what it ships on.

What is confirmed here is that the export path works at all: 16519, under the old code, returns 2 for grep -c "exporting cache to registry". That is the positive control for the instrument, not evidence about this change.

Merged as `14b7ee29`. Merge builds: cargo `16534`, docker `16536`, both green. The `Closes jlxq0/m365-mcp#4` in the body above did not fire. Cross-repository references do not close issues on Forgejo, and all four pull requests carried the same line, so the issue stayed open with nothing recording that the work was done. It is closed by hand now, and **[m365-mcp#4](https://forge.oddie.app/jlxq0/m365-mcp/issues/4) holds the record for all four repositories**: merge commits, what was verified, and two instruments that were proposed and withdrawn before anyone used them. **This repository is the one with a writer path still open.** Its `17 3 * * *` cron runs with `GITHUB_REF` of `refs/heads/main`, which is exactly the guard the export now sits behind, so the nightly run exports. Task `16519` is that cron on 2026-08-25, writing `sha256:e269c2ea...`. If it fires while a merge build is running, that is two writers again. Tracked in [m365-mcp#7](https://forge.oddie.app/jlxq0/m365-mcp/issues/7) with why a `concurrency:` group was not reached for. None of the other four repositories has a schedule. Evidence for this repository specifically, stated plainly: **there is none, and the change is unproven at runtime here.** `16536` wrote no log, and the `buildcache` digest is unchanged at `sha256:e269c2ea...` because the build was a full cache hit re-exporting identical bytes, which is precisely the case every available instrument is blind to. Three of the four repositories have a confirmed export on the merge build; this one does not. The static check in the body above is what it ships on. What is confirmed here is that the export path works at all: `16519`, under the old code, returns `2` for `grep -c "exporting cache to registry"`. That is the positive control for the instrument, not evidence about this change.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
deferred
No brief and no decision. Not scheduled. Needs an explicit go before any work.

  
waiting-on-julian
Blocked on Julian. The ask is in the issue.

No labels
deferred
waiting-on-julian
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
jlxq0/typst-mcp!4
No description provided.