fix(memory): use jemalloc and report heap statistics #161

Merged
jlxq0 merged 2 commits from fix/jemalloc-allocator into main 2026-11-09 14:48:58 +00:00
Owner

What changes

  • jemalloc replaces glibc malloc as the global allocator. Background threads purge freed pages, enabled at startup through tikv-jemalloc-ctl; this crate forbids unsafe, so the malloc_conf export isn't available.
  • New matrix_mcp_heap_bytes{kind=allocated|active|resident|metadata}, read on every scrape.
  • New matrix_mcp_client_builds_total{reason=new|replaced}.
  • matrix_mcp_active_clients is now kept current. It was registered but never set, so it always read 0.
  • New .cargo/audit.toml with three advisories that don't apply, each with its reason. main currently fails cargo audit on these three as well.

Why
v0.10.20 grew about 40 MiB/h on 9 Nov, with 39 tool calls in 10 hours. /proc/1/maps showed 30 glibc arena heaps (64 MiB reservations, 593 MiB committed) for 17 threads. Platform #1496 runs MALLOC_ARENA_MAX=2 as a parallel experiment. Either way, the heap gauges show retention (allocated rising) apart from allocator overhead (the gap to resident).

Gates, run locally with 1.98.0 (the CI toolchain)

  • fmt --check passes.
  • clippy -D warnings passes.
  • Tests pass: 390 + 6.
  • cargo audit passes.
  • cargo deny check bans licenses sources passes.
  • New test heap_stats_come_from_the_live_allocator.

🤖 Generated with Claude Code

https://claude.ai/code/session_01E2xQRZnoBbY9SEC5nF6J4U

**What changes** - jemalloc replaces glibc malloc as the global allocator. Background threads purge freed pages, enabled at startup through `tikv-jemalloc-ctl`; this crate forbids `unsafe`, so the `malloc_conf` export isn't available. - New `matrix_mcp_heap_bytes{kind=allocated|active|resident|metadata}`, read on every scrape. - New `matrix_mcp_client_builds_total{reason=new|replaced}`. - `matrix_mcp_active_clients` is now kept current. It was registered but never set, so it always read 0. - New `.cargo/audit.toml` with three advisories that don't apply, each with its reason. `main` currently fails `cargo audit` on these three as well. **Why** v0.10.20 grew about 40 MiB/h on 9 Nov, with 39 tool calls in 10 hours. `/proc/1/maps` showed 30 glibc arena heaps (64 MiB reservations, 593 MiB committed) for 17 threads. Platform #1496 runs `MALLOC_ARENA_MAX=2` as a parallel experiment. Either way, the heap gauges show retention (`allocated` rising) apart from allocator overhead (the gap to `resident`). **Gates, run locally with 1.98.0 (the CI toolchain)** - `fmt --check` passes. - `clippy -D warnings` passes. - Tests pass: 390 + 6. - `cargo audit` passes. - `cargo deny check bans licenses sources` passes. - New test `heap_stats_come_from_the_live_allocator`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01E2xQRZnoBbY9SEC5nF6J4U
fix(memory): use jemalloc and report heap statistics
All checks were successful
CI / cargo (pull_request) Successful in 3m22s
CI / docker (pull_request) Has been skipped
c8bedda7e8
v0.10.20 still grew about 40 MiB/h on 9 Nov with 39 tool calls in ten hours.
Its RSS sat in 30 glibc arena heaps (64 MiB reservations, 593 MiB committed)
for 17 threads. jemalloc becomes the global allocator, with background threads
purging freed pages.

matrix_mcp_heap_bytes{kind} reports jemalloc's allocated, active, resident and
metadata bytes on every scrape, so retention and allocator overhead can be told
apart. matrix_mcp_client_builds_total{reason} counts client builds. The
matrix_mcp_active_clients gauge was never set and is now kept current.

.cargo/audit.toml documents three advisories that do not apply to this build:
rmcp's OAuth and Streamable HTTP clients are not compiled in, and the bundled
SQLite fix needs a matrix-sdk release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2xQRZnoBbY9SEC5nF6J4U
chore(release): v0.10.21
All checks were successful
CI / cargo (pull_request) Successful in 4m25s
CI / docker (pull_request) Has been skipped
b4fbf215ae
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2xQRZnoBbY9SEC5nF6J4U
jlxq0 merged commit d2001cbd45 into main 2026-11-09 14:48:58 +00:00
jlxq0 deleted branch fix/jemalloc-allocator 2026-11-09 14:48:59 +00:00
Sign in to join this conversation.
No description provided.