Skip to content

docs: preloading jemalloc makes large builds 10-17% faster - #987

Open
dktue wants to merge 1 commit into
systemed:masterfrom
dktue:docs/jemalloc
Open

dktue wants to merge 1 commit into
systemed:masterfrom
dktue:docs/jemalloc

Conversation

@dktue

@dktue dktue commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Gain: none in code; it documents a way to make large builds 10–17% faster on Linux without rebuilding: preloading the jemalloc allocator. No new dependency.

While profiling the tile phase on the planet, about a fifth of its CPU time went into the memory allocator (malloc, free and their internals). Preloading a different allocator with LD_PRELOAD showed how much of that is recoverable, without changing any code:

planet in memory, .pmtiles glibc malloc (default) jemalloc change
master + #986, 96 threads (default) 24:43 / 23:50 20:32 −14% to −17%, peak RSS 162.8 → 155.9 GB
all open performance PRs, 96 threads 14:12 / 14:08 12:38 −11%
all open performance PRs, 32 threads 19:11 17:16 −10%
all open performance PRs, Germany, 32 threads 37.5 / 37.4 / 37.5 s 36.2 / 36.0 / 36.0 s −4%, peak RSS −4%

48-core / 96-thread server, OpenMapTiles profile, Debian/Ubuntu libjemalloc2. jemalloc was faster in every case and never needed more memory. For comparison, tcmalloc (gperftools 2.15) was 25% slower on the planet at 96 threads.

This PR only adds a short paragraph to the "Performance and memory tuning" section of docs/RUNNING.md. If you'd like tilemaker to use jemalloc without preloading, an optional CMake switch to link it would be a possible follow-up, but that adds a dependency, so I've left it out.

🤖 Generated with Claude Code

On Linux, LD_PRELOAD=libjemalloc.so.2 made the planet 14-17% faster at
96 threads (24:43 / 23:50 -> 20:32, 7 GB less memory) and a Germany
extract at 32 threads 4% faster. Documentation only; no new dependency.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant