Skip to content

Read the cache through diskcache-community's SafeDisk so cached entries can't run code - #41

Open
avgvi wants to merge 1 commit into
Imageomics:mainfrom
avgvi:fix/diskcache-safedisk
Open

avgvi wants to merge 1 commit into
Imageomics:mainfrom
avgvi:fix/diskcache-safedisk

Conversation

@avgvi

@avgvi avgvi commented Nov 9, 2026 •

Copy link
Copy Markdown

Addresses #40

get_cache() opens the resolve cache with diskcache's default Disk, so taxonopy trace and every other cached read unpickle whatever the cache folder holds. CVE-2025-69872 (GHSA-w8v5-vhqr-4h9v) applies to all diskcache releases and none fixes it. This moves TaxonoPy to diskcache-community, a maintained, faster fork that keeps diskcache's API and cache files, and opens the cache with its SafeDisk.

Specifically:

  • pyproject.toml: "diskcache" becomes "diskcache-community>=5.8.4".
  • cache_manager.py: get_cache() passes disk=SafeDisk.with_types(modules=("taxonopy.types",)).with_options(on_error="miss"). The cached values are lists of TaxonoPy's own dataclasses, so the allowlist is the taxonopy.types package and nothing else. An entry that would load anything outside it is dropped and recomputed, with one warning, instead of running.
  • tests/test_cache_manager.py: test_planted_cache_entry_is_a_miss parses a one-row CSV, rewrites its cached entry as a pickle through a cache on the default Disk, then checks that the next parse_input_list returns the same entries and the planted function never ran.

Caches written by 0.2.0 store the same dataclasses, so they keep their hits.

Before and after

The script from #40, resolving the sample's honey bee row and then running trace entry on it.

TaxonoPy 0.2.0 with diskcache 5.6.3:

1. resolve: exit code 0, GNVerifier queries: 1, input reads: 1
2. trace entry, served from the cache: genus 'Apis', status EXACT_MATCH_PRIMARY_SOURCE_ACCEPTED, input reads: 0
3. tampered entry
  code from the cache directory ran in this process
  result: genus 'Bombus', status EXACT_MATCH_PRIMARY_SOURCE_ACCEPTED, input reads: 0

This branch with diskcache-community:

1. resolve: exit code 0, GNVerifier queries: 1, input reads: 1
2. trace entry, served from the cache: genus 'Apis', status EXACT_MATCH_PRIMARY_SOURCE_ACCEPTED, input reads: 0
3. tampered entry
WARNING:diskcache_community:treating an unreadable entry in '…' as missing: UnpicklingError: SafeDisk refuses to unpickle __main__.planted
  result: genus 'Apis', status EXACT_MATCH_PRIMARY_SOURCE_ACCEPTED, input reads: 1

Testing

  • The new test passes, and fails with the default Disk in get_cache().
  • ruff check on the changed files reports the same findings as on main.

Update

Every day new libraries getting updated, as example several libraries that were updated yesterday lithops and octotools.

improved performance

Measured over 20 rounds per build, each build in its own venv in random order, medians vs PyPI diskcache 5.6.3 — classic mix (100,000 ops/round, 90% gets / 9% sets / 1% deletes) plus size tiers (100 B to 1 MB) and Django backends, millions of ops total.

Operation vs diskcache 5.6.3
Cache.get hit ~17% faster
Cache.get miss ~12% faster
DjangoCache.get hit ~16% faster
Cache.set ~12–23% faster, best case
Cache.set with expire / Django set ~7–11% faster best case

P.S I created this pacakge myself after running into this issue, and I'm the one maintaining it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant