Skip to content

feat(aggregator): new tools clear-immutables-archives-cache command - #3674

Merged
Alenar merged 14 commits into
mainfrom
djo/3653/aggregator/migration-of-immutables-archives
Nov 4, 2026
Merged

Alenar merged 14 commits into
mainfrom
djo/3653/aggregator/migration-of-immutables-archives

Conversation

@Alenar

@Alenar Alenar commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Content

This PR includes a new tools clear-immutables-archives-cache aggregator command.

The aggregator caches the immutable files archives it builds in {snapshot_directory}/cardano-database/immutable.
For an efficient IPFS distribution, aggregators must upload byte-identical archives, so this cache must be recomputed each time the archive compression changes (e.g. when the zstd library version is bumped).
This command clears it so the archives get rebuilt.

  • Dry run by default: prints the cache directory and a summary of the detected archives
  • --delete deletes the archives (other files are kept)
  • Warns when the local snapshot uploader is used, as this directory is then served as is
  • Also splits the tools command into one submodule per subcommand

Pre-submit checklist

  • Branch
    • Tests are provided (if possible)
    • Crates versions are updated (if relevant)
    • CHANGELOG file is updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
  • PR
    • All check jobs of the CI have succeeded
    • Self-reviewed the diff
    • Useful pull request description
    • Reviewer requested
  • Documentation
    • Update documentation website (if relevant)
    • No new TODOs introduced

Issue(s)

Relates to #3653

@Alenar Alenar self-assigned this Nov 2, 2026
@Alenar Alenar added the feature 🚀 New implemented feature label Nov 2, 2026
@jpraynaud
jpraynaud requested a lite review from Copilot November 2, 2026 17:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Active issues remain, including Clippy-failing unused imports and cache deletion logic that mishandles six-digit archive numbers.

4 open issues
What changed in this PR

Adds the tools clear-immutables-archives-cache aggregator command for inspecting and optionally deleting cached immutable archives, while reorganizing existing tools commands into dedicated submodules.

Changes:

  • Adds dry-run and --delete cache-clearing behavior with local-uploader warnings.
  • Introduces archive detection, summaries, deletion utilities, tests, and shared cache-path constants.
  • Splits existing tools commands into submodules and updates aggregator documentation.
File Summary Review notes
mithril-aggregator/​src/​tools/​mod.rs Exports immutable archive cache-cleaning utilities. No findings.
mithril-aggregator/​src/​tools/​immutables_archives_cache_cleaner.rs Detects, summarizes, deletes, and tests cached immutable archives. Moderate (2 votes): validation should accept archive numbers with at least five digits, including values ≥100000. Nit (2 votes): use grammatically correct singular/plural “immutable archive(s)” in summaries.
mithril-aggregator/​src/​dependency_injection/​builder/​protocol/​artifacts.rs Uses the shared immutable archive directory constant. No findings.
mithril-aggregator/​src/​dependency_injection/​builder/​mod.rs Defines shared artifact and immutable archive directory constants. No findings.
mithril-aggregator/​src/​commands/​tools_command/​recompute_certificates_hash.rs Extracts certificate hash recomputation into its own command module. Critical (1 vote): remove unused Map and Value imports to satisfy the no-warnings Clippy check.
mithril-aggregator/​src/​commands/​tools_command/​mod.rs Defines the modular tools command and subcommand dispatch. No findings.
mithril-aggregator/​src/​commands/​tools_command/​generate_keypair.rs Extracts keypair generation into its own module. No findings.
mithril-aggregator/​src/​commands/​tools_command/​clear_immutables_archives_cache.rs Implements the new cache inspection and deletion command. Critical (1 vote): remove unused Map and Value imports. Nit (1 vote): correct “immutables archives” to “immutable archives” in the user-facing message.
mithril-aggregator/​src/​commands/​tools_command.rs Removes the former monolithic tools command implementation. No findings.
docs/​website/​root/​manual/​develop/​nodes/​mithril-aggregator.md Documents the new command, options, warnings, and configuration. Nits (1 vote each): add the command to the subcommands table and command summary table, and mirror the updated help and configuration sections in the maintained versioned page.

Review effort: Lite

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Test Results

     5 files  ± 0     236 suites  ±0   47m 35s ⏱️ - 17m 44s
 4 184 tests +16   4 184 ✅ +16  0 💤 ±0  0 ❌ ±0 
13 259 runs  +48  13 259 ✅ +48  0 💤 ±0  0 ❌ ±0 

Results for commit 9344d21. ± Comparison against base commit 955d50f.

♻️ This comment has been updated with latest results.

@Alenar
Alenar deployed to testing-preview November 2, 2026 17:49 — with GitHub Actions Active
@Alenar
Alenar deployed to testing-2-preview November 2, 2026 17:49 — with GitHub Actions Active

@jpraynaud jpraynaud left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@damrobi damrobi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@Alenar
Alenar force-pushed the djo/3653/aggregator/migration-of-immutables-archives branch from fceebea to 35e6e6c Compare November 3, 2026 18:06
@Alenar
Alenar deployed to testing-2-preview November 3, 2026 18:19 — with GitHub Actions Active
@Alenar
Alenar deployed to testing-preview November 3, 2026 18:19 — with GitHub Actions Active

@turmelclem turmelclem left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Alenar added 13 commits November 4, 2026 12:14
…eCertificatesHashCommandConfiguration'

This config struct only applied to the certificate hash recomputation so
its naming was misleading.
this simplify the test and allow catching leftover or unexpected files.
- always warn in 'tools clear-immutables-archives-cache'
- handle immutable file archives with more than 5 digits
- clarify 'snapshot_directory' description.
- fix grammar
* mithril-aggregator from `0.10.29` to `0.10.30`
@Alenar
Alenar force-pushed the djo/3653/aggregator/migration-of-immutables-archives branch from 35e6e6c to 9344d21 Compare November 4, 2026 11:30
@Alenar
Alenar deployed to testing-2-preview November 4, 2026 11:42 — with GitHub Actions Active
@Alenar
Alenar deployed to testing-preview November 4, 2026 11:42 — with GitHub Actions Active
@Alenar
Alenar merged commit 5bffb73 into main Nov 4, 2026
53 checks passed
@Alenar
Alenar deleted the djo/3653/aggregator/migration-of-immutables-archives branch November 4, 2026 12:56

This branch was successfully deployed

2 active deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature 🚀 New implemented feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants