Skip to content

Reclaim local tier objects - #6581

Open
kim wants to merge 1 commit into
kim/tiered-storage-integrationfrom
kim/local-tier-reclamation
Open

kim wants to merge 1 commit into
kim/tiered-storage-integrationfrom
kim/local-tier-reclamation

Conversation

@kim

@kim kim commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Tracks backing storage ownership in the page manager and blob set. After transactions and under memory budget pressure, orphaned objects are reclaimed.

API and ABI breaking changes

N/A

Rollback safety impact

N/A

Expected complexity level and risk

3 - Makes the locking protocol in TieredStorage more complicated

Testing

Yes

@kim
kim requested a review from joshua-spacetime November 13, 2026 17:00
@kim
kim force-pushed the kim/local-tier-reclamation branch from 9a7fd8e to 89c0e6c Compare November 13, 2026 17:28
Tracks backing storage ownership in the page manager and blob set. After
transactions and under memory budget pressure, orphaned objects are
reclaimed.
@kim
kim force-pushed the kim/local-tier-reclamation branch from 89c0e6c to 0cff851 Compare November 13, 2026 17:31

let mut freed = 0;
for (key, slot) in slots {
let mut entry = slot.state.lock();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use try_lock() here instead of lock(), and set should_reclaim if the slot is in use? Note that write() can acquire locks in the opposite order: First object and then catalog.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this lock will ever be in use during reclaim, so this might not actually be a problem. But I don't think it hurts regardless.

Comment on lines +633 to +634
let mut slots = catalog
.iter()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a blocker, but I think we should at least add a TODO that reclaim not iterate the entire catalog.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants