Skip to content

🐛 OCPBUGS-76453: clean up orphaned temp dirs in catalog storage - #2537

Merged
openshift-merge-bot[bot] merged 1 commit into
operator-framework:mainfrom
tmshort:fix-OCPBUGS-76453
Mar 9, 2026
Merged

🐛 OCPBUGS-76453: clean up orphaned temp dirs in catalog storage#2537
openshift-merge-bot[bot] merged 1 commit into
operator-framework:mainfrom
tmshort:fix-OCPBUGS-76453

Conversation

@tmshort

Copy link
Copy Markdown
Contributor

LocalDirV1.Store creates a temp dir (.{catalog}-{random}) and renames it into place atomically. If the process is interrupted before the deferred RemoveAll runs, the temp dir persists. Each restart adds another, eventually filling the disk.

Primary fix: remove any matching temp dirs at the start of Store before creating a new one. Since Store holds the write mutex, any dirs found are guaranteed to be from a previous run.

Defense-in-depth: add /var/cache/catalogs to GarbageCollector's list of scanned paths (CachePath string -> CachePaths []string), so orphaned temp dirs are also removed by the periodic GC cycle.

Description

Reviewer Checklist

  • API Go Documentation
  • Tests: Unit Tests (and E2E Tests, if appropriate)
  • Comprehensive Commit Messages
  • Links to related GitHub Issue(s)

LocalDirV1.Store creates a temp dir (.{catalog}-{random}) and renames it
into place atomically. If the process is interrupted before the deferred
RemoveAll runs, the temp dir persists. Each restart adds another, eventually
filling the disk.
Primary fix: remove any matching temp dirs at the start of Store before
creating a new one. Since Store holds the write mutex, any dirs found are
guaranteed to be from a previous run.
Defense-in-depth: add /var/cache/catalogs to GarbageCollector's list of
scanned paths (CachePath string -> CachePaths []string), so orphaned temp
dirs are also removed by the periodic GC cycle.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Todd Short <tshort@redhat.com>
CopilotAI review requested due to automatic review settings March 4, 2026 19:16
@netlify

netlifyBot commented Mar 4, 2026

Copy link
Copy Markdown

Deploy Preview for olmv1 ready!

NameLink
🔨 Latest commitd027ce2
🔍 Latest deploy loghttps://app.netlify.com/projects/olmv1/deploys/69a8852cb8ef7600087ba038
😎 Deploy Previewhttps://deploy-preview-2537--olmv1.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses orphaned temporary directories left behind when LocalDirV1.Store is interrupted before its deferred cleanup runs, preventing disk usage from growing across restarts. It also extends periodic garbage collection so it can clean orphaned temp dirs under catalog storage in addition to the unpack cache.

Changes:

  • Clean up any leftover .{catalog}-* temp directories at the start of LocalDirV1.Store.
  • Update GarbageCollector to support scanning multiple cache directories (CachePathCachePaths) and include the catalog storage directory in catalogd’s GC configuration.
  • Add unit tests covering both the Store cleanup behavior and GC cleanup of orphaned temp dirs in the storage path.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
FileDescription
internal/catalogd/storage/localdir.goAdds pre-Store cleanup of orphaned .{catalog}-* temp dirs under the storage root.
internal/catalogd/storage/localdir_test.goAdds a test ensuring orphaned temp dirs for the same catalog are removed while unrelated ones remain.
internal/catalogd/garbagecollection/garbage_collector.goReworks GC config to accept multiple paths and runs GC per path with path-aware logging.
internal/catalogd/garbagecollection/garbage_collector_test.goAdds a test ensuring GC removes orphaned storage temp dirs while preserving the real catalog directory.
cmd/catalogd/main.goConfigures GC to scan both unpack cache and catalog storage directories.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@codecov

codecovBot commented Mar 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.83333% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.78%. Comparing base (55473d8) to head (d027ce2).
⚠️ Report is 6 commits behind head on main.

Files with missing linesPatch %Lines
internal/catalogd/storage/localdir.go71.42%2 Missing and 2 partials ⚠️
...al/catalogd/garbagecollection/garbage_collector.go66.66%2 Missing and 1 partial ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #2537 +/- ##
==========================================
+ Coverage 68.62% 68.78% +0.16% 
==========================================
Files 131 131 Lines 9288 9301 +13 ==========================================
+ Hits 6374 6398 +24 + Misses 2427 2419 -8 + Partials 487 484 -3 
FlagCoverage Δ
e2e42.40% <41.66%> (+0.05%)⬆️
experimental-e2e52.17% <41.66%> (+0.15%)⬆️
unit53.76% <62.50%> (+0.05%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pedjakpedjak left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ciopenshift-ciBot added the lgtm Indicates that a PR is ready to be merged. label Mar 5, 2026

@rashmigottipatirashmigottipati left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@pedjak

Copy link
Copy Markdown
Contributor

/approved

@pedjak

Copy link
Copy Markdown
Contributor

/approve

@openshift-ci

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pedjak, rashmigottipati

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ciopenshift-ciBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 5, 2026
@camilamacedo86

Copy link
Copy Markdown
Contributor

/override codecov/patch

@openshift-ci

Copy link
Copy Markdown

@camilamacedo86: Overrode contexts on behalf of camilamacedo86: codecov/patch

Details

In response to this:

/override codecov/patch

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@camilamacedo86

Copy link
Copy Markdown
Contributor

/override patch

@openshift-ci

Copy link
Copy Markdown

@camilamacedo86: /override requires failed status contexts, check run or a prowjob name to operate on.
The following unknown contexts/checkruns were given:

  • patch

Only the following failed contexts/checkruns were expected:

  • Verify PR title
  • crd-diff
  • e2e
  • experimental-e2e
  • extension-developer-e2e
  • go-apidiff
  • go-verdiff
  • goreleaser
  • lint
  • netlify/olmv1/deploy-preview
  • st2ex-e2e
  • tide
  • unit-test-basic
  • upgrade-st2st-e2e
  • verify

If you are trying to override a checkrun that has a space in it, you must put a double quote on the context.

Details

In response to this:

/override patch

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-merge-bot
openshift-merge-botBot merged commit 95afce0 into operator-framework:mainMar 9, 2026
37 of 39 checks passed
@tmshort
tmshort deleted the fix-OCPBUGS-76453 branch March 19, 2026 15:15
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.lgtmIndicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@tmshort@pedjak@camilamacedo86@rashmigottipati