Skip to content

Bug: [Brief description] PVCs aren't cleaned up #375

Description

@sallyom

Jira: RHOAIENG-51934

🐛 Bug Description

[Claude] Recommended Solution
The safest and most aligned with Kubernetes best practices:

Remove OwnerReferences from PVCs (Recommended)
Change: Don't set OwnerReferences on PVCs at all. Manage their lifecycle explicitly.

Pros:

  • PVCs won't be garbage collected when sessions are deleted
  • Users can safely delete session CRs without losing workspace data
  • Temp-content pods can access PVCs without multi-attach issues
  • Session restarts will work reliably

Cons:

  • Need explicit cleanup mechanism for orphaned PVCs
  • Need to add a cleanup controller or manual cleanup commands

Implementation:

// In operator/internal/handlers/sessions.go:220-232
// Change from:
ownerRefs = []v1.OwnerReference{
    {
        APIVersion: "vteam.ambient-code/v1",
        Kind:       "AgenticSession",
        Name:       currentObj.GetName(),
        UID:        currentObj.GetUID(),
        Controller: boolPtr(true),
    },
}

To:

ownerRefs = []v1.OwnerReference{}  // Empty - no garbage collection

// And in operator/internal/services/infrastructure.go:64-68
// Add labels for cleanup:
Labels: map[string]string{
    "app":                            "ambient-workspace",
    "agentic-session":                pvcName,
    "vteam.ambient-code/created-at":  time.Now().Format(time.RFC3339),
},

Then add a cleanup mechanism:

Background cleanup controller:

// Add to operator main.go
go cleanupOrphanedPVCs(30 * time.Minute)

func cleanupOrphanedPVCs(interval time.Duration) {
    for {
        time.Sleep(interval)
        // List all ambient-workspace PVCs
        // For each PVC, check if corresponding session exists
        // If session doesn't exist and PVC is > 24 hours old, delete it
    }
}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions