Add V2 namespace and physical container resources

[David Negstad (danegsta)]

Why

This starts the V2 resource model so new DCP resources can require namespaces, support namespace-scoped cleanup, and avoid repeating direct runtime lifecycle logic across V1 controllers.

V1 resources remain cluster-scoped and continue to work without namespaces. The new V2 model gives us a place to build cleaner physical runtime primitives first, then layer higher-level logical resources on top.

What changed

• Adds the V2 API group with Namespace, PhysicalContainerImage, and PhysicalContainer resources.
• Adds V2 namespace cleanup and side-effect blocking for missing, terminating, or inactive namespaces.
• Adds physical container/image controllers with queued long-running work, in-memory progress data, Ready conditions, runtime event watches, referenced image watches, and port mapping status.
• Defines physical-resource ownership semantics: existing runtime IDs are always retained, while persistent and replaceExisting apply only to newly created runtime objects.
• Moves shared container API fragments into pkg/commonapi and decouples container orchestrator packages from V1/V2 API packages.
• Adds shared cleanup ordering metadata and updates shutdown cleanup paths for V2 resources.
• Adds docs/v2-resource-plan.md with V2 guidelines and the follow-up roadmap.

Notes for reviewers

This is intentionally the initial V2 foundation rather than a complete V2 application model. Physical resources represent concrete runtime objects; future logical resources will own persistence/reuse policy and delegate runtime work to these physical primitives.

The V1 JSON API remains compatible. Shared fields added to common fragments are optional, and V1 resources remain cluster-scoped.

Validation

• make generate
• make test-prereqs
• Focused Go tests for changed API/controller/helper packages
• Targeted V2 integration tests
• Manual Docker smoke test for V2 namespace cleanup

[Copilot]

Pull request overview

Introduces the initial namespaced V2 resource model and physical container primitives while retaining V1 compatibility.

Changes:

• Adds V2 namespace, image, and container APIs and controllers.
• Moves reusable container definitions into commonapi.
• Extends cleanup, orchestration, generation, and integration coverage.

The namespace-finalization race, V1 lifecycle-key regression, and missing reference/protocol validation require changes.

Reviewed changes

Copilot reviewed 98 out of 103 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
Makefile	Generates API artifacts for V1 and V2.
api/v1/common_types.go	Re-exports shared API definitions.
api/v1/container_exec_types.go	Uses shared environment types.
api/v1/container_network_connection_types.go	Uses shared networking types.
api/v1/container_types.go	Migrates container fragments to commonapi.
api/v1/container_types_test.go	Updates V1 container tests.
api/v1/executable_replica_set_types.go	Uses shared environment definitions.
api/v1/executable_types.go	Uses shared environment definitions.
api/v1/executable_types_test.go	Updates shared-type assertions.
api/v1/groupversion_info.go	Shares resource-creation state.
api/v1/service_types.go	Uses shared protocol definitions.
api/v1/zz_generated.deepcopy.go	Regenerates V1 deep-copy code.
api/v1/zz_generated.model_name.go	Regenerates V1 model names.
api/v2/common_types.go	Adds V2 metadata helpers and validation.
api/v2/common_types_test.go	Tests V2 metadata validation.
api/v2/doc.go	Declares the V2 API package.
api/v2/groupversion_info.go	Registers the V2 API group.
api/v2/namespace_types.go	Defines the V2 Namespace resource.
api/v2/namespace_types_test.go	Tests Namespace validation.
api/v2/physical_container_image_types.go	Defines physical image APIs.
api/v2/physical_container_image_types_test.go	Tests physical image validation.
api/v2/physical_container_types.go	Defines physical container APIs.
api/v2/physical_container_types_test.go	Tests physical container validation.
api/v2/zz_generated.deepcopy.go	Adds V2 deep-copy implementations.
api/v2/zz_generated.model_name.go	Adds V2 model names.
controllers/container_common.go	Adapts shared container definitions.
controllers/container_controller.go	Converts V1 specs to runtime options.
controllers/container_exec_controller.go	Migrates shared environment types.
controllers/container_network_tunnel_proxy_controller.go	Updates shared runtime types.
controllers/container_watcher.go	Generalizes container event handling.
controllers/controller_common.go	Adds namespace and status helpers.
controllers/executable_controller.go	Migrates shared environment types.
controllers/executable_run_info.go	Uses common controller setters.
controllers/executable_start_result.go	Uses common controller setters.
controllers/namespace_controller.go	Implements namespace lifecycle cleanup.
controllers/object_state_map.go	Adds deferred operation tracking.
controllers/object_state_map_test.go	Tests deferred state behavior.
controllers/physical_container_controller.go	Implements physical container reconciliation.
controllers/physical_container_data.go	Stores physical container progress.
controllers/physical_container_image_controller.go	Implements image reconciliation.
controllers/physical_container_image_data.go	Stores image operation progress.
controllers/running_container_data.go	Updates shared runtime state.
controllers/service_controller_metrics.go	Uses shared protocol types.
controllers/volume_controller.go	Adopts common setter conventions.
docs/v2-resource-plan.md	Documents the V2 architecture roadmap.
internal/apiserver/admin_http_handler_test.go	Updates shutdown tests.
internal/apiserver/apiserver.go	Registers V2 API resources.
internal/appmgmt/shutdown.go	Includes V2 resources in shutdown.
internal/containers/container_orchestrator.go	Decouples orchestrator options from APIs.
internal/containers/containers_common.go	Adds API-neutral container types.
internal/containers/image_layers.go	Uses shared build definitions.
internal/containers/image_layers_test.go	Updates image-layer tests.
internal/containers/image_orchestrator.go	Adds image operation options.
internal/containers/test_container_orchestrator_client.go	Updates orchestrator test client.
internal/dcp/commands/cleanup_test.go	Covers revised cleanup metadata.
internal/dcpclient/client.go	Adds V2 namespace workload resolution.
internal/dcpclient/client_test.go	Tests V2 workload resolution.
internal/dcpclient/scheme.go	Registers the V2 scheme.
internal/dcpctrl/commands/run_controllers.go	Starts the V2 controllers.
internal/dcpproc/dcpproc_test.go	Updates shared-type usage.
internal/dcptun/image_builder.go	Uses API-neutral image options.
internal/dcptun/tunnel_test.go	Updates shared-type usage.
internal/docker/cli_orchestrator.go	Implements API-neutral Docker options.
internal/docker/cli_orchestrator_test.go	Tests revised Docker options.
internal/exerunners/ide_requests_responses.go	Uses shared environment types.
internal/exerunners/ide_requests_responses_test.go	Updates IDE protocol tests.
internal/exerunners/process_executable_runner.go	Uses shared environment types.
internal/networking/networking.go	Uses shared protocol definitions.
internal/networking/networking_test.go	Updates networking tests.
internal/podman/cli_orchestrator.go	Implements API-neutral Podman options.
internal/podman/cli_orchestrator_test.go	Tests revised Podman options.
internal/proxy/netproxy.go	Uses shared protocol definitions.
internal/proxy/proxy_api.go	Uses shared protocol definitions.
internal/proxy/proxy_test.go	Updates proxy tests.
internal/resourcecleanup/resources.go	Adds ordered V2 cleanup metadata.
internal/resourcecleanup/resources_test.go	Tests cleanup ordering.
internal/templating/value_templates.go	Uses shared container fragments.
internal/testutil/ctrlutil/test_container_orchestrator.go	Supports V2 controller testing.
internal/testutil/ctrlutil/test_proxy.go	Uses shared protocols.
pkg/commonapi/annotations.go	Adds common annotation definitions.
pkg/commonapi/annotations_test.go	Tests common annotations.
pkg/commonapi/container_deepcopy.go	Adds shared deep-copy helpers.
pkg/commonapi/container_types.go	Defines shared container API fragments.
pkg/commonapi/resource_creation.go	Shares shutdown creation state.
pkg/commonapi/types.go	Adds common workload identifiers.
pkg/generated/openapi/zz_generated.openapi.go	Regenerates V1/V2 OpenAPI schemas.
pkg/kubeconfig/kubeconfig.go	Includes V2 API configuration.
test/integration/advanced_test_env.go	Registers V2 test resources.
test/integration/container_controller_test.go	Migrates shared container types.
test/integration/container_network_connection_test.go	Migrates network types.
test/integration/container_network_tunnel_proxy_test.go	Migrates tunnel test types.
test/integration/controllers_common_test.go	Adds V2 controller test helpers.
test/integration/endpoints_test.go	Migrates shared protocol types.
test/integration/executable_controller_test.go	Migrates environment types.
test/integration/executable_replica_set_controller_test.go	Migrates environment types.
test/integration/resource_harvesting_test.go	Includes V2 cleanup behavior.
test/integration/service_controller_test.go	Migrates shared service types.
test/integration/service_controller_test_not_darwin.go	Migrates platform service tests.
test/integration/standard_test_env.go	Starts V2 reconcilers in tests.
test/integration/test_env_common.go	Adds V2 controller selection flags.
test/integration/v2_namespace_controller_test.go	Tests namespace activation and cleanup.
test/integration/v2_physical_container_controller_test.go	Tests physical container lifecycle behavior.
test/integration/v2_physical_container_image_controller_test.go	Tests image pull and build behavior.

Files not reviewed (4)

• api/v1/zz_generated.deepcopy.go: Generated file
• api/v1/zz_generated.model_name.go: Generated file
• api/v2/zz_generated.deepcopy.go: Generated file
• api/v2/zz_generated.model_name.go: Generated file

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Karol Zadora-Przylecki (karolz-ms)]

🤖 Copilot Code Review

Independent review of the code before reading the PR description, then reconciled against it. This is a review without approval — 19 findings below (5 ❌ error, 9 ⚠️ warning, 5 💡 suggestion), posted as inline comments.

I deliberately skipped the 4 findings already raised by the earlier automated review (containerPortEnd lifecycle key, namespace-finalization race, imageRef name validation, port protocol validation) to avoid duplicate feedback. Note that finding on api/v1/container_types.go:499 is a second, independent cause of the lifecycle-key regression that the existing containerPortEnd fix would not address.

Holistic assessment: ⚠️ Needs Changes

Motivation is sound: a namespaced V2 model with physical primitives gives DCP one place to implement runtime lifecycle instead of duplicating orchestrator calls across V1 controllers, and docs/v2-resource-plan.md lays out a credible roadmap.

Approach bundles two largely independent changes: (1) a mechanical move of shared container API fragments into pkg/commonapi plus a rework of the orchestrator CreateContainerOptions boundary, touching ~70 existing files; and (2) the genuinely new V2 API group + three controllers. The new controllers follow existing repo patterns closely (ReconcilerBase, ObjectStateMap, resiliency.WorkQueue, ContainerWatcher), which is good.

I'd suggest splitting this PR. The api/v1 -> pkg/commonapi move is self-contained and independently reviewable/revertable; separating it would make the V1 regression risk far easier to reason about, and would let the V2 foundation be evaluated on its own.

The blocking concerns are concentrated in the V1 blast radius and the steady-state cost of the new controllers, not in the V2 design itself:

• The commonapi move silently changes the V1 persistent-container lifecycle key (two independent causes), causing existing persistent containers to be removed and recreated on upgrade.
• Both new controllers requeue unconditionally every ~5-7s forever, each cycle shelling out to docker inspect.
• V2-created containers carry no creator-process labels, so the harvester can never reclaim them after a crash — and removeOnDeletion defaults to false, so they also survive a clean shutdown.

Where a human reviewer should focus / where I'm less confident:

• The lifecycle-key finding is the one I'd most want a second opinion on. I verified empirically that gob embeds struct name + field list in the hashed byte stream, and that container_controller.go:445 removes+recreates on mismatch — but whether the resulting churn is acceptable in practice is a product call.
• The api/v1 source-break + OpenAPI $ref rename: I don't know whether anything outside this repo consumes api/v1 as a Go package or generates clients from the DCP OpenAPI document. Someone who owns the Aspire/DCP client contract should confirm.
• The reconcile-frequency cost depends on typical container counts; I did not measure it.

Validation performed locally on this branch

Check	Result
go build ./api/... ./controllers/... ./pkg/... ./internal/resourcecleanup/...	✅
make lint	✅ 0 issues
make generate	✅ no drift in generated files
go test ./api/... ./pkg/commonapi/... ./controllers/... ./internal/dcpclient/... ./internal/resourcecleanup/...	✅
go test ./test/integration/... (full suite)	✅

No go.mod changes, so no new external dependencies.

Highest-risk files for human attention

File	Risk categories
api/v1/container_types.go, api/v1/common_types.go, api/v1/groupversion_info.go	Public API
api/v2/*_types.go, pkg/commonapi/container_types.go	Public API
pkg/generated/openapi/zz_generated.openapi.go	Public API, Network Communications
controllers/physical_container{,_image}_controller.go	Controller Code, Concurrency and Resiliency, Process Manipulation
controllers/namespace_controller.go, controllers/object_state_map.go	Controller Code, Concurrency and Resiliency
controllers/container_controller.go, container_common.go, running_container_data.go	Controller Code, Process Manipulation
internal/containers/*, internal/docker/cli_orchestrator.go, internal/podman/cli_orchestrator.go	Process Manipulation, Program Invocation
internal/dcpctrl/commands/run_controllers.go, internal/appmgmt/shutdown.go, internal/resourcecleanup/resources.go	Program Invocation, Controller Code
internal/apiserver/apiserver.go, internal/dcpclient/*	Public API, Network Communications
pkg/kubeconfig/kubeconfig.go	Security

[Copilot]

Pull request overview

Copilot reviewed 102 out of 108 changed files in this pull request and generated 3 comments.

Files not reviewed (4)

• api/v1/zz_generated.deepcopy.go: Generated file
• api/v1/zz_generated.model_name.go: Generated file
• api/v2/zz_generated.deepcopy.go: Generated file
• api/v2/zz_generated.model_name.go: Generated file