Skip to content

Add terraform utility - #103

Draft
Terri Modrakowski (Modrats) wants to merge 24 commits into
microsoft:mainfrom
Modrats:add-terraform-utility
Draft

Add terraform utility#103
Terri Modrakowski (Modrats) wants to merge 24 commits into
microsoft:mainfrom
Modrats:add-terraform-utility

Conversation

@Modrats

@ModratsTerri Modrakowski (Modrats) commented Jul 14, 2026

Copy link
Copy Markdown

Summary

A Terraform utility under utilities/terraform/ that provisions a complete Microsoft Discovery environment as an alternative to the Bicep path. It deploys the Discovery control plane (supercomputer with a node pool; workspace with a chat model deployment and project; storage container; optional bookshelf) together with the platform prerequisites it needs (VNets and subnets, four least-privilege managed identities with scoped RBAC, a storage account, and private networking).

Approach

  • Provider split. AzAPI drives every Microsoft.Discovery/* resource (not yet in the AzureRM catalog), pinned to GA API 2026-06-01; AzureRM handles all platform primitives for strongly-typed schemas and stable state.
  • Layered modules, single source of truth.modules/platform (network, identities, storage, RBAC, storage container) and modules/control-plane/* (supercomputer, workspace, bookshelf, tool). The root (main.tf) composes them into a single-apply end-to-end deployment — there is one definition of each resource.
  • Two deployment modes. Full stack by default, or bring-your-own: pass existing_supercomputer_id, existing_workspace_id, or existing_bookshelf_id to skip that module and wire your resource into the rest. A BYO supercomputer also skips its now-unused network (VNet, AKS/node-pool subnets, peering).
  • Tool module (extension).modules/control-plane/tool ships as a standalone control-plane module for provisioning Discovery Tools. It is intentionally not wired into the end-to-end root — it is an extension point for teams that provision Tools, callable directly with your own resource IDs.
  • Split-region managed resource groups. The control plane stays in one region while each resource's managed infrastructure — for example the GPU supercomputer — can run in another via the discovery.overridemrgregion tag, backed by a peered supercomputer VNet and cross-VNet private DNS.
  • Tagging.common_tags plus per-resource tag inputs merge so required platform tags are always preserved.
  • Tooling. A deterministic preflight.sh (RP registration, region and VM-SKU allowlists, compute-cores quota, pluggable checks under preflight-checks/) and an idempotent destroy.sh that handles Discovery-specific teardown ordering.

Type of change

  • Other (describe below)

Other: New IaC deployment utility under utilities/terraform/. Parallels the existing Bicep deployment path; no agent, starter-kit, schema, or workflow surface is touched.

Related issue / tracking

n/a

Schema impact

  • No schema changes

Validation checklist

  • terraform fmt and terraform validate pass for the root and every module.
  • Documentation is updated: README.md covers the quickstart, architecture, and deployment modes; ADRs live under docs/adr/.
  • No OS/editor artifacts or secrets committed. State, .terraform/, *.tfplan, and real *.tfvars are gitignored.
  • No model-weight files added.
  • All Markdown links resolve.

Reviewer notes

  • Provider split is intentionalMicrosoft.Discovery/* types are not in the AzureRM catalog yet. See the README intro for the rationale.
  • Modules are the single source of truth. The root and any BYO call use the same modules/control-plane/*; there is no inlined duplicate of a Discovery resource.
  • Workspace children ordering. The project depends on a Succeeded chat model, which the reference graph does not otherwise express; the workspace module encodes this with depends_on between its children.
  • Preflight is deterministic-only by design — no capacity or throttling probes that could give false confidence. Add-on checks under preflight-checks/NN-*.sh are auto-sourced; the contract is documented in the README.
  • Supported regions:eastus, eastus2, uksouth, swedencentral.

@github-actionsgithub-actionsBot added update-branch PR branch is behind base — please update needs-human-review Awaiting human approval labels Jul 14, 2026
@Modrats
Terri Modrakowski (Modrats) marked this pull request as draft July 14, 2026 07:56
@github-actionsgithub-actionsBot added the pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. label Jul 14, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

@github-actionsgithub-actionsBot added pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Jul 14, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

@github-actionsgithub-actionsBot added pr-validation-failed The pr-review workflow's validator failed; see inline comments. and removed pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Jul 21, 2026
@github-actions

Copy link
Copy Markdown

❌ Automated Check Results — 2 issue(s) found

All issues are listed below. Please address all of them before requesting human review.

Rule IDFileIssue
SEC-SCAN(repository)Secret scan failed — potential credentials detected. Check the TruffleHog job for details.
INTERNAL(workflow)Validation job was skipped or did not produce results — check the Validate PR job logs for details.

Once all issues are resolved, push a new commit to re-trigger this check.
When the pr-review validator passes, the pr-validation-passed label will be applied and the maintainers will be auto-requested for review. Note: other status checks (unit-tests, schema regression, etc.) report separately — see the full status rollup at the bottom of this PR.

See the authoring guides for end-to-end walkthroughs.

@github-actionsgithub-actionsBot added pr-validation-failed The pr-review workflow's validator failed; see inline comments. and removed pr-validation-failed The pr-review workflow's validator failed; see inline comments. labels Jul 21, 2026
@github-actions

Copy link
Copy Markdown

❌ Automated Check Results — 2 issue(s) found

All issues are listed below. Please address all of them before requesting human review.

Rule IDFileIssue
SEC-SCAN(repository)Secret scan failed — potential credentials detected. Check the TruffleHog job for details.
INTERNAL(workflow)Validation job was skipped or did not produce results — check the Validate PR job logs for details.

Once all issues are resolved, push a new commit to re-trigger this check.
When the pr-review validator passes, the pr-validation-passed label will be applied and the maintainers will be auto-requested for review. Note: other status checks (unit-tests, schema regression, etc.) report separately — see the full status rollup at the bottom of this PR.

See the authoring guides for end-to-end walkthroughs.

@github-actionsgithub-actionsBot added pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed pr-validation-failed The pr-review workflow's validator failed; see inline comments. labels Jul 27, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

Adds utilities/terraform/, a self-contained Terraform module that provisions
a Microsoft Discovery services environment:
* Virtual network with five subnets (workspaceSubnet + agentSubnet
delegated to Microsoft.App/environments)
* User-assigned managed identity + three role assignments
(Storage Blob Data Contributor, Microsoft Discovery Platform Contributor,
AcrPull)
* Storage account (shared-key auth disabled, public network disabled,
TLS1_2, Studio/VS Code CORS) with one AzureRM-managed blob container
* Discovery Supercomputer + node pool
* Discovery Workspace with one chat model deployment + project
* Discovery StorageContainer bound to the AzureRM storage account
Provider split:
* hashicorp/azurerm ~> 4.20 for every platform primitive
* Azure/azapi ~> 2.0 for all Microsoft.Discovery/* resources
(pinned to @2026-02-01-preview) and for the blob container
control-plane call so a data-plane RBAC grant is not required
Also updates utilities/README.md inventory with the new entry.
New scripts in utilities/terraform/:
- preflight.sh: deterministic pre-apply checks (RP registration, region
allowlist, VM SKU allowlist, compute cores quota) with auto-sourced
add-on modules under preflight-checks/ (one concern per file).
- destroy.sh: idempotent teardown that pre-drains Discovery children
serially, detaches AKS-orphaned NSGs, detects the unrecoverable
legionservicelink SAL, and force-deletes surviving MRGs.
Terraform changes:
- discovery.tf: add explicit depends_on so project creation waits for
the chat model deployment to reach Succeeded (RP-side ordering the
reference graph does not express).
- variables.tf: default location -> uksouth (matches current test region).
- .gitignore: exclude bare 'tfplan' and local TESTING-PLAN.md.
Docs:
- utilities/terraform/README.md: Quickstart TL;DR and preflight
callouts in the setup walkthrough.
- README.md: trailing newline.
…oy.sh
The utility now lets Azure surface the VNet/RG delete failure directly when
an orphaned Microsoft.App/environments serviceAssociationLink
(legionservicelink) is present. Orchestrating support-ticket workflows and
maintaining a quarantined-RG tagging scheme is out of scope for this module.
Removes:
- --abandon-orphan-vnet CLI flag and ABANDON_ORPHAN_VNET variable.
- Step 1.7 (detect_orphaned_sals, print_orphan_sal_guidance) and its ~170
lines of support-ticket template.
- SKIP_RG_DELETE branch in step 2.
- References to stuck-SAL deadlocks in the header comment block.
Kept:
- Step 1.5 serial pre-drain of workspaces/supercomputers (still prevents
the race that creates orphaned SALs in the first place).
- Step 1.6 AKS-orphaned NSG detachment (self-service, unrelated concern).
Bring the Terraform utility up to parity with uami.bicep through the search
subnet:
- Set NetworkIsolation, discovery.workbench.enableGhcpAiFeatures, and
discovery.workbench.enableExtensions tags on the workspace (was version=v2
only). Fixes the broken-hybrid posture where subnet IDs were passed without
NetworkIsolation=true, which disabled Cosmos public access with no private
endpoint and deadlocked teardown.
- Add network_isolation, enable_ghcp_ai_features, enable_extensions variables
(default true) and search_subnet_prefix.
- Add delegated searchSubnet (sixth subnet, 10.0.6.0/24) to match the bicep VNet.
- Update README and terraform.tfvars.example for parity.
Out of scope (post-search-subnet in bicep): four-identity least-privilege
model and its seven role assignments remain a follow-up.
Port the per-identity least-privilege split from uami.bicep:
- Add cluster, kubelet, and workload UAMIs (were all collapsed onto the single
workspace identity).
- Expand to seven scoped role assignments: workspace gets Discovery Platform
Contributor (RG) + Storage Blob Data Contributor (storage); cluster gets
Network Contributor (AKS subnet only); kubelet gets Managed Identity Operator
(cluster identity) + AcrPull (RG) + Storage Blob Data Contributor (storage);
workload gets Storage Blob Data Contributor (storage) only.
- Wire the supercomputer clusterIdentity/kubeletIdentity/workloadIdentities to
the dedicated identities and update create-ordering depends_on.
- Add cluster/kubelet/workload identity name variables, locals, and outputs.
- Update README (four identities, seven roles, 27-resource plan).
- Correct README plan summary: thirteen outputs (was twelve) after adding the
cluster/kubelet/workload identity outputs.
- Add ADR 0001 documenting the network isolation posture decision (default
network_isolation=true, always provision the private topology) and link it
from the README.
Patch isolationScope = "Regional" onto all four user-assigned managed
identities via azapi_update_resource (azurerm 4.x does not expose the property),
matching uami.bicep. Regional isolation restricts each identity to source
resources in its own region, shrinking the blast radius on compromise and
containing identity-plane failures to one region; all source resources here
live in var.location. Update README plan summary (31 resources) and the
identity section.
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

- add existing_supercomputer_id; skip the module and link the workspace to it
- resolve supercomputer_id via coalesce for created-or-existing paths
- document the two deployment modes (full stack vs BYO) in README
🔌 - Generated by Copilot
@github-actionsgithub-actionsBot added pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Aug 10, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

…twork
- add existing_workspace_id and existing_bookshelf_id (skip their modules)
- gate platform supercomputer network behind create_supercomputer_network
- root skips the SC network when existing_supercomputer_id is set
- resolve created-or-existing IDs for all three; document BYO knobs
🧩 - Generated by Copilot
@github-actionsgithub-actionsBot added pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Aug 10, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

- key chat_model_deployments and projects by a stable logical key
- derive the Azure name from an optional name value (defaults to key)
- fixes "Invalid for_each argument" on first plan when names derive from the random suffix
🐛 - Generated by Copilot
@github-actionsgithub-actionsBot added pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Aug 11, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

…elf_id
- Drop eastus2 from location validation allowlist so terraform validate
fails fast instead of misleading users into a ~30m apply failure; mirrors
preflight KNOWN_BAD_REGIONS and 06-approved-regions.sh (RI-2)
- Point providers.tf comment at the real ADR
docs/adr/0001-network-isolation-posture.md (RI-1)
- Explain why bookshelf_id uses a ternary instead of coalesce (RI-3)
- Sync README region references with the tightened allowlist
@github-actionsgithub-actionsBot added pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Aug 13, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

@github-actionsgithub-actionsBot added pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Aug 14, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

Teardown is out of scope for this deployment utility; deletion is
handled by the root delete-discovery-deployment utility.
@github-actionsgithub-actionsBot added pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Aug 14, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

@Modrats

Copy link
Copy Markdown
Author

Terri Modrakowski (Terri Modrakowski (@Modrats)) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

  • (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
  • (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree company="Microsoft"

@github-actionsgithub-actionsBot added pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. and removed pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Aug 14, 2026
@github-actions

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-human-reviewAwaiting human approvalpr-validation-passedThe pr-review workflow's validator passed. Other status checks report separately.update-branchPR branch is behind base — please update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Modrats