Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -83,9 +83,11 @@ Publish local artifact files to an Azure APIM service.
| `--service-name <name>` | *(required)* | APIM service name |
| `--source <dir>` | `./apim-artifacts` | Source artifacts directory |
| `--overrides <path>` | | Path to overrides file |
| `--filter <path>` | | Filter YAML file (same format as `extract`) |
| `--no-transitive` | | Publish only filter matches, without referenced dependencies |
| `--commit-id <sha>` | | Git commit SHA for incremental publish |
| `--dry-run` | | Preview changes without applying |
| `--delete-unmatched` | | Delete resources not in artifacts (mutually exclusive with `--commit-id`) |
| `--delete-unmatched` | | Delete resources not in artifacts (mutually exclusive with `--filter` and `--commit-id`) |

```bash
apiops publish --help
Expand All@@ -107,8 +109,19 @@ apiops publish \
--resource-group <rg> \
--service-name <name> \
--commit-id <sha>

# Publish a filtered subset; transitive dependencies are included by default
apiops publish \
--resource-group <rg> \
--service-name <name> \
--source ./apim-artifacts \
--filter ./filter.yaml
```

The publish filter uses the same YAML file and matching rules as `apiops extract --filter`.
Referenced named values, backends (including backend pool members), policy fragments, and version
sets are included automatically. Add `--no-transitive` to publish only the exact filter matches.

### `apiops init`

Scaffold a new APIM artifacts repository with CI/CD pipelines.
Expand Down
1 change: 1 addition & 0 deletions docs/architecture.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -109,6 +109,7 @@ flowchart TB
extract_svc --> apim_client
extract_svc --> store

publish_svc --> filter_svc
publish_svc --> override_svc
publish_svc --> git_svc
publish_svc --> dry_svc
Expand Down
18 changes: 17 additions & 1 deletion docs/commands/publish.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -44,6 +44,20 @@ apiops publish \
--commit-id abc123def456
```

### Publish a filtered subset

```bash
apiops publish \
--resource-group my-rg \
--service-name my-apim \
--filter ./configuration.extractor.yaml
```

The filter uses the same YAML format as `apiops extract --filter`. Referenced named values,
backends (including backend pool members), policy fragments, and version sets are included by
default; use `--no-transitive` to publish only exact matches. Product, gateway, and subscription
links do not pull their API or Product targets into the publish set.

### Delete resources not in source

```bash
Expand DownExpand Up@@ -73,11 +87,13 @@ apiops publish \
| `--service-name <name>` | string | — | Yes | APIM service instance name |
| `--source <dir>` | string | `./apim-artifacts` | No | Source directory containing artifacts |
| `--overrides <path>` | string | — | No | Override configuration YAML file |
| `--filter <path>` | string | — | No | Filter YAML file shared with `extract` |
| `--no-transitive` | boolean | `false` | No | Publish only exact filter matches |
| `--commit-id <sha>` | string | env: `COMMIT_ID` | No | Git commit SHA for incremental publish |
| `--dry-run` | boolean | `false` | No | Preview changes without applying |
| `--delete-unmatched` | boolean | `false` | No | Delete APIM resources not present in source |

> **Note:** `--commit-id` and `--delete-unmatched` are **mutually exclusive**. The CLI will error if both are specified.
> **Note:** `--filter` and `--commit-id` can be combined; `--delete-unmatched` cannot be combined with either option.

### Global flags

Expand Down
58 changes: 39 additions & 19 deletions docs/guides/filtering-resources.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -64,7 +64,7 @@ documentations: []
workspaces: []
```

With transitive resolution enabled (the default), any version set, backend, named value, policy fragment, or tag directly referenced by `my-own-api` or its policies is still pulled in automatically — even though those keys are set to `[]`. Use `--no-transitive` to disable that behavior.
With transitive resolution enabled (the default), any version set, backend, named value, or policy fragment directly referenced by `my-own-api` or its policies is still pulled in automatically — even though those keys are set to `[]`. Use `--no-transitive` to disable that behavior.

The three states for every key are:

Expand DownExpand Up@@ -200,16 +200,14 @@ namedValues:
- '!keyvault-*'
```

Exclusions work anywhere a string list is accepted, including sub-filter fields inside `apiSubFilters` and `workspaceSubFilters`:
Exclusions work anywhere a string list is accepted, including nested API and workspace sub-filters:

```yaml
apis:
- 'my-api'
apiSubFilters:
my-api:
operations:
- 'get-*'
- '!get-internal-*' # keep all get-* operations except internal ones
- 'my-api':
operations:
- 'get-*'
- '!get-internal-*' # keep all get-* operations except internal ones
```

---
Expand DownExpand Up@@ -260,7 +258,7 @@ Supported workspace sub-filter keys: `apis`, `backends`, `diagnostics`, `groups`

---

## All 16 Filterable Resource Types
## All 17 Filterable Resource Types

| Filter Field | APIM Resource | Example Values |
|-------------|---------------|----------------|
Expand DownExpand Up@@ -295,10 +293,9 @@ flowchart TD
A[Filtered API] --> B[Backends referenced in policies]
A --> C[Named Values referenced in policies]
A --> D[Policy Fragments included in policies]
A --> E[Tags attached to the API]
A --> F[Products containing the API]
A --> G[Diagnostics configured on the API]
A --> H[Version Set the API belongs to]
A --> E[Version Set the API belongs to]
B --> F[Backend pool members]
D --> G[Resources referenced by fragment content]
```

### Example
Expand All@@ -315,11 +312,23 @@ If `petstore-api` has a policy that references:
- Named value `petstore-api-key` → **auto-included**
- Policy fragment `rate-limit-fragment` → **auto-included**

And `petstore-api` is assigned to:
- Product `starter` → **auto-included**
- Tag `production` → **auto-included**
The extract output includes these dependencies even though only `apis` was specified in the filter.
Product, gateway, tag, and subscription links do not reverse the dependency direction or
auto-include composite API or Product targets.

### API Children Are Still Extracted

Tags and diagnostics were not removed from API extraction. They are API child artifacts, not
transitive dependencies:

The extract output includes all of these, even though only `apiNames` was specified in the filter.
- Selecting an API also extracts its API tag associations and API diagnostics.
- Use a nested API `diagnostics` filter to narrow or exclude API diagnostics.
- Product selection remains independent. Selecting an API does not find and include Products that
contain it; omit `products` to include all Products, list specific Products, or use `products: []`
to include none.

Top-level `tags` and `diagnostics` filter their resource definitions independently from API child
artifacts. This follows the general rule that every top-level filter key is independent.

### Why Transitive Matters

Expand All@@ -329,7 +338,7 @@ Without transitive resolution, publishing the extracted artifacts to a new APIM

## Disabling Transitive Dependencies

Use `--no-transitive` to extract **only** the explicitly listed resources:
Use `--no-transitive` to disable referenced dependency expansion:

```bash
apiops extract \
Expand All@@ -345,6 +354,9 @@ apiops extract \
- You want a minimal extract and will handle missing references manually
- Debugging — to see exactly what was explicitly filtered

Selected parent resources still include their normal child artifacts. For example, selecting an API
still extracts its tags and diagnostics unless nested API sub-filters exclude them.

> ⚠️ **Caution:** Extracted artifacts without transitive dependencies may not be publishable standalone. You'll need to ensure all referenced resources exist in the target APIM instance.

---
Expand DownExpand Up@@ -401,7 +413,15 @@ backends:

### Full Extract Minus Specific Resources

There is no "exclude" syntax. To extract everything except certain resources, list all the resources you _do_ want. For large instances, it's often easier to extract everything and use `.gitignore` or separate branches to manage visibility.
Use `!`-prefixed entries to extract everything except specific resources. For example:

```yaml
apis:
- '*'
- '!legacy-api'
```

For large instances, it can still be easier to extract everything and use `.gitignore` or separate branches to manage visibility.

### Pattern-Based Team Filtering

Expand Down
28 changes: 22 additions & 6 deletions src/cli/publish-command.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,15 +3,15 @@
/**
* Publish command CLI registration
* Commander subcommand with --resource-group, --service-name, --source,
* --overrides, --dry-run, --delete-unmatched flags.
* --overrides, --filter, --no-transitive, --dry-run, --delete-unmatched flags.
* Includes --format json: machine-readable JSON output mode.
*/

import { Command } from 'commander';
import { PublishConfig } from '../models/config.js';
import { ApimServiceContext } from '../models/types.js';
import { runPublish, PublishResult } from '../services/publish-service.js';
import { loadOverrideConfig } from '../lib/config-loader.js';
import { loadFilterConfig, loadOverrideConfig } from '../lib/config-loader.js';
import { logger, parseLogLevel } from '../lib/logger.js';
import { ApimClient } from '../clients/apim-client.js';
import { ArtifactStore } from '../clients/artifact-store.js';
Expand All@@ -25,6 +25,8 @@ interface PublishOptions {
serviceName: string;
source: string;
overrides?: string;
filter?: string;
transitive: boolean;
commitId?: string;
dryRun: boolean;
deleteUnmatched: boolean;
Expand All@@ -40,6 +42,8 @@ export function createPublishCommand(): Command {
.requiredOption('--service-name <name>', 'APIM service instance name')
.option('--source <dir>', 'Source directory with artifacts', './apim-artifacts')
.option('--overrides <path>', 'Override configuration YAML file')
.option('--filter <path>', 'Filter configuration YAML file')
.option('--no-transitive', 'Disable transitive dependency inclusion')
.option(
'--commit-id <sha>',
'Git commit SHA for incremental publish (overrides COMMIT_ID env var)'
Expand DownExpand Up@@ -121,15 +125,24 @@ async function executePublish(
}
}

let filterConfig;
if (options.filter) {
filterConfig = await loadFilterConfig(options.filter);
if (!filterConfig) {
logger.error(`Filter file not found: ${options.filter}`);
process.exit(2);
}
}

// Resolve commit ID for incremental publish
const commitId = options.commitId ?? process.env.COMMIT_ID;
if (commitId) {
logger.debug(`Using incremental publish with commit ID: ${commitId}`);
}

if (hasMutuallyExclusivePublishOptions(options.deleteUnmatched, commitId)) {
if (hasMutuallyExclusivePublishOptions(options.deleteUnmatched, commitId, Boolean(options.filter))) {
logger.error(
'Options --commit-id (or COMMIT_ID) and --delete-unmatched are mutually exclusive.'
'Option --delete-unmatched cannot be combined with --filter or --commit-id (or COMMIT_ID).'
);
process.exit(2);
}
Expand All@@ -138,6 +151,8 @@ async function executePublish(
const publishConfig: PublishConfig = {
service: context,
sourceDir: options.source,
filter: filterConfig,
includeTransitive: options.transitive,
overrides: overrideConfig,
dryRun: options.dryRun,
deleteUnmatched: options.deleteUnmatched,
Expand DownExpand Up@@ -167,9 +182,10 @@ async function executePublish(
*/
export function hasMutuallyExclusivePublishOptions(
deleteUnmatched: boolean,
commitId?: string
commitId?: string,
hasFilter = false
): boolean {
return deleteUnmatched && Boolean(commitId);
return deleteUnmatched && (Boolean(commitId) || hasFilter);
}

/**
Expand Down
17 changes: 17 additions & 0 deletions src/lib/resource-path.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -143,6 +143,23 @@ export function getNamePart(nameParts: string[], index: number): string {
return value;
}

/**
* Create a case-insensitive identity key for a resource descriptor.
*/
export function getResourceDescriptorKey(descriptor: ResourceDescriptor): string {
return `${descriptor.type}:${descriptor.workspace ?? ''}:${descriptor.nameParts.join('/')}`.toLowerCase();
}

/**
* Compare resource descriptors using APIM's case-insensitive names.
*/
export function sameResourceDescriptor(
left: ResourceDescriptor,
right: ResourceDescriptor
): boolean {
return getResourceDescriptorKey(left) === getResourceDescriptorKey(right);
}

/**
* Converts a positional template string to a capturing regex.
* Each `{i}` placeholder becomes a `([^/]+)` capture group; all other
Expand Down
2 changes: 2 additions & 0 deletions src/models/config.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -75,6 +75,8 @@ export interface FilterConfig {
export interface PublishConfig {
service: ApimServiceContext;
sourceDir: string;
filter?: FilterConfig;
includeTransitive?: boolean;
overrides?: OverrideConfig;
dryRun: boolean;
deleteUnmatched: boolean;
Expand Down
Loading