Skip to content

direct: grants: Fix "Duplicate privileges" error and drift on duplicates #4801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
denik merged 15 commits into from
Mar 23, 2026
Merged

direct: grants: Fix "Duplicate privileges" error and drift on duplicates #4801

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
ee6b7a6
Add acceptance test reproducing duplicate grants error with ALL_PRIVI…
denik Mar 20, 2026
8c8abd6
testserver: Reject duplicate privileges in Add and Remove for grants
denik Mar 20, 2026
85b4d50
Add duplicate_principals grants test; add Badness annotations
denik Mar 20, 2026
8660651
Add duplicate_privileges grants test; use cleanup() trap in all grant…
denik Mar 20, 2026
cbfc4d3
Add per-engine grants request logging to duplicate grant tests
denik Mar 20, 2026
a283eb1
Merge branch 'main' into denik/duplicate-grants
denik Mar 20, 2026
fb384b4
Fix duplicate grants issues in direct engine's applyGrants()
denik Mar 20, 2026
5b64180
clean up test
denik Mar 23, 2026
c4abccc
use json plan
denik Mar 23, 2026
ebb2fd2
Add MergeGrants mutator to deduplicate grants during initialization
denik Mar 23, 2026
190b3b2
Remove redundant mergeGrantAssignments from applyGrants
denik Mar 23, 2026
11b0db8
Simplify MergeGrants: inline principalString, avoid panics
denik Mar 23, 2026
39aa69d
lint fix
denik Mar 23, 2026
16481c8
remove mutator unit test
denik Mar 23, 2026
0f8d5e6
Update NEXT_CHANGELOG with grants fixes
denik Mar 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions NEXT_CHANGELOG.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,6 +8,8 @@

### Bundles
* engine/direct: Fix drift in grants resource due to privilege reordering ([#4794](https://github.com/databricks/cli/pull/4794))
* engine/direct: Fix 400 error when deploying grants with ALL_PRIVILEGES ([#4801](https://github.com/databricks/cli/pull/4801))
* Deduplicate grant entries with duplicate principals or privileges during initialization ([#4801](https://github.com/databricks/cli/pull/4801))

### Dependency updates

Expand Down
12 changes: 12 additions & 0 deletions acceptance/bundle/resources/grants/schemas/all_privileges/databricks.yml.tmpl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
bundle:
name: schema-dup-grants-$UNIQUE_NAME

resources:
schemas:
apps_schema:
name: schema_dup_grants_$UNIQUE_NAME
catalog_name: main
grants:
- principal: deco-test-user@databricks.com
privileges:
- ALL_PRIVILEGES
14 changes: 14 additions & 0 deletions acceptance/bundle/resources/grants/schemas/all_privileges/out.requests.direct.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]",
"body": {
"changes": [
{
"add": [
"ALL_PRIVILEGES"
],
"principal": "deco-test-user@databricks.com"
}
]
}
}
26 changes: 26 additions & 0 deletions acceptance/bundle/resources/grants/schemas/all_privileges/out.requests.terraform.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
{
"method": "GET",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]"
}
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]",
"body": {
"changes": [
{
"add": [
"ALL_PRIVILEGES"
],
"principal": "deco-test-user@databricks.com"
}
]
}
}
{
"method": "GET",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]"
}
{
"method": "GET",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]"
}
6 changes: 6 additions & 0 deletions acceptance/bundle/resources/grants/schemas/all_privileges/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 16 additions & 0 deletions acceptance/bundle/resources/grants/schemas/all_privileges/output.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/schema-dup-grants-[UNIQUE_NAME]/default/files...
Deploying resources...
Updating deployment state...
Deployment complete!

>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
delete resources.schemas.apps_schema

This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
delete resources.schemas.apps_schema

All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/schema-dup-grants-[UNIQUE_NAME]/default

Deleting files...
Destroy complete!
12 changes: 12 additions & 0 deletions acceptance/bundle/resources/grants/schemas/all_privileges/script
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
envsubst < databricks.yml.tmpl > databricks.yml

cleanup() {
trace $CLI bundle destroy --auto-approve
rm -f out.requests.txt
}
trap cleanup EXIT

# The direct engine puts ALL_PRIVILEGES in both the Add and Remove lists in the PATCH request,
# which the backend rejects with "Duplicate privileges to add and delete".
$CLI bundle deploy
print_requests.py --get //permissions --keep > out.requests.$DATABRICKS_BUNDLE_ENGINE.txt
1 change: 1 addition & 0 deletions acceptance/bundle/resources/grants/schemas/all_privileges/test.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
RecordRequests = true
15 changes: 15 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_principals/databricks.yml.tmpl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
bundle:
name: schema-dup-grants-$UNIQUE_NAME

resources:
schemas:
apps_schema:
name: schema_dup_grants_$UNIQUE_NAME
catalog_name: main
grants:
- principal: deco-test-user@databricks.com
privileges:
- CREATE_TABLE
- principal: deco-test-user@databricks.com
privileges:
- CREATE_TABLE
44 changes: 44 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_principals/out.plan.direct.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
{
"plan_version": 2,
"cli_version": "[DEV_VERSION]",
"lineage": "[UUID]",
"serial": 1,
"plan": {
"resources.schemas.apps_schema": {
"action": "skip",
"remote_state": {
"browse_only": false,
"catalog_name": "main",
"catalog_type": "MANAGED_CATALOG",
"created_at": [UNIX_TIME_MILLIS][0],
"created_by": "[USERNAME]",
"full_name": "main.schema_dup_grants_[UNIQUE_NAME]",
"name": "schema_dup_grants_[UNIQUE_NAME]",
"owner": "[USERNAME]",
"updated_at": [UNIX_TIME_MILLIS][0],
"updated_by": "[USERNAME]"
}
},
"resources.schemas.apps_schema.grants": {
"depends_on": [
{
"node": "resources.schemas.apps_schema",
"label": "${resources.schemas.apps_schema.id}"
}
],
"action": "skip",
"remote_state": {
"securable_type": "schema",
"full_name": "main.schema_dup_grants_[UNIQUE_NAME]",
"__embed__": [
{
"principal": "deco-test-user@databricks.com",
"privileges": [
"CREATE_TABLE"
]
}
]
}
}
}
}
11 changes: 11 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_principals/out.plan.terraform.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
{
"cli_version": "[DEV_VERSION]",
"plan": {
"resources.schemas.apps_schema": {
"action": "skip"
},
"resources.schemas.apps_schema.grants": {
"action": "skip"
}
}
}
17 changes: 17 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_principals/out.requests.direct.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]",
"body": {
"changes": [
{
"add": [
"CREATE_TABLE"
],
"principal": "deco-test-user@databricks.com",
"remove": [
Comment thread
denik marked this conversation as resolved.
"ALL_PRIVILEGES"
]
}
]
}
}
26 changes: 26 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_principals/out.requests.terraform.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
{
"method": "GET",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]"
}
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]",
"body": {
"changes": [
{
"add": [
"CREATE_TABLE"
],
"principal": "deco-test-user@databricks.com"
}
]
}
}
{
"method": "GET",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]"
}
{
"method": "GET",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]"
}
6 changes: 6 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_principals/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_principals/output.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/schema-dup-grants-[UNIQUE_NAME]/default/files...
Deploying resources...
Updating deployment state...
Deployment complete!

>>> [CLI] bundle plan -o json

>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
delete resources.schemas.apps_schema

This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
delete resources.schemas.apps_schema

All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/schema-dup-grants-[UNIQUE_NAME]/default

Deleting files...
Destroy complete!
12 changes: 12 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_principals/script
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
envsubst < databricks.yml.tmpl > databricks.yml

cleanup() {
trace $CLI bundle destroy --auto-approve
rm -f out.requests.txt
}
trap cleanup EXIT

# Same principal listed twice with the same privilege.
trace $CLI bundle deploy
print_requests.py --get //permissions --keep > out.requests.$DATABRICKS_BUNDLE_ENGINE.txt
trace $CLI bundle plan -o json > out.plan.$DATABRICKS_BUNDLE_ENGINE.json
1 change: 1 addition & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_principals/test.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
RecordRequests = true
13 changes: 13 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_privileges/databricks.yml.tmpl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
bundle:
name: schema-dup-grants-$UNIQUE_NAME

resources:
schemas:
apps_schema:
name: schema_dup_grants_$UNIQUE_NAME
catalog_name: main
grants:
- principal: deco-test-user@databricks.com
privileges:
- CREATE_TABLE

@andrewnesterandrewnesterMar 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might worth showing a warning to users about such configs

- CREATE_TABLE
3 changes: 3 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_privileges/out.plan.direct.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@

>>> [CLI] bundle plan
Plan: 0 to add, 0 to change, 0 to delete, 2 unchanged
3 changes: 3 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_privileges/out.plan.terraform.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@

>>> [CLI] bundle plan
Plan: 0 to add, 0 to change, 0 to delete, 2 unchanged
17 changes: 17 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_privileges/out.requests.direct.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]",
"body": {
"changes": [
{
"add": [
"CREATE_TABLE"
],
"principal": "deco-test-user@databricks.com",
"remove": [
"ALL_PRIVILEGES"
]
}
]
}
}
26 changes: 26 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_privileges/out.requests.terraform.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
{
"method": "GET",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]"
}
{
"method": "PATCH",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]",
"body": {
"changes": [
{
"add": [
"CREATE_TABLE"
],
"principal": "deco-test-user@databricks.com"
}
]
}
}
{
"method": "GET",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]"
}
{
"method": "GET",
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_dup_grants_[UNIQUE_NAME]"
}
6 changes: 6 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_privileges/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_privileges/output.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/schema-dup-grants-[UNIQUE_NAME]/default/files...
Deploying resources...
Updating deployment state...
Deployment complete!

>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
delete resources.schemas.apps_schema

This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
delete resources.schemas.apps_schema

All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/schema-dup-grants-[UNIQUE_NAME]/default

Deleting files...
Destroy complete!
12 changes: 12 additions & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_privileges/script
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
envsubst < databricks.yml.tmpl > databricks.yml

cleanup() {
trace $CLI bundle destroy --auto-approve
rm -f out.requests.txt
}
trap cleanup EXIT

# Same privilege listed twice for the same principal in a single entry.
trace $CLI bundle deploy
print_requests.py --get //permissions --keep > out.requests.$DATABRICKS_BUNDLE_ENGINE.txt
trace $CLI bundle plan > out.plan.$DATABRICKS_BUNDLE_ENGINE.txt 2>&1
1 change: 1 addition & 0 deletions acceptance/bundle/resources/grants/schemas/duplicate_privileges/test.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
RecordRequests = true
Loading
Loading