Skip to content

Commit 8cbd384

Browse files
JingsongLiJingsongLi
authored
[docs] Add REST management OpenAPI (#9406)
1 parent 5d6a49c commit 8cbd384

6 files changed

Lines changed: 1853 additions & 136 deletions

File tree

‎docs/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ The site will be available at http://localhost:3000/docs/master/.
2222
## Build
2323

2424
```bash
25-
# Validate the REST Catalog OpenAPI contract
25+
# Validate the REST OpenAPI contracts
2626
yarn test:rest-openapi
2727

2828
# Production build

‎docs/docs/concepts/rest/index.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,4 +63,5 @@ RESTCatalog supports multiple access authentication methods, including the follo
6363

6464
## REST Open API
6565

66-
See [REST API](./rest-api).
66+
See [REST Catalog API](./rest-api) for catalog operations and
67+
[REST Management API](./management-api) for permission and data-policy management.

‎docs/docs/concepts/rest/management-api.md‎

Lines changed: 232 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,232 @@
1+
---
2+
title: "REST Management API"
3+
hide_table_of_contents: true
4+
---
5+
6+
<!--
7+
Licensed to the Apache Software Foundation (ASF) under one or more
8+
contributor license agreements. See the NOTICE file distributed with
9+
this work for additional information regarding copyright ownership.
10+
The ASF licenses this file to You under the Apache License, Version 2.0
11+
(the "License"); you may not use this file except in compliance with
12+
the License. You may obtain a copy of the License at
13+
14+
http://www.apache.org/licenses/LICENSE-2.0
15+
16+
Unless required by applicable law or agreed to in writing, software
17+
distributed under the License is distributed on an "AS IS" BASIS,
18+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
19+
See the License for the specific language governing permissions and
20+
limitations under the License.
21+
-->
22+
23+
The REST Management API is an experimental OpenAPI 3.1 control-plane extension for object
24+
privileges, row filters, and column masks in a Paimon REST Catalog. Its current contract version is
25+
`1.0` and may evolve incompatibly while the design is being validated.
26+
27+
`RESTCatalog` exposes `permissionManagement()` and `policyManagement()` directly. These methods are
28+
intentionally not part of the generic `Catalog` interface. Other catalog implementations do not
29+
expose this management contract.
30+
31+
## Catalog addressing
32+
33+
All management endpoints use the opaque `prefix` returned by the REST Catalog config endpoint. It
34+
is not a catalog name in a payload and is independent of local engine catalog aliases.
35+
36+
```
37+
GET /v1/{prefix}/permissions
38+
POST /v1/{prefix}/permissions/grant
39+
POST /v1/{prefix}/permissions/revoke
40+
41+
GET /v1/{prefix}/databases/{database}/tables/{table}/policies
42+
POST /v1/{prefix}/databases/{database}/tables/{table}/policies
43+
POST /v1/{prefix}/databases/{database}/tables/{table}/policies/drop
44+
```
45+
46+
Policies are currently attached only to tables. The path is the attachment identity, so policy
47+
request bodies do not repeat a catalog, database, table, or resource type. Catalog- and
48+
database-level matching can be added later with explicit matching semantics instead of implied
49+
path inheritance.
50+
51+
The complete wire contract is available in
52+
[`rest-management-open-api.yaml`](/rest-management-open-api.yaml).
53+
54+
## Privileges and policies are independent
55+
56+
A permission grants one access on one resource to one principal. A data policy restricts rows or
57+
columns visible through an already-authorized read. Creating a policy never grants `SELECT`, and
58+
revoking `SELECT` does not delete policies.
59+
60+
This separation also defines the expected query path:
61+
62+
1. The server evaluates object privileges.
63+
2. The server resolves all row-filter and column-masking policies applicable to the caller.
64+
3. The existing REST Catalog table authorization endpoint returns the stored Paimon predicate and
65+
column transforms to the engine.
66+
4. The engine applies those restrictions when planning the scan.
67+
68+
Management payloads use the same serialized Paimon `Predicate` and `Transform` representation as
69+
the existing `AuthTableQueryResponse`. Policy conflict detection, schema validation, and principal
70+
resolution are server responsibilities.
71+
72+
## Permission model
73+
74+
Permission resources are structured objects:
75+
76+
| Resource type | Required locator | Example |
77+
| --- | --- | --- |
78+
|`CATALOG`| none |`{"type":"CATALOG"}`|
79+
|`CATALOG_ALL`| none |`{"type":"CATALOG_ALL"}`|
80+
|`DATABASE`|`database`|`{"type":"DATABASE","database":"sales"}`|
81+
|`DATABASE_ALL`|`database`|`{"type":"DATABASE_ALL","database":"sales"}`|
82+
|`TABLE`|`database`, `table`|`{"type":"TABLE","database":"sales","table":"orders"}`|
83+
|`COLUMN`|`database`, `table`|`{"type":"COLUMN","database":"sales","table":"orders"}`|
84+
|`FUNCTION`|`database`, `function`|`{"type":"FUNCTION","database":"sales","function":"calculate_tax"}`|
85+
|`VIEW`|`database`, `view`|`{"type":"VIEW","database":"sales","view":"daily_orders"}`|
86+
87+
Principals are opaque, canonical strings that are globally unique in the server namespace. Their
88+
format is server-defined and may encode a user, group, role, or service identity, for example
89+
`role:analyst` or an external identity-provider ARN. Principal type and membership resolution are
90+
server responsibilities. Access values are limited to 32 characters and principals to 128
91+
characters. An implementation may resolve wire locators and principals to different stable
92+
persistence identifiers; those internal ids are not exposed by this API.
93+
94+
The built-in accesses use a common data-authorization vocabulary. Creation accesses intentionally
95+
use their persisted names without underscores:
96+
97+
| Access | Meaning |
98+
| --- | --- |
99+
|`ALL`| All accesses applicable to the resource. |
100+
|`CREATEDATABASE`| Create a database in a catalog. |
101+
|`DESCRIBE`| Read database metadata or select the current database. |
102+
|`ALTER`| Modify resource metadata. |
103+
|`DROP`| Drop the resource. |
104+
|`CREATETABLE`| Create a table in a database. |
105+
|`CREATEFUNCTION`| Create a function in a database. |
106+
|`CREATEVIEW`| Create a view in a database. |
107+
|`LIST`| List resources in a database. |
108+
|`SELECT`| Read table or view data, or use a function. |
109+
|`UPDATE`| Write table data, including insert, update, and delete operations. |
110+
|`GRANT`| Grant or revoke assignments on the resource. |
111+
112+
Java helpers accept access names case-insensitively and normalize them before sending. The REST
113+
wire format uses upper case. Built-in accesses are resource-specific:
114+
115+
| Resource | Accesses |
116+
| --- | --- |
117+
|`CATALOG`|`ALL`, `ALTER`, `DROP`, `GRANT`, `CREATEDATABASE`|
118+
|`CATALOG_ALL`|`ALL`, `DESCRIBE`, `ALTER`, `DROP`, `GRANT`, `CREATETABLE`, `CREATEVIEW`, `CREATEFUNCTION`, `LIST`, `SELECT`, `UPDATE`|
119+
|`DATABASE`|`ALL`, `DESCRIBE`, `ALTER`, `DROP`, `GRANT`, `CREATETABLE`, `CREATEVIEW`, `CREATEFUNCTION`, `LIST`|
120+
|`DATABASE_ALL`|`ALL`, `SELECT`, `UPDATE`, `ALTER`, `DROP`, `GRANT`|
121+
|`TABLE`|`ALL`, `SELECT`, `UPDATE`, `ALTER`, `DROP`, `GRANT`|
122+
|`COLUMN`|`SELECT`|
123+
|`VIEW`|`ALL`, `SELECT`, `ALTER`, `DROP`, `GRANT`|
124+
|`FUNCTION`|`ALL`, `SELECT`, `ALTER`, `DROP`, `GRANT`|
125+
126+
An assignment identity is `resource`, `access`, and `principal`. Granting the same identity replaces
127+
its expiry, and revocation is idempotent. `CATALOG`, `DATABASE`, `TABLE`, `COLUMN`, `VIEW`, and
128+
`FUNCTION` apply only to the exact referenced resource. `CATALOG_ALL` is an explicit scope over the
129+
configured catalog's database, table, view, and function descendants; `DATABASE_ALL` is an explicit
130+
scope over the named database's table, view, and function descendants. These scope assignments also
131+
apply to descendants created later. They remain direct assignments in listing responses; the server
132+
does not synthesize inherited assignments. Resolving group membership and role inheritance remains a
133+
server responsibility.
134+
135+
### Column permissions
136+
137+
A column permission uses a `COLUMN` resource whose locator is the containing table, `SELECT`
138+
access, and one `columns` object. Exactly one non-empty list is allowed:
139+
140+
-`columnNames` is an allowlist. Only the named top-level columns are readable.
141+
-`excludedColumnNames` is a denylist. Every current top-level column except the named columns is
142+
readable.
143+
144+
For example, this assignment allows only `order_id` and `region`:
145+
146+
```json
147+
{
148+
"resource": {
149+
"type": "COLUMN",
150+
"database": "sales",
151+
"table": "orders"
152+
},
153+
"access": "SELECT",
154+
"principal": "role:analyst",
155+
"columns": {
156+
"columnNames": ["order_id", "region"]
157+
}
158+
}
159+
```
160+
161+
The assignment identity remains `(resource, access, principal)`; `columns` is not part of the
162+
identity. Granting the same identity replaces the entire previous allowlist or denylist rather than
163+
merging individual names. Revocation therefore omits `columns` and removes the whole column
164+
assignment.
165+
166+
All named columns must exist when granted, and the table must enforce query authorization. A server
167+
may enable `query-auth.enabled` atomically with the grant; otherwise it must reject the grant. Column
168+
names refer only to top-level fields. For every effective caller principal, applicable column ranges
169+
are intersected. If any applicable range rejects a selected column, the query fails rather than
170+
silently dropping that column.
171+
172+
Schema evolution keeps the assignment attached to the stable table identity. Renaming a referenced
173+
column updates its stored name. Dropping a referenced column removes it from the range; if that
174+
would leave the stored list empty, the assignment is removed. An allowlist denies columns added
175+
later, while a denylist allows them, so allowlists are safer when new columns may contain sensitive
176+
data.
177+
178+
`expireTime`, when present, is an exclusive upper bound evaluated against the REST server clock.
179+
At `now >= expireTime`, the assignment must not authorize access. Expired direct assignments may
180+
remain visible in listings until server cleanup. Timestamps must not be more precise than
181+
milliseconds; the wire value uses UTC `Z` and contains at most three fractional digits.
182+
183+
Resource objects in this API are wire locators, not persistence identities. Servers must bind direct
184+
assignments to a stable internal resource identity: renaming a database, table, function, or view
185+
retains its assignments and subsequent responses use the new locator; dropping it removes its direct
186+
assignments; recreating the same locator does not restore them.
187+
188+
## Data policy model
189+
190+
A data policy is attached directly to one table and one principal. It applies whenever that
191+
principal is effective for the caller after the server resolves group and role membership. A
192+
principal can have at most one row filter on a table and at most one column mask on each table
193+
column. A row-filter identity is `(table, ROW_FILTER, principal)`; a column-mask identity is
194+
`(table, COLUMN_MASKING, principal, onColumn)`.
195+
196+
Each policy contains exactly one typed definition:
197+
198+
| Definition | Required fields | Result |
199+
| --- | --- | --- |
200+
|`rowFilter`|`predicate`| One serialized Paimon `Predicate`, applied to every scan. |
201+
|`columnMask`|`onColumn`, `transform`| One serialized Paimon `Transform` whose result replaces the protected column. |
202+
203+
The common field is one `principal`. `rowFilter.predicate` maps directly to one entry in
204+
`AuthTableQueryResponse.filter`. `columnMask.onColumn` and `columnMask.transform` map directly to
205+
one key and value in `AuthTableQueryResponse.columnMasking`. Each JSON value is limited to 60 KiB
206+
in UTF-8. This is Paimon's versioned serialization format rather than SQL text or a portable policy
207+
DSL; clients and servers must use compatible Paimon versions.
208+
209+
Policy creation must be rejected unless all of these conditions hold:
210+
211+
1. The target database and table exist.
212+
2. The table has `query-auth.enabled=true`; otherwise a stored policy could be silently bypassed.
213+
3. The referenced principal exists.
214+
4. The predicate or transform is recognized by the server, deserializes to a non-null Paimon
215+
object, and is canonicalized before storage.
216+
5. Every referenced field and `onColumn` exists in the target table, and a transform's output type
217+
matches its protected column.
218+
219+
These invariants continue to apply for the whole table lifecycle. Servers must bind policies to a
220+
stable table identity, preserve that binding across table renames, and remove the policies when the
221+
table is dropped. A table with policies must reject changes that disable `query-auth.enabled` or
222+
remove or rename a protected or referenced column, unless the policy update and schema change are
223+
performed atomically. If an implementation persists all masks for one principal in one document,
224+
creating or dropping one column mask must atomically preserve masks for other columns.
225+
226+
At authorization time, all applicable row filters must be combined with logical `AND`. More than
227+
one applicable column mask targeting the same column must fail closed. An invalid, unsupported, or
228+
schema-incompatible predicate or transform must also fail closed rather than omit a restriction.
229+
230+
This experimental contract deliberately does not define governed tags, catalog/database policy
231+
inheritance, or tag-driven matching. Those features need explicit match conditions and conflict
232+
rules before being added.

0 commit comments

Comments
 (0)