Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 20 additions & 9 deletions iotdb-thingsboard-table/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -74,7 +74,8 @@ selector:
`telemetry_latest` overlay. Enabled by `database.ts_latest.type=iotdb-table`
(see the latest-telemetry section below).
- `IoTDBTableAttributesDao`: entity attributes, **inert by default**. Enabled
only by the independent `database.attributes.type=iotdb-table` opt-in.
only by the `database.attributes.type=iotdb-table` opt-in, which is separate
from the two timeseries selectors.

> **This is an incremental / experimental backend.** Nothing routes through
> IoTDB Table Mode unless the matching selector is set explicitly; with no
Expand All@@ -91,11 +92,19 @@ explicitly with `database.attributes.type=iotdb-table`.
This attribute selector is **independent** of `database.ts.type` /
`database.ts_latest.type` — the attribute DAO routes separately from the
time-series DAOs (a piggy-back on the timeseries selector was deliberately
rejected). No shipped ThingsBoard release exposes a `database.attributes.type`
selector yet, so a real Phase-1 deployment never sets it; the activation condition
rejected). Leaving it unset is the default posture: the activation condition
stays false, no attribute bean or session pool is created, and attributes keep
flowing to the host entity-DB `AttributesDao`.

`database.attributes.type` is a selector this module supplies rather than one
ThingsBoard offers. ThingsBoard switches its timeseries DAOs by configuration but
has no equivalent for attributes — at v4.3.1.2 its `JpaAttributeDao` is an
unconditional `@Component`, so no property can stand it down. Setting this
selector therefore has the module withdraw that one bean at startup, matched on
both its bean name and its fully-qualified class name, logging a WARN that names
it; any other competing `AttributesDao` fails startup untouched. See
`docs/user-guide.md` for the full semantics and their boundary.

When activated, each identity tuple
`(tenant_id, entity_type, entity_id, attribute_scope, key)` holds exactly one
current row: `save` is a tag-only `DELETE` (no time predicate) followed by an
Expand DownExpand Up@@ -302,7 +311,7 @@ Key activation and operational flags:
| --- | --- | --- |
| `database.ts.type` | _(unset)_ | Set to `iotdb-table` as the ThingsBoard historical-timeseries backend selector. |
| `iotdb.ts.experimental-raw-only` | `false` | Explicit opt-in for this backend. Must be `true` together with `database.ts.type=iotdb-table`. The name predates the aggregation support and is kept for compatibility: write, raw read, delete **and** time-bucketed aggregation are all served when it is enabled. |
| `database.attributes.type` | _(unset)_ | Set to `iotdb-table` to opt in to the entity-attribute DAO. Independent of the timeseries selectors. Unset in a real Phase-1 deployment, so the attribute DAO is inert by default. |
| `database.attributes.type` | _(unset)_ | Set to `iotdb-table` to opt in to the entity-attribute DAO. Independent of the timeseries selectors. Unset by default, and while unset the attribute DAO is inert. Setting it withdraws ThingsBoard's own `jpaAttributeDao` bean — see the Entity attributes section. |
| `iotdb.attributes.cluster_mode` | _(empty)_ | Required when `database.attributes.type=iotdb-table`. Must be `sticky-routing` (per-identity writes pinned to one node) or `disabled` (single-node / acknowledged best-effort); any other value (including the empty default) fails construction fast, because the attribute write path converges only within a single JVM. |
| `iotdb.ts_latest.cluster_mode` | _(empty)_ | Required when `database.ts_latest.type=iotdb-table` (the latest-overlay DAO is active). Must be `sticky-routing` (per-identity latest writes pinned to one node) or `disabled` (single-node / acknowledged best-effort); any other value (including the empty default) fails construction fast, because the latest-overlay write path converges only within a single JVM. This is the symmetric acknowledgement to `iotdb.attributes.cluster_mode`. |
| `iotdb.attributes.executor.threads` | `4` | Worker-thread count for the attribute DAO's bounded IO executor. Sized independently of `iotdb.ts.read.*` so the attribute path's concurrency can be tuned on its own; the default matches `iotdb.ts.read`. |
Expand DownExpand Up@@ -380,8 +389,10 @@ behind its own `database.ts_latest.type=iotdb-table` selector. Physical
retention is a table property the operator sets on the schema; see
Retention / TTL above.

`IoTDBTableAttributesDao` is **inert by default** and activated only by the independent
`database.attributes.type=iotdb-table` opt-in (see the Entity attributes section
above and its Phase-1 limitations). In a real Phase-1 deployment the selector is
unset, so the attribute DAO never activates and attributes stay in the host
entity database.
`IoTDBTableAttributesDao` is **inert by default** and activated only by the
`database.attributes.type=iotdb-table` opt-in, which is separate from the two
timeseries selectors (see the Entity attributes section above and its Phase-1
limitations). While the selector is unset — the default posture — the attribute
DAO never activates and attributes stay in the host entity database. Setting it
has the module withdraw ThingsBoard's own attributes bean, which is why that
section describes the matching rule and its boundary.
35 changes: 25 additions & 10 deletions iotdb-thingsboard-table/docs/migration-guide.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -319,12 +319,25 @@ Each route is independently activated and guarded:
- **Independent activation.** Enabling telemetry does not enable attributes, and
vice versa. You can route telemetry + latest to IoTDB while attributes stay in
the host entity database (the default Phase-1 posture).
- **Fail-fast conflict guard.** When a route is enabled but a conflicting
non-IoTDB host DAO bean of the same SPI type is also present, startup fails
fast with a clear message rather than silently shadowing one DAO with another.
The historical (`TimeseriesDao`), latest (`TimeseriesLatestDao`), and attribute
(`AttributesDao`) routes each have their own guard. Make sure the host backend
for a route is removed/disabled when you point that route at IoTDB.
- **Conflict guards, and they are not all fail-fast.** The historical
(`TimeseriesDao`), latest (`TimeseriesLatestDao`), and attribute
(`AttributesDao`) routes each have their own guard, but they resolve the
conflict differently.

The timeseries and latest guards fail startup when a conflicting non-IoTDB host
DAO of the same SPI type is present, rather than silently shadowing one DAO with
another. Remove or disable the host backend for that route before pointing it at
IoTDB.

The attribute guard cannot ask for that, because ThingsBoard registers
`JpaAttributeDao` unconditionally and no configuration stands it down. So when
`database.attributes.type=iotdb-table` is set, the module withdraws that one bean
definition itself and logs a WARN naming it. The match is on both the bean name
and the exact fully-qualified class name, so nothing else is ever removed: any
other competing `AttributesDao` visible to the guard when it runs fails startup
untouched. Definitions registered after it, supplied by a `FactoryBean` that
does not report its type until initialisation, or inherited from a parent
context are outside its reach.

### Rollback

Expand DownExpand Up@@ -370,10 +383,12 @@ module `README.md` for the authoritative list):
with table-wide IoTDB TTL; the module uses it only for ThingsBoard's
storage-accounting, never as a physical-retention directive. Set physical
retention on the table (Step 5).
- **The attributes route is a stretch / Phase-2 opt-in.** No shipped ThingsBoard
release exposes a `database.attributes.type` selector yet (open question,
tracked upstream), so in a real Phase-1 deployment the selector is unset and
attributes stay in the host entity database. When activated, `save` is a
- **The attributes route is a stretch / Phase-2 opt-in.** `database.attributes.type`
is a selector this module supplies rather than one ThingsBoard offers; leaving it
unset is the default posture, and while unset attributes stay in the host entity
database. Setting it makes the module withdraw ThingsBoard's own attributes bean
— see the conflict-guard bullet above for the matching rule and its boundary.
When activated, `save` is a
non-atomic tag-only delete-then-insert under a per-identity in-JVM lock that
converges only within one JVM; `findNextBatch` is unsupported
(`UnsupportedOperationException`), and `findAllKeysByDeviceProfileId` with a
Expand Down
56 changes: 45 additions & 11 deletions iotdb-thingsboard-table/docs/user-guide.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -76,22 +76,56 @@ Notes:
timeseries selectors are required as well because the latest value is derived
from the `telemetry` table that only the IoTDB writer populates — the latest
path can never activate without that writer.
- **Attributes** is an opt-in stretch feature. No shipped
ThingsBoard release exposes a `database.attributes.type` selector yet (open
question Q6 / ThingsBoard Discussion #15296), so a real deployment normally
leaves it unset and attributes keep flowing to the host entity database. When
it is activated, `iotdb.attributes.cluster-mode` must also be set or the DAO
fails fast at startup (see [§6](#6-configuration-reference)).
- **Attributes** is an opt-in stretch feature, and `database.attributes.type` is
a selector this module supplies rather than one ThingsBoard offers. ThingsBoard
switches its timeseries DAOs by configuration but has no equivalent for
attributes: `JpaAttributeDao` is an unconditional `@Component` (verified at
v4.3.1.2), so nothing in `thingsboard.yml` can stand it down. Setting this
selector therefore has the module withdraw that one bean at startup, logging a
WARN that names it. Leaving the selector unset is the default posture and
attributes keep flowing to the host entity database. When it is activated,
`iotdb.attributes.cluster-mode` must also be set or the DAO fails fast at
startup (see [§6](#6-configuration-reference)). Open question Q6 / ThingsBoard
Discussion #15296 tracks a native selector; if one ships, this module should
use it instead.

When activated, the DAOs share a single module-owned IoTDB table session pool.

### Conflict guards

When a selector is on, the module fails startup fast if a conflicting non-IoTDB
DAO bean of the same SPI type is also present (for example, another
`TimeseriesDao` while `database.ts.type=iotdb-table`). This is deliberate: it
prevents the module from silently shadowing, or being shadowed by, a different
backend. Remove the conflicting backend or unset the IoTDB selector.
When a selector is on, the module refuses to share its SPI slot with another
backend. The timeseries and latest guards do this by failing startup: if a
conflicting non-IoTDB `TimeseriesDao` or `TimeseriesLatestDao` bean is present
while the matching selector is set, startup stops with a message naming it.
Remove the conflicting backend or unset the IoTDB selector.

The attributes guard behaves differently, because the conflict it faces is not
one you can resolve from a configuration file. ThingsBoard registers
`JpaAttributeDao` unconditionally, so "remove the conflicting backend" is not
advice an operator can act on. When `database.attributes.type=iotdb-table` is
set, the module therefore withdraws that one bean definition itself and logs the
line below. `AttributesDaoConflictGuardTest` asserts that the module emits it;
it was also observed in a live run against `thingsboard/tb-node:4.3.1.2` on
2026-08-20:

```text
WARN Removed ThingsBoard bean 'jpaAttributeDao'
(org.thingsboard.server.dao.sql.attributes.JpaAttributeDao)
because database.attributes.type=iotdb-table selects the IoTDB attributes
backend; ...
```

The withdrawal is narrow on purpose. It matches on both the bean name and the
exact fully-qualified class name, so **only** ThingsBoard's own component is
removed. Any other
competing `AttributesDao` — a third-party backend, a decorator, or a subclass of
this module's DAO under a different bean name — is left in place and startup
fails instead, naming it. A bean your application registered deliberately is not
the module's to delete.

The guard reads the bean definitions present when it runs. A definition
registered later, supplied by a `FactoryBean` that does not report its type until
initialisation, or inherited from a parent context is outside its reach.

## 4. The three DAOs

Expand Down
39 changes: 33 additions & 6 deletions iotdb-thingsboard-table/pom.xml
Original file line numberDiff line numberDiff line change
Expand Up@@ -99,6 +99,24 @@
<artifactId>spring-context</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<!--
Test scope only. Without a binding the module's own slf4j calls resolve to the NOP
logger, so no test can observe them at all; AttributesDaoConflictGuardTest asserts
the WARN that records which ThingsBoard bean was withdrawn, and that assertion is
the only automated check that the disclosure actually happens. Version is managed
by the parent, and logback is already used by other modules in this repository.
-->
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<!-- Test scope only: ListAppender, used to capture the withdrawal WARN. -->
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
Expand DownExpand Up@@ -130,14 +148,23 @@
<scope>test</scope>
</dependency>
<dependency>
<!-- test-scope only: the unit tests construct StatementExecutionException(TSStatus) with
a specific status code to exercise the retry / idempotency classification. TSStatus
lives in iotdb-thrift-commons (a transitive runtime dependency of iotdb-session);
declared here at test scope so it is a declared, test-only use. Version inherited
from the parent reactor (iotdb.version). -->
<!-- Runtime scope, deliberately. iotdb-thrift-commons carries TEndPoint and TSStatus:
the session pool needs TEndPoint at RUNTIME, and the unit tests construct
StatementExecutionException(TSStatus) to exercise the retry / idempotency
classification. No main source references it, so compile scope would overstate the
dependency; runtime scope puts it on both the runtime and the test classpaths,
which is exactly where it is used.

The scope must be declared: the artifact arrives transitively from iotdb-session,
but a direct declaration wins under Maven's nearest-definition rule, so declaring
it at TEST scope (as this block once did) demoted the transitive runtime dependency
and dropped it from the runtime classpath entirely. Deploying the module then
failed at first session creation with NoClassDefFoundError on TEndPoint. Verified
with dependency:build-classpath -DincludeScope=runtime. Version inherited from the
parent reactor (iotdb.version). -->
<groupId>org.apache.iotdb</groupId>
<artifactId>iotdb-thrift-commons</artifactId>
<scope>test</scope>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -55,14 +55,15 @@
/**
* Entity-attribute DAO for the IoTDB Table Mode backend.
*
* <p>Spring activation: {@code database.attributes.type=iotdb-table}. NOTE: this activation
* property is the Phase-1 selector pending upstream ThingsBoard confirmation; upstream ThingsBoard
* does not yet expose an {@code AttributesDao} selector, so the DAO is <b>inert by default</b> (no
* real Phase-1 deployment sets {@code database.attributes.type}, so the {@link
* IoTDBTableAttributesEnabledCondition} stays false and the bean is never instantiated). The
* selector is independent of {@code database.ts.type} / {@code database.ts_latest.type} (the
* attribute DAO routes separately); if upstream resolves to a different property, the condition is
* updated. Phase-1 attributes stay in the host entity DB.
* <p>Spring activation: {@code database.attributes.type=iotdb-table}. This is a selector this
* module supplies rather than one upstream ThingsBoard offers -- upstream exposes no {@code
* AttributesDao} selector of its own -- so the DAO is <b>inert by default</b>: while the property
* is unset, {@link IoTDBTableAttributesEnabledCondition} stays false, the bean is never
* instantiated, and attributes stay in the host entity DB. Setting it makes {@code
* AttributesDaoConflictGuard} withdraw ThingsBoard's own attributes bean; see that guard's javadoc
* for the matching rule and the boundary of what it can see. The selector is independent of {@code
* database.ts.type} / {@code database.ts_latest.type} (the attribute DAO routes separately); if
* upstream ever exposes a native selector, this module should use it instead.
*
* <p>This DAO is wired as an explicit {@code @Bean} in {@link IoTDBTableConfiguration} (guarded by
* the activation property) rather than via component scanning, so the {@code ITableSessionPool}
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -28,15 +28,16 @@
*
* <p>This selector is INDEPENDENT of {@code database.ts.type} / {@code database.ts_latest.type}:
* the attribute DAO routes separately from the time-series DAOs (a piggy-back on the timeseries
* selector was deliberately rejected). Because upstream ThingsBoard does not expose an {@code
* AttributesDao} selector yet, no real Phase-1 deployment sets {@code database.attributes.type};
* the property is therefore absent in practice, this condition returns false, the attribute bean is
* never instantiated, and attributes keep flowing to the host entity-DB {@code AttributesDao}. The
* DAO is thus inert by default and only activates when an operator opts in explicitly.
* selector was deliberately rejected). Upstream ThingsBoard exposes no {@code AttributesDao}
* selector of its own, so {@code database.attributes.type} is one this module supplies. Leaving it
* unset is the default posture: this condition returns false, the attribute bean is never
* instantiated, and attributes keep flowing to the host entity-DB {@code AttributesDao}. The DAO is
* inert by default and activates only when an operator opts in explicitly -- at which point {@code
* AttributesDaoConflictGuard} withdraws ThingsBoard's own attributes bean.
*/
final class IoTDBTableAttributesEnabledCondition implements Condition {
private static final String SELECTOR_PROPERTY = "database.attributes.type";
private static final String SELECTOR_VALUE = "iotdb-table";
static final String SELECTOR_PROPERTY = "database.attributes.type";
static final String SELECTOR_VALUE = "iotdb-table";

@Override
public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) {
Expand Down
Loading