Skip to content

Make the attributes selector usable and correct a runtime dependency's scope - #125

Open
PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix/attributes-selector-and-runtime-dependency
Open

Make the attributes selector usable and correct a runtime dependency's scope#125
PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix/attributes-selector-and-runtime-dependency

Conversation

@PDGGK

@PDGGKPDGGK commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Deploying the module into a live ThingsBoard for the first time surfaced two defects the existing tests could not reach. Until now the module had been compiled and integration-tested against ThingsBoard's types, but never run inside a running instance. That gap was known and recorded; this is the work that closes it.

The two defects were seen on different deployment attempts, so each names its own below. The end-to-end run, and everything in the first section, is thingsboard/tb-node:4.3.1.2 against apache/iotdb:2.0.8-standalone, with the module jar and its runtime dependencies placed on ThingsBoard's extension classpath, driven through ThingsBoard's own REST API.

The attributes selector could never take effect

ThingsBoard gates its timeseries DAOs on database.ts.type, so for ts and ts_latest it steps aside by itself. It offers no equivalent for attributes: at v4.3.1.2 (dao/src/main/java/org/thingsboard/server/dao/sql/attributes/JpaAttributeDao.java:58-61) JpaAttributeDao is a bare @Component, so it registers unconditionally. I have checked that one version. I have not surveyed the release history, so this PR claims it for v4.3.1.2 and no wider.

So AttributesDaoConflictGuard always found a competing bean and failed startup:

database.attributes.type=iotdb-table, but a non-IoTDB AttributesDao bean 'jpaAttributeDao'
is present; remove it or unset the IoTDB attributes selector

Its own advice cannot be followed — a @Component cannot be un-registered from a properties file. So on that build the documented attributes opt-in was not an option a deployment could take: setting the selector produced a startup failure, every time.

The guard now withdraws ThingsBoard's bean instead of refusing to start, and the withdrawal is deliberately narrow. Candidates are sorted into three classes and nothing is mutated until every reason to stop has been evaluated:

classidentified byoutcome
oursbean name ioTDBTableAttributesDao, looked up directlyrequired; startup fails if absent or if something else holds the name
ThingsBoard'sbean name jpaAttributeDaoand resolved type org.thingsboard.server.dao.sql.attributes.JpaAttributeDaowithdrawn, with a WARN naming it
anything elseevery other AttributesDao — a third-party backend, a decorator, a subclass of ours under another namestartup fails, untouched

The third row is the part I care most about. A bean the operator registered deliberately is not this module's to delete, so the module refuses rather than guesses — and unlike the old message, "remove it or unset the selector" is now advice the operator can actually act on, because the bean belongs to their application.

Matching on the class name and not on assignability means a rename upstream fails closed: the bean becomes unrecognised and startup stops naming it, rather than the guard removing something it should not have.

The message it emits, excerpted and wrapped for width. AttributesDaoConflictGuardTest asserts that the current code emits a WARN naming the bean, its class, the property and its value; the same message was observed in a live run against thingsboard/tb-node:4.3.1.2 on 2026-08-20, which exercised the earlier revision of the guard:

WARN Removed ThingsBoard bean 'jpaAttributeDao'
(org.thingsboard.server.dao.sql.attributes.JpaAttributeDao) because
database.attributes.type=iotdb-table selects the IoTDB attributes backend; ThingsBoard
provides no configuration switch for attributes, so the conflicting bean is deregistered
rather than left to conflict.

Our own DAO bean also had to lose @ConditionalOnMissingBean(type = ATTRIBUTES_DAO_CLASS_NAME) — the string form, chosen so that evaluating the condition does not load ThingsBoard classes. It is evaluated while configuration classes are parsed, strictly before the post-processor runs, so on a stock 4.3.1.2 it skipped our bean on every deployment.

What the guard guarantees, and where that stops. After it runs, exactly one AttributesDao definition remains among the candidates it could see, and it is ours. Candidates come from one getBeanNamesForType(type, true, false) snapshot, which does not initialise FactoryBeans and does not consult a parent factory — so a definition registered by a later post-processor, produced by an opaque FactoryBean whose getObjectType() is null until initialisation, or inherited from an ancestor context is outside the guarantee. That boundary is in the javadoc rather than only here.

A runtime dependency was demoted out of the runtime classpath

iotdb-thrift-commons was declared at test scope, with a comment describing it as "a transitive runtime dependency of iotdb-session" — which is precisely what the declaration broke, since a direct declaration wins under Maven's nearest-definition rule. It carries TEndPoint, which the session pool needs at runtime. The runtime dependency set produced from the POM omitted that artifact, so the deployed module could not create a session — observed as a TEndPoint load failure during an earlier tb-postgres:4.2.1.1 deployment attempt, not during the 4.3.1.2 run above. I have not preserved that container's log, so I am stating the observation rather than quoting a trace.

It is now declared at runtime scope, which is what it actually is: no main source references it, the unit tests construct StatementExecutionException(TSStatus), and the session pool needs it at runtime. Runtime scope puts it on the test and runtime classpaths and off the compile classpath — exactly where it is used. Verified with dependency:build-classpath -DincludeScope=runtime: absent before, present after. dependency:analyze-only no longer reports it, so the analyzer suppression an earlier revision of this PR carried has been deleted rather than kept.

One new compile-only file

src/provided/java/org/thingsboard/server/dao/sql/attributes/JpaAttributeDao.java joins the existing Strategy F surface. ThingsBoard's dao artifact is not on Maven Central, so without a class of that exact fully-qualified name on the test classpath the conjunctive match above could not be exercised by any test at all.

It is excluded from the packaged jar by the existing org/thingsboard/** rule, so it cannot shadow the real class on a deployment classpath. I verified that by building the jar and listing its entries rather than by reading the configuration: zero org/thingsboard entries.

To be explicit about what it does not prove: no test in this module establishes that org.thingsboard.server.dao.sql.attributes.JpaAttributeDao is the right name. That comes from reading ThingsBoard's source at v4.3.1.2.

Documentation in this PR

Three in-repo documents change, because the behaviour they describe changed. They are a third of the diff and worth reading as part of it.

docs/user-guide.md — the "Conflict guards" section now separates the two behaviours. The timeseries and latest guards still fail startup on a competing DAO, and that advice is actionable. The attributes guard cannot ask an operator to remove JpaAttributeDao, so it describes the withdrawal instead, quotes the WARN, states that the match is on bean name and fully-qualified class name, and states the visibility boundary. The attributes bullet in the selector list is rewritten for the same reason.

docs/migration-guide.md — the "Fail-fast conflict guard" bullet was accurate for all three routes and is no longer. It is now split: fail-fast for timeseries and latest, narrow withdrawal for attributes, with the same boundary sentence.

Two production javadoc blocks change for the same reason — IoTDBTableAttributesDao and IoTDBTableAttributesEnabledCondition each argued that "no real Phase-1 deployment sets database.attributes.type", which is the rationale this change overturns. Both now describe the selector as one this module supplies and point at the guard for what setting it does.

README.md — this one is the reason it is here rather than in a follow-up. Its attributes section argued "no shipped ThingsBoard release exposes a database.attributes.type selector yet, so a real Phase-1 deployment never sets it". That rationale is what this change overturns: the selector is one this module supplies, and after this PR setting it does something. The paragraph now says what is true instead. Retained unchanged: the statement that the attributes selector is independent of database.ts.type / database.ts_latest.type — that is independence among this module's own three routes, and it remains correct.

Verification

mvn -P with-thingsboard,iotdb-table-it clean verify204 unit tests + 58 integration tests, rat / checkstyle / spotless / dependency-analyze / enforcer clean.

Two constants in IoTDBTableAttributesEnabledCondition widen from private to package-private so the guard's WARN can name the property and value from their single definition; the class is itself package-private, so nothing a consumer can see changes.

IoTDBTableTimeseriesAggregationIT gains non-UTC calendar coverage in this PR — a boundary-straddling sample where 2023-01-31T20:00Z is January under UTC and February under Asia/Shanghai, so an implementation that drops the query timezone produces a different bucket count and fails. Confirmed by forcing the zone to UTC and watching it go red.

The guard's own class now carries 14 tests. Each of the three guarantees was mutation-checked: reverting the class-name half of the match, identifying our bean by assignability instead of by name, and moving the unknown-bean check after the removal each reddened exactly one test, with an unrelated test staying green as a contamination control.

The module builds against Spring 5.3 while ThingsBoard 4.3.1.2 runs Spring 6. One run of the full unit suite with -Dspring.version=6.2.18 -Dspring-boot.version=3.5.14 is also green (204/204). Making that a standing matrix rather than a one-off check is a reasonable follow-up.

End to end on 4.3.1.2, through ThingsBoard's REST API: telemetry and attributes written and confirmed in IoTDB; five read shapes correct — raw range, latest, attributes, fixed-window aggregation (AVG and COUNT arithmetic exact), and calendar aggregation in a non-UTC zone. ThingsBoard's own internal telemetry flows through the module as well (cpuUsage, discUsage, successfulMsgs, activeDevicesCountHourly), so what the module serves is the platform's telemetry persistence, not only the test device I created. Entities, relations, users and dashboards do not touch TimeseriesDao and stay in PostgreSQL. That run exercised the earlier revision of the guard; the current one is covered by the unit and integration suites above.

Points I'd appreciate review on

  • May an extension module withdraw one named bean definition from its host, on the strength of a single explicit non-default selector plus a WARN? That is the whole question, and it is the one thing I would most like ruled on.

    The mechanics, precisely: the guard is a BeanFactoryPostProcessor activated by database.attributes.type=iotdb-table alone. iotdb.attributes.cluster_mode is validated when the DAO is constructed, which is after the post-processor has run, so it is not a precondition of the withdrawal — though a successful attributes deployment still requires it, since a failed validation fails the context and the in-memory withdrawal never becomes external state.

    My reasoning for saying yes: ThingsBoard switches its timeseries DAOs by configuration and offers nothing equivalent for attributes, so this supplies a switch it does not have rather than fighting its design; and narrowing the target to one bean matched on name and fully-qualified class keeps it from ever touching a bean the operator wrote. In Add ThingsBoard latest-telemetry and attributes DAOs on IoTDB Table Mode #113 this guard was reviewed as "fail loudly on a conflict"; it is now "withdraw ThingsBoard's own attributes bean, fail loudly on anything else".

    Two alternatives, both live. Retreat — mark attributes as pending ThingsBoard-side selector support: honest, but gives up the capability. Gate the withdrawal behind its own property (say iotdb.attributes.replace-existing-dao), so that mutating the host requires its own explicit acknowledgement rather than riding on the backend selector.

    I am happy to take any of the three, or to take this to dev@ first if that is the better route.

  • Whether the compile-only stub is the right way to make that match testable. It follows the pattern already in the module, but it is the first stub added for a concrete ThingsBoard class rather than an SPI interface. If there is a way you would rather see this pinned, I will take it.

Happy to iterate.

scope
Deploying the module into a live ThingsBoard for the first time surfaced
two defects that the existing tests could not: the module had never been
run inside a running ThingsBoard, only compiled and integration-tested
against its types. The two were seen on different deployment attempts,
and each is attributed to its own below.
The attributes selector could never take effect. This one is from the
end-to-end run on thingsboard/tb-node:4.3.1.2 against
apache/iotdb:2.0.8-standalone. ThingsBoard gates its timeseries DAOs on
database.ts.type, so for ts and ts_latest it steps aside by itself, but
it offers no equivalent for attributes: at v4.3.1.2, JpaAttributeDao is
a bare @component and so registers unconditionally.
AttributesDaoConflictGuard therefore always found a competing bean and
failed startup, and its own advice -- remove the bean or unset the
selector -- could not be followed, because a @component cannot be
un-registered from a properties file. So on that build the documented
attributes opt-in was not an option a deployment could take: setting the
selector produced a startup failure, every time. I have checked 4.3.1.2
and have not surveyed the release history, so this is stated for
v4.3.1.2 and no wider.
The guard now withdraws ThingsBoard's bean instead of refusing to start,
and the withdrawal is narrow. Candidates are sorted into three classes,
and nothing is mutated until every reason to stop has been evaluated:
- ours, found by looking up the bean name this configuration registers
rather than by filtering the type scan, so a bean that took the name while
implementing something else is caught too;
- ThingsBoard's own, matched on BOTH the bean name jpaAttributeDao and the
resolved type
org.thingsboard.server.dao.sql.attributes.JpaAttributeDao -- this one is
withdrawn, with a WARN naming it, its class and the property that
caused it;
- anything else implementing AttributesDao -- a third-party backend, a
decorator, a subclass of our own DAO under another name -- which fails
startup untouched.
The third class is the point. A bean the operator registered
deliberately is not this module's to delete, so the guard refuses rather
than guesses, and unlike the old message "remove it or unset the
selector" is now advice that can actually be followed. Matching on the
class name rather than on assignability also means an upstream rename
fails closed: the bean becomes unrecognised and startup stops naming it,
instead of the guard withdrawing something it should not have.
What the guard guarantees is bounded by what it can see. Candidates come
from one getBeanNamesForType(type, true, false) snapshot, which does not
initialise FactoryBeans and does not consult a parent factory, so a
definition registered by a later post-processor, produced by an opaque
FactoryBean, or inherited from an ancestor context is outside it. The
javadoc says so rather than claiming more.
Our own DAO bean also had to lose
@ConditionalOnMissingBean(type = ATTRIBUTES_DAO_CLASS_NAME) -- the string
form, chosen so that evaluating the condition does not load ThingsBoard
classes. It is evaluated while
configuration classes are parsed, strictly before the post-processor
runs, so on a stock 4.3.1.2 it skipped our bean on every deployment.
The second defect is in the POM. iotdb-thrift-commons was declared at
test scope with a comment calling it "a transitive runtime dependency of
iotdb-session" -- which is exactly what the declaration broke, since a
direct declaration wins under Maven's nearest-definition rule. It
carries TEndPoint, which the session pool needs at runtime, so the
runtime dependency set produced from the POM omitted it and the deployed
module could not create a session -- observed as a TEndPoint load
failure during an earlier tb-postgres:4.2.1.1 deployment attempt, not
during the 4.3.1.2 run above. That container's log was not preserved, so
this is the observation rather than a quotation. It is now declared at
runtime scope, which is what it is: no main source references it, the
unit tests do, and the session pool needs it at runtime. That also
retires the dependency:analyze suppression -- analyze-only no longer
reports the artifact, so the correct scope replaces the suppression
rather than sitting alongside it.
One new compile-only file,
src/provided/java/org/thingsboard/server/dao/sql/
attributes/JpaAttributeDao.java, joins the existing Strategy F surface.
ThingsBoard's dao artifact is not on Maven Central, so without a class
of that exact fully-qualified name on the test classpath the conjunctive
match above could not be exercised at all. It is excluded from the
packaged jar by the existing org/thingsboard/** rule and cannot shadow
the real class; verified by building the jar and listing its entries. No
test establishes that the name is correct -- that comes from
ThingsBoard's source at v4.3.1.2, and a rename fails closed.
Three in-repo documents and two production javadoc blocks change with
it, because the behaviour they describe changed. IoTDBTableAttributesDao
and IoTDBTableAttributesEnabledCondition each argued that no real
Phase-1 deployment sets database.attributes.type -- the rationale this
change overturns -- and now describe it as a selector this module
supplies, pointing at the guard for what setting it does.
docs/user-guide.md and docs/migration-guide.md both described a single
fail-fast conflict guard for all three routes; that is still true of
timeseries and latest and is no longer true of attributes, so each now
describes the two behaviours separately, with the withdrawal's matching
rule and its visibility boundary. README.md is included for a different
reason: its attributes section argued that no shipped ThingsBoard
release exposes the selector, "so a real Phase-1 deployment never sets
it" -- a rationale this change overturns, since the selector is one this
module supplies. Its statement that the attributes selector is
independent of database.ts.type / database.ts_latest.type is retained:
that is independence among this module's own routes and it remains
correct.
Two constants in IoTDBTableAttributesEnabledCondition widen from private
to package-private so the guard's WARN can name the property and value
from their single definition; the class is itself package-private, so
nothing a consumer can see changes.
Tests: AttributesDaoConflictGuardTest now carries 14 cases, covering the
withdrawal, the zero-replacement refusal, an IoTDB peer under another
name, a third-party DAO alongside a removable ThingsBoard one,
right-name/wrong-type and right-type/wrong-name, a pre-built singleton
with no definition to withdraw, the WARN's four fields, the no-op path
and the ordering. Each of the three guarantees was mutation-checked:
reverting the class-name half of the match, identifying our bean by
assignability, and moving the unknown-bean check after the removal each
reddened exactly one test, with an unrelated test green throughout as a
contamination control. IoTDBTableTimeseriesAggregationIT gains non-UTC
calendar coverage with a boundary-straddling sample -- 2023-01-31T20:00Z
is January under UTC and February under Asia/Shanghai -- so an
implementation that drops the query timezone changes the bucket count
and fails; confirmed by forcing the zone to UTC and watching it go red.
204 unit tests and 58 integration tests pass. The full unit suite is
also green against Spring 6.2.18 / Spring Boot 3.5.14, the line
ThingsBoard 4.3.1.2 runs.
@PDGGK
PDGGKforce-pushed the fix/attributes-selector-and-runtime-dependency branch from c22b721 to 86daf89CompareAugust 21, 2026 09:33
PDGGK added a commit to PDGGK/iotdb-docs that referenced this pull request Aug 21, 2026
Adds a user-guide page for the iotdb-thingsboard-table module in
apache/iotdb-extras: stock ThingsBoard storing telemetry and attributes in
IoTDB 2.x table model, by placing the module and its runtime dependencies on
ThingsBoard's classpath and setting a few properties. The page lists the exact
jar set, including the ones that must NOT be copied because ThingsBoard bundles
newer copies of them.
Four new pages -- Thingsboard.md in latest-Table and Master/Table, English and
Chinese -- plus the two sidebar groups and the four Ecosystem-Overview index
rows. Structure follows the existing Spark-IoTDB page.
The attributes selector is documented as not yet usable on a stock ThingsBoard:
ThingsBoard registers its JPA attributes bean unconditionally and exposes no
switch for it, so setting the selector currently fails startup. The fix is
apache/iotdb-extras#125; the page says so and will be updated when a build
containing it is released.
@PDGGKPDGGK changed the title Make the attributes selector usable and restore a runtime dependencyMake the attributes selector usable and correct a runtime dependency's scopeAug 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@PDGGK