Uh oh!
There was an error while loading. Please reload this page.
Make the attributes selector usable and correct a runtime dependency's scope - #125
Open
PDGGK wants to merge 1 commit into
Open
Make the attributes selector usable and correct a runtime dependency's scope#125PDGGK wants to merge 1 commit into
PDGGK wants to merge 1 commit into
Conversation
scope Deploying the module into a live ThingsBoard for the first time surfaced two defects that the existing tests could not: the module had never been run inside a running ThingsBoard, only compiled and integration-tested against its types. The two were seen on different deployment attempts, and each is attributed to its own below. The attributes selector could never take effect. This one is from the end-to-end run on thingsboard/tb-node:4.3.1.2 against apache/iotdb:2.0.8-standalone. ThingsBoard gates its timeseries DAOs on database.ts.type, so for ts and ts_latest it steps aside by itself, but it offers no equivalent for attributes: at v4.3.1.2, JpaAttributeDao is a bare @component and so registers unconditionally. AttributesDaoConflictGuard therefore always found a competing bean and failed startup, and its own advice -- remove the bean or unset the selector -- could not be followed, because a @component cannot be un-registered from a properties file. So on that build the documented attributes opt-in was not an option a deployment could take: setting the selector produced a startup failure, every time. I have checked 4.3.1.2 and have not surveyed the release history, so this is stated for v4.3.1.2 and no wider. The guard now withdraws ThingsBoard's bean instead of refusing to start, and the withdrawal is narrow. Candidates are sorted into three classes, and nothing is mutated until every reason to stop has been evaluated: - ours, found by looking up the bean name this configuration registers rather than by filtering the type scan, so a bean that took the name while implementing something else is caught too; - ThingsBoard's own, matched on BOTH the bean name jpaAttributeDao and the resolved type org.thingsboard.server.dao.sql.attributes.JpaAttributeDao -- this one is withdrawn, with a WARN naming it, its class and the property that caused it; - anything else implementing AttributesDao -- a third-party backend, a decorator, a subclass of our own DAO under another name -- which fails startup untouched. The third class is the point. A bean the operator registered deliberately is not this module's to delete, so the guard refuses rather than guesses, and unlike the old message "remove it or unset the selector" is now advice that can actually be followed. Matching on the class name rather than on assignability also means an upstream rename fails closed: the bean becomes unrecognised and startup stops naming it, instead of the guard withdrawing something it should not have. What the guard guarantees is bounded by what it can see. Candidates come from one getBeanNamesForType(type, true, false) snapshot, which does not initialise FactoryBeans and does not consult a parent factory, so a definition registered by a later post-processor, produced by an opaque FactoryBean, or inherited from an ancestor context is outside it. The javadoc says so rather than claiming more. Our own DAO bean also had to lose @ConditionalOnMissingBean(type = ATTRIBUTES_DAO_CLASS_NAME) -- the string form, chosen so that evaluating the condition does not load ThingsBoard classes. It is evaluated while configuration classes are parsed, strictly before the post-processor runs, so on a stock 4.3.1.2 it skipped our bean on every deployment. The second defect is in the POM. iotdb-thrift-commons was declared at test scope with a comment calling it "a transitive runtime dependency of iotdb-session" -- which is exactly what the declaration broke, since a direct declaration wins under Maven's nearest-definition rule. It carries TEndPoint, which the session pool needs at runtime, so the runtime dependency set produced from the POM omitted it and the deployed module could not create a session -- observed as a TEndPoint load failure during an earlier tb-postgres:4.2.1.1 deployment attempt, not during the 4.3.1.2 run above. That container's log was not preserved, so this is the observation rather than a quotation. It is now declared at runtime scope, which is what it is: no main source references it, the unit tests do, and the session pool needs it at runtime. That also retires the dependency:analyze suppression -- analyze-only no longer reports the artifact, so the correct scope replaces the suppression rather than sitting alongside it. One new compile-only file, src/provided/java/org/thingsboard/server/dao/sql/ attributes/JpaAttributeDao.java, joins the existing Strategy F surface. ThingsBoard's dao artifact is not on Maven Central, so without a class of that exact fully-qualified name on the test classpath the conjunctive match above could not be exercised at all. It is excluded from the packaged jar by the existing org/thingsboard/** rule and cannot shadow the real class; verified by building the jar and listing its entries. No test establishes that the name is correct -- that comes from ThingsBoard's source at v4.3.1.2, and a rename fails closed. Three in-repo documents and two production javadoc blocks change with it, because the behaviour they describe changed. IoTDBTableAttributesDao and IoTDBTableAttributesEnabledCondition each argued that no real Phase-1 deployment sets database.attributes.type -- the rationale this change overturns -- and now describe it as a selector this module supplies, pointing at the guard for what setting it does. docs/user-guide.md and docs/migration-guide.md both described a single fail-fast conflict guard for all three routes; that is still true of timeseries and latest and is no longer true of attributes, so each now describes the two behaviours separately, with the withdrawal's matching rule and its visibility boundary. README.md is included for a different reason: its attributes section argued that no shipped ThingsBoard release exposes the selector, "so a real Phase-1 deployment never sets it" -- a rationale this change overturns, since the selector is one this module supplies. Its statement that the attributes selector is independent of database.ts.type / database.ts_latest.type is retained: that is independence among this module's own routes and it remains correct. Two constants in IoTDBTableAttributesEnabledCondition widen from private to package-private so the guard's WARN can name the property and value from their single definition; the class is itself package-private, so nothing a consumer can see changes. Tests: AttributesDaoConflictGuardTest now carries 14 cases, covering the withdrawal, the zero-replacement refusal, an IoTDB peer under another name, a third-party DAO alongside a removable ThingsBoard one, right-name/wrong-type and right-type/wrong-name, a pre-built singleton with no definition to withdraw, the WARN's four fields, the no-op path and the ordering. Each of the three guarantees was mutation-checked: reverting the class-name half of the match, identifying our bean by assignability, and moving the unknown-bean check after the removal each reddened exactly one test, with an unrelated test green throughout as a contamination control. IoTDBTableTimeseriesAggregationIT gains non-UTC calendar coverage with a boundary-straddling sample -- 2023-01-31T20:00Z is January under UTC and February under Asia/Shanghai -- so an implementation that drops the query timezone changes the bucket count and fails; confirmed by forcing the zone to UTC and watching it go red. 204 unit tests and 58 integration tests pass. The full unit suite is also green against Spring 6.2.18 / Spring Boot 3.5.14, the line ThingsBoard 4.3.1.2 runs.
PDGGKforce-pushed
the
fix/attributes-selector-and-runtime-dependency
branch
from
August 21, 2026 09:33
c22b721 to
86daf89ComparePDGGK added a commit
to PDGGK/iotdb-docs
that referenced
this pull request
Aug 21, 2026
Adds a user-guide page for the iotdb-thingsboard-table module in apache/iotdb-extras: stock ThingsBoard storing telemetry and attributes in IoTDB 2.x table model, by placing the module and its runtime dependencies on ThingsBoard's classpath and setting a few properties. The page lists the exact jar set, including the ones that must NOT be copied because ThingsBoard bundles newer copies of them. Four new pages -- Thingsboard.md in latest-Table and Master/Table, English and Chinese -- plus the two sidebar groups and the four Ecosystem-Overview index rows. Structure follows the existing Spark-IoTDB page. The attributes selector is documented as not yet usable on a stock ThingsBoard: ThingsBoard registers its JPA attributes bean unconditionally and exposes no switch for it, so setting the selector currently fails startup. The fix is apache/iotdb-extras#125; the page says so and will be updated when a build containing it is released.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Deploying the module into a live ThingsBoard for the first time surfaced two defects the existing tests could not reach. Until now the module had been compiled and integration-tested against ThingsBoard's types, but never run inside a running instance. That gap was known and recorded; this is the work that closes it.
The two defects were seen on different deployment attempts, so each names its own below. The end-to-end run, and everything in the first section, is
thingsboard/tb-node:4.3.1.2againstapache/iotdb:2.0.8-standalone, with the module jar and its runtime dependencies placed on ThingsBoard's extension classpath, driven through ThingsBoard's own REST API.The attributes selector could never take effect
ThingsBoard gates its timeseries DAOs on
database.ts.type, so fortsandts_latestit steps aside by itself. It offers no equivalent for attributes: at v4.3.1.2 (dao/src/main/java/org/thingsboard/server/dao/sql/attributes/JpaAttributeDao.java:58-61)JpaAttributeDaois a bare@Component, so it registers unconditionally. I have checked that one version. I have not surveyed the release history, so this PR claims it for v4.3.1.2 and no wider.So
AttributesDaoConflictGuardalways found a competing bean and failed startup:Its own advice cannot be followed — a
@Componentcannot be un-registered from a properties file. So on that build the documented attributes opt-in was not an option a deployment could take: setting the selector produced a startup failure, every time.The guard now withdraws ThingsBoard's bean instead of refusing to start, and the withdrawal is deliberately narrow. Candidates are sorted into three classes and nothing is mutated until every reason to stop has been evaluated:
ioTDBTableAttributesDao, looked up directlyjpaAttributeDaoand resolved typeorg.thingsboard.server.dao.sql.attributes.JpaAttributeDaoAttributesDao— a third-party backend, a decorator, a subclass of ours under another nameThe third row is the part I care most about. A bean the operator registered deliberately is not this module's to delete, so the module refuses rather than guesses — and unlike the old message, "remove it or unset the selector" is now advice the operator can actually act on, because the bean belongs to their application.
Matching on the class name and not on assignability means a rename upstream fails closed: the bean becomes unrecognised and startup stops naming it, rather than the guard removing something it should not have.
The message it emits, excerpted and wrapped for width.
AttributesDaoConflictGuardTestasserts that the current code emits a WARN naming the bean, its class, the property and its value; the same message was observed in a live run againstthingsboard/tb-node:4.3.1.2on 2026-08-20, which exercised the earlier revision of the guard:Our own DAO bean also had to lose
@ConditionalOnMissingBean(type = ATTRIBUTES_DAO_CLASS_NAME)— the string form, chosen so that evaluating the condition does not load ThingsBoard classes. It is evaluated while configuration classes are parsed, strictly before the post-processor runs, so on a stock 4.3.1.2 it skipped our bean on every deployment.What the guard guarantees, and where that stops. After it runs, exactly one
AttributesDaodefinition remains among the candidates it could see, and it is ours. Candidates come from onegetBeanNamesForType(type, true, false)snapshot, which does not initialiseFactoryBeans and does not consult a parent factory — so a definition registered by a later post-processor, produced by an opaqueFactoryBeanwhosegetObjectType()is null until initialisation, or inherited from an ancestor context is outside the guarantee. That boundary is in the javadoc rather than only here.A runtime dependency was demoted out of the runtime classpath
iotdb-thrift-commonswas declared at test scope, with a comment describing it as "a transitive runtime dependency of iotdb-session" — which is precisely what the declaration broke, since a direct declaration wins under Maven's nearest-definition rule. It carriesTEndPoint, which the session pool needs at runtime. The runtime dependency set produced from the POM omitted that artifact, so the deployed module could not create a session — observed as aTEndPointload failure during an earliertb-postgres:4.2.1.1deployment attempt, not during the 4.3.1.2 run above. I have not preserved that container's log, so I am stating the observation rather than quoting a trace.It is now declared at
runtimescope, which is what it actually is: no main source references it, the unit tests constructStatementExecutionException(TSStatus), and the session pool needs it at runtime. Runtime scope puts it on the test and runtime classpaths and off the compile classpath — exactly where it is used. Verified withdependency:build-classpath -DincludeScope=runtime: absent before, present after.dependency:analyze-onlyno longer reports it, so the analyzer suppression an earlier revision of this PR carried has been deleted rather than kept.One new compile-only file
src/provided/java/org/thingsboard/server/dao/sql/attributes/JpaAttributeDao.javajoins the existing Strategy F surface. ThingsBoard's dao artifact is not on Maven Central, so without a class of that exact fully-qualified name on the test classpath the conjunctive match above could not be exercised by any test at all.It is excluded from the packaged jar by the existing
org/thingsboard/**rule, so it cannot shadow the real class on a deployment classpath. I verified that by building the jar and listing its entries rather than by reading the configuration: zeroorg/thingsboardentries.To be explicit about what it does not prove: no test in this module establishes that
org.thingsboard.server.dao.sql.attributes.JpaAttributeDaois the right name. That comes from reading ThingsBoard's source at v4.3.1.2.Documentation in this PR
Three in-repo documents change, because the behaviour they describe changed. They are a third of the diff and worth reading as part of it.
docs/user-guide.md— the "Conflict guards" section now separates the two behaviours. The timeseries and latest guards still fail startup on a competing DAO, and that advice is actionable. The attributes guard cannot ask an operator to removeJpaAttributeDao, so it describes the withdrawal instead, quotes the WARN, states that the match is on bean name and fully-qualified class name, and states the visibility boundary. The attributes bullet in the selector list is rewritten for the same reason.docs/migration-guide.md— the "Fail-fast conflict guard" bullet was accurate for all three routes and is no longer. It is now split: fail-fast for timeseries and latest, narrow withdrawal for attributes, with the same boundary sentence.Two production javadoc blocks change for the same reason —
IoTDBTableAttributesDaoandIoTDBTableAttributesEnabledConditioneach argued that "no real Phase-1 deployment setsdatabase.attributes.type", which is the rationale this change overturns. Both now describe the selector as one this module supplies and point at the guard for what setting it does.README.md— this one is the reason it is here rather than in a follow-up. Its attributes section argued "no shipped ThingsBoard release exposes adatabase.attributes.typeselector yet, so a real Phase-1 deployment never sets it". That rationale is what this change overturns: the selector is one this module supplies, and after this PR setting it does something. The paragraph now says what is true instead. Retained unchanged: the statement that the attributes selector is independent ofdatabase.ts.type/database.ts_latest.type— that is independence among this module's own three routes, and it remains correct.Verification
mvn -P with-thingsboard,iotdb-table-it clean verify— 204 unit tests + 58 integration tests, rat / checkstyle / spotless / dependency-analyze / enforcer clean.Two constants in
IoTDBTableAttributesEnabledConditionwiden fromprivateto package-private so the guard's WARN can name the property and value from their single definition; the class is itself package-private, so nothing a consumer can see changes.IoTDBTableTimeseriesAggregationITgains non-UTC calendar coverage in this PR — a boundary-straddling sample where2023-01-31T20:00Zis January under UTC and February underAsia/Shanghai, so an implementation that drops the query timezone produces a different bucket count and fails. Confirmed by forcing the zone to UTC and watching it go red.The guard's own class now carries 14 tests. Each of the three guarantees was mutation-checked: reverting the class-name half of the match, identifying our bean by assignability instead of by name, and moving the unknown-bean check after the removal each reddened exactly one test, with an unrelated test staying green as a contamination control.
The module builds against Spring 5.3 while ThingsBoard 4.3.1.2 runs Spring 6. One run of the full unit suite with
-Dspring.version=6.2.18 -Dspring-boot.version=3.5.14is also green (204/204). Making that a standing matrix rather than a one-off check is a reasonable follow-up.End to end on 4.3.1.2, through ThingsBoard's REST API: telemetry and attributes written and confirmed in IoTDB; five read shapes correct — raw range, latest, attributes, fixed-window aggregation (AVG and COUNT arithmetic exact), and calendar aggregation in a non-UTC zone. ThingsBoard's own internal telemetry flows through the module as well (
cpuUsage,discUsage,successfulMsgs,activeDevicesCountHourly), so what the module serves is the platform's telemetry persistence, not only the test device I created. Entities, relations, users and dashboards do not touchTimeseriesDaoand stay in PostgreSQL. That run exercised the earlier revision of the guard; the current one is covered by the unit and integration suites above.Points I'd appreciate review on
May an extension module withdraw one named bean definition from its host, on the strength of a single explicit non-default selector plus a WARN? That is the whole question, and it is the one thing I would most like ruled on.
The mechanics, precisely: the guard is a
BeanFactoryPostProcessoractivated bydatabase.attributes.type=iotdb-tablealone.iotdb.attributes.cluster_modeis validated when the DAO is constructed, which is after the post-processor has run, so it is not a precondition of the withdrawal — though a successful attributes deployment still requires it, since a failed validation fails the context and the in-memory withdrawal never becomes external state.My reasoning for saying yes: ThingsBoard switches its timeseries DAOs by configuration and offers nothing equivalent for attributes, so this supplies a switch it does not have rather than fighting its design; and narrowing the target to one bean matched on name and fully-qualified class keeps it from ever touching a bean the operator wrote. In Add ThingsBoard latest-telemetry and attributes DAOs on IoTDB Table Mode #113 this guard was reviewed as "fail loudly on a conflict"; it is now "withdraw ThingsBoard's own attributes bean, fail loudly on anything else".
Two alternatives, both live. Retreat — mark attributes as pending ThingsBoard-side selector support: honest, but gives up the capability. Gate the withdrawal behind its own property (say
iotdb.attributes.replace-existing-dao), so that mutating the host requires its own explicit acknowledgement rather than riding on the backend selector.I am happy to take any of the three, or to take this to dev@ first if that is the better route.
Whether the compile-only stub is the right way to make that match testable. It follows the pattern already in the module, but it is the first stub added for a concrete ThingsBoard class rather than an SPI interface. If there is a way you would rather see this pinned, I will take it.
Happy to iterate.