Skip to content

branch-4.1: [fix](fe) fix Azure resource persistence #65052 - #65177

Merged
yiguolei merged 1 commit into
branch-4.1from
auto-pick-65052-branch-4.1
Jul 3, 2026
Merged

branch-4.1: [fix](fe) fix Azure resource persistence #65052#65177
yiguolei merged 1 commit into
branch-4.1from
auto-pick-65052-branch-4.1

Conversation

@github-actions

@github-actionsgithub-actionsBot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Cherry-picked from #65052

@github-actions
github-actionsBot requested a review from yiguolei as a code ownerJuly 2, 2026 15:21
@hello-stephen

hello-stephen commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hello-stephen

hello-stephen commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

run buildall

@hello-stephen

hello-stephen commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 50.00% (23/46) 🎉
Increment coverage report
Complete coverage report

@gavinchou@yiguolei
[fix](fe) fix Azure resource persistence (#65052)
96c0b32
### What problem does this PR solve?
Issue Number: DORIS-26664
Related PR: None
Problem Summary:
`AzureResource` was not registered in the `Resource` runtime Gson
adapter. As a result, persisted Azure resources could be serialized
without the `clazz` field, and FE replay from edit log or image failed
when deserializing them as `Resource.class`.
Observed failure stack:
```text
2026-06-26 10:16:34,576 ERROR (leaderCheckpointer|100) [BDBJournalCursor.next():122] fail to read journal entity key=136567, will exit
com.google.gson.JsonParseException: cannot deserialize class org.apache.doris.catalog.Resource because it does not define a field named clazz
at org.apache.doris.persist.gson.RuntimeTypeAdapterFactory$1.read(RuntimeTypeAdapterFactory.java:321)
at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:204)
at com.google.gson.Gson.fromJson(Gson.java:1227)
at com.google.gson.Gson.fromJson(Gson.java:1137)
at com.google.gson.Gson.fromJson(Gson.java:1047)
at com.google.gson.Gson.fromJson(Gson.java:982)
at org.apache.doris.catalog.Resource.read(Resource.java:264)
at org.apache.doris.journal.JournalEntity.readFields(JournalEntity.java:585)
at org.apache.doris.journal.bdbje.BDBJournalCursor.next(BDBJournalCursor.java:119)
at org.apache.doris.catalog.Env.replayJournal(Env.java:3091)
at org.apache.doris.master.Checkpoint.doCheckpoint(Checkpoint.java:161)
at org.apache.doris.master.Checkpoint.runAfterCatalogReady(Checkpoint.java:89)
at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58)
at org.apache.doris.common.util.Daemon.run(Daemon.java:119)
```
This PR registers `AzureResource` in `GsonUtils` and makes both
`Resource.read` and `ResourceMgr.read` compatible with legacy resource
JSON that has `type` but no `clazz`, deriving the subtype label from
`ResourceType`.
Why the persisted JSON could miss `clazz`:
`clazz` is not a real field of `Resource`; it is the discriminator added
by `RuntimeTypeAdapterFactory` only when that factory handles
serialization. `Resource.write()` currently uses
`GsonUtils.GSON.toJson(this)`. When `this` is an `AzureResource` and
`AzureResource` has not been registered as a `Resource` subtype, Gson
looks up serialization by the runtime type `AzureResource.class`, the
resource runtime type adapter does not match, and Gson falls back to
normal reflective serialization. That reflective path writes real fields
such as `name`, `type`, `references`, `id`, and `version`, but it does
not add `clazz`.
The read path is different: `Resource.read()` deserializes the same JSON
as `Resource.class`, so `RuntimeTypeAdapterFactory` is used and requires
the `clazz` discriminator before it can choose a concrete subtype. The
resources image path also deserializes nested `Resource` values through
`ResourceMgr.read()` and `GsonUtils.GSON.fromJson(json,
ResourceMgr.class)`, so those nested resources do not pass through
`Resource.read()`. Registering `AzureResource` fixes new writes and
reads of JSON that already has `clazz:"AzureResource"`, but it cannot
repair already persisted legacy JSON that has only `type:"AZURE"` and no
`clazz`.
@yiguolei
yiguoleiforce-pushed the auto-pick-65052-branch-4.1 branch from d184e9e to 96c0b32CompareJuly 3, 2026 07:43
@yiguolei

yiguolei commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

run buildall

@hello-stephen

hello-stephen commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 32.61% (15/46) 🎉
Increment coverage report
Complete coverage report

@yiguolei

yiguolei commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

skip buildall

@yiguolei
yiguolei merged commit 8ef0476 into branch-4.1Jul 3, 2026
31 of 34 checks passed
@morningman
morningman deleted the auto-pick-65052-branch-4.1 branch July 8, 2026 14:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguoleiyiguoleiyiguolei approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@hello-stephen@yiguolei@gavinchou