Skip to content

[Bug] Fe cannot be restarted, when the stream load record manager takes effect #5729

Description

@EmmyMiao87

Describe the bug

The persistence logic for FetchStreamLoadRecord is wrong which will cause the metadata to fail to read.

To Reproduce
Version 0.14.8
Steps to reproduce the behavior:

  1. execute stream load
  2. stop fe
  3. start fe
  4. fe cannot be restarted with error:
2021-04-27 20:05:02,233 ERROR (replayer|80) [BDBJournalCursor.next():117] fail to read journal entity key=349360906, will exit
java.io.EOFException: null
at java.io.DataInputStream.readBoolean(DataInputStream.java:244) ~[?:1.8.0_131]
at org.apache.doris.load.StreamLoadRecordMgr$FetchStreamLoadRecord.read(StreamLoadRecordMgr.java:162) ~[palo-fe.jar:3.4.0]
at org.apache.doris.journal.JournalEntity.readFields(JournalEntity.java:525) ~[palo-fe.jar:3.4.0]
at org.apache.doris.journal.bdbje.BDBJournalCursor.next(BDBJournalCursor.java:115) [palo-fe.jar:3.4.0]
at org.apache.doris.catalog.Catalog.replayJournal(Catalog.java:2478) [palo-fe.jar:3.4.0]
at org.apache.doris.catalog.Catalog$3.runOneCycle(Catalog.java:2267) [palo-fe.jar:3.4.0]
at org.apache.doris.common.util.Daemon.run(Daemon.java:116) [palo-fe.jar:3.4.0]

Expected behavior

Fe can be restarted normally.
The record of StreamLoadAudit could be replayed normally.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/catalogIssues or PRs related to catalog managementkind/fixCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions