Skip to content

[To dev/1.3] Load: Support Async Load in SQL (#15208) - #15320

Merged
SteveYurongSu merged 3 commits into
apache:dev/1.3from
luoluoyuyu:cp-async-load
Apr 22, 2025
Merged

[To dev/1.3] Load: Support Async Load in SQL (#15208)#15320
SteveYurongSu merged 3 commits into
apache:dev/1.3from
luoluoyuyu:cp-async-load

Conversation

@luoluoyuyu

Copy link
Copy Markdown
Member

Description

As the title said


This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

…Load in SQL & Support Async Load Strategy in Pipe (apache#15208)
Co-authored-by: Steve Yurong Su <rong@apache.org>

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

# Conflicts:
#	iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/utils/FileUtils.java

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for asynchronous loading in the SQL LOAD TSFILE command by introducing a new boolean parameter and corresponding processing logic. Key changes include:

  • Adding a new parameter ("async") to the load configuration and validation in LoadTsFileConfigurator.
  • Introducing an asynchronous load flag in LoadTsFileStatement with appropriate initialization and string representation.
  • Updating LoadTsFileAnalyzer to branch to an asynchronous file loading mechanism using retry utilities and logging.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

FileDescription
LoadTsFileConfigurator.javaAdded new configuration key, default value, and parameter validation for async load.
LoadTsFileStatement.javaIntroduced new boolean flag for async load and integrated it into attribute initialization and toString().
LoadTsFileAnalyzer.javaImplemented async load handling logic with file transfers and retry mechanism.

@SteveYurongSu
SteveYurongSu merged commit cb48b67 into apache:dev/1.3Apr 22, 2025
@luoluoyuyu
luoluoyuyu deleted the cp-async-load branch April 28, 2025 09:39
Caideyipi pushed a commit to Caideyipi/iotdb that referenced this pull request Mar 25, 2026
* Load & Pipe: Support Active Load Table Model TsFiles & Support Async Load in SQL & Support Async Load Strategy in Pipe (apache#15208)
---------
Co-authored-by: Steve Yurong Su <rong@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@luoluoyuyu@SteveYurongSu