Uh oh!
There was an error while loading. Please reload this page.
Load: Support multi-disks awareness for IoTV2 and Pipe - #15181
Conversation
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Pull Request Overview
This PR introduces multi-disks awareness for load operations by adding different disk selection strategies for both ordinary load and Pipe/IoTV2 load. It adds a new MinIOSelector implementation, integrates disk selector configuration into IoTDBConfig and IoTDBDescriptor, and updates DataRegion to choose between selectors based on file origin.
Reviewed Changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/disk/MinIOSelector.java | New disk selector implementation using MinIO strategy. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/disk/InheritSystemMultiDisksStrategySelector.java | Base implementation for a multi-disks selection strategy. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/disk/ILoadDiskSelector.java | Added interface for load disk selectors along with a new enum option. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/DataRegion.java | Updated to initialize and use different disk selectors based on file origin. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/conf/IoTDBDescriptor.java | Updated to load disk selector configurations and adjust consensus configuration. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java | Added new configuration properties for disk selector strategies. |
Files not reviewed (1)
- iotdb-core/node-commons/src/assembly/resources/conf/iotdb-system.properties.template: Language not supported
Comments suppressed due to low confidence (1)
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/disk/MinIOSelector.java:50
- [nitpick] Consider renaming the lambda parameter 'v' to a more descriptive name (e.g., 'dataDir') to improve code readability.
.map(v -> fsFactory.getFile(v, IoTDBConstant.UNSEQUENCE_FOLDER_NAME).getPath())
(cherry picked from commit a81c5c6)
* multi-disk aware * get mount point * enhance log * fix up * fix up * fix import * add test code * use file store * add strategy for load disk selector * fix description * fix review * fix default * spotless * fix review * adopt to multi-layer storage (cherry picked from commit a81c5c6)
…apache#15213) (cherry picked from commit a81c5c6)
as title.
Usage of
FileStoreto get disk root of one specific file has been self tested.Test on MacOS Machine:
Test on Linux Machine: