Skip to content

Pipe: Add metrics for tsfile to tablets invocation call count and time - #15202

Merged
SteveYurongSu merged 4 commits into
masterfrom
tsfile-2-tablet-metrics
Mar 27, 2025
Merged

Pipe: Add metrics for tsfile to tablets invocation call count and time#15202
SteveYurongSu merged 4 commits into
masterfrom
tsfile-2-tablet-metrics

Conversation

@SteveYurongSu

Copy link
Copy Markdown
Member

As title.

@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecovBot commented Mar 26, 2025

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 13.68421% with 82 lines in your changes missing coverage. Please review.

Project coverage is 39.35%. Comparing base (a5c1d11) to head (3737cd9).
Report is 4 commits behind head on master.

Files with missing linesPatch %Lines
...pe/metric/overview/PipeTsFileToTabletsMetrics.java0.00%60 Missing ⚠️
...otdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java0.00%6 Missing ⚠️
...mmon/tsfile/parser/TsFileInsertionEventParser.java50.00%5 Missing ⚠️
...ile/parser/TsFileInsertionEventParserProvider.java0.00%5 Missing ⚠️
.../parser/table/TsFileInsertionEventTableParser.java0.00%3 Missing ⚠️
...ache/iotdb/db/pipe/metric/PipeDataNodeMetrics.java0.00%2 Missing ⚠️
...extractor/dataregion/IoTDBDataRegionExtractor.java0.00%1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## master #15202 +/- ##
=========================================
Coverage 39.35% 39.35% Complexity 193 193 =========================================
Files 4619 4620 +1 Lines 297275 297368 +93 Branches 37114 37122 +8 =========================================
+ Hits 116980 117023 +43 - Misses 180295 180345 +50 

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@SteveYurongSuSteveYurongSu self-assigned this Mar 26, 2025

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds new metrics for monitoring tsfile-to-tablets operations, tracking both invocation count and execution time. Key changes include:

  • Adding two new metric enums in Metric.java.
  • Implementing the PipeTsFileToTabletsMetrics class for metric management.
  • Integrating these metrics into various parser and task components.

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
FileDescription
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/service/metric/enums/Metric.javaAdded new metric enums for tsfile-to-tablets time and rate.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/metric/overview/PipeTsFileToTabletsMetrics.javaIntroduced a new metric class for managing tsfile-to-tablets metrics.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/metric/PipeDataNodeMetrics.javaIntegrated new metrics into the data node metrics binding.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/extractor/dataregion/IoTDBDataRegionExtractor.javaRegistered tsfile-to-tablets metrics during data region extraction.
Various parser and provider filesUpdated constructors and method calls to pass pipeName and creationTime for metric tracking.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.javaDeregistered the new metrics during pipe task cleanup.
Comments suppressed due to low confidence (2)

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/metric/overview/PipeTsFileToTabletsMetrics.java:130

  • [nitpick] Consider renaming 'markTsFileToTabletInvocation' to 'markTsFileToTabletsInvocation' to match the plural form used in the metric enums for consistency.
public void markTsFileToTabletInvocation(final String taskID) {

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/metric/overview/PipeTsFileToTabletsMetrics.java:143

  • [nitpick] Consider renaming 'recordTsFileToTabletTime' to 'recordTsFileToTabletsTime' to maintain consistency with the metric enum naming.
public void recordTsFileToTabletTime(final String taskID, long costTimeInNanos) {

@SteveYurongSu
SteveYurongSu merged commit e86ec17 into masterMar 27, 2025
@SteveYurongSu
SteveYurongSu deleted the tsfile-2-tablet-metrics branch March 27, 2025 03:24
SteveYurongSu added a commit that referenced this pull request Apr 1, 2025
#15202) & Pipe: Prevent Duplicate Time Usage Reporting on Close in PipeTsFileToTabletsMetrics (#15220) (#15224)
Co-authored-by: Steve Yurong Su <rong@apache.org>
JackieTien97 pushed a commit that referenced this pull request Apr 14, 2025
Caideyipi pushed a commit to Caideyipi/iotdb that referenced this pull request Mar 25, 2026
apache#15202) & Pipe: Prevent Duplicate Time Usage Reporting on Close in PipeTsFileToTabletsMetrics (apache#15220) (apache#15224)
Co-authored-by: Steve Yurong Su <rong@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@SteveYurongSu