Skip to content
This repository was archived by the owner on May 8, 2026. It is now read-only.

Add CSM for batch write flow control - #2685

Merged
mutianf merged 1 commit into
googleapis:mainfrom
kongweihan:csm-batch-write-flow-control
Dec 15, 2025
Merged

Add CSM for batch write flow control#2685
mutianf merged 1 commit into
googleapis:mainfrom
kongweihan:csm-batch-write-flow-control

Conversation

@kongweihan

@kongweihankongweihan commented Oct 6, 2025

Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)
  • Rollback plan is reviewed and LGTMed
  • All new data plane features have a completed end to end testing plan

Fixes #<issue_number_goes_here> ☕️

If you write sample code, please follow the samples format.

@kongweihan
kongweihan requested review from a teamOctober 6, 2025 16:18
@product-auto-labelproduct-auto-labelBot added size: m Pull request size is medium. api: bigtable Issues related to the googleapis/java-bigtable API. labels Oct 6, 2025
@kongweihan
kongweihanforce-pushed the csm-batch-write-flow-control branch 2 times, most recently from 2ca884e to b4006a2CompareOctober 6, 2025 16:54
mutianf
mutianf reviewed Oct 6, 2025
View reviewed changes
Comment thread...rc/main/java/com/google/cloud/bigtable/data/v2/stub/RateLimitingServerStreamingCallable.java Outdated
Comment thread...rc/main/java/com/google/cloud/bigtable/data/v2/stub/RateLimitingServerStreamingCallable.java Outdated
Comment thread...rc/main/java/com/google/cloud/bigtable/data/v2/stub/RateLimitingServerStreamingCallable.java Outdated
Comment thread...ud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BigtableTracer.java Outdated
Comment thread...ud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BigtableTracer.java Outdated
Comment thread...le/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsConstants.java Outdated
Comment thread...le/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsConstants.java Outdated
@kongweihan
kongweihanforce-pushed the csm-batch-write-flow-control branch from b4006a2 to e8a246fCompareOctober 7, 2025 03:03
mutianf
mutianf reviewed Oct 8, 2025
View reviewed changes
Comment thread...rc/main/java/com/google/cloud/bigtable/data/v2/stub/RateLimitingServerStreamingCallable.java Outdated
Comment thread...ud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BigtableTracer.java
Comment thread...le/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsConstants.java
Comment thread...le/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsConstants.java Outdated
Comment thread...le/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsConstants.java Outdated
Comment thread...le/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsConstants.java Outdated
Comment thread...rc/main/java/com/google/cloud/bigtable/data/v2/stub/RateLimitingServerStreamingCallable.java Outdated
@kongweihan
kongweihanforce-pushed the csm-batch-write-flow-control branch from e8a246f to ae0ed5aCompareOctober 8, 2025 14:27
@kongweihan
kongweihanforce-pushed the csm-batch-write-flow-control branch 7 times, most recently from 7d3fdd2 to 5f07efcCompareDecember 10, 2025 00:37
@product-auto-labelproduct-auto-labelBot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Dec 10, 2025
mutianf
mutianf reviewed Dec 10, 2025
View reviewed changes
Comment thread...rc/main/java/com/google/cloud/bigtable/data/v2/stub/RateLimitingServerStreamingCallable.java
Comment thread...rc/main/java/com/google/cloud/bigtable/data/v2/stub/RateLimitingServerStreamingCallable.java Outdated
Comment thread...table/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracer.java Outdated
Comment thread...rc/main/java/com/google/cloud/bigtable/data/v2/stub/RateLimitingServerStreamingCallable.java Outdated
Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
}
builder.addEntriesBuilder().setIndex(i);

com.google.protobuf.Duration duration = builder.getRateLimitInfoBuilder().getPeriodBuilder().setSeconds(10).build();

@mutianfmutianfDec 10, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can you import com.google.bigtable.v2.RateLimitInfo so we don't need to write the full path here?

@kongweihankongweihanDec 15, 2025

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You meant Duration? I didn't see a full path here for RateLimitInfo?

Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java
@kongweihan
kongweihanforce-pushed the csm-batch-write-flow-control branch 3 times, most recently from 8e84e9f to 7981240CompareDecember 11, 2025 22:36
mutianf
mutianf reviewed Dec 12, 2025
View reviewed changes

@mutianfmutianf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding the tests, left some more nits but otherwise looks good!

Comment thread...le/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsConstants.java Outdated
Comment thread...rc/main/java/com/google/cloud/bigtable/data/v2/stub/RateLimitingServerStreamingCallable.java
Comment threadgoogle-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/Util.java
Comment thread...le/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTestUtils.java Outdated
Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
batcher.add(
RowMutationEntry.create("batch-write-flow-control-success-08").setCell("f", "q", "v"));

// closing the batcher to trigger the third flush

@mutianfmutianfDec 12, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
// closing the batcher to trigger the third flush
// closing the batcher to trigger the flush

Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
batcher.add(
RowMutationEntry.create("batch-write-flow-control-success-05").setCell("f", "q", "v"));

// closing the batcher to trigger the third flush

@mutianfmutianfDec 12, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// closing the batcher to trigger the third flush
// closing the batcher to trigger the flush

Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
Comment thread...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
@kongweihan
kongweihanforce-pushed the csm-batch-write-flow-control branch from 7981240 to caf5a50CompareDecember 15, 2025 17:36
@mutianf
mutianf merged commit 62ffd1b into googleapis:mainDec 15, 2025
32 of 34 checks passed
@release-pleaserelease-pleaseBot mentioned this pull request Dec 15, 2025
Merged
blakeli0 added a commit that referenced this pull request Feb 3, 2026
@blakeli0@diegomarquezp
@cloud-java-bot@renovate-bot@release-please@sushanb@kongweihan@trollyxia@mutianf
chore: Merge release v2.71.0 into protobuf-3.x (#2771)
2e10f0b
* tests: Remove unnecessary null check for protobuf lists (#2723)
ErrorProne will complain ImpossibleNullComparison once protobuf runtime is upgraded to 4.x. See the downstream tests in sdk-platform-java.
* chore: add release-please config for protobuf-4.x (#2722)
* chore: add release-please config for protobuf-4.x
* chore: generate libraries at Wed Dec 3 18:40:37 UTC 2025
* chore: enable manifest option in release config
---------
Co-authored-by: cloud-java-bot <cloud-java-bot@google.com>
* deps: update shared dependencies (#2734)
* chore(main): release 2.70.1-SNAPSHOT (#2715)
* chore(main): release 2.70.1-SNAPSHOT
* chore: generate libraries at Fri Dec 12 14:04:07 UTC 2025
---------
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: cloud-java-bot <cloud-java-bot@google.com>
* chore(main): release 2.70.1 (#2735)
* chore(main): release 2.70.1
* chore: generate libraries at Fri Dec 12 15:48:36 UTC 2025
---------
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: cloud-java-bot <cloud-java-bot@google.com>
* fix(bigtable): add handling for gauge metrics (#2719)
* fix(bigtable): add handling for gauge metrics
* add it
* chore: generate libraries at Tue Nov 25 20:15:38 UTC 2025
---------
Co-authored-by: cloud-java-bot <cloud-java-bot@google.com>
* chore(main): release 2.70.2-SNAPSHOT (#2736)
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* feat: Add CSM for batch write flow control (#2685)
* test: Add missing test table cleanup for MaterializedViewIT (#2721)
Change-Id: I1281568894ac2d04f0aafe0aa7f4efa7570ae58a
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a [bug/issue](https://togithub.com/googleapis/java-bigtable/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [ ] Rollback plan is reviewed and LGTMed
- [ ] All new data plane features have a completed end to end testing plan
Fixes #<issue_number_goes_here> ☕️
If you write sample code, please follow the [samples format](
https://togithub.com/GoogleCloudPlatform/java-docs-samples/blob/main/SAMPLE_FORMAT.md).
* tests: Migrate usage of GeneratedMessageV3 to Message (#2745)
* tests: Migrate usage of GeneratedMessageV3 to Message
* chore: generate libraries at Mon Dec 22 22:29:53 UTC 2025
---------
Co-authored-by: cloud-java-bot <cloud-java-bot@google.com>
* fix: create stub with BigtableClientContext so otels are closed (#2747)
* fix: create stub with BigtableClientContext so otels are closed
* address comment
* test: Reduce ExecuteQueryIT flakiness with a retry loop (#2748)
Change-Id: I4171e5b11d2278a8b3dc19d2dec3f03d784759ce
* fix: use the same background executor in otel reader and monitoring c… (#2746)
* fix: use the same background executor in otel reader and monitoring client
* add public api doc
* fix tests
* close background executor
* fix test
* address comments
* fix
* deps: update shared dependencies (#2752)
* fix: update BigtableChannelPool to use the background executor (#2753)
* deps: update dependency com.google.cloud:gapic-libraries-bom to v1.76.0 (#2754)
* chore(main): release 2.71.0 (#2737)
* chore(main): release 2.71.0
* chore: generate libraries at Thu Jan 15 16:06:19 UTC 2026
---------
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: cloud-java-bot <cloud-java-bot@google.com>
---------
Co-authored-by: Diego Marquez <diegomarquezp@google.com>
Co-authored-by: cloud-java-bot <cloud-java-bot@google.com>
Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: Sushan Bhattarai <sushantsusan@google.com>
Co-authored-by: Weihan Kong <kongweihan@users.noreply.github.com>
Co-authored-by: Lixia Chen <trollyxia@gmail.com>
Co-authored-by: Mattie Fu <mattiefu@google.com>
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer
@mutianfmutianfmutianf approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

api: bigtableIssues related to the googleapis/java-bigtable API.size: lPull request size is large.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kongweihan@mutianf