feat(bigtable): add client side metric instrumentation to basic rpcs by daniel-sanche · Pull Request #16712 · googleapis/google-cloud-python

daniel-sanche · 2026-04-17T20:19:17Z

Migration of googleapis/python-bigtable#1188 to the monorepo

This PR builds off of googleapis/python-bigtable#1187 to add instrumentation to basic data client rpcs (check_and_mutate, read_modify_write, sample_row_keys, mutate_row)

Metrics are not currently being exported anywhere, just collected and dropped. A future PR will add a GCP exporter to the system

gemini-code-assist

Code Review

This pull request integrates client-side metrics tracking into the Bigtable data client for both asynchronous and synchronous implementations. It wraps key operations—including sample_row_keys, mutate_row, check_and_mutate_row, and read_modify_write_row—with metrics collection logic and introduces a tracked_retry wrapper to monitor retry attempts. Additionally, the PR refactors system tests by consolidating fixtures into a shared SystemTestRunner class and adds new system tests specifically for metrics. Feedback focuses on regressions in retry logic where critical arguments like sleep_generator and exception_factory were omitted in the transition to tracked_retry. There are also suggestions to improve resource cleanup in test fixtures and to relax restrictive timing assertions in tests to prevent flakiness.

gemini-code-assist · 2026-04-17T20:26:15Z

+ ),
+ clusters=cluster_config,
+ )
+ operation.result(timeout=240)


If operation.result(timeout=240) raises an exception (e.g., a TimeoutError), the fixture will stop execution and the delete_instance call in the teardown phase will never be reached. This can lead to leaked Bigtable instances in the test project. Consider wrapping the creation and yield in a try...finally block or ensuring cleanup happens even on creation failure.

mutianf

some final nit otherwise lgtm

mutianf · 2026-04-23T15:09:20Z

+ # validate operation
+ operation = handler.completed_operations[0]
+ assert isinstance(operation, CompletedOperationMetric)
+ assert operation.final_status.value[0] == 0


why is this final_status.value[0] and not just final_status.value? or final_status.name = 'OK'?

value is a tuple of (0, "ok"). We could just do final_status.name == 'OK' if you think that's clearer though

i just noticed that some other tests are using final_status.name :)

mutianf · 2026-04-23T15:13:32Z

+ for i in range(num_retryable):
+ attempt = handler.completed_attempts[i]
+ assert isinstance(attempt, CompletedAttemptMetric)
+ assert attempt.end_status.name == "ABORTED"


carry over my comment from before, I don't think we retry aborted error for mutate row by default, so maybe use unavailable instead? And also a little suprised that this test passes?

We allow the user to set custom retryable errors, which is what the test is doing here. So that's why it passes.
I can't remember for sure why I chose to use explicit retryable errors here instead of the default. Maybe just so the test makes sense even if you don't know the defaults. I could probably switch it to UNAVAILABLE though if you prefer

mutianf · 2026-04-29T15:16:26Z

+ )
+ return [(s.row_key, s.offset_bytes) async for s in results]
+
+ return await tracked_retry(


where did sleep_generator and exception_factory go?

tracked_retry contains them both. needed a custom versions of sleep_generator to report backoff, and exception_factory to report terminal errors metrics module. See go/bigtable-csm-python

mutianf · 2026-04-29T15:18:28Z

+ timeout=operation_timeout,
+ retry=None,
+ )
+ return result.predicate_matched


this is not wrapped in tracked_retry. Will the attempt level metrics (attempt latencies, server latencies and connectivity error count) still be recorded?

check_and_mutate shouldn't have retries, right?
But yes, a sngle attempt is will be recorded when this completes. The duration/gfe_latency data is captured, and will be exported as those metrics in the follow-up PR

mutianf · 2026-04-29T15:18:40Z

+ retry=None,
+ )
+ # construct Row from result
+ return Row._from_pb(result.row)


same question as check and mutate.

Yes, those are also captured here

…async.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…e_rows (#16758) Migration of googleapis/python-bigtable#1256 to the monorepo Builds off of #16712 to add instrumentation to read_rows and mutate_rows, along with the mutation batcher

🤖 I have created a release *beep* *boop* --- ## [2.39.0](google-cloud-bigtable-v2.38.0...google-cloud-bigtable-v2.39.0) (2026-06-22) ### Features * added client side metric instrumentation to read_rows and mutate_rows ([#16758](#16758)) ([4005e66](4005e66)) * **bigtable:** add client side metric instrumentation to basic rpcs ([#16712](#16712)) ([636af26](636af26)) * **bigtable:** add view_parameters support to execute_query ([#17382](#17382)) ([2695aad](2695aad)) * support row_range in sample_row_keys method ([#17330](#17330)) ([384724c](384724c)), closes [#17329](#17329) * update API sources and regenerate ([#17431](#17431)) ([2e75c78](2e75c78)) * update googleapis and regenerate ([33ba3af](33ba3af)) ### Bug Fixes * **bigtable:** ensure deadline is respected for read_rows_sharded ([#17352](#17352)) ([6cc890b](6cc890b)) * require Python 3.10+ ([#17245](#17245)) ([200b0d3](200b0d3)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

daniel-sanche added 3 commits April 17, 2026 12:38

moved over instrumentation code
8a3af45

refactored system tests
cf58c57

updated tests
00618f8

daniel-sanche requested a review from a team as a code owner April 17, 2026 20:19

gemini-code-assistBot reviewed Apr 17, 2026
View reviewed changes

fixed lint
ead3bf1

daniel-sanche added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 21, 2026

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 21, 2026

daniel-sancheand others added 4 commits April 21, 2026 12:39

Merge branch 'main' into bigtable_csm_1_basic_instrumentation
af0beba

regenerated files
d0c03d0

fixed mtls tests
01f1739

fixed format
d40fa0a

daniel-sanche mentioned this pull request Apr 21, 2026
feat: added client side metric instrumentation to read_rows and mutate_rows #16758
Merged

mutianf reviewed Apr 23, 2026
View reviewed changes

mutianf reviewed Apr 29, 2026
View reviewed changes

parthea assigned daniel-sanche May 1, 2026

daniel-sancheand others added 2 commits May 4, 2026 16:00

Update packages/google-cloud-bigtable/tests/system/data/test_metrics_…
febdaac
…async.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

improve system test cleanup
c7b142d

mutianf approved these changes May 7, 2026
View reviewed changes

fix lint
e936535

daniel-sanche enabled auto-merge (squash) May 8, 2026 22:57

daniel-sanche merged commit 636af26 into mainMay 8, 2026
34 checks passed

daniel-sanche deleted the bigtable_csm_1_basic_instrumentation branch May 8, 2026 23:07

release-pleaseBot mentioned this pull request Jun 17, 2026
chore(main): release google-cloud-bigtable 2.39.0 #17497
Merged

release-pleaseBot mentioned this pull request Jul 15, 2026
chore(main): release google-cloud-bigtable 0.1.0 #17719
Closed

Uh oh!

Conversation

daniel-sanche commented Apr 17, 2026

Uh oh!

gemini-code-assistBot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gemini-code-assistBotApr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mutianf left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants