BigTable: row.commit throwing _Rendezvous exception instead of ServiceUnavailable

[zakons]

We are using retry.py and creating a retry exception predicate as follows:

predicate=retry.if_exception_type(exceptions.ServiceUnavailable)

However, since we are only getting _Rendezvous exceptions with StatusCode.UNAVAILABLE (see below stack trace), we had to change this to:

predicate=retry.if_exception_type(exceptions.GrpcRendezvous)

I see a different issue in cloud datastore for mapping gRPC codes to service level exceptions:

#2590

Will this also be done for cloud bigtable?

Stack Trace:

 File "single_key_read_write.py", line 135, in write
 row.commit()
 File "/home/jenkins/.local/lib/python3.5/site-packages/google/cloud/bigtable/row.py", line 445, in commit
 retry_(retry_commit)()
 File "/home/jenkins/.local/lib/python3.5/site-packages/google/api_core/retry.py", line 261, in retry_wrapped_func
 on_error=on_error,
 File "/home/jenkins/.local/lib/python3.5/site-packages/google/api_core/retry.py", line 178, in retry_target
 return target()
 File "/home/jenkins/.local/lib/python3.5/site-packages/google/cloud/bigtable/row.py", line 256, in __call__
 client._data_stub.MutateRow(self.request_pb)
 File "/home/jenkins/.local/lib/python3.5/site-packages/grpc/_channel.py", line 492, in __call__
 return _end_unary_response_blocking(state, call, False, deadline)
 File "/home/jenkins/.local/lib/python3.5/site-packages/grpc/_channel.py", line 440, in _end_unary_response_blocking
 raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Transport closed)>

[dhermes]

@zakons In order for the "new" exceptions to be in place, the Bigtable gRPC / GAPIC surface will need to be regenerated.

@lukesneeringer Can we make this happen?

[sduskis]

@dhermes: IIUC, the Bigtable client doesn't actually use GAPIC.

FWIW, I'm all for regenerating the protobuf and gRPC artifacts. I'd also be happy if we can migrate to GAPIC.

[dhermes]

IIUC, the Bigtable client doesn't actually use GAPIC.

That's correct. Bigtable was the very first package to use protoc with the grpc extension and we still have a Makefile to build the google.cloud.bigtable._generated package.

This very much should be retired in favor of the "modern" autogen infrastructure.

[zakons]

Thanks. Just to clarify, after the regeneration we would get a mapping of the Rendezvous exception to a ServiceUnavailable exception? Curious to know how this code generation works so that the code in api_core/google/api_core/exceptions.py for ServiceUnavailable is used.

[theacodes]

Just to clarify, after the regeneration we would get a mapping of the Rendezvous exception to a ServiceUnavailable exception?

Yes

Curious to know how this code generation works so that the code in api_core/google/api_core/exceptions.py for ServiceUnavailable is used.

The generator generates wrappers for the underlying grpc client (stub in grpc terminology). One of the wrappers applied to all RPC methods is google.api_core.grpc_helpers.wrap_errors which handles re-mapping grpc.RpcErrors to appropriate exception classes. wrap_errors is applied to RPC methods by google.api_core.gapic_v1.method.wrap_method which is called by the generated code (example).