TypeError('Object of type ListValue is not JSON serializable')

[SebastianSong]

Hello I am Sebastian Song. When I upgrade google-cloud-logging
I have faced an exception while upgrading the google-cloud-logging version from 1.15.3 to 3.50
As you can see in the attached screenshot, logging was working well in version 1.15.3

Environment details

• OS type and version: docker image From python:3.11.1-slim-bullseye
• Python version: 3.11.1
• pip version: 22.3.1
• google-cloud-logging version: 3.5.0

Code example

# example
 def commit_log(self):
 from google.cloud import logging_v2 as google_logger
 from google.cloud.logging_v2.logger import Batch
 from google.cloud.logging_v2.resource import Resource
 from google.protobuf.struct_pb2 import Struct, Value, ListValue
 from datetime import datetime
 logger = google_logger.Client(_use_grpc=False).logger(
 'appengine.googleapis.com%2Frequest_log')
 batch_client: Batch = logger.batch()
 try:
 line1 = Struct(fields={'severity': Value(string_value="INFO"),'logMessage': Value(string_value="test message1")})
 line2 = Struct(fields={'severity': Value(string_value="INFO"),'logMessage': Value(string_value="test message2")})
 log_lines = ListValue()
 log_lines.values.add(struct_value=line1)
 log_lines.values.add(struct_value=line2)
 message = Struct(fields={
 '@type': Value(string_value='type.googleapis.com/google.appengine.logging.v1.RequestLog'),
 'appId': Value(string_value='test'),
 'latency': Value(string_value='0.019886s'),
 'responseSize': Value(string_value='128'),
 'resource': Value(string_value='/test'),
 'userAgent': Value(string_value='test'),
 'ip': Value(string_value='192.128.0.1'),
 'line': Value(list_value=log_lines)
 })

 batch_client.log_proto(message,
 log_name="projects/banded-totality-629/logs/test",
 severity="INFO",
 resource=Resource(type='gae_app',
 labels={'module_id': "staging-test",
 'project_id': "banded-totality-629",
 'version_id': "v6"}),
 operation={'id': "test_request_id_56599",
 'first': True,
 'last': True,
 },
 timestamp=datetime.utcnow())
 batch_client.commit()
 except Exception as e:
 print(repr(e))

Stack trace

 in batch_client.commit()
TypeError('Object of type ListValue is not JSON serializable')

Thanks!

[daniel-sanche]

It looks like there is an error in the proto parsing logic

It converts the top-level struct to a dict, but it doesn't recurse into the lists.

We could fix this by improving the dict conversion logic, or by using MessageToDict when the input is a Struct type, which also seems to work.

We have system tests in place that should have caught this, but it looks like the test sample message doesn't include lists