What happened?
Since beam 2.49 after #26320, we can see the following exception
"java.lang.NullPointerException
at org.apache.avro.generic.GenericData.addLogicalTypeConversion(GenericData.java:116)
at org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.addLogicalTypeConversions(AvroUtils.java:169)
at org.apache.beam.sdk.extensions.avro.io.AvroDatumFactory$ReflectDatumFactory.apply(AvroDatumFactory.java:188)
at org.apache.beam.sdk.extensions.avro.io.AvroIO$Sink.open(AvroIO.java:2126)
at org.apache.beam.sdk.extensions.smb.FileOperations$Writer.prepareWrite(FileOperations.java:219)
at org.apache.beam.sdk.extensions.smb.FileOperations$Writer.access$100(FileOperations.java:206)
at org.apache.beam.sdk.extensions.smb.FileOperations.createWriter(FileOperations.java:133)
Either in the AvroIO or in the AvroCoder.
This seems to happen due to a concurrency access on the same GenericData (During initialization of the datum reader and writer, there is a high chance for avro returning the singleton instance).
The ReflectDatumFactory should avoid concurrent access to avro's model data.
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components
What happened?
Since beam
2.49after #26320, we can see the following exceptionEither in the
AvroIOor in theAvroCoder.This seems to happen due to a concurrency access on the same
GenericData(During initialization of the datum reader and writer, there is a high chance for avro returning the singleton instance).The
ReflectDatumFactoryshould avoid concurrent access to avro's model data.Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components