Skip to content

Revert "Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils" - #24489

Merged
Abacn merged 2 commits into
masterfrom
revert-23969-sqlttypes
Dec 2, 2022
Merged

Revert "Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils"#24489
Abacn merged 2 commits into
masterfrom
revert-23969-sqlttypes

Conversation

@Abacn

@AbacnAbacn commented Dec 2, 2022

Copy link
Copy Markdown
Contributor

Fixes#24415

Reverts #23969

#23969 introduced

  • a one direction mapping from LogicalType(Date) to Avro Logical Type Data
  • and a bidirectional mapping between LoficalType(SqlTypes.TIMESTAMP) Avro Logical type TimestampMicros

However, SqlTypes.TIMESTAMP (portable micros_instant) is not yet supported in Beam SQL (though its comments say Beam LogicalType corresponding to ZetaSQL TIMESTAMP type, which is not true yet)

@Abacn

Abacn commented Dec 2, 2022

Copy link
Copy Markdown
ContributorAuthor

R: @apilloud
CC: @kennknowles

@github-actions

Copy link
Copy Markdown
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

@Abacn

Abacn commented Dec 2, 2022

Copy link
Copy Markdown
ContributorAuthor

This is a regression of Beam 2.44.0. Though It should be safe to make mapping of LoficalType(SqlTypes.TIMESTAMP) -> Avro Logical type TimestampMicros one directional as Date, however, defer this to future releases.

@Abacn

Abacn commented Dec 2, 2022

Copy link
Copy Markdown
ContributorAuthor

R: @robertwb

@Abacn

Abacn commented Dec 2, 2022

Copy link
Copy Markdown
ContributorAuthor

Java Precommit fails known flakes

org.apache.beam.runners.samza.runtime.AsyncDoFnRunnerTest.testPipelineWithState
org.apache.beam.sdk.io.pulsar.PulsarIOTest.testReadFromSimpleTopic

#21714; #23745 (same test class) not relevant to this PR

@Abacn

Abacn commented Dec 2, 2022

Copy link
Copy Markdown
ContributorAuthor

Kotlin example actually passed but not updated to GitHub UI: https://ci-beam.apache.org/job/beam_PreCommit_Kotlin_Examples_Commit/2800/

Will get this in and need a cherry pick to release-2.44.0 branch

@Abacn
Abacn merged commit d873220 into masterDec 2, 2022
@Abacn
Abacn deleted the revert-23969-sqlttypes branch December 2, 2022 15:32
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT

2 participants

@Abacn@kennknowles