Skip to content

HIVE-27455: Iceberg: Set COLUMN_STATS_ACCURATE after writing stats for Iceberg tables - #4440

Merged
deniskuzZ merged 1 commit into
apache:masterfrom
SourabhBadhya:HIVE-27455
Jul 12, 2023
Merged

HIVE-27455: Iceberg: Set COLUMN_STATS_ACCURATE after writing stats for Iceberg tables#4440
deniskuzZ merged 1 commit into
apache:masterfrom
SourabhBadhya:HIVE-27455

Conversation

@SourabhBadhya

@SourabhBadhyaSourabhBadhya commented Jun 20, 2023

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Iceberg: Set COLUMN_STATS_ACCURATE after writing stats for Iceberg tables

Why are the changes needed?

Currently, we are writing the stats to puffin files but we are not setting the column stats accurate to the desired values.
The focus of the Jira would be to update the field whenever non-native tables update stats in their format.

Does this PR introduce any user-facing change?

No

Is the change a dependency upgrade?

No

How was this patch tested?

Unit test

deniskuzZ
deniskuzZ reviewed Jun 22, 2023
View reviewed changes
Comment threadql/src/java/org/apache/hadoop/hive/ql/stats/ColStatsProcessor.java Outdated
@SourabhBadhyaSourabhBadhya mentioned this pull request Jun 23, 2023
Merged
kasakrisz
kasakrisz requested changes Jun 30, 2023
View reviewed changes
Comment threadiceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java
Comment threadql/src/test/results/clientpositive/llap/autoColumnStats_10.q.out Outdated
kasakrisz
kasakrisz approved these changes Jun 30, 2023
View reviewed changes

@kasakriszkasakrisz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 pending tests

deniskuzZ
deniskuzZ reviewed Jul 10, 2023
View reviewed changes
Comment threadql/src/java/org/apache/hadoop/hive/ql/stats/ColStatsProcessor.java Outdated
deniskuzZ
deniskuzZ approved these changes Jul 11, 2023
View reviewed changes

@deniskuzZdeniskuzZ left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, pending tests

@sonarqubecloud

sonarqubecloudBot commented Jul 12, 2023

Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed! Quality Gate passed

BugA0 Bugs
VulnerabilityA0 Vulnerabilities
Security HotspotA0 Security Hotspots
Code SmellA3 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@deniskuzZdeniskuzZdeniskuzZ approved these changes
@kasakriszkasakriszkasakrisz approved these changes

Assignees

No one assigned

Labels

tests passed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@SourabhBadhya@deniskuzZ@kasakrisz@kgyrtkirk