ORC-1858: [C++] Add support to get StripeStatistics without row index

[wushap]

What changes were proposed in this pull request?

add a C++ API in reader to get stripe level statistics without reading row group index.

Why are the changes needed?

To #2137

How was this patch tested?

UT PASS

Was this patch authored or co-authored using generative AI tooling?

NO

[dongjoon-hyun]

Thank you for making a PR, @wushap .

cc @wgtmac

[ffacs]

@wushap Gently ping, could you make the CI happy?

[ffacs]

std::make_unique would be better

[wgtmac]

Thanks for adding this! Could you please create a JIRA issue for this? If you don't have one, please request one from https://selfserve.apache.org/jira-account.html.

[wushap]

Thanks for adding this! Could you please create a JIRA issue for this? If you don't have one, please request one from https://selfserve.apache.org/jira-account.html.

https://issues.apache.org/jira/browse/ORC-1858
here

[wgtmac]

I'd prefer not to introduce a new API, especially when the name of getStripeStatisticsOnly is a little bit misleading. This is not your fault, getStripeStatistics does not clearly state it carries row index statistics as well :(

What about changing the existing API like below:

virtual std::unique_ptr<StripeStatistics> getStripeStatistics(uint64_t stripeIndex, bool includeRowIndex = true) const = 0;

Then return different StripeStatistics subclasses depending on the value of includeRowIndex.

 class StripeStatisticsImpl : public StripeStatistics {
private:
std::unique_ptr<StatisticsImpl> columnStats_;
// DELIBERATELY NOT IMPLEMENTED
StripeStatisticsImpl(const StripeStatisticsImpl&);
StripeStatisticsImpl& operator=(const StripeStatisticsImpl&);
public:
StripeStatisticsImpl(const proto::StripeStatistics& stripeStats,
const StatContext& statContext);
virtual const ColumnStatistics* getColumnStatistics(uint32_t columnId) const override {
return columnStats_->getColumnStatistics(columnId);
}
uint32_t getNumberOfColumns() const override {
return columnStats_->getNumberOfColumns();
}
virtual const ColumnStatistics* getRowIndexStatistics(uint32_t columnId,
uint32_t rowIndex) const override {
throw NotImplementedYet("getRowIndexStatistics");
}
virtual ~StripeStatisticsImpl() override;
uint32_t getNumberOfRowIndexStats(uint32_t columnId) const override {
throw NotImplementedYet("getNumberOfRowIndexStats");
}
};
class StripeStatisticsWithRowIndexImpl : public StripeStatisticsImpl {
private:
std::vector<std::vector<std::shared_ptr<const ColumnStatistics> > > rowIndexStats_;
public:
StripeStatisticsWithRowIndexImpl(const proto::StripeStatistics& stripeStats,
std::vector<std::vector<proto::ColumnStatistics> >& indexStats,
const StatContext& statContext);
virtual const ColumnStatistics* getRowIndexStatistics(uint32_t columnId,
uint32_t rowIndex) const override {
// check id indices are valid
return rowIndexStats_[columnId][rowIndex].get();
}
virtual ~StripeStatisticsWithRowIndexImpl() override;
uint32_t getNumberOfRowIndexStats(uint32_t columnId) const override {
return static_cast<uint32_t>(rowIndexStats_[columnId].size());
}
};

[wushap]

This looks better

[wgtmac]

+1

Thanks @wushap!

[dongjoon-hyun]

+1, LGTM. Thank you, @wushap and @wgtmac .
Merged to main for Apache ORC 2.2.0.

[dongjoon-hyun]

Welcome to the Apache ORC community and congratulation for your first commit, @wushap .

I added you to the Apache ORC contributor group and assigned ORC-1858 to you.