[Feature](Variant) support variant load by eldenmoon · Pull Request #24554 · apache/doris

eldenmoon · 2023-09-18T10:13:29Z

Proposed changes

This PR is the load part of variant column

related #26225
Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

github-actions

clang-tidy made some suggestions

eldenmoon · 2023-09-18T10:24:56Z

run buildall

doris-robot · 2023-09-18T11:23:51Z

TeamCity be ut coverage result:
Function Coverage: 36.57% (7990/21847)
Line Coverage: 28.64% (64068/223725)
Region Coverage: 27.52% (33283/120926)
Branch Coverage: 24.16% (17054/70590)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1b2f9420f2d4ddbda3c129dc92b04b7d62b2e124_1b2f9420f2d4ddbda3c129dc92b04b7d62b2e124/report/index.html

eldenmoon · 2023-09-19T04:15:37Z

run buildall

doris-robot · 2023-09-19T05:07:37Z

TeamCity be ut coverage result:
Function Coverage: 36.58% (7990/21845)
Line Coverage: 28.64% (64079/223736)
Region Coverage: 27.53% (33286/120930)
Branch Coverage: 24.17% (17062/70594)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cff9e79c025a3418f0b18d1af247051e0ac6b53b_cff9e79c025a3418f0b18d1af247051e0ac6b53b/report/index.html

eldenmoon · 2023-09-19T06:21:22Z

run buildall

eldenmoon · 2023-09-19T06:21:32Z

run buildall

eldenmoon · 2023-09-19T06:29:01Z

run buildall

doris-robot · 2023-09-19T06:38:39Z

TeamCity be ut coverage result:
Function Coverage: 36.58% (7990/21845)
Line Coverage: 28.64% (64084/223738)
Region Coverage: 27.52% (33286/120931)
Branch Coverage: 24.17% (17062/70594)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ddd048c7006e1e3a5305b12a1b5a251a952dca6c_ddd048c7006e1e3a5305b12a1b5a251a952dca6c/report/index.html

doris-robot · 2023-09-19T06:39:43Z

TeamCity be ut coverage result:
Function Coverage: 36.58% (7990/21845)
Line Coverage: 28.64% (64077/223738)
Region Coverage: 27.53% (33290/120931)
Branch Coverage: 24.17% (17061/70594)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ddd048c7006e1e3a5305b12a1b5a251a952dca6c_ddd048c7006e1e3a5305b12a1b5a251a952dca6c/report/index.html

doris-robot · 2023-09-19T07:05:28Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.39 seconds
stream load tsv: 600 seconds loaded 74807831229 Bytes, about 118 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162367314 Bytes

eldenmoon · 2023-09-20T02:10:19Z

run buildall

doris-robot · 2023-09-20T02:39:13Z

TeamCity be ut coverage result:
Function Coverage: 36.43% (7990/21933)
Line Coverage: 28.52% (64080/224697)
Region Coverage: 27.42% (33296/121434)
Branch Coverage: 24.08% (17062/70850)
Coverage Report: http://coverage.selectdb-in.cc/coverage/812d71c1980ddfe12aed2b2820cea13a78f613ac_812d71c1980ddfe12aed2b2820cea13a78f613ac/report/index.html

doris-robot · 2023-09-20T03:07:45Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.26 seconds
stream load tsv: 612 seconds loaded 74807831229 Bytes, about 116 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162433259 Bytes

xiaokang

olap/rowset reviewed

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 25 out of 65. Check the log or trigger a new build to see more.

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 25 out of 44. Check the log or trigger a new build to see more.

github-actions · 2023-10-07T04:36:27Z

+ subcolumns.get_root()->data.get_least_common_type()->get_name(), path.get_path()); \
+ }
+
+Status ColumnObject::extract_root(const PathInData& path) {


warning: method 'extract_root' can be made const [readability-make-member-function-const]
be/src/vec/columns/column_object.h:457:
- Status extract_root(const PathInData& path);+ Status extract_root(const PathInData& path) const;
Suggested change
Status ColumnObject::extract_root(const PathInData& path) {
Status ColumnObject::extract_root(const PathInData& path) const{

github-actions · 2023-10-07T04:36:29Z

 return TPrimitiveType::INVALID_TYPE;
 }

+ doris::FieldType get_type_as_field_type() const override {


warning: function 'get_type_as_field_type' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
doris::FieldTypeget_type_as_field_type() constoverride {
[[nodiscard]] doris::FieldTypeget_type_as_field_type() constoverride {

github-actions · 2023-10-07T04:36:29Z

 return TPrimitiveType::HLL;
 }

+ doris::FieldType get_type_as_field_type() const override {


warning: function 'get_type_as_field_type' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
doris::FieldTypeget_type_as_field_type() constoverride {
[[nodiscard]] doris::FieldTypeget_type_as_field_type() constoverride {

github-actions · 2023-10-07T04:36:29Z

 return TPrimitiveType::JSONB;
 }

+ doris::FieldType get_type_as_field_type() const override {


warning: function 'get_type_as_field_type' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
doris::FieldTypeget_type_as_field_type() constoverride {
[[nodiscard]] doris::FieldTypeget_type_as_field_type() constoverride {

github-actions · 2023-10-07T04:36:29Z

 __builtin_unreachable();
 }
+
+ doris::FieldType get_type_as_field_type() const override {


warning: function 'get_type_as_field_type' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
doris::FieldTypeget_type_as_field_type() constoverride {
[[nodiscard]] doris::FieldTypeget_type_as_field_type() constoverride {

github-actions · 2023-10-07T04:36:29Z

 // and modified by Doris

 #pragma once
+#include <gen_cpp/Exprs_types.h>


warning: 'gen_cpp/Exprs_types.h' file not found [clang-diagnostic-error]
#include<gen_cpp/Exprs_types.h> ^

eldenmoon · 2023-10-07T05:46:56Z

run buildall

doris-robot · 2023-10-07T06:48:13Z

TeamCity be ut coverage result:
Function Coverage: 36.05% (8161/22641)
Line Coverage: 28.22% (65240/231182)
Region Coverage: 27.15% (33822/124562)
Branch Coverage: 23.84% (17250/72360)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b8ed00c82e9f407035f0b580ae79062642a5dabb_b8ed00c82e9f407035f0b580ae79062642a5dabb/report/index.html

doris-robot · 2023-10-07T07:16:34Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.49 seconds
stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162544698 Bytes

github-actions

clang-tidy made some suggestions

github-actions · 2023-10-07T12:43:06Z

 TPrimitiveType::type get_type_as_tprimitive_type() const override {
 return TPrimitiveType::VARIANT;
 }
+ doris::FieldType get_type_as_field_type() const override {


warning: function 'get_type_as_field_type' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
doris::FieldTypeget_type_as_field_type() constoverride {
[[nodiscard]] doris::FieldTypeget_type_as_field_type() constoverride {

github-actions · 2023-10-07T12:43:07Z

 bool get_is_parametric() const override { return true; }
- bool can_be_inside_nullable() const override { return true; }
 bool have_subtypes() const override { return true; };
+ bool can_be_inside_nullable() const override { return false; }


warning: function 'can_be_inside_nullable' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
boolcan_be_inside_nullable() constoverride { return false; }
[[nodiscard]] boolcan_be_inside_nullable() constoverride { return false; }

github-actions · 2023-10-07T12:43:07Z

- const auto& column_object = assert_cast<const ColumnObject&>(column);
- return "Variant: " + column_object.get_keys_str();
- }
+ std::string to_string(const IColumn& column, size_t row_num) const override;


warning: function 'to_string' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
std::stringto_string(constIColumn&column, size_trow_num) constoverride;
[[nodiscard]] std::stringto_string(constIColumn&column, size_trow_num) constoverride;

github-actions · 2023-10-07T12:43:07Z

- [[noreturn]] Field get_default() const override {
- LOG(FATAL) << "Method getDefault() is not implemented for data type " << get_name();
- }
+ Field get_default() const override { return VariantMap(); }


warning: function 'get_default' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
Fieldget_default() constoverride { returnVariantMap(); }
[[nodiscard]] Fieldget_default() constoverride { returnVariantMap(); }

github-actions · 2023-10-07T12:43:07Z


- [[noreturn]] Field get_field(const TExprNode& node) const override {
- LOG(FATAL) << "Unimplemented get_field for object";
+ Field get_field(const TExprNode& node) const override {


warning: function 'get_field' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
Fieldget_field(constTExprNode&node) constoverride {
[[nodiscard]] Fieldget_field(constTExprNode&node) constoverride {

github-actions · 2023-10-07T12:43:08Z

+ };
+
+ // create cresponding variant value to wrap from_type
+ WrapperType create_variant_wrapper(const DataTypePtr& from_type,


warning: function 'create_variant_wrapper' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
WrapperTypecreate_variant_wrapper(constDataTypePtr&from_type,
[[nodiscard]] WrapperTypecreate_variant_wrapper(constDataTypePtr&from_type,

github-actions · 2023-10-07T12:43:08Z

+ }
+
+ // create cresponding type convert from variant
+ WrapperType create_variant_wrapper(const DataTypeObject& from_type,


warning: function 'create_variant_wrapper' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
WrapperTypecreate_variant_wrapper(constDataTypeObject&from_type,
[[nodiscard]] WrapperTypecreate_variant_wrapper(constDataTypeObject&from_type,

github-actions · 2023-10-07T12:43:08Z


+ class OlapColumnDataConvertorVariant : public OlapColumnDataConvertorBase {
+ public:
+ OlapColumnDataConvertorVariant()


warning: use '= default' to define a trivial default constructor [modernize-use-equals-default]
be/src/vec/olap/olap_data_convertor.h:488:
- : _root_data_convertor(std::make_unique<OlapColumnDataConvertorVarChar>(true)) {}+ : _root_data_convertor(std::make_unique<OlapColumnDataConvertorVarChar>(true)) = default;

github-actions · 2023-10-07T12:43:09Z

+ size_t num_rows) override;
+ Status convert_to_olap() override;
+
+ const void* get_data() const override;


warning: function 'get_data' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
constvoid*get_data() constoverride;
[[nodiscard]] constvoid*get_data() constoverride;

github-actions · 2023-10-07T12:43:09Z

+ Status convert_to_olap() override;
+
+ const void* get_data() const override;
+ const void* get_data_at(size_t offset) const override;


warning: function 'get_data_at' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
constvoid*get_data_at(size_toffset) constoverride;
[[nodiscard]] constvoid*get_data_at(size_toffset) constoverride;

eldenmoon · 2023-10-23T11:02:36Z

run buildall

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 25 out of 46. Check the log or trigger a new build to see more.

github-actions · 2023-10-23T11:20:04Z

+ return std::string(buffer.GetString());
+}
+
+void JsonFunctions::merge_objects(rapidjson::Value& dst_object, rapidjson::Value& src_object,


warning: method 'merge_objects' can be made static [readability-convert-member-functions-to-static]
Suggested change
voidJsonFunctions::merge_objects(rapidjson::Value& dst_object, rapidjson::Value& src_object,
staticvoidJsonFunctions::merge_objects(rapidjson::Value& dst_object, rapidjson::Value& src_object,

github-actions · 2023-10-23T11:20:05Z

 }
 }

+void BaseTablet::update_by_least_common_schema(const TabletSchemaSPtr& update_schema) {


warning: method 'update_by_least_common_schema' can be made static [readability-convert-member-functions-to-static]
be/src/olap/base_tablet.h:67:
- void update_by_least_common_schema(const TabletSchemaSPtr& update_schema);+ static void update_by_least_common_schema(const TabletSchemaSPtr& update_schema);

github-actions · 2023-10-23T11:20:05Z

+// Eg. rowset schema: A(int), B(float), C(int), D(int)
+// _tabelt->tablet_schema: A(bigint), B(double)
+// => update_schema: A(bigint), B(double), C(int), D(int)
+void BetaRowsetWriter::update_rowset_schema(TabletSchemaSPtr flush_schema) {


warning: method 'update_rowset_schema' can be made static [readability-convert-member-functions-to-static]
be/src/olap/rowset/beta_rowset_writer.h:174:
- void update_rowset_schema(TabletSchemaSPtr flush_schema);+ static void update_rowset_schema(TabletSchemaSPtr flush_schema);

github-actions · 2023-10-23T11:20:05Z


 int64_t num_rows() const override { return _total_key_group_rows; }

+ virtual const RowsetWriterContext& context() const override { LOG(FATAL) << "Not implemented"; }


warning: 'virtual' is redundant since the function is already declared 'override' [modernize-use-override]
Suggested change
virtualconstRowsetWriterContext&context() constoverride { LOG(FATAL) << "Not implemented"; }
constRowsetWriterContext&context() constoverride { LOG(FATAL) << "Not implemented"; }

github-actions · 2023-10-23T11:20:05Z

 }
 }

+double ColumnArray::get_ratio_of_default_rows(double sample_ratio) const {


warning: method 'get_ratio_of_default_rows' can be made static [readability-convert-member-functions-to-static]
Suggested change
doubleColumnArray::get_ratio_of_default_rows(double sample_ratio) const{
doubleColumnArray::get_ratio_of_default_rows(double sample_ratio) {
be/src/vec/columns/column_array.h:267:
- double get_ratio_of_default_rows(double sample_ratio) const override;+ static double get_ratio_of_default_rows(double sample_ratio) override;

github-actions · 2023-10-23T11:20:07Z

 return get_ptr();
 }

+ double get_ratio_of_default_rows(double sample_ratio) const override {


warning: method 'get_ratio_of_default_rows' can be made static [readability-convert-member-functions-to-static]
Suggested change
doubleget_ratio_of_default_rows(doublesample_ratio)constoverride {
staticdoubleget_ratio_of_default_rows(doublesample_ratio) override {

github-actions · 2023-10-23T11:20:07Z

+ [&](const auto& subcolumn) { return subcolumn.index(indexes, limit); });
+}
+
+bool ColumnObject::Subcolumn::check_if_sparse_column(size_t num_rows) {


warning: method 'check_if_sparse_column' can be made static [readability-convert-member-functions-to-static]
be/src/vec/columns/column_object.h:143:
- bool check_if_sparse_column(size_t num_rows);+ static bool check_if_sparse_column(size_t num_rows);

github-actions · 2023-10-23T11:20:08Z

 }

-void ColumnObject::try_insert_range_from(const IColumn& src, size_t start, size_t length) {
+FieldInfo ColumnObject::Subcolumn::get_subcolumn_field_info() const {


warning: method 'get_subcolumn_field_info' can be made static [readability-convert-member-functions-to-static]
be/src/vec/columns/column_object.h:148:
- FieldInfo get_subcolumn_field_info() const;+ static FieldInfo get_subcolumn_field_info() ;
Suggested change
FieldInfo ColumnObject::Subcolumn::get_subcolumn_field_info() const{
FieldInfo ColumnObject::Subcolumn::get_subcolumn_field_info() {

github-actions · 2023-10-23T11:20:08Z

+ return std::regex_match(path.get_path(), COLUMN_NAME_REGEX);
+}
+
+void ColumnObject::Subcolumn::wrapp_array_nullable() {


warning: method 'wrapp_array_nullable' can be made static [readability-convert-member-functions-to-static]
be/src/vec/columns/column_object.h:190:
- void wrapp_array_nullable();+ static void wrapp_array_nullable();

github-actions · 2023-10-23T11:20:08Z

+ finalize(true);
+}
+
+void ColumnObject::ensure_root_node_type(const DataTypePtr& expected_root_type) {


warning: method 'ensure_root_node_type' can be made static [readability-convert-member-functions-to-static]
be/src/vec/columns/column_object.h:257:
- void ensure_root_node_type(const DataTypePtr& type);+ static void ensure_root_node_type(const DataTypePtr& type);

doris-robot · 2023-10-23T12:07:31Z

TeamCity be ut coverage result:
Function Coverage: 36.74% (8305/22607)
Line Coverage: 28.94% (66656/230322)
Region Coverage: 27.52% (34651/125893)
Branch Coverage: 24.20% (17601/72722)
Coverage Report: http://coverage.selectdb-in.cc/coverage/94b38ae9a01034d1c4ca6691cedac92f3472e016_94b38ae9a01034d1c4ca6691cedac92f3472e016/report/index.html

doris-robot · 2023-10-23T13:04:57Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.67 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162797382 Bytes

github-actions · 2023-11-07T03:37:20Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-11-07T03:37:23Z

PR approved by anyone and no changes requested.

xiaokang

LGTM

qidaye

LGTM

eldenmoon · 2023-11-07T10:18:09Z

run buildall

github-actions

clang-tidy made some suggestions

github-actions · 2023-11-07T10:31:38Z

+ return true;
+}
+
+void ColumnObject::merge_sparse_to_root_column() {


warning: method 'merge_sparse_to_root_column' can be made const [readability-make-member-function-const]
be/src/vec/columns/column_object.h:252:
- void merge_sparse_to_root_column();+ void merge_sparse_to_root_column() const;
Suggested change
voidColumnObject::merge_sparse_to_root_column() {
voidColumnObject::merge_sparse_to_root_column() const{

github-actions · 2023-11-07T10:31:39Z

+ subcolumns.get_mutable_root()->data.get_finalized_column_ptr() = mresult->get_ptr();
+}
+
+void ColumnObject::finalize(bool ignore_sparse) {


warning: method 'finalize' can be made const [readability-make-member-function-const]
be/src/vec/columns/column_object.h:326:
- void finalize(bool ignore_sparse);+ void finalize(bool ignore_sparse) const;
Suggested change
voidColumnObject::finalize(bool ignore_sparse) {
voidColumnObject::finalize(bool ignore_sparse) const{

github-actions · 2023-11-07T10:31:39Z

+ finalize(true);
+}
+
+void ColumnObject::ensure_root_node_type(const DataTypePtr& expected_root_type) {


warning: method 'ensure_root_node_type' can be made static [readability-convert-member-functions-to-static]
be/src/vec/columns/column_object.h:255:
- void ensure_root_node_type(const DataTypePtr& type);+ static void ensure_root_node_type(const DataTypePtr& type);

github-actions · 2023-11-07T10:31:39Z

+ add_sub_column({}, std::move(column), type);
+}
+
+bool ColumnObject::is_null_root() const {


warning: method 'is_null_root' can be made static [readability-convert-member-functions-to-static]
be/src/vec/columns/column_object.h:264:
- bool is_null_root() const;+ static bool is_null_root() ;
Suggested change
boolColumnObject::is_null_root() const{
boolColumnObject::is_null_root() {

doris-robot · 2023-11-07T11:26:56Z

TeamCity be ut coverage result:
Function Coverage: 36.68% (8379/22845)
Line Coverage: 29.17% (67927/232845)
Region Coverage: 27.85% (35161/126269)
Branch Coverage: 24.67% (17979/72892)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f32a5b7164427ff4a1e1db1b54e1b1c96a817ffe_f32a5b7164427ff4a1e1db1b54e1b1c96a817ffe/report/index.html

doris-robot · 2023-11-07T14:44:02Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.73 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162380854 Bytes

yiguolei

LGTM

github-actions · 2023-11-08T02:15:58Z

PR approved by at least one committer and no changes requested.

xiaokang

LGTM

eldenmoon · 2023-11-08T02:37:55Z

run buildall

doris-robot · 2023-11-08T03:48:16Z

TeamCity be ut coverage result:
Function Coverage: 36.70% (8380/22832)
Line Coverage: 29.19% (67921/232721)
Region Coverage: 27.86% (35151/126187)
Branch Coverage: 24.68% (17976/72844)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c52d069029b2906ffbb10263e6c749692d2599ca_c52d069029b2906ffbb10263e6c749692d2599ca/report/index.html

doris-robot · 2023-11-08T04:12:45Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.4 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17162308898 Bytes

eldenmoon · 2023-11-08T06:14:50Z

For some reason this PR can not pass github action.I will push another PR

github-actionsBot reviewed Sep 18, 2023
View reviewed changes

Comment threadbe/src/udf/udf.h
Comment threadbe/src/udf/udf.h

eldenmoon force-pushed the var-load branch from cff9e79 to a4e766dCompare September 19, 2023 06:20

eldenmoon added the variant label Sep 19, 2023

xiaokang reviewed Sep 26, 2023
View reviewed changes

eldenmoon requested a review from xiaokang September 27, 2023 06:49

eldenmoon force-pushed the var-load branch from 812d71c to 950ea5cCompare October 7, 2023 04:19

github-actionsBot reviewed Oct 7, 2023
View reviewed changes

eldenmoon force-pushed the var-load branch from 1b19068 to 94b38aeCompare October 23, 2023 11:01

github-actionsBot reviewed Oct 23, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 7, 2023

github-actionsBot added the reviewed label Nov 7, 2023

xiaokang previously approved these changes Nov 7, 2023
View reviewed changes

eldenmoon removed the request for review from yiguolei November 7, 2023 03:58

qidaye previously approved these changes Nov 7, 2023
View reviewed changes

yiguolei reviewed Nov 7, 2023
View reviewed changes

Comment threadbe/src/vec/columns/column_string.h

remove can_be_inside_nullable
f32a5b7

eldenmoon dismissed stale reviews from qidaye, xiaokang, and dataroaring via f32a5b7November 7, 2023 10:17

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Nov 7, 2023

github-actionsBot reviewed Nov 7, 2023
View reviewed changes

yiguolei approved these changes Nov 8, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 8, 2023

xiaokang approved these changes Nov 8, 2023
View reviewed changes

Merge branch 'master' into var-load
c52d069

zhangstar333 approved these changes Nov 8, 2023
View reviewed changes

xiaokang mentioned this pull request Nov 8, 2023
[Feature](Variant) support variant load #26572
Merged

hello-stephen mentioned this pull request Nov 8, 2023
[fix](action) Update pr-approve-status.yml #26577
Merged

eldenmoon closed this Nov 9, 2023

	Status ColumnObject::extract_root(const PathInData& path) {
	Status ColumnObject::extract_root(const PathInData& path) const{

	doris::FieldTypeget_type_as_field_type() constoverride {
	[[nodiscard]] doris::FieldTypeget_type_as_field_type() constoverride {

	boolcan_be_inside_nullable() constoverride { return false; }
	[[nodiscard]] boolcan_be_inside_nullable() constoverride { return false; }

	std::stringto_string(constIColumn&column, size_trow_num) constoverride;
	[[nodiscard]] std::stringto_string(constIColumn&column, size_trow_num) constoverride;

	Fieldget_default() constoverride { returnVariantMap(); }
	[[nodiscard]] Fieldget_default() constoverride { returnVariantMap(); }

	Fieldget_field(constTExprNode&node) constoverride {
	[[nodiscard]] Fieldget_field(constTExprNode&node) constoverride {

	WrapperTypecreate_variant_wrapper(constDataTypePtr&from_type,
	[[nodiscard]] WrapperTypecreate_variant_wrapper(constDataTypePtr&from_type,

	constvoid*get_data() constoverride;
	[[nodiscard]] constvoid*get_data() constoverride;

	constvoid*get_data_at(size_toffset) constoverride;
	[[nodiscard]] constvoid*get_data_at(size_toffset) constoverride;

	voidJsonFunctions::merge_objects(rapidjson::Value& dst_object, rapidjson::Value& src_object,
	staticvoidJsonFunctions::merge_objects(rapidjson::Value& dst_object, rapidjson::Value& src_object,


		int64_t num_rows() const override { return _total_key_group_rows; }

		virtual const RowsetWriterContext& context() const override { LOG(FATAL) << "Not implemented"; }

	virtualconstRowsetWriterContext&context() constoverride { LOG(FATAL) << "Not implemented"; }
	constRowsetWriterContext&context() constoverride { LOG(FATAL) << "Not implemented"; }

	doubleColumnArray::get_ratio_of_default_rows(double sample_ratio) const{
	doubleColumnArray::get_ratio_of_default_rows(double sample_ratio) {

	doubleget_ratio_of_default_rows(doublesample_ratio)constoverride {
	staticdoubleget_ratio_of_default_rows(doublesample_ratio) override {

	FieldInfo ColumnObject::Subcolumn::get_subcolumn_field_info() const{
	FieldInfo ColumnObject::Subcolumn::get_subcolumn_field_info() {

	voidColumnObject::merge_sparse_to_root_column() {
	voidColumnObject::merge_sparse_to_root_column() const{

	voidColumnObject::finalize(bool ignore_sparse) {
	voidColumnObject::finalize(bool ignore_sparse) const{

	boolColumnObject::is_null_root() const{
	boolColumnObject::is_null_root() {

Uh oh!

Conversation

eldenmoon commented Sep 18, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

eldenmoon commented Sep 18, 2023

Uh oh!

doris-robot commented Sep 18, 2023

Uh oh!

eldenmoon commented Sep 19, 2023

Uh oh!

doris-robot commented Sep 19, 2023

Uh oh!

eldenmoon commented Sep 19, 2023

Uh oh!

eldenmoon commented Sep 19, 2023

Uh oh!

eldenmoon commented Sep 19, 2023

Uh oh!

doris-robot commented Sep 19, 2023

Uh oh!

doris-robot commented Sep 19, 2023

Uh oh!

doris-robot commented Sep 19, 2023

Uh oh!

eldenmoon commented Sep 20, 2023

Uh oh!

doris-robot commented Sep 20, 2023

Uh oh!

doris-robot commented Sep 20, 2023

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actionsBotOct 7, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actionsBotOct 7, 2023

Choose a reason for hiding this comment

Uh oh!

github-actionsBotOct 7, 2023

Choose a reason for hiding this comment

Uh oh!

eldenmoon commented Sep 18, 2023•
edited
Loading