[Enhancement] Support some spatial functions by koi2000 · Pull Request #48695 · apache/doris

koi2000 · 2025-03-05T10:02:22Z

What problem does this PR solve?

Issue Number: close#48203

Problem Summary:

Support for ST_Intersects, ST_Disjoint, ST_Touches sql functions.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes. [feat] suppor the spatial function docs, like ST_Intersects, ST_Disjo… doris-website#2157

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

…_Touches function and test (apache#48203)

…T_touches function (apache#48203)

Thearas · 2025-03-05T10:02:27Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

doris-robot · 2025-03-06T07:08:43Z

TeamCity cloud ut coverage result:
Function Coverage: 82.15% (1063/1294)
Line Coverage: 65.72% (17638/26840)
Region Coverage: 65.16% (8692/13340)
Branch Coverage: 55.17% (4695/8510)
Coverage Report: http://coverage.selectdb-in.cc/coverage/20e36804dccf1e258ac43ac8b94c069ab08141ad_20e36804dccf1e258ac43ac8b94c069ab08141ad_cloud/report/index.html

doris-robot · 2025-03-06T08:01:58Z

TPC-H: Total hot run time: 32854 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 20e36804dccf1e258ac43ac8b94c069ab08141ad, data reload: false
------ Round 1 ----------------------------------
q1	17586	5301	5117	5117
q2	2065	301	174	174
q3	10462	1234	754	754
q4	10222	1025	533	533
q5	7586	2399	2346	2346
q6	194	172	136	136
q7	902	772	627	627
q8	9325	1296	1106	1106
q9	4859	4816	4904	4816
q10	6844	2300	1911	1911
q11	508	286	258	258
q12	349	359	232	232
q13	17776	3669	3111	3111
q14	226	225	221	221
q15	534	486	491	486
q16	637	606	574	574
q17	575	857	346	346
q18	7014	6449	6382	6382
q19	1969	1097	609	609
q20	330	321	192	192
q21	2892	2227	1951	1951
q22	1031	1035	972	972
Total cold run time: 103886 ms
Total hot run time: 32854 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5324	5156	5219	5156
q2	242	324	232	232
q3	2167	2687	2326	2326
q4	1436	1817	1357	1357
q5	4265	4108	4162	4108
q6	215	169	124	124
q7	1996	1930	1822	1822
q8	2631	2648	2549	2549
q9	7307	7233	7210	7210
q10	3035	3235	2904	2904
q11	590	524	480	480
q12	652	726	575	575
q13	3461	3941	3291	3291
q14	271	284	268	268
q15	528	486	481	481
q16	632	689	651	651
q17	1160	1569	1373	1373
q18	7775	7645	7534	7534
q19	840	816	913	816
q20	1946	2025	1924	1924
q21	5436	4936	4651	4651
q22	1080	1051	1040	1040
Total cold run time: 52989 ms
Total hot run time: 50872 ms

doris-robot · 2025-03-06T08:14:13Z

TPC-DS: Total hot run time: 193344 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 20e36804dccf1e258ac43ac8b94c069ab08141ad, data reload: false
query1	1418	1060	1013	1013
query2	6203	1921	1890	1890
query3	11025	4534	4542	4534
query4	54329	24656	23202	23202
query5	5228	558	509	509
query6	413	213	196	196
query7	5089	514	295	295
query8	327	274	242	242
query9	6293	2585	2607	2585
query10	439	332	278	278
query11	15194	15090	15010	15010
query12	159	113	109	109
query13	1149	520	408	408
query14	10590	7437	6844	6844
query15	212	199	192	192
query16	7115	686	503	503
query17	1079	771	620	620
query18	1546	417	328	328
query19	210	207	190	190
query20	140	125	166	125
query21	213	120	115	115
query22	4415	4631	4410	4410
query23	34066	33393	33447	33393
query24	5786	2440	2491	2440
query25	448	475	410	410
query26	696	275	161	161
query27	1806	524	346	346
query28	2818	2499	2446	2446
query29	601	581	464	464
query30	271	212	193	193
query31	869	860	804	804
query32	75	61	62	61
query33	458	374	306	306
query34	770	902	530	530
query35	832	829	775	775
query36	965	1014	891	891
query37	126	104	79	79
query38	4316	4338	4259	4259
query39	1490	1517	1683	1517
query40	210	122	114	114
query41	56	54	53	53
query42	137	119	111	111
query43	514	541	496	496
query44	1367	819	825	819
query45	179	179	170	170
query46	873	1058	669	669
query47	1833	1875	1783	1783
query48	408	428	329	329
query49	701	512	459	459
query50	742	779	435	435
query51	4221	4380	4258	4258
query52	117	109	106	106
query53	241	269	202	202
query54	501	506	449	449
query55	88	88	88	88
query56	278	279	291	279
query57	1169	1165	1121	1121
query58	247	240	257	240
query59	2761	2836	2658	2658
query60	292	302	271	271
query61	156	121	123	121
query62	752	758	701	701
query63	247	209	199	199
query64	1845	1048	706	706
query65	4537	4392	4324	4324
query66	726	392	300	300
query67	15933	15548	15287	15287
query68	7062	892	521	521
query69	533	304	273	273
query70	1235	1133	1118	1118
query71	504	309	324	309
query72	5754	3615	3436	3436
query73	1356	743	345	345
query74	9234	9097	9112	9097
query75	3887	3173	2759	2759
query76	4196	1202	776	776
query77	591	386	291	291
query78	10104	10007	9290	9290
query79	2481	841	594	594
query80	642	529	456	456
query81	492	278	228	228
query82	689	126	94	94
query83	227	172	153	153
query84	286	93	76	76
query85	770	380	318	318
query86	390	311	288	288
query87	4559	4722	4467	4467
query88	3608	2171	2197	2171
query89	435	327	277	277
query90	1798	223	211	211
query91	144	145	111	111
query92	72	62	63	62
query93	1913	1053	575	575
query94	676	407	311	311
query95	346	269	262	262
query96	481	570	273	273
query97	3390	3385	3287	3287
query98	235	207	199	199
query99	1462	1405	1295	1295
Total cold run time: 299862 ms
Total hot run time: 193344 ms

doris-robot · 2025-03-06T08:19:31Z

ClickBench: Total hot run time: 30.75 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 20e36804dccf1e258ac43ac8b94c069ab08141ad, data reload: false
query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.07	0.06
query4	1.62	0.10	0.11
query5	0.56	0.54	0.57
query6	1.20	0.73	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.54	0.52
query10	0.58	0.58	0.57
query11	0.16	0.11	0.11
query12	0.16	0.12	0.12
query13	0.61	0.59	0.60
query14	2.68	2.82	2.81
query15	0.94	0.85	0.87
query16	0.39	0.38	0.38
query17	1.03	1.00	1.04
query18	0.21	0.20	0.20
query19	1.93	1.85	1.90
query20	0.02	0.01	0.02
query21	15.36	0.92	0.55
query22	0.75	1.13	0.64
query23	14.99	1.40	0.62
query24	6.89	1.52	0.58
query25	0.47	0.26	0.14
query26	0.59	0.16	0.14
query27	0.06	0.05	0.04
query28	9.54	0.93	0.44
query29	12.54	3.99	3.34
query30	0.25	0.09	0.06
query31	2.82	0.61	0.39
query32	3.24	0.55	0.48
query33	2.98	2.98	3.04
query34	15.88	5.12	4.50
query35	4.50	4.52	4.51
query36	0.67	0.49	0.48
query37	0.09	0.06	0.07
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.16	0.13	0.13
query41	0.09	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.12 s
Total hot run time: 30.75 s

hello-stephen · 2025-03-06T08:51:02Z

BE UT Coverage Report

Increment line coverage 41.30% (304/736) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	46.08% (12342/26781)
Line Coverage	35.61% (104555/293631)
Region Coverage	34.76% (53520/153965)
Branch Coverage	30.48% (27166/89140)

HappenLee · 2025-03-10T04:30:11Z

+ if (shapes[i] == nullptr) {
+ null_map[row] = 1;
+ res->insert_default();
+ break;


here better directly return, not need check if (i == 2)

Sorry, I didn't understand the purpose of checking if i is equal to 2. Checking if i is 2 is actually checking for the presence of a nullptr, I've fixed it now.

…ode duplication

zclllyybb · 2025-03-23T19:21:50Z

+ const S2Point& p = loop->vertex(j);
+ S2Point closest_point = line->polyline()->Project(p, &next);
+ S1Angle distance(closest_point, p);
+ if (distance.radians() < 1e-2) {


why use this as an error limit? It seems too great

btw, change it with the constexpr variable

zclllyybb · 2025-03-23T19:22:46Z

+bool GeoPolygon::intersects(const GeoShape* rhs) const {
+ switch (rhs->type()) {
+ case GEO_SHAPE_POINT: {
+ const GeoPoint* point = (const GeoPoint*)rhs;


why do C-style cast here? seems useless

I need to get to the point variable in the subclass to determine if it's Intersects or not.

sorry, I mistook the type of the rhs. but here you should use assert_cast

zclllyybb · 2025-03-23T19:23:28Z

+ implements BinaryExpression, ExplicitlyCastableSignature, AlwaysNullable, PropagateNullLiteral {
+
+ public static final List<FunctionSignature> SIGNATURES = ImmutableList.of(
+ FunctionSignature.ret(BooleanType.INSTANCE).args(VarcharType.SYSTEM_DEFAULT, VarcharType.SYSTEM_DEFAULT)


add StringType for args also

zclllyybb · 2025-03-23T19:31:32Z

+ std::unique_ptr<s2builderutil::S2PolygonLayer> layer(
+ new s2builderutil::S2PolygonLayer(&result));
+ S2BooleanOperation op(S2BooleanOperation::OpType::INTERSECTION, std::move(layer));
+ S2Error error;


deal with the potiential error

zclllyybb · 2025-03-23T19:37:35Z

- res->insert_data(const_cast<const char*>((char*)&contains_value), 0);
- }
+ auto relation_value = Func::evaluate(shapes[0].get(), shapes[1].get());
+ res->insert_data(const_cast<const char*>((char*)&relation_value), 0);


why do these cast? seems weird.
dont use insert_data interface. even if we allocated enough memory, it needs to do an unnecessary check for length. just get the Container(PODArray) from the column and change its data directly.

zclllyybb · 2025-03-23T19:43:42Z


 static void loop_do(StringRef& lhs_value, StringRef& rhs_value,
- std::vector<std::shared_ptr<GeoShape>>& shapes, int& i,
+ std::vector<std::shared_ptr<GeoShape>>& shapes,


shapes seems could be unique_ptr. could you confirm and fix it by the way?

…ive number scenarios (apache#48203)

zclllyybb · 2025-03-26T08:09:46Z

no need to modify this file anymore since it's not useful now

zclllyybb · 2025-03-26T16:23:12Z

+ return false;
+}
+
+bool IsSegmentsIntersect(const S2Point& point1, const S2Point& point2, const S2Point& line_point1,


all non-const variable or function should be named with underscore naming convention.

zclllyybb · 2025-03-26T16:25:17Z

all cases just add in nereids_p0 is enough. but make sure all type-pair's all relationship are covered in this file

The regression test now contains all relationships for all type pairs. Do I need to add more corner cases?

…re not precise enough and follows the underscore naming convention

…ession tests

…gression testing

doris-robot · 2025-04-04T07:45:27Z

BE UT Coverage Report

Increment line coverage 83.30% (454/545) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.26% (14022/26831)
Line Coverage	41.04% (121015/294889)
Region Coverage	39.78% (61601/154854)
Branch Coverage	34.45% (30839/89520)

HappenLee

LGTM

github-actions · 2025-04-07T03:01:50Z

PR approved by at least one committer and no changes requested.

Support for ST_Intersects, ST_Disjoint, ST_Touches sql functions.

…functions (#50073) pick: #37003, #48695 and #49665 --------- Co-authored-by: Mryange <59914473+Mryange@users.noreply.github.com> Co-authored-by: koi <koi20000@163.com>

…0061) pick: #48695 and #49665 --------- Co-authored-by: koi <koi20000@163.com>

Support for ST_Intersects, ST_Disjoint, ST_Touches sql functions.

…ype, ST_Length (#60170) Related PR: #48695 Problem Summary: Support for ST_Distance, ST_GeometryType, ST_Length sql functions.

koi2000 added 2 commits March 5, 2025 11:17

[feat][test](nereids)(vec) support the ST_Intersects, ST_Disjoint, ST…
78ad92b
…_Touches function and test (apache#48203)

[Fix] Fix linestring vs linestring and linestring vs polygon bug in S…
5579716
…T_touches function (apache#48203)

Merge branch 'master' into master
20e3680

HappenLee reviewed Mar 10, 2025
View reviewed changes

Comment threadbe/src/vec/functions/functions_geo.cpp

HappenLee reviewed Mar 10, 2025
View reviewed changes

Comment threadbe/src/vec/functions/functions_geo.cpp

koi2000 added 2 commits March 10, 2025 19:31

[feat] refactor geo relationship functions using template to remove c…
9f0cabd
…ode duplication

Merge branch 'master' ofhttps://github.com/koi2000/doris
a7cd68a

koi2000 requested a review from HappenLee March 10, 2025 13:21

HappenLee reviewed Mar 12, 2025
View reviewed changes

Comment threadbe/src/vec/functions/functions_geo.cpp Outdated

HappenLee reviewed Mar 12, 2025
View reviewed changes

Comment threadbe/src/geo/geo_types.cpp Outdated

[chore](geo) remove unused comment
1cbab4e

koi2000 requested a review from HappenLee March 13, 2025 06:40

zclllyybb reviewed Mar 23, 2025
View reviewed changes

[fix][test](geo) fix the bug of insufficient query precision in negat…
24a416f
…ive number scenarios (apache#48203)

koi2000 requested a review from zclllyybb March 26, 2025 03:16

zclllyybb reviewed Mar 26, 2025
View reviewed changes

koi2000 added 4 commits March 27, 2025 11:06

[chore](geo) using assert_cast to replace the C-style cast
701a7b1

[chore][fix](geo) fix the bug that intersects and touches in circle a…
25aee35
…re not precise enough and follows the underscore naming convention

[fix][test](geo) fix line bugs in intersects and touches and add regr…
92cdacf
…ession tests

[test](geo) add some new corner cases for intesects and touches in re…
e2d26ae
…gression testing

koi2000 requested a review from zclllyybb March 28, 2025 00:32

koi2000 requested a review from HappenLee April 4, 2025 08:18

HappenLee approved these changes Apr 7, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Apr 7, 2025

HappenLee merged commit 08da6a7 into apache:masterApr 7, 2025

HappenLee added dev/2.1.x dev/3.0.x labels Apr 15, 2025

github-actionsBot added the dev/2.1.x-conflict label Apr 15, 2025

github-actionsBot pushed a commit that referenced this pull request Apr 15, 2025
[Enhancement] Support some spatial functions (#48695)
069be95
Support for ST_Intersects, ST_Disjoint, ST_Touches sql functions.

github-actionsBot mentioned this pull request Apr 15, 2025
branch-3.0: [Enhancement] Support some spatial functions #48695 #50059
Closed

linrrzqqq pushed a commit to linrrzqqq/doris that referenced this pull request Apr 15, 2025
[Enhancement] Support some spatial functions (apache#48695)
1ddd011
Support for ST_Intersects, ST_Disjoint, ST_Touches sql functions.

linrrzqqq mentioned this pull request Apr 15, 2025
branch-3.0: [Enhancement] Support some spatial functions (#48695) #50061
Merged

linrrzqqq pushed a commit to linrrzqqq/doris that referenced this pull request Apr 15, 2025
[Enhancement] Support some spatial functions (apache#48695)
9d22aee
Support for ST_Intersects, ST_Disjoint, ST_Touches sql functions.

linrrzqqq mentioned this pull request Apr 15, 2025
branch-2.1: [Enhancement](GEO) Support Multipolygon and some spatial functions #50073
Merged

linrrzqqq pushed a commit to linrrzqqq/doris that referenced this pull request Apr 16, 2025
[Enhancement] Support some spatial functions (apache#48695)
f72a88e
Support for ST_Intersects, ST_Disjoint, ST_Touches sql functions.

yiguolei added usercase Important user case type label dev/2.1.10-merged and removed dev/2.1.x dev/2.1.x-conflict labels Apr 17, 2025

dataroaring pushed a commit that referenced this pull request Apr 22, 2025
branch-3.0: [Enhancement] Support some spatial functions (#48695) (#5…
e7d4b9f
…0061) pick: #48695 and #49665 --------- Co-authored-by: koi <koi20000@163.com>

dataroaring added dev/3.0.6-merged and removed dev/3.0.x labels Apr 22, 2025

yiguolei mentioned this pull request May 13, 2025
2.1.10 Release Note #50853
Closed

zclllyybb mentioned this pull request May 26, 2025
[feat] suppor the spatial function docs, like ST_Intersects, ST_Disjo… apache/doris-website#2157
Merged
8 tasks

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
[Enhancement] Support some spatial functions (apache#48695)
98b832f
Support for ST_Intersects, ST_Disjoint, ST_Touches sql functions.

zxc20041 mentioned this pull request Jan 23, 2026
[feature](geo) support 3 spatial functions: ST_Distance, ST_GeometryType, ST_Length #60170
Merged
16 tasks

zclllyybb pushed a commit that referenced this pull request Feb 13, 2026
[feature](geo) support 3 spatial functions: ST_Distance, ST_GeometryT…
d563436
…ype, ST_Length (#60170) Related PR: #48695 Problem Summary: Support for ST_Distance, ST_GeometryType, ST_Length sql functions.

github-actionsBot pushed a commit that referenced this pull request Feb 13, 2026
[feature](geo) support 3 spatial functions: ST_Distance, ST_GeometryT…
e54f717
…ype, ST_Length (#60170) Related PR: #48695 Problem Summary: Support for ST_Distance, ST_GeometryType, ST_Length sql functions.

Uh oh!

Conversation

koi2000 commented Mar 5, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Mar 5, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

hello-stephen commented Mar 6, 2025

BE UT Coverage Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

koi2000Apr 4, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

doris-robot commented Apr 4, 2025

BE UT Coverage Report

Uh oh!

HappenLee left a comment

koi2000Apr 4, 2025•
edited
Loading