[bugfix](k8s) fix log output for K8S environment in cloud and add unit tests by deardeng · Pull Request #60490 · apache/doris

deardeng · 2026-02-04T07:28:02Z

K8S uses stdout to capture logs. When service crashes, it tries to output crash stack to .out file. Previously, it could not output to .out file correctly. This patch standardizes the log output behavior for both BE and Cloud components:

--daemon mode: output logs to .info file and crash stack to .out file
--console mode: 2.1 enable_file_logger = true: output all logs to console (stdout) and to .info file, crash stack to .out file 2.2 enable_file_logger = false: output all logs to console (stdout) and crash stack to .out file

Changes:

Implement StdoutLogSink for custom log format output to stdout in K8S
Modify console mode redirection in start scripts to separate stdout/stderr
Add shutdown_logging() function to properly clean up LogSink resources
Add shutdown_logging() calls in main() functions for graceful shutdown
Add comprehensive unit tests for both Cloud and BE components

This ensures crash stack is always saved to .out file for debugging, while normal logs can be captured by K8S logging infrastructure.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

…t tests K8S uses stdout to capture logs. When service crashes, it tries to output crash stack to .out file. Previously, it could not output to .out file correctly. This patch standardizes the log output behavior for both BE and Cloud components: 1. --daemon mode: output logs to .info file and crash stack to .out file 2. --console mode: 2.1 enable_file_logger = true: output all logs to console (stdout) and to .info file, crash stack to .out file 2.2 enable_file_logger = false: output all logs to console (stdout) and crash stack to .out file Changes: - Implement StdoutLogSink for custom log format output to stdout in K8S - Modify console mode redirection in start scripts to separate stdout/stderr - Add shutdown_logging() function to properly clean up LogSink resources - Add shutdown_logging() calls in main() functions for graceful shutdown - Add comprehensive unit tests for both Cloud and BE components This ensures crash stack is always saved to .out file for debugging, while normal logs can be captured by K8S logging infrastructure.

hello-stephen · 2026-02-04T07:28:08Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

deardeng · 2026-02-04T07:28:14Z

run buildall

deardeng · 2026-02-04T15:29:43Z

run buildall

doris-robot · 2026-02-04T16:10:13Z

TPC-H: Total hot run time: 31755 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 957cd28e8cf4983c02c41148f976481cb882fc6e, data reload: false
------ Round 1 ----------------------------------
q1	17632	5347	5073	5073
q2	2061	308	197	197
q3	10216	1299	732	732
q4	10213	890	331	331
q5	7562	2133	1927	1927
q6	202	185	155	155
q7	865	743	600	600
q8	9264	1385	1082	1082
q9	5162	4897	4913	4897
q10	6855	1942	1586	1586
q11	517	299	283	283
q12	330	378	234	234
q13	17767	4034	3205	3205
q14	236	234	223	223
q15	918	842	814	814
q16	669	666	613	613
q17	645	833	468	468
q18	6796	6355	6277	6277
q19	1226	982	617	617
q20	390	345	230	230
q21	2631	2084	1932	1932
q22	353	312	279	279
Total cold run time: 102510 ms
Total hot run time: 31755 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5336	5289	5299	5289
q2	255	339	250	250
q3	2162	2666	2270	2270
q4	1356	1728	1325	1325
q5	4286	4188	4285	4188
q6	261	188	139	139
q7	2248	2099	1901	1901
q8	2633	2551	2402	2402
q9	7489	7546	7542	7542
q10	2806	3019	2612	2612
q11	552	478	451	451
q12	668	824	651	651
q13	3881	4676	3394	3394
q14	293	339	284	284
q15	891	862	822	822
q16	704	707	689	689
q17	1142	1354	1336	1336
q18	8220	7883	7877	7877
q19	870	826	831	826
q20	2045	2153	2077	2077
q21	4879	4277	4028	4028
q22	578	565	500	500
Total cold run time: 53555 ms
Total hot run time: 50853 ms

doris-robot · 2026-02-04T16:14:18Z

Cloud UT Coverage Report

Increment line coverage 2.94% (1/34) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.26% (1792/2261)
Line Coverage	64.71% (31831/49189)
Region Coverage	65.42% (15888/24287)
Branch Coverage	55.93% (8438/15088)

doris-robot · 2026-02-04T16:26:48Z

ClickBench: Total hot run time: 28.62 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 957cd28e8cf4983c02c41148f976481cb882fc6e, data reload: false
query1	0.06	0.05	0.05
query2	0.09	0.04	0.04
query3	0.25	0.09	0.08
query4	1.60	0.11	0.11
query5	0.26	0.25	0.25
query6	1.17	0.70	0.67
query7	0.03	0.02	0.02
query8	0.05	0.03	0.04
query9	0.57	0.49	0.50
query10	0.54	0.55	0.54
query11	0.15	0.09	0.10
query12	0.14	0.10	0.10
query13	0.64	0.61	0.61
query14	1.08	1.07	1.05
query15	0.88	0.87	0.86
query16	0.40	0.41	0.40
query17	1.15	1.13	1.12
query18	0.23	0.23	0.22
query19	2.08	2.05	1.98
query20	0.02	0.02	0.01
query21	15.40	0.28	0.15
query22	5.09	0.06	0.05
query23	15.93	0.28	0.10
query24	1.48	0.54	0.70
query25	0.08	0.08	0.08
query26	0.14	0.14	0.14
query27	0.09	0.08	0.06
query28	4.79	1.15	0.97
query29	12.63	3.90	3.14
query30	0.27	0.13	0.12
query31	2.82	0.63	0.40
query32	3.23	0.60	0.49
query33	3.23	3.28	3.27
query34	16.23	5.45	4.73
query35	4.71	4.80	4.84
query36	0.66	0.50	0.48
query37	0.11	0.07	0.07
query38	0.07	0.05	0.04
query39	0.05	0.03	0.03
query40	0.20	0.15	0.14
query41	0.09	0.03	0.04
query42	0.05	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 98.78 s
Total hot run time: 28.62 s

doris-robot · 2026-02-04T16:53:24Z

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.56% (19369/36854)
Line Coverage	36.04% (179901/499196)
Region Coverage	32.40% (139444/430400)
Branch Coverage	33.41% (60401/180796)

hello-stephen · 2026-02-04T18:11:47Z

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.58% (25853/36118)
Line Coverage	54.21% (269971/498001)
Region Coverage	51.58% (224291/434816)
Branch Coverage	53.15% (96477/181526)

github-actions · 2026-02-10T10:28:01Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-02-10T10:28:04Z

PR approved by anyone and no changes requested.

…t tests (#60490) K8S uses stdout to capture logs. When service crashes, it tries to output crash stack to .out file. Previously, it could not output to .out file correctly. This patch standardizes the log output behavior for both BE and Cloud components: 1. --daemon mode: output logs to .info file and crash stack to .out file 2. --console mode: 2.1 enable_file_logger = true: output all logs to console (stdout) and to .info file, crash stack to .out file 2.2 enable_file_logger = false: output all logs to console (stdout) and crash stack to .out file Changes: - Implement StdoutLogSink for custom log format output to stdout in K8S - Modify console mode redirection in start scripts to separate stdout/stderr - Add shutdown_logging() function to properly clean up LogSink resources - Add shutdown_logging() calls in main() functions for graceful shutdown - Add comprehensive unit tests for both Cloud and BE components This ensures crash stack is always saved to .out file for debugging, while normal logs can be captured by K8S logging infrastructure.

… and add unit tests #60490 (#60678) Cherry-picked from #60490 Co-authored-by: deardeng <dengxin@selectdb.com>

… and add unit tests apache#60490 (apache#60678) Cherry-picked from apache#60490 Co-authored-by: deardeng <dengxin@selectdb.com>

due to #60490, ms not print ms log in k8s `start.sh --conf=/opt/apache/doris/ms/conf/doris_cloud.conf --console`

deardeng requested review from dataroaring, gavinchou and w41ter as code owners February 4, 2026 07:28

deardeng force-pushed the fix-cloud-log-console branch 3 times, most recently from 549c6fe to 49a3cf7Compare February 4, 2026 10:51

fix
957cd28

deardeng force-pushed the fix-cloud-log-console branch from 49a3cf7 to 957cd28Compare February 4, 2026 15:29

gavinchou approved these changes Feb 10, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 10, 2026

github-actionsBot added the reviewed label Feb 10, 2026

morningman approved these changes Feb 10, 2026
View reviewed changes

hello-stephen merged commit 5446faa into apache:masterFeb 11, 2026
29 of 30 checks passed

gavinchou added dev/3.1.x dev/4.0.x labels Feb 11, 2026

github-actionsBot mentioned this pull request Feb 11, 2026
branch-4.0: [bugfix](k8s) fix log output for K8S environment in cloud and add unit tests #60490 #60678
Merged

yiguolei pushed a commit that referenced this pull request Feb 13, 2026
branch-4.0: [bugfix](k8s) fix log output for K8S environment in cloud…
85a1ea9
… and add unit tests #60490 (#60678) Cherry-picked from #60490 Co-authored-by: deardeng <dengxin@selectdb.com>

yiguolei removed the dev/4.0.x label Feb 13, 2026

yiguolei added the dev/4.0.4-merged label Feb 13, 2026

deardeng mentioned this pull request Mar 26, 2026
[fix](cloud) Fix cloud not print ms log #61766
Merged
16 tasks

gavinchou pushed a commit that referenced this pull request Apr 1, 2026
[fix](cloud) Fix cloud not print ms log (#61766)
3cdc482
due to #60490, ms not print ms log in k8s `start.sh --conf=/opt/apache/doris/ms/conf/doris_cloud.conf --console`

github-actionsBot pushed a commit that referenced this pull request Apr 1, 2026
[fix](cloud) Fix cloud not print ms log (#61766)
76d0566
due to #60490, ms not print ms log in k8s `start.sh --conf=/opt/apache/doris/ms/conf/doris_cloud.conf --console`

github-actionsBot pushed a commit that referenced this pull request Apr 1, 2026
[fix](cloud) Fix cloud not print ms log (#61766)
5d1231e
due to #60490, ms not print ms log in k8s `start.sh --conf=/opt/apache/doris/ms/conf/doris_cloud.conf --console`

deardeng removed the dev/3.1.x label Apr 2, 2026

morningman pushed a commit that referenced this pull request Apr 2, 2026
[fix](cloud) Fix cloud not print ms log (#61766)
4b1e55c
due to #60490, ms not print ms log in k8s `start.sh --conf=/opt/apache/doris/ms/conf/doris_cloud.conf --console`

Navigation Menu

Uh oh!

[bugfix](k8s) fix log output for K8S environment in cloud and add unit tests - #60490

[bugfix](k8s) fix log output for K8S environment in cloud and add unit tests#60490
hello-stephen merged 2 commits into
apache:masterfrom
deardeng:fix-cloud-log-console

deardeng commented Feb 4, 2026

Uh oh!

hello-stephen commented Feb 4, 2026

Uh oh!

deardeng commented Feb 4, 2026

Uh oh!

deardeng commented Feb 4, 2026

Uh oh!

doris-robot commented Feb 4, 2026

Uh oh!

doris-robot commented Feb 4, 2026

Uh oh!

doris-robot commented Feb 4, 2026

Uh oh!

doris-robot commented Feb 4, 2026

Uh oh!

hello-stephen commented Feb 4, 2026

Uh oh!

github-actionsBot commented Feb 10, 2026

Uh oh!

github-actionsBot commented Feb 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

deardeng commented Feb 4, 2026

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Feb 4, 2026

Uh oh!

deardeng commented Feb 4, 2026

Uh oh!

deardeng commented Feb 4, 2026

Uh oh!

doris-robot commented Feb 4, 2026

Uh oh!

doris-robot commented Feb 4, 2026

Cloud UT Coverage Report

Uh oh!

doris-robot commented Feb 4, 2026

Uh oh!

doris-robot commented Feb 4, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Feb 4, 2026

BE Regression && UT Coverage Report

Uh oh!

github-actionsBot commented Feb 10, 2026

Uh oh!

github-actionsBot commented Feb 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants