[test](ms) Add injection point to randomly fail in `txn->commit()` for test by bobhan1 · Pull Request #47336 · apache/doris

bobhan1 · 2025-01-23T03:15:43Z

What problem does this PR solve?

Add random fault injection in txn->commit()

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-01-23T03:15:48Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

bobhan1 · 2025-02-05T06:34:34Z

run buildall

bobhan1 · 2025-02-11T10:54:41Z

run buildall

doris-robot · 2025-02-11T10:59:51Z

TeamCity cloud ut coverage result:
Function Coverage: 82.25% (1061/1290)
Line Coverage: 65.77% (17554/26690)
Region Coverage: 65.31% (8659/13259)
Branch Coverage: 55.22% (4668/8454)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4da8f4f9392a6023f97232f169cdd91fe5900250_4da8f4f9392a6023f97232f169cdd91fe5900250_cloud/report/index.html

bobhan1 · 2025-02-11T12:20:19Z

run cloud_p0

bobhan1 · 2025-02-11T12:20:44Z

run vault_p0

zhannngchen

LGTM

dataroaring

LGTM

bobhan1 · 2025-02-12T11:49:10Z

run buildall

doris-robot · 2025-02-12T11:54:23Z

TeamCity cloud ut coverage result:
Function Coverage: 82.25% (1061/1290)
Line Coverage: 65.83% (17584/26712)
Region Coverage: 65.39% (8669/13257)
Branch Coverage: 55.29% (4673/8452)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0bc4a43b986df00dc21af16002317145015df872_0bc4a43b986df00dc21af16002317145015df872_cloud/report/index.html

zhannngchen

LGTM

gavinchou · 2025-02-12T06:44:22Z

+ bool enable_inject_random_fault {false};
+ TEST_INJECTION_POINT_CALLBACK("Transaction::commit.inject_random_fault",
+ &enable_inject_random_fault);
+ if (enable_inject_random_fault) [[unlikely]] {
+ std::mt19937 gen {std::random_device {}()};
+ double p {-1.0};
+ TEST_INJECTION_POINT_CALLBACK("Transaction::commit.inject_random_fault.set_p", &p);
+ if (p < 0 || p > 1.0) {
+ p = 0.01; // default injection possibility is 1%
+ }
+ std::bernoulli_distribution inject_fault {p};
+ if (inject_fault(gen)) {
+ std::bernoulli_distribution err_type {0.5};
+ TxnErrorCode inject_err =
+ (err_type(gen) ? TxnErrorCode::TXN_CONFLICT : TxnErrorCode::TXN_TOO_OLD);
+ LOG_WARNING("inject {} err when txn->commit()", inject_err);
+ return inject_err;
+ }
+ }
+


Suggested change
bool enable_inject_random_fault {false};
TEST_INJECTION_POINT_CALLBACK("Transaction::commit.inject_random_fault",
&enable_inject_random_fault);
if (enable_inject_random_fault) [[unlikely]] {
std::mt19937 gen {std::random_device {}()};
double p {-1.0};
TEST_INJECTION_POINT_CALLBACK("Transaction::commit.inject_random_fault.set_p", &p);
if (p < 0 || p > 1.0) {
p = 0.01; // default injection possibility is 1%
}
std::bernoulli_distribution inject_fault {p};
if (inject_fault(gen)) {
std::bernoulli_distribution err_type {0.5};
TxnErrorCode inject_err =
(err_type(gen) ? TxnErrorCode::TXN_CONFLICT : TxnErrorCode::TXN_TOO_OLD);
LOG_WARNING("inject {} err when txn->commit()", inject_err);
return inject_err;
}
}
TEST_INJECTION_POINT_RETURN_WITH_VALUE()

gavinchou · 2025-02-12T12:19:31Z

run buildall

github-actions · 2025-02-12T12:19:51Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-02-12T12:19:53Z

PR approved by anyone and no changes requested.

doris-robot · 2025-02-12T12:33:52Z

TeamCity cloud ut coverage result:
Function Coverage: 82.25% (1061/1290)
Line Coverage: 65.80% (17577/26713)
Region Coverage: 65.35% (8663/13257)
Branch Coverage: 55.25% (4670/8452)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ef1f59c96897551ccc388f5f2f2af426584d8430_ef1f59c96897551ccc388f5f2f2af426584d8430_cloud/report/index.html

zhannngchen

LGTM

…r test (#47336) Add random fault injection in `txn->commit()`

…r test (apache#47336) Add random fault injection in `txn->commit()`

…commit()` for test #47336 (#47854) Cherry-picked from #47336 Co-authored-by: bobhan1 <baohan@selectdb.com>

…r test (apache#47336) Add random fault injection in `txn->commit()`

bobhan1 requested review from dataroaring, gavinchou and w41ter as code owners January 23, 2025 03:15

bobhan1 force-pushed the txn-commit-inject-failure branch from 4a450e5 to 8e47f36Compare January 23, 2025 03:56

gavinchou requested changes Jan 23, 2025
View reviewed changes

Comment threadcloud/src/meta-service/txn_kv.cpp Outdated
Comment threadcloud/src/common/config.h Outdated

bobhan1 force-pushed the txn-commit-inject-failure branch 2 times, most recently from 2200210 to d68b384Compare February 5, 2025 05:54

bobhan1 requested a review from gavinchou February 5, 2025 06:34

bobhan1 changed the title ~~[test](ms) Add a config to randomly fail in txn->commit() for test~~[test](ms) Add injection point to randomly fail in txn->commit() for testFeb 6, 2025

dataroaring added the dev/3.0.x label Feb 10, 2025

zhannngchen reviewed Feb 11, 2025
View reviewed changes

Comment threadregression-test/suites/fault_injection_p0/cloud/test_txn_commit_inject.groovy

bobhan1 force-pushed the txn-commit-inject-failure branch from 520964c to 4da8f4fCompare February 11, 2025 10:53

bobhan1 requested a review from zhannngchen February 11, 2025 10:54

zhannngchen previously approved these changes Feb 11, 2025
View reviewed changes

dataroaring previously approved these changes Feb 12, 2025
View reviewed changes

bobhan1 added 4 commits February 12, 2025 14:56

update
3b556cb

change config to injection point
caaf7af

add regression case
fdccd2b

update
87a267f

bobhan1 dismissed stale reviews from dataroaring and zhannngchen via a97986fFebruary 12, 2025 08:32

bobhan1 force-pushed the txn-commit-inject-failure branch 3 times, most recently from 78db136 to d8eee49Compare February 12, 2025 11:44

bobhan1 force-pushed the txn-commit-inject-failure branch from d8eee49 to 48da2acCompare February 12, 2025 11:45

update
0bc4a43

bobhan1 force-pushed the txn-commit-inject-failure branch from 48da2ac to 0bc4a43Compare February 12, 2025 11:46

zhannngchen previously approved these changes Feb 12, 2025
View reviewed changes

update
ef1f59c

bobhan1 dismissed zhannngchen’s stale review via ef1f59cFebruary 12, 2025 11:57

gavinchou approved these changes Feb 12, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 12, 2025

github-actionsBot added the reviewed label Feb 12, 2025

zhannngchen approved these changes Feb 13, 2025
View reviewed changes

zhannngchen merged commit 135d233 into apache:masterFeb 13, 2025

github-actionsBot pushed a commit that referenced this pull request Feb 13, 2025
[test](ms) Add injection point to randomly fail in txn->commit() fo…
52c5dac
…r test (#47336) Add random fault injection in `txn->commit()`

github-actionsBot mentioned this pull request Feb 13, 2025
branch-3.0: [test](ms) Add injection point to randomly fail in txn->commit() for test #47336 #47854
Merged

lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
[test](ms) Add injection point to randomly fail in txn->commit() fo…
f6a0fd2
…r test (apache#47336) Add random fault injection in `txn->commit()`

dataroaring pushed a commit that referenced this pull request Feb 24, 2025
branch-3.0: [test](ms) Add injection point to randomly fail in `txn->…
8a04ff9
…commit()` for test #47336 (#47854) Cherry-picked from #47336 Co-authored-by: bobhan1 <baohan@selectdb.com>

dataroaring added dev/3.0.5-merged and removed dev/3.0.x labels Feb 24, 2025

hust-hhb mentioned this pull request Apr 2, 2025
[improve](cloud-mow) MS can choose use new way or old way when processing delete bitmap lock #49223
Merged
16 tasks

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
[test](ms) Add injection point to randomly fail in txn->commit() fo…
3923b17
…r test (apache#47336) Add random fault injection in `txn->commit()`

Uh oh!

Conversation

bobhan1 commented Jan 23, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jan 23, 2025

Uh oh!

Uh oh!

Uh oh!

bobhan1 commented Feb 5, 2025

Uh oh!

Uh oh!

bobhan1 commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

bobhan1 commented Feb 11, 2025

Uh oh!

bobhan1 commented Feb 11, 2025

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

bobhan1 commented Feb 12, 2025

Uh oh!

doris-robot commented Feb 12, 2025

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

gavinchouFeb 12, 2025

Choose a reason for hiding this comment

Uh oh!

gavinchou commented Feb 12, 2025

Uh oh!

github-actionsBot commented Feb 12, 2025

Uh oh!

github-actionsBot commented Feb 12, 2025

Uh oh!

doris-robot commented Feb 12, 2025

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bobhan1 commented Jan 23, 2025•
edited
Loading