Skip to content

Avoid raising debuggee not finished error if assertions failed - #923

Merged
ko1 merged 1 commit into
ruby:masterfrom
Shopify:st0012-improve-protocol-test-case
Mar 14, 2023
Merged

Avoid raising debuggee not finished error if assertions failed#923
ko1 merged 1 commit into
ruby:masterfrom
Shopify:st0012-improve-protocol-test-case

Conversation

@st0012

Copy link
Copy Markdown
Member

When assertions failed, we should let the AssertionFailedError surface and not calling flunk so we can get correct failure messages.

Assume we have this test that fails the assertion:

run_protocol_scenarioPROGRAMdoreq_add_breakpoint5req_continue# should be 3 instead of 5assert_repl_result({value: '5',type: 'Integer'},'1+2')req_terminate_debuggeeend

Before

The failure message is completely overridden by this flunk call and becomes misleading.

 -------------------
| Failure Message |
-------------------
Expected the debuggee program to finish.
/Users/hung-wulo/src/github.com/ruby/debug/test/support/assertions.rb:97:in `assert_block'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:340:in `ensure in execute_dap_scenario'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:344:in `execute_dap_scenario'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:46:in `block in run_protocol_scenario'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `block in catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:198:in `timeout'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:44:in `run_protocol_scenario'
test/protocol/eval_test.rb:17:in `test_eval_evaluates_arithmetic_expressions'
14: RUBY
15: 16: def test_eval_evaluates_arithmetic_expressions
=> 17: run_protocol_scenario PROGRAM do
18: req_add_breakpoint 5
19: req_continue
20: assert_repl_result({value: '5', type: 'Integer'}, '1+2') # should be 3
======================================================================================================================================

After

F
==============================================================================================================================================================
Failure: test_eval_evaluates_arithmetic_expressions(DEBUGGER__::EvalTest)
/Users/hung-wulo/src/github.com/ruby/debug/test/support/assertions.rb:97:in `assert_block'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:480:in `assert_eval_result'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:302:in `assert_repl_result'
test/protocol/eval_test.rb:22:in `block in test_eval_evaluates_arithmetic_expressions'
19: req_continue
20: assert_repl_result({value: '2', type: 'Integer'}, 'a')
21: assert_repl_result({value: '4', type: 'Integer'}, 'd')
=> 22: assert_repl_result({value: '5', type: 'Integer'}, '1+2')
23: req_terminate_debuggee
24: end
25: end
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:336:in `execute_dap_scenario'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:46:in `block in run_protocol_scenario'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `block in catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:198:in `timeout'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:44:in `run_protocol_scenario'
test/protocol/eval_test.rb:17:in `test_eval_evaluates_arithmetic_expressions'
-------------------------
| All Protocol Messages |
-------------------------
(protocol messages removed as they're not important here)
--------------------------
| Last Protocol Messages |
--------------------------
(protocol messages removed as they're not important here)
--------------------
| Debuggee Session |
--------------------
> DEBUGGER: Debugger can attach via UNIX domain socket (/var/folders/lf/4xqm_gk10ts83_mxrhsz9qf80000gn/T/ruby-debug-sock-501/ruby-debug-hung-wulo-61875-1)
> DEBUGGER: wait for debugger connection...
> DEBUGGER: Connected.
-------------------
| Failure Message |
-------------------
result:
{
"type": "response",
"command": "evaluate",
"request_seq": 14,
"success": true,
"message": "Success",
"body": {
"result": "3",
"type": "Integer",
"variablesReference": 4,
"indexedVariables": 0,
"namedVariables": 1
},
"seq": 17
}
<"5"> expected but was
<"3">
==============================================================================================================================================================

@ko1
ko1 requested a review from ono-maxMarch 10, 2023 08:12

attach_to_dap_server
scenario.call
rescue Test::Unit::AssertionFailedError => e

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add this implementation to cdp, too?

@ono-max

ono-max commented Mar 12, 2023

Copy link
Copy Markdown
Member

I would be happy to write a test for this change in https://github.com/ruby/debug/blob/master/test/support/protocol_test_case_test.rb. I'll leave it up to you whether to write the test in a separate PR or the same PR.

@st0012
st0012force-pushed the st0012-improve-protocol-test-case branch from 2d91f5d to 0940cccCompareMarch 13, 2023 09:17
When assertions failed, we should let the AssertionFailedError surface
and not calling flunk so we can get correct failure messages.
@st0012
st0012force-pushed the st0012-improve-protocol-test-case branch from 0940ccc to 2154375CompareMarch 13, 2023 09:22
@st0012

Copy link
Copy Markdown
MemberAuthor

@ono-max Both updated 👍

@st0012
st0012 requested a review from ono-maxMarch 13, 2023 09:29

@ono-maxono-max left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thank you.

@ko1
ko1 merged commit c258ee0 into ruby:masterMar 14, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@st0012@ono-max@ko1