CFE-3041: testall should fail a regular acceptance test if it prints errors

[sarakthon]

Ticket: CFE-3041

Running testall before changes (just a snippet from terminal):

./00_basics/01_compiler/string_contexts.cf Pass
./00_basics/01_compiler/string_contexts.x.cf Pass
./00_basics/01_compiler/trailing_comma_args FAIL (UNEXPECTED FAILURE)
2.cf FAIL (UNEXPECTED FAILURE)
./00_basics/01_compiler/trailing_comma_paramlist FAIL (UNEXPECTED FAILURE)
2.cf FAIL (UNEXPECTED FAILURE)
./00_basics/01_compiler/trailing_commas FAIL (UNEXPECTED FAILURE)
2.cf FAIL (UNEXPECTED FAILURE)
./00_basics/01_compiler/trailing_commas.cf Pass
./00_basics/01_compiler/with_iteration.cf Pass
./00_basics/02_switches/001.cf Pass
./00_basics/02_switches/002.cf Pass
./00_basics/02_switches/003.cf Pass
./00_basics/02_switches/004.cf Pass
./00_basics/02_switches/005.cf Pass
./00_basics/02_switches/006.cf Pass
./00_basics/02_switches/008.cf Pass
./00_basics/02_switches/010.cf Pass
./00_basics/02_switches/dry_run_perms_doesnt_lie.cf Pass
./00_basics/02_switches/show-evaluated-vars.cf Pass
./00_basics/02_switches/staging/007.cf Skipped (Staging tests are disabled)
./00_basics/02_switches/staging/009.cf Skipped (Staging tests are disabled)
./00_basics/02_switches/staging/negate_hard_class.cf Skipped (Staging tests are disabled)
./00_basics/03_bodies/001.cf Pass
./00_basics/03_bodies/002.cf Pass
./00_basics/03_bodies/003.cf Pass
./00_basics/03_bodies/101.cf Pass
./00_basics/03_bodies/102.cf Pass
./00_basics/03_bodies/103.cf Pass
./00_basics/03_bodies/104.cf Pass
./00_basics/03_bodies/109.cf Pass
./00_basics/03_bodies/112.cf Pass
./00_basics/03_bodies/113.cf Pass
./00_basics/03_bodies/114.cf Pass
./00_basics/03_bodies/115.cf Pass
./00_basics/03_bodies/116.cf Pass
./00_basics/03_bodies/117.cf Pass
./00_basics/03_bodies/118.cf Pass
./00_basics/03_bodies/119.cf Pass
./00_basics/03_bodies/120.cf Pass
./00_basics/03_bodies/121.cf Pass
...
...
Passed tests: 1632
Failed tests: 130
Skipped tests: 235
Soft failures: 75
Flakey failures: 0
Total tests: 2072

After changes:

./00_basics/01_compiler/string_contexts.cf FAIL (Test passed, but printed error message(s)) (UNEXPECTED FAILURE)
./00_basics/01_compiler/string_contexts.x.cf Pass
./00_basics/01_compiler/trailing_comma_args FAIL (UNEXPECTED FAILURE)
2.cf FAIL (UNEXPECTED FAILURE)
./00_basics/01_compiler/trailing_comma_paramlist FAIL (UNEXPECTED FAILURE)
2.cf FAIL (UNEXPECTED FAILURE)
./00_basics/01_compiler/trailing_commas FAIL (UNEXPECTED FAILURE)
2.cf FAIL (UNEXPECTED FAILURE)
./00_basics/01_compiler/trailing_commas.cf Pass
./00_basics/01_compiler/with_iteration.cf Pass
./00_basics/02_switches/001.cf Pass
./00_basics/02_switches/002.cf Pass
./00_basics/02_switches/003.cf Pass
./00_basics/02_switches/004.cf Pass
./00_basics/02_switches/005.cf Pass
./00_basics/02_switches/006.cf Pass
./00_basics/02_switches/008.cf Pass
./00_basics/02_switches/010.cf Pass
./00_basics/02_switches/dry_run_perms_doesnt_lie.cf Pass
./00_basics/02_switches/show-evaluated-vars.cf Pass
./00_basics/02_switches/staging/007.cf Skipped (Staging tests are disabled)
./00_basics/02_switches/staging/009.cf Skipped (Staging tests are disabled)
./00_basics/02_switches/staging/negate_hard_class.cf Skipped (Staging tests are disabled)
./00_basics/03_bodies/001.cf Pass
./00_basics/03_bodies/002.cf FAIL (Test passed, but printed error message(s)) (UNEXPECTED FAILURE)
./00_basics/03_bodies/003.cf Pass
./00_basics/03_bodies/101.cf Pass
./00_basics/03_bodies/102.cf Pass
./00_basics/03_bodies/103.cf Pass
./00_basics/03_bodies/104.cf Pass
./00_basics/03_bodies/109.cf Pass
./00_basics/03_bodies/112.cf FAIL (Test passed, but printed error message(s)) (UNEXPECTED FAILURE)
./00_basics/03_bodies/113.cf FAIL (Test passed, but printed error message(s)) (UNEXPECTED FAILURE)
./00_basics/03_bodies/114.cf Pass
./00_basics/03_bodies/115.cf Pass
./00_basics/03_bodies/116.cf Pass
./00_basics/03_bodies/117.cf Pass
./00_basics/03_bodies/118.cf FAIL (Test passed, but printed error message(s)) (UNEXPECTED FAILURE)
./00_basics/03_bodies/119.cf FAIL (Test passed, but printed error message(s)) (UNEXPECTED FAILURE)
./00_basics/03_bodies/120.cf FAIL (Test passed, but printed error message(s)) (UNEXPECTED FAILURE)
./00_basics/03_bodies/121.cf FAIL (Test passed, but printed error message(s)) (UNEXPECTED FAILURE)
...
...
Passed tests: 1319
Failed tests: 443
Skipped tests: 235
Soft failures: 75
Flakey failures: 0
Total tests: 2072

Next I'm going to look into the tests that now fail with these new changes, and investigate the error messages.

[CLAassistant]

All committers have signed the CLA.

[olehermanse]

Looks good so far :) Marking this PR as draft since it is not ready to be merged yet :)