gh-105540: Convert `pytest` tests of `cases_generator` to regular tests by sobolevn · Pull Request #106713 · python/cpython

sobolevn · 2023-07-13T08:38:19Z

I went ahead and converted them :)

Now they will be executed the same way as regular tests.
I've taken the approach we use for https://github.com/python/cpython/blob/main/Lib/test/test_clinic.py

Local run:

» ./python.exe -m test -v test_generated_cases
== CPython 3.13.0a0 (heads/main:da98ed0aa04, Jul 12 2023, 23:50:11) [Clang 14.0.3 (clang-1403.0.22.14.1)]
== macOS-13.4.1-arm64-arm-64bit little-endian
== Python build: debug
== cwd: /Users/sobolev/Desktop/cpython/build/test_python_77465æ
== CPU count: 12
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.56 Run tests sequentially
0:00:00 load avg: 1.56 [1/1] test_generated_cases
test_effect_sizes (test.test_generated_cases.TestEffects.test_effect_sizes) ... ok
test_array_error_if (test.test_generated_cases.TestGeneratedCases.test_array_error_if) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpnwp2ip6m
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp83kj4oij
ok
test_array_input (test.test_generated_cases.TestGeneratedCases.test_array_input) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpyq_8mt3h
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmptssmvzyp
ok
test_array_input_output (test.test_generated_cases.TestGeneratedCases.test_array_input_output) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp9ej806j9
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp_rqt1jh7
ok
test_array_output (test.test_generated_cases.TestGeneratedCases.test_array_output) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp1j80dud3
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp0a1lgx20
ok
test_binary_op (test.test_generated_cases.TestGeneratedCases.test_binary_op) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp6tkafxpr
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp8nc34ekw
ok
test_cache_effect (test.test_generated_cases.TestGeneratedCases.test_cache_effect) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp0cbo72dj
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp8qhl_sfc
ok
test_cond_effect (test.test_generated_cases.TestGeneratedCases.test_cond_effect) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpx4adjvf8
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpq81jd5gj
ok
test_error_if_plain (test.test_generated_cases.TestGeneratedCases.test_error_if_plain) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmplae6z0p3
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpiritl87t
ok
test_error_if_plain_with_comment (test.test_generated_cases.TestGeneratedCases.test_error_if_plain_with_comment) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpc9w7n9zq
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmposc9ygia
ok
test_error_if_pop (test.test_generated_cases.TestGeneratedCases.test_error_if_pop) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpyu93upkp
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpcual0bhu
ok
test_inst_no_args (test.test_generated_cases.TestGeneratedCases.test_inst_no_args) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpcchbpx7s
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpbqgi_phs
ok
test_inst_one_pop (test.test_generated_cases.TestGeneratedCases.test_inst_one_pop) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpn9cco114
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp1s9r5wbx
ok
test_inst_one_push (test.test_generated_cases.TestGeneratedCases.test_inst_one_push) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp7c48tflr
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpda_z7uoc
ok
test_inst_one_push_one_pop (test.test_generated_cases.TestGeneratedCases.test_inst_one_push_one_pop) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp7alht925
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp_0m7n5zw
ok
test_macro_cond_effect (test.test_generated_cases.TestGeneratedCases.test_macro_cond_effect) ... Read 2 instructions/ops, 1 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpm55x7lbt
Wrote 0 instructions, 1 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp2mad7oyz
ok
test_macro_instruction (test.test_generated_cases.TestGeneratedCases.test_macro_instruction) ... Read 3 instructions/ops, 1 macros, 0 pseudos, and 1 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp7igsdpls
Wrote 2 instructions, 1 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpzqxw_efb
ok
test_overlap (test.test_generated_cases.TestGeneratedCases.test_overlap) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp_ct08u9k
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmphn9i5p0i
ok
test_predictions_and_eval_breaker (test.test_generated_cases.TestGeneratedCases.test_predictions_and_eval_breaker) ... Read 2 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp8w5ljdry
Wrote 2 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp6jjdnwrr
ok
test_suppress_dispatch (test.test_generated_cases.TestGeneratedCases.test_suppress_dispatch) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpbdh1a1wn
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpygejojvf
ok
----------------------------------------------------------------------
Ran 20 tests in 0.021s
OK
== Tests result: SUCCESS ==
1 test OK.
Total duration: 81 ms
Tests result: SUCCESS

Issue: cases_generator tests aren't run by CI, and are broken #105540

…ar tests

sobolevn · 2023-07-13T09:06:42Z

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

Fixing tests failure. I don't have access to Win, so I will experiment :)

gvanrossum

Thank you, thank you, thank you!

gvanrossum · 2023-07-14T00:00:17Z


+def prettify_filename(filename: str) -> str:
+ # Make filename more user-friendly and less platform-specific,
+ # it is only used for error reporting at this point.


That is incorrect, and is not the reason why we go through this much effort. The prettified filename is incorporated in the output (as part of the header explaining that this was a generated file) and must be the same regardless of where it was generated, otherwise you get spurious diffs when e.g. running the tool on Windows.

sobolevn · 2023-07-14T06:54:35Z

Looks like something broke our tests :(

gvanrossum · 2023-07-16T22:32:11Z

Huh, tests pass for me locally. Let me see if I can merge.

gvanrossum · 2023-07-16T22:39:49Z

If the tests pass this time I will merge this upstream. I don't care enough about the one comment to hold it up, and I'd like to add some new tests.

gvanrossum · 2023-07-16T23:08:57Z

Oh dang. I wish the closing """ of the input/output text fragments were one tab stop to the right. But it doesn't matter enough for another PR.

sobolevn · 2023-07-17T08:15:29Z

Thanks! I will continue to make them better, since they now are a part of Lib/test :)

gvanrossum · 2023-07-17T14:16:19Z

Great, we need more eyeballs on this code generator.

pythongh-105540: Convertpytesttests ofcases_generatorto regul…
96ebd50
…ar tests

sobolevn requested a review from gvanrossum July 13, 2023 08:38

bedevere-bot added awaiting review tests Tests in the Lib/test dir labels Jul 13, 2023

bedevere-bot mentioned this pull request Jul 13, 2023
cases_generator tests aren't run by CI, and are broken #105540
Closed
4 tasks

sobolevn added the skip news label Jul 13, 2023

sobolevn added 5 commits July 13, 2023 12:24

Fix test run on Windows
b7ea637

Fix Linux build
f22fd88

Fix Windows build
0bfb1e1

Fix Windows build
79f01b8

Fix Windows build
2286f99

gvanrossum reviewed Jul 14, 2023
View reviewed changes

Add unittest.main
58ebcc7

Merge branch 'main' into issue-105540
e5ed260

gvanrossum merged commit c413207 into python:mainJul 16, 2023

bedevere-bot removed the awaiting review label Jul 16, 2023

efimov-mikhail mentioned this pull request Oct 14, 2024
gh-119786: [doc] broken link and typo fix in interpreter_definition.md #125455
Merged

Navigation Menu

Uh oh!

Uh oh!

gh-105540: Convert `pytest` tests of `cases_generator` to regular tests - #106713

gh-105540: Convert `pytest` tests of `cases_generator` to regular tests#106713
gvanrossum merged 8 commits into
python:mainfrom
sobolevn:issue-105540

sobolevn commented Jul 13, 2023•
edited
Loading

Uh oh!

sobolevn commented Jul 13, 2023•
edited
Loading

Uh oh!

gvanrossum left a comment

Uh oh!

Uh oh!

gvanrossumJul 14, 2023

Uh oh!

Uh oh!

sobolevn commented Jul 14, 2023

Uh oh!

gvanrossum commented Jul 16, 2023

Uh oh!

gvanrossum commented Jul 16, 2023

Uh oh!

gvanrossum commented Jul 16, 2023

Uh oh!

sobolevn commented Jul 17, 2023

Uh oh!

gvanrossum commented Jul 17, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

sobolevn commented Jul 13, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sobolevn commented Jul 13, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gvanrossum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gvanrossumJul 14, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sobolevn commented Jul 14, 2023

Uh oh!

gvanrossum commented Jul 16, 2023

Uh oh!

gvanrossum commented Jul 16, 2023

Uh oh!

gvanrossum commented Jul 16, 2023

Uh oh!

sobolevn commented Jul 17, 2023

Uh oh!

gvanrossum commented Jul 17, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sobolevn commented Jul 13, 2023•
edited
Loading

sobolevn commented Jul 13, 2023•
edited
Loading