3.7.0 object of type 'Package' has no len()

[bluzi]

Hey,

Our tests kept failing today due to an error inside pytest, reverting to 3.6.4 solved it.

00:00:11.460 [test] ============================= test session starts ==============================
00:00:11.462 [test] platform linux2 -- Python 2.7.14, pytest-3.7.0, py-1.5.4, pluggy-0.7.1
00:00:11.463 [test] rootdir: /app, inifile:
00:00:11.465 [test] plugins: flake8-1.0.1
00:00:11.466 [test] collected 0 items / 1 errors
00:00:11.467 [test] 
00:00:11.468 [test] ==================================== ERRORS ====================================
00:00:11.470 [test] ______________________________ ERROR collecting  _______________________________
00:00:11.471 [test] /usr/lib/python2.7/site-packages/_pytest/runner.py:201: in __init__
00:00:11.473 [test]     self.result = func()
00:00:11.474 [test] /usr/lib/python2.7/site-packages/_pytest/runner.py:261: in <lambda>
00:00:11.475 [test]     call = CallInfo(lambda: list(collector.collect()), "collect")
00:00:11.477 [test] /usr/lib/python2.7/site-packages/_pytest/main.py:475: in collect
00:00:11.478 [test]     for x in self._collect(arg):
00:00:11.480 [test] /usr/lib/python2.7/site-packages/_pytest/main.py:539: in _collect
00:00:11.481 [test]     for y in self.matchnodes(col, names):
00:00:11.482 [test] /usr/lib/python2.7/site-packages/_pytest/main.py:605: in matchnodes
00:00:11.483 [test]     num = len(nodes)
00:00:11.485 [test] E   TypeError: object of type 'Package' has no len()
00:00:11.486 [test] ----------------- generated xml file: /test-reports/pytest.xml -----------------
00:00:11.487 [test] =============================== warnings summary ===============================
00:00:11.489 [test] <undetermined location>
00:00:11.490 [test]   could not create cache path /app/.pytest_cache/v/cache/nodeids
00:00:11.491 [test]   could not create cache path /app/.pytest_cache/v/cache/lastfailed
00:00:11.492 [test] 
00:00:11.493 [test] -- Docs: http://doc.pytest.org/en/latest/warnings.html
00:00:11.495 [test] !!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
00:00:11.496 [test] ===================== 2 warnings, 1 error in 0.13 seconds ======================

It's running on Alpine Linux, if you guys need more information feel free to ask.

[lost-a-tooth]

Seeing the same issue on 3.7.0 and 3.6.4 fixed it, definitely a regression.

[nicoddemus]

Thanks for the report, both of you. This is definitely a problem with the recent addition of package-scoped fixtures.

The current workaround is indeed to pin to !=3.7.0 until 3.7.1 is released with a fix. 👍

[nicoddemus]

Can any one of you test the nicoddemus:collect-file-bug branch to see if this fixes the issue?

It can be installed with:

pip install git+https://github.com/nicoddemus/pytest@collect-file-bug

[bluzi]

Looks like 3.7.1 is bugged too, this is the error now:

00:00:15.413 [test] ============================= test session starts ==============================
00:00:15.415 [test] platform linux2 -- Python 2.7.14, pytest-3.7.1, py-1.5.4, pluggy-0.7.1
00:00:15.417 [test] rootdir: /app, inifile:
00:00:15.418 [test] plugins: flake8-1.0.1
00:00:15.420 [test] collected 0 items / 1 errors
00:00:15.672 [test] 
00:00:15.673 [test] ==================================== ERRORS ====================================
00:00:15.675 [test] ______________________________ ERROR collecting  _______________________________
00:00:15.676 [test] /usr/lib/python2.7/site-packages/_pytest/runner.py:201: in __init__
00:00:15.678 [test]     self.result = func()
00:00:15.679 [test] /usr/lib/python2.7/site-packages/_pytest/runner.py:261: in <lambda>
00:00:15.681 [test]     call = CallInfo(lambda: list(collector.collect()), "collect")
00:00:15.682 [test] /usr/lib/python2.7/site-packages/_pytest/main.py:475: in collect
00:00:15.683 [test]     for x in self._collect(arg):
00:00:15.685 [test] /usr/lib/python2.7/site-packages/_pytest/main.py:538: in _collect
00:00:15.686 [test]     col = root._collectfile(argpath)
00:00:15.688 [test] E   AttributeError: 'list' object has no attribute '_collectfile'
00:00:15.689 [test] ----------------- generated xml file: /test-reports/pytest.xml -----------------
00:00:15.690 [test] =============================== warnings summary ===============================
00:00:15.692 [test] <undetermined location>
00:00:15.693 [test]   could not create cache path /app/.pytest_cache/v/cache/nodeids
00:00:15.695 [test]   could not create cache path /app/.pytest_cache/v/cache/lastfailed
00:00:15.696 [test] 
00:00:15.697 [test] -- Docs: http://doc.pytest.org/en/latest/warnings.html
00:00:15.698 [test] !!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
00:00:15.700 [test] ===================== 2 warnings, 1 error in 0.11 seconds ======================

[nicoddemus]

Thanks for confirming @bluzi

[nicoddemus]

@bluzi and @lost-a-tooth, any chance you can provide a snort reproducible example? I'm having trouble to reproduce this. Appreciate it!

[bluzi]

I'll try to find time, need to reproduce that in another environment since I cannot post here the actual code.

[SarLecobee]

Just want to add this seems to be broken also in pytest 3.8.0. Reverting to 3.6.4 fixes it.

====================================================== test session starts ======================================================
platform linux -- Python 3.5.2, pytest-3.8.0, py-1.6.0, pluggy-0.7.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: <cut>, inifile: pytest.ini
plugins: allure-pytest-2.5.1, flaky-3.3.0
collecting 0 items / 1 errors                                                                                                   2018-09-18T17:50:40.244103 - WARNING - No test plan configuration information provided. All tests will be run
collected 0 items / 1 errors

============================================================ ERRORS =============================================================
_______________________________________________________ ERROR collecting  _______________________________________________________
../../../.local/lib/python3.5/site-packages/_pytest/runner.py:201: in __init__
    self.result = func()
../../../.local/lib/python3.5/site-packages/_pytest/runner.py:261: in <lambda>
    call = CallInfo(lambda: list(collector.collect()), "collect")
../../../.local/lib/python3.5/site-packages/_pytest/main.py:477: in collect
    for x in self._collect(arg):
../../../.local/lib/python3.5/site-packages/_pytest/main.py:544: in _collect
    for y in self.matchnodes(col, names):
../../../.local/lib/python3.5/site-packages/_pytest/main.py:609: in matchnodes
    num = len(nodes)
E   TypeError: object of type 'Package' has no len()
 generated xml file: <cut>
===Flaky Test Report===


===End Flaky Test Report===
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================================== 1 error in 0.14 seconds ====================================================

[nicoddemus]

Thanks for the ping, unfortunately this has not really been fixed yet. I believe @jonozzz was working on this, but I'm not sure.

[mdmintz]

@nicoddemus @bluzi @jonozzz I have a way to easily reproduce this. Using pytest==3.8.0, if you clone SeleniumBase and cd into the examples folder and run pytest --collect-only *.py, you'll see the following (tested on my MacBook):

(sbase6) DrSeleniums-MacBook-Pro:examples michael$ pytest --collect-only *.py

============================= test session starts ==============================
platform darwin -- Python 3.7.0, pytest-3.8.0, py-1.6.0, pluggy-0.7.1
rootdir: /Users/michael/github/SeleniumBase, inifile: pytest.ini
plugins: xdist-1.23.0, metadata-1.7.0, html-1.19.0, forked-0.2, seleniumbase-1.15.10
collected 0 items / 1 errors                                                   

==================================== ERRORS ====================================
______________________________ ERROR collecting  _______________________________
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/runner.py:201: in __init__
    self.result = func()
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/runner.py:261: in <lambda>
    call = CallInfo(lambda: list(collector.collect()), "collect")
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/main.py:477: in collect
    for x in self._collect(arg):
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/main.py:544: in _collect
    for y in self.matchnodes(col, names):
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/main.py:609: in matchnodes
    num = len(nodes)
E   TypeError: object of type 'Package' has no len()
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.16 seconds ============================

---

In summary, you get: TypeError: object of type 'Package' has no len()

---

If you then install pytest==3.6.4 and run pytest --collect-only *.py again, it works:

(sbase6) DrSeleniums-MacBook-Pro:examples michael$ pytest --collect-only *.py
==================================== test session starts =====================================
platform darwin -- Python 3.7.0, pytest-3.6.4, py-1.6.0, pluggy-0.7.1
rootdir: /Users/michael/github/SeleniumBase, inifile: pytest.ini
plugins: xdist-1.23.0, metadata-1.7.0, html-1.19.0, forked-0.2, seleniumbase-1.15.10
collected 14 items                                                                           
<Module 'examples/basic_script.py'>
  <UnitTestCase 'MyTestClass'>
    <TestCaseFunction 'test_basic'>
<Module 'examples/delayed_assert_test.py'>
  <UnitTestCase 'MyTestClass'>
    <TestCaseFunction 'test_delayed_asserts'>
<Module 'examples/github_test.py'>
  <UnitTestCase 'GitHubTests'>
    <TestCaseFunction 'test_github'>
<Module 'examples/my_first_test.py'>
  <UnitTestCase 'MyTestClass'>
    <TestCaseFunction 'test_basic'>
<Module 'examples/my_test_suite.py'>
  <UnitTestCase 'MyTestSuite'>
    <TestCaseFunction 'test_1'>
    <TestCaseFunction 'test_2'>
    <TestCaseFunction 'test_3'>
    <TestCaseFunction 'test_4'>
<Module 'examples/parameterized_test.py'>
  <UnitTestCase 'GoogleTestClass'>
    <TestCaseFunction 'test_parameterized_google_search_0_pypi'>
    <TestCaseFunction 'test_parameterized_google_search_1_wikipedia'>
    <TestCaseFunction 'test_parameterized_google_search_2_seleniumbase'>
<Module 'examples/proxy_test.py'>
  <UnitTestCase 'MyTestClass'>
    <TestCaseFunction 'test_proxy'>
<Module 'examples/rate_limiting_test.py'>
  <UnitTestCase 'MyTestClass'>
    <TestCaseFunction 'test_rate_limited_printing'>
<Module 'examples/test_fail.py'>
  <UnitTestCase 'MyTestClass'>
    <TestCaseFunction 'test_find_army_of_robots_on_xkcd_desert_island'>

================================ no tests ran in 0.17 seconds ================================

If you need me to help you fix it, I'm happy to help. It's been an issue for a few customers of mine.

[mdmintz]

Upon further investigation, it's the __init__.py file that causes the issues in pytest==3.8.0, but not when using pytest==3.6.4 . With 3.6.4, pytest is smart enough to not try parsing the file, yet with 3.8.0, it fails. Can reproduce more easily with pytest --collect-only __init__.py.

pytest --collect-only __init__.py 
========================================= test session starts =========================================
platform darwin -- Python 3.7.0, pytest-3.8.0, py-1.6.0, pluggy-0.7.1
rootdir: /Users/michael/github/SeleniumBase, inifile: pytest.ini
plugins: xdist-1.23.0, metadata-1.7.0, html-1.19.0, forked-0.2, seleniumbase-1.15.10
collected 0 items / 1 errors                                                                          

=============================================== ERRORS ================================================
__________________________________________ ERROR collecting  __________________________________________
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/runner.py:201: in __init__
    self.result = func()
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/runner.py:261: in <lambda>
    call = CallInfo(lambda: list(collector.collect()), "collect")
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/main.py:477: in collect
    for x in self._collect(arg):
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/main.py:544: in _collect
    for y in self.matchnodes(col, names):
../../../.virtualenvs/sbase6/lib/python3.7/site-packages/_pytest/main.py:609: in matchnodes
    num = len(nodes)
E   TypeError: object of type 'Package' has no len()

[nicoddemus]

Thanks @mdmintz for trying to reproduce the issue! 👍

Why does pytest fail to parse the __init__ file? An import error, syntax error, something else? Can you provide a minimal example that reproduces the issue?

[mdmintz]

@nicoddemus I think all you need is an empty __init__.py file. Just make one and put it anywhere. Then run pytest __init__.py with 3.8.0 (where it fails), and with 3.6.4 (where it properly ignores it).

[nicoddemus]

Oh I see, passing an __init__.py explicitly indeed reproduces the problem, thanks!