Multiple test failures on Alpine 3.15 / musl-1.2.2-r7

[tiran]

BPO	46390
Nosy	@brettcannon, @terryjreedy, @tiran, @zware, @ncopa
Files	alpine315-tests.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-01-15.12:56:24.197>
labels = ['type-bug', 'tests', 'build', '3.11']
title = 'Multiple test failures on Alpine 3.15 / musl-1.2.2-r7'
updated_at = <Date 2022-03-31.17:55:03.007>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2022-03-31.17:55:03.007>
actor = 'brett.cannon'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build', 'Tests']
creation = <Date 2022-01-15.12:56:24.197>
creator = 'christian.heimes'
dependencies = []
files = ['50566']
hgrepos = []
issue_num = 46390
keywords = []
message_count = 7.0
messages = ['410645', '410848', '410849', '411187', '411191', '411193', '411905']
nosy_count = 5.0
nosy_names = ['brett.cannon', 'terry.reedy', 'christian.heimes', 'zach.ware', 'ncopa']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46390'
versions = ['Python 3.11']

Linked PRs

• gh-90548: Make musl test skips smarter (fixes Alpine errors) #131313
• gh-90548: Allow Alpine/MUSL to pass test_c_locale_coercion. #134454
• gh-90548: Fix musl version detection with --strip-all #137864
• [3.14] gh-90548: Fix musl version detection with --strip-all (GH-137864) #138348
• gh-90548: Skip ctypes test_null_dlsym when linked to musl #138592
• gh-90548: Skip NODEV portion of test_makedev when linked to musl #138593
• [3.14] gh-90548: Allow Alpine/MUSL to pass test_c_locale_coercion. (GH-134454) #138600
• [3.14] gh-90548: Skip ctypes test_null_dlsym when linked to musl (GH-138592) #138601
• [3.14] gh-90548: Skip NODEV portion of test_makedev when linked to musl (GH-138593) #138602
• [3.13] gh-90548: Skip NODEV portion of test_makedev when linked to musl (GH-138593) #138713

[tiran]

I'm getting multiple test failures with latest Alpine 3.15 and musl-1.2.2-r7. Some test failures may be caused by wrong assumptions in our tests, some might be bugs in musl lib.c

9 tests failed:
test__locale test_c_locale_coercion test_cmd_line test_gdb
test_locale test_os test_posix test_re test_selectors

I have attached the output of

./python -m test -v test__locale test_c_locale_coercion test_cmd_line test_gdb test_locale test_os test_posix test_re test_selectors 2>&1 | tee alpine315-tests.txt

You can use my container to reproduce the test failures:

$ podman run --privileged -ti --rm -v $(pwd):/cpython:Z quay.io/tiran/cpythonbuild:alpine-3.15 /bin/sh
# /cmd.sh
# cd /cpython/builddep/alpine-3.15-x86_64/
# make test

[kumaraditya303]

These tests seems to be expected to fail on alpine.

See https://github.com/alpinelinux/aports/blob/b36ed9bba2fdbf49a98dfdc3377c29271525082f/main/python3/APKBUILD#L123

[tiran]

I would put it differently: The package maintainer of Python on Alpine decided to ignore all test failures in these test modules.

[terryjreedy]

The first alpine315-tests.txt appears to be a truncated version of the second. Were you expecting the first to be automatically replaced? Should it be unlinked?

https://www.alpinelinux.org "Alpine Linux is a security-oriented, lightweight Linux distribution based on musl libc and busybox." Fron the doc linked above:

# Maintainer: Natanael Copa <ncopa@alpinelinux.org> ## I nosied Natanael at his CLA-signed Alpine id.
# Contributor: Sheila Aman <sheila@vulpine.house>
...
# musl related
fail="test__locale test_locale test_strptime test_re" # various musl locale deficiencies
fail="$fail test_c_locale_coercion"
fail="$fail test_datetime" # hangs if 'tzdata' installed
fail="$fail test_os" # fpathconf, ttyname errno values
fail="$fail test_posix" # sched_[gs]etscheduler not impl
fail="$fail test_shutil" # lchmod, requires real unzip

Should we change CPython tests to accommodate things that are missing (versus buggy). Should the tests requiring sched_[gs]etscheduler be skipped if missing? Or are they required to be 'posix' and is test_posix meant to test completeness as well as correctness of what is present?

[tiran]

In my opinion we should treat these issues as Alpine / musl libc platform bugs and ask the Alpine maintainers to look into the issue. The tests are passing on Linux with glibc and BSD platforms (FreeBSD, macOS, ...) with BSD libc. It is reasonable to assume that failing test are caused by incompatibilities or deficiencies in musl libc, or by a different interpretation of POSIX and Open Group standards.

I would not ignore or skip any test unless we have a thorough understanding of the problem and the deviation is documented. The issue can also affect user code.

Python's test suite is exhaustive. Our tests have found a fair amount of problems in e.g, libm. A couple of years ago ine of my AF_ALG socket tests even found a Kernel bug by triggered a Kernel fault.

[tiran]

The comment about sched_[gs]etscheduler seems to be outdated. For one CPython's test suite has a @requires_sched decorator that performs a check for sched_getscheduler and the Kernel syscall. musl libc in Alpine 3.13 and 3.15 have sched_setscheduler.

[zware]

BTW, we do have an Alpine buildbot worker in the unstable set, running only on the main branch: https://buildbot.python.org/all/#/workers/19