Skip to content

GH-110793: add GitHub Actions job that runs tests after installing#110794

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
FFY00 wants to merge 6 commits into
base:main
Choose a base branch
from
Open

GH-110793: add GitHub Actions job that runs tests after installing #110794

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8753599
GH-110793: add Github action job that runs tests after installing
FFY00 Oct 13, 2023
096ad48
debug
FFY00 Oct 13, 2023
d7c4fc3
debug
FFY00 Oct 13, 2023
c379860
debug
FFY00 Oct 13, 2023
7cdc98f
try --with-openssl-rpath
FFY00 Oct 13, 2023
0775658
don't cd into the builddir when running the tests
FFY00 Oct 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -334,6 +334,73 @@ jobs:
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: xvfb-run make test

build_ubuntu_installed:
name: 'Ubuntu (installed)'
runs-on: ubuntu-20.04
timeout-minutes: 60
needs: check_source
if: needs.check_source.outputs.run_tests == 'true'
env:
OPENSSL_VER: 3.0.11
PYTHONSTRICTEXTENSIONBUILD: 1
steps:
- uses: actions/checkout@v4
- name: Register gcc problem matcher
run: echo "::add-matcher::.github/problem-matchers/gcc.json"
- name: Install Dependencies
run: sudo ./.github/workflows/posix-deps-apt.sh
- name: Configure OpenSSL env vars
run: |
echo "MULTISSL_DIR=${GITHUB_WORKSPACE}/multissl" >> $GITHUB_ENV
echo "OPENSSL_DIR=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> $GITHUB_ENV
- name: 'Restore OpenSSL build'
id: cache-openssl
uses: actions/cache@v3
with:
path: ./multissl/openssl/${{ env.OPENSSL_VER }}
key: ${{ runner.os }}-multissl-openssl-${{ env.OPENSSL_VER }}
- name: Install OpenSSL
if: steps.cache-openssl.outputs.cache-hit != 'true'
run: python3 Tools/ssl/multissltests.py --steps=library --base-directory $MULTISSL_DIR --openssl $OPENSSL_VER --system Linux
- name: Add ccache to PATH
run: |
echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV
- name: Configure ccache action
uses: hendrikmuhs/ccache-action@v1.2
- name: Setup directory envs for out-of-tree builds
run: |
echo "CPYTHON_RO_SRCDIR=$(realpath -m ${GITHUB_WORKSPACE}/../cpython-ro-srcdir)" >> $GITHUB_ENV
echo "CPYTHON_BUILDDIR=$(realpath -m ${GITHUB_WORKSPACE}/../cpython-builddir)" >> $GITHUB_ENV
- name: Create directories for read-only out-of-tree builds
run: mkdir -p $CPYTHON_RO_SRCDIR $CPYTHON_BUILDDIR
- name: Bind mount sources read-only
run: sudo mount --bind -o ro $GITHUB_WORKSPACE $CPYTHON_RO_SRCDIR
- name: Restore config.cache
uses: actions/cache@v3
with:
path: ${{ env.CPYTHON_BUILDDIR }}/config.cache
key: ${{ github.job }}-${{ runner.os }}-${{ needs.check_source.outputs.config_hash }}
- name: Configure CPython out-of-tree
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: |
../cpython-ro-srcdir/configure \
--config-cache \
--with-pydebug \
--with-openssl=$OPENSSL_DIR \
--with-openssl-rpath
- name: Build CPython out-of-tree
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: make -j4

@sobolevnsobolevnOct 13, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we assume that 4 is a good number (other build jobs also do this)?
Снимок экрана 2023-10-13 в 11 04 43

Should we switch to -j$(nproc) / $(sysctl -n hw.ncpu)?

@FFY00FFY00Oct 13, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's the number of threads for Linux and Windows.

In this PR, I just copied the other jobs, so this comes from there. We can do the improvements you are suggesting (would also be nicer when running locally with act for eg), but it should probably be in a different PR.

@hugovkhugovkOct 15, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed something for another PR, but worth investigating.

I've sometimes done a test on my fork with different numbers of jobs in a matrix.

For example, with macOS: make buildbottest TESTOPTS="-j${{ matrix.jobs }} -uall,-cpu"

Shows ~4 is about right:

image

- name: Display build info
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: make pythoninfo
- name: Install CPython to the system
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: sudo make install
- name: Tests
run: xvfb-run /usr/local/bin/python3 -m test

build_ubuntu_ssltests:
name: 'Ubuntu SSL tests with OpenSSL'
runs-on: ubuntu-20.04
Expand DownExpand Up@@ -603,6 +670,7 @@ jobs:
- build_win_arm64
- build_macos
- build_ubuntu
- build_ubuntu_installed
- build_ubuntu_ssltests
- test_hypothesis
- build_asan
Expand DownExpand Up@@ -638,6 +706,7 @@ jobs:
build_win_arm64,
build_macos,
build_ubuntu,
build_ubuntu_installed,
build_ubuntu_ssltests,
build_asan,
'
Expand Down