Skip to content

pip install firedrake - #4011

Merged
connorjward merged 119 commits into
masterfrom
connorjward/pip-install
Feb 26, 2025
Merged

connorjward merged 119 commits into
masterfrom
connorjward/pip-install

Conversation

@connorjward

@connorjward connorjward commented Feb 5, 2025

Copy link
Copy Markdown
Contributor

Die firedrake-install! Die! Die! Die!

Remaining tasks

  • Decide what to do about make check

User-facing changes

pip install firedrake

  • Changes the officially supported installation method for Firedrake to using pip.
  • Adds a firedrake-configure script to facilitate installation.
  • firedrake-install/firedrake-update remain but spit out a deprecation warning.
  • Updates the install page to describe the new approach.
  • firedrake-install --doi will no longer work.

Docker

  • firedrake-vanilla and firedrake-complex have been renamed to firedrake-vanilla-default and firedrake-vanilla-complex respectively.
  • firedrake is now installed directly in the container. It is no longer necessary to activate a virtual environment.
  • The firedrake repository is now installed in ~/firedrake instead of ~/firedrake/src/firedrake. The same applies to other packages like gusto and thetis inside the firedrakeproject/firedrake container.
  • The Docker containers now ship OpenMPI instead of MPICH. This means that running parallel tests alongside serial ones with a single pytest invocation will no longer work.
  • VTK is no longer installed by default into the firedrake-vanilla containers. If your package needs it then please add it to your own pyproject.toml or run pip install vtk.

Please see firedrakeproject/gusto#618 as an example switching a workflow to using the new containers.

Developer-facing changes

  • The firedrake-env container no longer contains PETSc, which is instead built in firedrake-vanilla.
  • CI now rebuilds PETSc every time instead of pre-building it inside the firedrake-env container.
  • The linting and zenodo canary workflows have been merged into the same workflow file as the Firedrake tests. This means that we will get fewer notifications if (when) things fail.

Design choices

Unlike firedrake-install, firedrake-configure is designed to be very, very dumb. All it does is take in a specification of a desired package manager (apt-x86_64, apt-aarch64, brew-arm64) and "arch" (default or complex) and spits out a string of appropriate system dependencies (--show-system-dependencies) or PETSc configure options (--show-petsc-configure-options). You can also pass --no-package-manager if you aren't using a supported system and this will give you a set of PETSc configure options where everything is downloaded.

@github-actions

github-actions Bot commented Feb 5, 2025

Copy link
Copy Markdown
TestsPassed ✅Skipped ⏭️Failed ❌
Firedrake complex8270 ran6574 passed1696 skipped0 failed

@github-actions

github-actions Bot commented Feb 5, 2025

Copy link
Copy Markdown
TestsPassed ✅Skipped ⏭️Failed ❌
Firedrake real0 ran0 passed0 skipped0 failed

Comment thread scripts/firedrake-configure
Comment thread scripts/firedrake-configure Outdated
Comment thread docs/source/install.rst Outdated
Comment thread scripts/firedrake-configure Outdated
@connorjward
connorjward marked this pull request as ready for review February 26, 2025 10:58
Comment thread .github/workflows/build.yml Outdated
Comment thread .github/workflows/build.yml Outdated
Comment thread .github/workflows/build.yml Outdated
Comment thread .github/workflows/docker_build.yml Outdated
Comment thread .github/workflows/docker_build.yml Outdated
Comment thread .github/workflows/docs.yml Outdated
Comment thread docker/Dockerfile.docdeps Outdated
Comment thread docker/Dockerfile.vanilla Outdated
Comment thread docker/Dockerfile.vanilla Outdated
Comment thread pyproject.toml Outdated
@Ig-dolci

Copy link
Copy Markdown
Contributor

I just tried installing petsc using the most recent firedrake-configure file. Follow the steps

curl -O https://raw.githubusercontent.com/firedrakeproject/firedrake/connorjward/pip-install/scripts/firedrake-configure
brew update
brew upgrade
brew install $(python3 firedrake-configure --show-system-packages)
git clone https://github.com/firedrakeproject/petsc.git
cd petsc
./configure $(python3 ../firedrake-configure --show-petsc-configure-options)

Here is the attached configure.log file:
configure.log

@JHopeCollins

Copy link
Copy Markdown
Member

I just tried installing petsc using the most recent firedrake-configure file. Follow the steps

curl -O https://raw.githubusercontent.com/firedrakeproject/firedrake/connorjward/pip-install/scripts/firedrake-configure
brew update
brew upgrade
brew install $(python3 firedrake-configure --show-system-packages)
git clone https://github.com/firedrakeproject/petsc.git
cd petsc
./configure $(python3 ../firedrake-configure --show-petsc-configure-options)

Here is the attached configure.log file: configure.log

It should be:

python3 ../firedrake-configure --show-petsc-configure-options | xargs -L1 ./configure

See the discussion here and the updated installation doc page

@JHopeCollins
JHopeCollins dismissed their stale review February 26, 2025 16:15

Changed updted

@connorjward
connorjward merged commit 7d97c49 into master Feb 26, 2025
@connorjward
connorjward deleted the connorjward/pip-install branch February 26, 2025 17:19
ddundo added a commit to mesh-adaptation/docs that referenced this pull request Mar 4, 2025
… install` (#114)

Firedrake is now installable via `pip`, with `firedrake-install` now
giving a `FutureWarning`. For an overview of changes on Firedrake side,
see firedrakeproject/firedrake#4011. This PR
updates the `firedrake-parmmg` image and `reusable_test_suite.yml`
workflows to conform to those changes, as described below.

General:
- The `firedrake-parmmg` image is now only 2.4GB and the build time is
almost twice as fast (25 min faster)!
- Mesh adaptation repos are now cloned to the home directory (they used
to be under `/home/firedrake/firedrake/src`). Firedrake made this change
in their containers too so I am just following their structure.
- It is no longer necessary to create a virtual environment, so I don't
create one (to be consistent with Firedrake's containers again).

Dependencies:
- We used to install
[hypre](https://petsc.org/release/manualpages/PC/PCHYPRE/) (it's listed
in our
[petsc-configure-options.txt](https://github.com/mesh-adaptation/docs/blob/main/install/petsc_configure_options.txt))
but I don't think we need this so I do not install it here.
- Firedrake installs https://github.com/firedrakeproject/slepc in their
vanilla container, but we didn't have this in our old containers so I
don't install it now either.
- Firedrake no longer installs `vtk` by default, so I added it to
dependencies of packages.

Parallel tests:
- I realised that parallel tests did not contribute to the coverage
report. This is now fixed.
- The container now ships OpenMPI, so we can't run serial and parallel
tests with a single `pytest` call.
- Parallel tests are now run with `mpiexec -n N python3 -m pytest ...`,
which is strongly discouraged to be executed as root (see
https://docs.open-mpi.org/en/v5.0.x/man-openmpi/man1/mpirun.1.html#the-allow-run-as-root-option),
which we do in `reusable_test_suite.yml`. Long story short, I think this
is fine to do in our workflow since it uses a container so we're not
really concerned about something going wrong.
- The `reusable_test_suite.yml` now runs serial and parallel tests in
two separate steps, and the coverage report is generated in a further
separate step.

Related PRs:
- Animate: mesh-adaptation/animate#175
- Goalie: mesh-adaptation/goalie#291
- Movement: mesh-adaptation/movement#159
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants