Build NetCDF without DAP and HDF4: halve the Linux wheel

[HugoFara]

Halves the Linux wheel by building NetCDF without features ForeFire does not use.

	Before	After
Compressed	8.6 MB	4.0 MB
Unpacked	23.3 MB	12.0 MB
Vendored libraries	32	6

What was in there

EPEL's netcdf package enables DAP (reading datasets over HTTP) and HDF4. auditwheel correctly vendors their entire dependency closure, so a wheel whose own code is 2.6 MB was shipping:

libcurl, libcrypto, libssl, libkrb5, libgssapi_krb5, libk5crypto,
libkrb5support, libcom_err, libkeyutils, libldap, liblber, libsasl2,
libssh, libnghttp2, libbrotlicommon, libbrotlidec, libpsl, libidn2,
libunistring, libselinux, libpcre2, libcrypt <- 10.2 MB, DAP
libmfhdf, libdf, libjpeg, libtirpc <- 1.4 MB, HDF4

26 of 32 libraries. ForeFire opens local .nc files and Meso-NH PGD files, so neither feature is reachable from it.

The security side matters more than the size. That list includes OpenSSL 1.1.1k, which is past end of life, plus Kerberos, LDAP and libssh. Any scanner pointed at an installed forefire flags them, and we would be respinning wheels for CVEs in code the simulator never calls.

What changed

tools/devops/install-netcdf-manylinux.sh now builds netcdf-c 4.9.3 from source, checksum pinned, rather than installing netcdf-devel from EPEL:

--disable-dap --disable-byterange --disable-hdf4 --disable-libxml2

--disable-byterange is needed as well as --disable-dap, because byterange is a second remote-access path that links libcurl on its own. HDF5 still comes from EPEL. netcdf-cxx4 was already built from source and is unchanged, other than skipping its examples/ directory: they call nc_set_log_level, which netcdf-c only exports when built --enable-logging, and 4.3.1 has no --disable-examples.

The script now fails if libnetcdf regains a libcurl dependency, so a future change to the base image cannot quietly undo this.

Effect on platform consistency

Mixed, and worth a decision rather than a nod:

• HDF4: this removes an inconsistency. Homebrew's NetCDF has never enabled it, so the macOS wheel already could not read HDF4 files while the Linux one could. They now agree.
• DAP: this introduces one. macOS keeps DAP, because its libnetcdf resolves libcurl to /usr/lib/libcurl.4.dylib, a system library delocate leaves alone. It costs nothing there, so there was no reason to build NetCDF from source on macOS just to remove it. If you would rather the two match exactly, say so and I will do the same on macOS.

The remaining 6 libraries are libnetcdf, libnetcdf_c++4, libhdf5, libhdf5_hl, libsz and libaec, which is the same core set the macOS wheel carries.

Verification

Built in a clean manylinux_2_28_x86_64 container with podman, installed into a fresh venv on a Fedora 44 host that has no NetCDF of its own:

Check	Result
Smoke test tests/python/test_wheel.py	passes, 122 fire nodes
forefire -v	v2.5.0
Repository example tests/runff/run.ff	exit 0, GeoJSON 1625 bytes, byte-identical to the current wheel
libnetcdf links libcurl	no, and the script now asserts it

Open question

Does anything in ForeFire or the Meso-NH coupling read datasets over OPeNDAP, or open HDF4 files? I could not find a path to either, but the coupling is the part I know least. If one of them is genuinely used, the right answer is to enable it deliberately on both platforms instead, and this should be closed.

---

This pull request, including its code changes and this description, was generated by Claude Opus 5.

[filippi]

Hi On mobile... sorry for typos Thanks ! No opendap, and in fact netcdf is a pain.. will probably drop it in some ways in future releases.. using a custom but optimized dataformat and pre-processor. If used from Python it could even not be called at all. As for MesoNH it links with its own separated netcdf built from source anyway.. no real problem. Creers and thanks so much for the effort ! ******************************************** Jean-Baptiste Filippi Chargé de Recherches SPE - UMR 6134 CNRS Université de Corse Quartier grossetti BP 52, 20250 Corte Tel. +33 495 450 158 ****************************************** Le mar. 11 août 2026, 18:32, Hugo ***@***.***> a écrit :

…

Halves the Linux wheel by building NetCDF without features ForeFire does not use. Before After Compressed 8.6 MB *4.0 MB* Unpacked 23.3 MB *12.0 MB* Vendored libraries 32 *6* What was in there EPEL's netcdf package enables DAP (reading datasets over HTTP) and HDF4. auditwheel correctly vendors their entire dependency closure, so a wheel whose own code is 2.6 MB was shipping: libcurl, libcrypto, libssl, libkrb5, libgssapi_krb5, libk5crypto, libkrb5support, libcom_err, libkeyutils, libldap, liblber, libsasl2, libssh, libnghttp2, libbrotlicommon, libbrotlidec, libpsl, libidn2, libunistring, libselinux, libpcre2, libcrypt <- 10.2 MB, DAP libmfhdf, libdf, libjpeg, libtirpc <- 1.4 MB, HDF4 26 of 32 libraries. ForeFire opens local .nc files and Meso-NH PGD files, so neither feature is reachable from it. The security side matters more than the size. That list includes OpenSSL 1.1.1k, which is past end of life, plus Kerberos, LDAP and libssh. Any scanner pointed at an installed forefire flags them, and we would be respinning wheels for CVEs in code the simulator never calls. What changed tools/devops/install-netcdf-manylinux.sh now builds netcdf-c 4.9.3 from source, checksum pinned, rather than installing netcdf-devel from EPEL: --disable-dap --disable-byterange --disable-hdf4 --disable-libxml2 --disable-byterange is needed as well as --disable-dap, because byterange is a second remote-access path that links libcurl on its own. HDF5 still comes from EPEL. netcdf-cxx4 was already built from source and is unchanged, other than skipping its examples/ directory: they call nc_set_log_level, which netcdf-c only exports when built --enable-logging, and 4.3.1 has no --disable-examples. The script now fails if libnetcdf regains a libcurl dependency, so a future change to the base image cannot quietly undo this. Effect on platform consistency Mixed, and worth a decision rather than a nod: - *HDF4: this removes an inconsistency.* Homebrew's NetCDF has never enabled it, so the macOS wheel already could not read HDF4 files while the Linux one could. They now agree. - *DAP: this introduces one.* macOS keeps DAP, because its libnetcdf resolves libcurl to /usr/lib/libcurl.4.dylib, a system library delocate leaves alone. It costs nothing there, so there was no reason to build NetCDF from source on macOS just to remove it. If you would rather the two match exactly, say so and I will do the same on macOS. The remaining 6 libraries are libnetcdf, libnetcdf_c++4, libhdf5, libhdf5_hl, libsz and libaec, which is the same core set the macOS wheel carries. Verification Built in a clean manylinux_2_28_x86_64 container with podman, installed into a fresh venv on a Fedora 44 host that has no NetCDF of its own: Check Result Smoke test tests/python/test_wheel.py passes, 122 fire nodes forefire -v v2.5.0 Repository example tests/runff/run.ff exit 0, GeoJSON 1625 bytes, byte-identical to the current wheel libnetcdf links libcurl no, and the script now asserts it Open question Does anything in ForeFire or the Meso-NH coupling read datasets over OPeNDAP, or open HDF4 files? I could not find a path to either, but the coupling is the part I know least. If one of them is genuinely used, the right answer is to enable it deliberately on both platforms instead, and this should be closed. ------------------------------ *This pull request, including its code changes and this description, was generated by Claude Opus 5.* ------------------------------ You can view, comment on, or merge this pull request online at: #154 Commit Summary - 75e8f26 <75e8f26> perf(packaging): build NetCDF without DAP and HDF4 for Linux wheels File Changes (1 file <https://github.com/forefireAPI/forefire/pull/154/files>) - *M* tools/devops/install-netcdf-manylinux.sh <https://github.com/forefireAPI/forefire/pull/154/files#diff-8633cfae7082064f7d48e6954d6b9d212e3258a8e40a54cb9f59ae9cae007322> (86) Patch Links: - https://github.com/forefireAPI/forefire/pull/154.patch - https://github.com/forefireAPI/forefire/pull/154.diff — Reply to this email directly, view it on GitHub <#154?email_source=notifications&email_token=AAUZ7CV3YMCVHRYDDWSHX5L5JL76DA5CNFSNUABEM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UF42DENJSG42TQNZRHCTHEZLBONXW5KTTOVRHGY3SNFRGKZFFMV3GK3TUVRTG633UMVZF6Y3MNFRWW>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAUZ7CRWLJEE4T2F56NVH5T5JL76DAVCNFSNUABEKJSXA33TNF2G64TZHMYTOMRUHE3TKMJ3JFZXG5LFHM2TCMJZGY4DMNRYG6QXMAQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[HugoFara]

@antonio-leblanc I'll need your help to accept this one. GitHub won't let me merge, maybe because I was added to the project after opening the PR.

---

EDIT

@filippi I am asking NetCDF C++ if the project continues, they haven't released in 7 years and that is causing considerable drift with other tools. Otherwise, yes we align on Méso-NH. Currently NetCDF is making the code complex to serve only a fraction of users.

[pantera-bot]

Code Review Summary

Verdict: Approved

✅ Looks Good

• Dropping DAP and HDF4 significantly reduces the manylinux wheel size and its attack surface (no more bundled OpenSSL, Kerberos, libssh).
• The script is well-structured: pinning the source tarballs and checksums is a solid security practice.
• Fallback checks are correctly implemented (failing if is dynamically linked).
• Consistent behavior between Linux and macOS regarding HDF4 is a plus.

💡 Suggestions

• The PR description states that the pkg-config file (netcdf-cxx4.pc) is still installed. However, because make install SUBDIRS=cxx4 only recurses into the cxx4 directory, the root-level pkgconfig_DATA rule is skipped and the .pc file is actually omitted. This is not a blocker for ForeFire, as its CMakeLists.txt uses find_path and find_library rather than pkg_check_modules, but it's worth noting.
• Regarding the open question on DAP and HDF4: ForeFire's NetCDF interactions (including Meso-NH coupling) operate exclusively on local files (.nc, PGD), so OPENDAP and HDF4 features are indeed safe to remove.

---

Reviewed by Hermes Agent

[pantera-bot]

Code Review Summary

Verdict: Approved

✅ Looks Good

• Dropping DAP and HDF4 significantly reduces the manylinux wheel size and its attack surface (no more bundled OpenSSL, Kerberos, libssh).
• The install-netcdf-manylinux.sh script is well-structured: pinning the source tarballs and checksums is a solid security practice.
• Fallback checks are correctly implemented (failing if libcurl is dynamically linked).
• Consistent behavior between Linux and macOS regarding HDF4 is a plus.

💡 Suggestions

• The PR description states that the pkg-config file (netcdf-cxx4.pc) is still installed. However, because make install SUBDIRS=cxx4 only recurses into the cxx4 directory, the root-level pkgconfig_DATA rule is skipped and the .pc file is actually omitted. This is not a blocker for ForeFire, as its CMakeLists.txt uses find_path and find_library rather than pkg_check_modules, but it's worth noting.
• Regarding the open question on DAP and HDF4: ForeFire's NetCDF interactions (including Meso-NH coupling) operate exclusively on local files (.nc, PGD), so OPENDAP and HDF4 features are indeed safe to remove.

---

Reviewed by Hermes Agent