Allow Docker image build using HTTP proxy in corporate networks

[wolfdn]

Problem

Building Airflow Docker images fails in environments where GPG's dirmngr cannot reach keyservers — most commonly behind corporate HTTP proxies. This makes commands like breeze start-airflow fail, if it is attempted to build a new Docker image - even if the Docker daemon and client have correct HTTP proxy configuration.

The issue is that dirmngr (GnuPG's network daemon, used for gpg --recv-keys) resolves DNS locally before connecting, even when --keyserver-options http-proxy=... is configured. In corporate networks where an intercepting DNS server returns NXDOMAIN for external domains (routing all internet access exclusively through an HTTP proxy), dirmngr fails with keyserver receive failed: No keyserver available before it ever attempts a connection.

Tools like wget and curl work in these environments because they delegate DNS resolution to the proxy via the CONNECT method when http_proxy/https_proxy environment variables are set.

This affects two code paths:

1. Python source verification (install_os_dependencies.sh) — fetching the Python release manager's GPG key from keys.openpgp.org when building Python 3.10 from source.
2. APT repository key import (common.sh → common::import_trusted_gpg) — fetching GPG keys for MariaDB, MySQL, PostgreSQL, and MSSQL APT repositories from keyserver.ubuntu.com / pgp.surf.nl.

I'm not sure how many developers of Airflow it would help to fix this - so how many developers (are forced to) work behind a corporate HTTP proxy. At least for us it would be nice to have this fixed, because it would make setting up a development environment a bit more straightforward.

Proposed Solution

Add a wget fallback after each gpg --recv-keys call. If GPG fails to fetch the key natively, the same key is fetched via HTTPS using wget and imported with gpg --import. This keeps the existing GPG-native path as the primary method (no changes for environments without proxy issues) while providing a working fallback for proxy-constrained environments.

Why the wget command differs between the two code paths

The fallback uses different URLs depending on the keyserver API:

• keys.openpgp.org (Python key) uses the VKS API, which has a clean endpoint:

  https://keys.openpgp.org/vks/v1/by-fingerprint/<fingerprint>
  

  The key fingerprint is used directly — no prefix manipulation needed.

• keyserver.ubuntu.com / pgp.surf.nl (APT repo keys) use the HKP protocol's /pks/lookup endpoint:

  https://<host>/pks/lookup?op=get&options=mr&search=<keyid>
  

  This requires:

  • Extracting the hostname from the hkps:// or hkp:// URL scheme
  • Stripping the 0x prefix from the key ID (some callers pass a 0x prefix for the key IDs, others don't) and re-adding it as a query parameter to avoid a double-0x bug
  • Verifying the import succeeded with gpg --list-keys before breaking out of the keyserver loop, because gpg --import exits 0 even on empty input

Changes

• scripts/docker/common.sh — common::import_trusted_gpg(): added wget fallback with HKP lookup after gpg --recv-keys in the keyserver loop
• scripts/docker/install_os_dependencies.sh — install_python(): added wget fallback with VKS lookup after gpg --recv-keys for the Python GPG key
• Dockerfile.ci, Dockerfile — updated inlined copies of both scripts

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  GitHub Copilot - Claude Opus 4.6

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

[jscheffl]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

What do your mean with "embedding"? That the key is cloned in Airflow repo and COPY is made into docker or that wget is used in general to download? I mean the docker is anyway built dynamically... so what does "persist" mean? Git?

[wolfdn]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

I like the idea - do you mean something like this? #65408

[wolfdn]

Closing in favor of #65408

[wolfdn]

Problem

Building Airflow Docker images fails in environments where GPG's dirmngr cannot reach keyservers — most commonly behind corporate HTTP proxies. This makes commands like breeze start-airflow fail, if it is attempted to build a new Docker image - even if the Docker daemon and client have correct HTTP proxy configuration.

The issue is that dirmngr (GnuPG's network daemon, used for gpg --recv-keys) resolves DNS locally before connecting, even when --keyserver-options http-proxy=... is configured. In corporate networks where an intercepting DNS server returns NXDOMAIN for external domains (routing all internet access exclusively through an HTTP proxy), dirmngr fails with keyserver receive failed: No keyserver available before it ever attempts a connection.

Tools like wget and curl work in these environments because they delegate DNS resolution to the proxy via the CONNECT method when http_proxy/https_proxy environment variables are set.

This affects two code paths:

1. Python source verification (install_os_dependencies.sh) — fetching the Python release manager's GPG key from keys.openpgp.org when building Python 3.10 from source.
2. APT repository key import (common.sh → common::import_trusted_gpg) — fetching GPG keys for MariaDB, MySQL, PostgreSQL, and MSSQL APT repositories from keyserver.ubuntu.com / pgp.surf.nl.

I'm not sure how many developers of Airflow it would help to fix this - so how many developers (are forced to) work behind a corporate HTTP proxy. At least for us it would be nice to have this fixed, because it would make setting up a development environment a bit more straightforward.

Proposed Solution

Add a wget fallback after each gpg --recv-keys call. If GPG fails to fetch the key natively, the same key is fetched via HTTPS using wget and imported with gpg --import. This keeps the existing GPG-native path as the primary method (no changes for environments without proxy issues) while providing a working fallback for proxy-constrained environments.

Why the wget command differs between the two code paths

The fallback uses different URLs depending on the keyserver API:

• keys.openpgp.org (Python key) uses the VKS API, which has a clean endpoint:

  https://keys.openpgp.org/vks/v1/by-fingerprint/<fingerprint>
  

  The key fingerprint is used directly — no prefix manipulation needed.

• keyserver.ubuntu.com / pgp.surf.nl (APT repo keys) use the HKP protocol's /pks/lookup endpoint:

  https://<host>/pks/lookup?op=get&options=mr&search=<keyid>
  

  This requires:

  • Extracting the hostname from the hkps:// or hkp:// URL scheme
  • Stripping the 0x prefix from the key ID (some callers pass a 0x prefix for the key IDs, others don't) and re-adding it as a query parameter to avoid a double-0x bug
  • Verifying the import succeeded with gpg --list-keys before breaking out of the keyserver loop, because gpg --import exits 0 even on empty input

Changes

• scripts/docker/common.sh — common::import_trusted_gpg(): added wget fallback with HKP lookup after gpg --recv-keys in the keyserver loop
• scripts/docker/install_os_dependencies.sh — install_python(): added wget fallback with VKS lookup after gpg --recv-keys for the Python GPG key
• Dockerfile.ci, Dockerfile — updated inlined copies of both scripts

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  GitHub Copilot - Claude Opus 4.6

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

[jscheffl]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

What do your mean with "embedding"? That the key is cloned in Airflow repo and COPY is made into docker or that wget is used in general to download? I mean the docker is anyway built dynamically... so what does "persist" mean? Git?

[wolfdn]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

I like the idea - do you mean something like this? #65408

[wolfdn]

Closing in favor of #65408

[wolfdn]

Problem

Building Airflow Docker images fails in environments where GPG's dirmngr cannot reach keyservers — most commonly behind corporate HTTP proxies. This makes commands like breeze start-airflow fail, if it is attempted to build a new Docker image - even if the Docker daemon and client have correct HTTP proxy configuration.

The issue is that dirmngr (GnuPG's network daemon, used for gpg --recv-keys) resolves DNS locally before connecting, even when --keyserver-options http-proxy=... is configured. In corporate networks where an intercepting DNS server returns NXDOMAIN for external domains (routing all internet access exclusively through an HTTP proxy), dirmngr fails with keyserver receive failed: No keyserver available before it ever attempts a connection.

Tools like wget and curl work in these environments because they delegate DNS resolution to the proxy via the CONNECT method when http_proxy/https_proxy environment variables are set.

This affects two code paths:

1. Python source verification (install_os_dependencies.sh) — fetching the Python release manager's GPG key from keys.openpgp.org when building Python 3.10 from source.
2. APT repository key import (common.sh → common::import_trusted_gpg) — fetching GPG keys for MariaDB, MySQL, PostgreSQL, and MSSQL APT repositories from keyserver.ubuntu.com / pgp.surf.nl.

I'm not sure how many developers of Airflow it would help to fix this - so how many developers (are forced to) work behind a corporate HTTP proxy. At least for us it would be nice to have this fixed, because it would make setting up a development environment a bit more straightforward.

Proposed Solution

Add a wget fallback after each gpg --recv-keys call. If GPG fails to fetch the key natively, the same key is fetched via HTTPS using wget and imported with gpg --import. This keeps the existing GPG-native path as the primary method (no changes for environments without proxy issues) while providing a working fallback for proxy-constrained environments.

Why the wget command differs between the two code paths

The fallback uses different URLs depending on the keyserver API:

• keys.openpgp.org (Python key) uses the VKS API, which has a clean endpoint:

  https://keys.openpgp.org/vks/v1/by-fingerprint/<fingerprint>
  

  The key fingerprint is used directly — no prefix manipulation needed.

• keyserver.ubuntu.com / pgp.surf.nl (APT repo keys) use the HKP protocol's /pks/lookup endpoint:

  https://<host>/pks/lookup?op=get&options=mr&search=<keyid>
  

  This requires:

  • Extracting the hostname from the hkps:// or hkp:// URL scheme
  • Stripping the 0x prefix from the key ID (some callers pass a 0x prefix for the key IDs, others don't) and re-adding it as a query parameter to avoid a double-0x bug
  • Verifying the import succeeded with gpg --list-keys before breaking out of the keyserver loop, because gpg --import exits 0 even on empty input

Changes

• scripts/docker/common.sh — common::import_trusted_gpg(): added wget fallback with HKP lookup after gpg --recv-keys in the keyserver loop
• scripts/docker/install_os_dependencies.sh — install_python(): added wget fallback with VKS lookup after gpg --recv-keys for the Python GPG key
• Dockerfile.ci, Dockerfile — updated inlined copies of both scripts

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  GitHub Copilot - Claude Opus 4.6

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

[jscheffl]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

What do your mean with "embedding"? That the key is cloned in Airflow repo and COPY is made into docker or that wget is used in general to download? I mean the docker is anyway built dynamically... so what does "persist" mean? Git?

[wolfdn]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

I like the idea - do you mean something like this? #65408

[wolfdn]

Closing in favor of #65408

[wolfdn]

Problem

Building Airflow Docker images fails in environments where GPG's dirmngr cannot reach keyservers — most commonly behind corporate HTTP proxies. This makes commands like breeze start-airflow fail, if it is attempted to build a new Docker image - even if the Docker daemon and client have correct HTTP proxy configuration.

The issue is that dirmngr (GnuPG's network daemon, used for gpg --recv-keys) resolves DNS locally before connecting, even when --keyserver-options http-proxy=... is configured. In corporate networks where an intercepting DNS server returns NXDOMAIN for external domains (routing all internet access exclusively through an HTTP proxy), dirmngr fails with keyserver receive failed: No keyserver available before it ever attempts a connection.

Tools like wget and curl work in these environments because they delegate DNS resolution to the proxy via the CONNECT method when http_proxy/https_proxy environment variables are set.

This affects two code paths:

1. Python source verification (install_os_dependencies.sh) — fetching the Python release manager's GPG key from keys.openpgp.org when building Python 3.10 from source.
2. APT repository key import (common.sh → common::import_trusted_gpg) — fetching GPG keys for MariaDB, MySQL, PostgreSQL, and MSSQL APT repositories from keyserver.ubuntu.com / pgp.surf.nl.

I'm not sure how many developers of Airflow it would help to fix this - so how many developers (are forced to) work behind a corporate HTTP proxy. At least for us it would be nice to have this fixed, because it would make setting up a development environment a bit more straightforward.

Proposed Solution

Add a wget fallback after each gpg --recv-keys call. If GPG fails to fetch the key natively, the same key is fetched via HTTPS using wget and imported with gpg --import. This keeps the existing GPG-native path as the primary method (no changes for environments without proxy issues) while providing a working fallback for proxy-constrained environments.

Why the wget command differs between the two code paths

The fallback uses different URLs depending on the keyserver API:

• keys.openpgp.org (Python key) uses the VKS API, which has a clean endpoint:

  https://keys.openpgp.org/vks/v1/by-fingerprint/<fingerprint>
  

  The key fingerprint is used directly — no prefix manipulation needed.

• keyserver.ubuntu.com / pgp.surf.nl (APT repo keys) use the HKP protocol's /pks/lookup endpoint:

  https://<host>/pks/lookup?op=get&options=mr&search=<keyid>
  

  This requires:

  • Extracting the hostname from the hkps:// or hkp:// URL scheme
  • Stripping the 0x prefix from the key ID (some callers pass a 0x prefix for the key IDs, others don't) and re-adding it as a query parameter to avoid a double-0x bug
  • Verifying the import succeeded with gpg --list-keys before breaking out of the keyserver loop, because gpg --import exits 0 even on empty input

Changes

• scripts/docker/common.sh — common::import_trusted_gpg(): added wget fallback with HKP lookup after gpg --recv-keys in the keyserver loop
• scripts/docker/install_os_dependencies.sh — install_python(): added wget fallback with VKS lookup after gpg --recv-keys for the Python GPG key
• Dockerfile.ci, Dockerfile — updated inlined copies of both scripts

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  GitHub Copilot - Claude Opus 4.6

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

[jscheffl]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

What do your mean with "embedding"? That the key is cloned in Airflow repo and COPY is made into docker or that wget is used in general to download? I mean the docker is anyway built dynamically... so what does "persist" mean? Git?

[wolfdn]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

I like the idea - do you mean something like this? #65408

[wolfdn]

Closing in favor of #65408

[wolfdn]

Problem

Building Airflow Docker images fails in environments where GPG's dirmngr cannot reach keyservers — most commonly behind corporate HTTP proxies. This makes commands like breeze start-airflow fail, if it is attempted to build a new Docker image - even if the Docker daemon and client have correct HTTP proxy configuration.

The issue is that dirmngr (GnuPG's network daemon, used for gpg --recv-keys) resolves DNS locally before connecting, even when --keyserver-options http-proxy=... is configured. In corporate networks where an intercepting DNS server returns NXDOMAIN for external domains (routing all internet access exclusively through an HTTP proxy), dirmngr fails with keyserver receive failed: No keyserver available before it ever attempts a connection.

Tools like wget and curl work in these environments because they delegate DNS resolution to the proxy via the CONNECT method when http_proxy/https_proxy environment variables are set.

This affects two code paths:

1. Python source verification (install_os_dependencies.sh) — fetching the Python release manager's GPG key from keys.openpgp.org when building Python 3.10 from source.
2. APT repository key import (common.sh → common::import_trusted_gpg) — fetching GPG keys for MariaDB, MySQL, PostgreSQL, and MSSQL APT repositories from keyserver.ubuntu.com / pgp.surf.nl.

I'm not sure how many developers of Airflow it would help to fix this - so how many developers (are forced to) work behind a corporate HTTP proxy. At least for us it would be nice to have this fixed, because it would make setting up a development environment a bit more straightforward.

Proposed Solution

Add a wget fallback after each gpg --recv-keys call. If GPG fails to fetch the key natively, the same key is fetched via HTTPS using wget and imported with gpg --import. This keeps the existing GPG-native path as the primary method (no changes for environments without proxy issues) while providing a working fallback for proxy-constrained environments.

Why the wget command differs between the two code paths

The fallback uses different URLs depending on the keyserver API:

• keys.openpgp.org (Python key) uses the VKS API, which has a clean endpoint:

  https://keys.openpgp.org/vks/v1/by-fingerprint/<fingerprint>
  

  The key fingerprint is used directly — no prefix manipulation needed.

• keyserver.ubuntu.com / pgp.surf.nl (APT repo keys) use the HKP protocol's /pks/lookup endpoint:

  https://<host>/pks/lookup?op=get&options=mr&search=<keyid>
  

  This requires:

  • Extracting the hostname from the hkps:// or hkp:// URL scheme
  • Stripping the 0x prefix from the key ID (some callers pass a 0x prefix for the key IDs, others don't) and re-adding it as a query parameter to avoid a double-0x bug
  • Verifying the import succeeded with gpg --list-keys before breaking out of the keyserver loop, because gpg --import exits 0 even on empty input

Changes

• scripts/docker/common.sh — common::import_trusted_gpg(): added wget fallback with HKP lookup after gpg --recv-keys in the keyserver loop
• scripts/docker/install_os_dependencies.sh — install_python(): added wget fallback with VKS lookup after gpg --recv-keys for the Python GPG key
• Dockerfile.ci, Dockerfile — updated inlined copies of both scripts

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  GitHub Copilot - Claude Opus 4.6

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

[jscheffl]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

What do your mean with "embedding"? That the key is cloned in Airflow repo and COPY is made into docker or that wget is used in general to download? I mean the docker is anyway built dynamically... so what does "persist" mean? Git?

[wolfdn]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

I like the idea - do you mean something like this? #65408

[wolfdn]

Closing in favor of #65408

[wolfdn]

Problem

Building Airflow Docker images fails in environments where GPG's dirmngr cannot reach keyservers — most commonly behind corporate HTTP proxies. This makes commands like breeze start-airflow fail, if it is attempted to build a new Docker image - even if the Docker daemon and client have correct HTTP proxy configuration.

The issue is that dirmngr (GnuPG's network daemon, used for gpg --recv-keys) resolves DNS locally before connecting, even when --keyserver-options http-proxy=... is configured. In corporate networks where an intercepting DNS server returns NXDOMAIN for external domains (routing all internet access exclusively through an HTTP proxy), dirmngr fails with keyserver receive failed: No keyserver available before it ever attempts a connection.

Tools like wget and curl work in these environments because they delegate DNS resolution to the proxy via the CONNECT method when http_proxy/https_proxy environment variables are set.

This affects two code paths:

1. Python source verification (install_os_dependencies.sh) — fetching the Python release manager's GPG key from keys.openpgp.org when building Python 3.10 from source.
2. APT repository key import (common.sh → common::import_trusted_gpg) — fetching GPG keys for MariaDB, MySQL, PostgreSQL, and MSSQL APT repositories from keyserver.ubuntu.com / pgp.surf.nl.

I'm not sure how many developers of Airflow it would help to fix this - so how many developers (are forced to) work behind a corporate HTTP proxy. At least for us it would be nice to have this fixed, because it would make setting up a development environment a bit more straightforward.

Proposed Solution

Add a wget fallback after each gpg --recv-keys call. If GPG fails to fetch the key natively, the same key is fetched via HTTPS using wget and imported with gpg --import. This keeps the existing GPG-native path as the primary method (no changes for environments without proxy issues) while providing a working fallback for proxy-constrained environments.

Why the wget command differs between the two code paths

The fallback uses different URLs depending on the keyserver API:

• keys.openpgp.org (Python key) uses the VKS API, which has a clean endpoint:

  https://keys.openpgp.org/vks/v1/by-fingerprint/<fingerprint>
  

  The key fingerprint is used directly — no prefix manipulation needed.

• keyserver.ubuntu.com / pgp.surf.nl (APT repo keys) use the HKP protocol's /pks/lookup endpoint:

  https://<host>/pks/lookup?op=get&options=mr&search=<keyid>
  

  This requires:

  • Extracting the hostname from the hkps:// or hkp:// URL scheme
  • Stripping the 0x prefix from the key ID (some callers pass a 0x prefix for the key IDs, others don't) and re-adding it as a query parameter to avoid a double-0x bug
  • Verifying the import succeeded with gpg --list-keys before breaking out of the keyserver loop, because gpg --import exits 0 even on empty input

Changes

• scripts/docker/common.sh — common::import_trusted_gpg(): added wget fallback with HKP lookup after gpg --recv-keys in the keyserver loop
• scripts/docker/install_os_dependencies.sh — install_python(): added wget fallback with VKS lookup after gpg --recv-keys for the Python GPG key
• Dockerfile.ci, Dockerfile — updated inlined copies of both scripts

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  GitHub Copilot - Claude Opus 4.6

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

[jscheffl]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

What do your mean with "embedding"? That the key is cloned in Airflow repo and COPY is made into docker or that wget is used in general to download? I mean the docker is anyway built dynamically... so what does "persist" mean? Git?

[wolfdn]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

I like the idea - do you mean something like this? #65408

[wolfdn]

Closing in favor of #65408

[wolfdn]

Problem

Building Airflow Docker images fails in environments where GPG's dirmngr cannot reach keyservers — most commonly behind corporate HTTP proxies. This makes commands like breeze start-airflow fail, if it is attempted to build a new Docker image - even if the Docker daemon and client have correct HTTP proxy configuration.

The issue is that dirmngr (GnuPG's network daemon, used for gpg --recv-keys) resolves DNS locally before connecting, even when --keyserver-options http-proxy=... is configured. In corporate networks where an intercepting DNS server returns NXDOMAIN for external domains (routing all internet access exclusively through an HTTP proxy), dirmngr fails with keyserver receive failed: No keyserver available before it ever attempts a connection.

Tools like wget and curl work in these environments because they delegate DNS resolution to the proxy via the CONNECT method when http_proxy/https_proxy environment variables are set.

This affects two code paths:

1. Python source verification (install_os_dependencies.sh) — fetching the Python release manager's GPG key from keys.openpgp.org when building Python 3.10 from source.
2. APT repository key import (common.sh → common::import_trusted_gpg) — fetching GPG keys for MariaDB, MySQL, PostgreSQL, and MSSQL APT repositories from keyserver.ubuntu.com / pgp.surf.nl.

I'm not sure how many developers of Airflow it would help to fix this - so how many developers (are forced to) work behind a corporate HTTP proxy. At least for us it would be nice to have this fixed, because it would make setting up a development environment a bit more straightforward.

Proposed Solution

Add a wget fallback after each gpg --recv-keys call. If GPG fails to fetch the key natively, the same key is fetched via HTTPS using wget and imported with gpg --import. This keeps the existing GPG-native path as the primary method (no changes for environments without proxy issues) while providing a working fallback for proxy-constrained environments.

Why the wget command differs between the two code paths

The fallback uses different URLs depending on the keyserver API:

• keys.openpgp.org (Python key) uses the VKS API, which has a clean endpoint:

  https://keys.openpgp.org/vks/v1/by-fingerprint/<fingerprint>
  

  The key fingerprint is used directly — no prefix manipulation needed.

• keyserver.ubuntu.com / pgp.surf.nl (APT repo keys) use the HKP protocol's /pks/lookup endpoint:

  https://<host>/pks/lookup?op=get&options=mr&search=<keyid>
  

  This requires:

  • Extracting the hostname from the hkps:// or hkp:// URL scheme
  • Stripping the 0x prefix from the key ID (some callers pass a 0x prefix for the key IDs, others don't) and re-adding it as a query parameter to avoid a double-0x bug
  • Verifying the import succeeded with gpg --list-keys before breaking out of the keyserver loop, because gpg --import exits 0 even on empty input

Changes

• scripts/docker/common.sh — common::import_trusted_gpg(): added wget fallback with HKP lookup after gpg --recv-keys in the keyserver loop
• scripts/docker/install_os_dependencies.sh — install_python(): added wget fallback with VKS lookup after gpg --recv-keys for the Python GPG key
• Dockerfile.ci, Dockerfile — updated inlined copies of both scripts

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  GitHub Copilot - Claude Opus 4.6

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

[jscheffl]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

What do your mean with "embedding"? That the key is cloned in Airflow repo and COPY is made into docker or that wget is used in general to download? I mean the docker is anyway built dynamically... so what does "persist" mean? Git?

[wolfdn]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

I like the idea - do you mean something like this? #65408

[wolfdn]

Closing in favor of #65408

[wolfdn]

Problem

Building Airflow Docker images fails in environments where GPG's dirmngr cannot reach keyservers — most commonly behind corporate HTTP proxies. This makes commands like breeze start-airflow fail, if it is attempted to build a new Docker image - even if the Docker daemon and client have correct HTTP proxy configuration.

The issue is that dirmngr (GnuPG's network daemon, used for gpg --recv-keys) resolves DNS locally before connecting, even when --keyserver-options http-proxy=... is configured. In corporate networks where an intercepting DNS server returns NXDOMAIN for external domains (routing all internet access exclusively through an HTTP proxy), dirmngr fails with keyserver receive failed: No keyserver available before it ever attempts a connection.

Tools like wget and curl work in these environments because they delegate DNS resolution to the proxy via the CONNECT method when http_proxy/https_proxy environment variables are set.

This affects two code paths:

1. Python source verification (install_os_dependencies.sh) — fetching the Python release manager's GPG key from keys.openpgp.org when building Python 3.10 from source.
2. APT repository key import (common.sh → common::import_trusted_gpg) — fetching GPG keys for MariaDB, MySQL, PostgreSQL, and MSSQL APT repositories from keyserver.ubuntu.com / pgp.surf.nl.

I'm not sure how many developers of Airflow it would help to fix this - so how many developers (are forced to) work behind a corporate HTTP proxy. At least for us it would be nice to have this fixed, because it would make setting up a development environment a bit more straightforward.

Proposed Solution

Add a wget fallback after each gpg --recv-keys call. If GPG fails to fetch the key natively, the same key is fetched via HTTPS using wget and imported with gpg --import. This keeps the existing GPG-native path as the primary method (no changes for environments without proxy issues) while providing a working fallback for proxy-constrained environments.

Why the wget command differs between the two code paths

The fallback uses different URLs depending on the keyserver API:

• keys.openpgp.org (Python key) uses the VKS API, which has a clean endpoint:

  https://keys.openpgp.org/vks/v1/by-fingerprint/<fingerprint>
  

  The key fingerprint is used directly — no prefix manipulation needed.

• keyserver.ubuntu.com / pgp.surf.nl (APT repo keys) use the HKP protocol's /pks/lookup endpoint:

  https://<host>/pks/lookup?op=get&options=mr&search=<keyid>
  

  This requires:

  • Extracting the hostname from the hkps:// or hkp:// URL scheme
  • Stripping the 0x prefix from the key ID (some callers pass a 0x prefix for the key IDs, others don't) and re-adding it as a query parameter to avoid a double-0x bug
  • Verifying the import succeeded with gpg --list-keys before breaking out of the keyserver loop, because gpg --import exits 0 even on empty input

Changes

• scripts/docker/common.sh — common::import_trusted_gpg(): added wget fallback with HKP lookup after gpg --recv-keys in the keyserver loop
• scripts/docker/install_os_dependencies.sh — install_python(): added wget fallback with VKS lookup after gpg --recv-keys for the Python GPG key
• Dockerfile.ci, Dockerfile — updated inlined copies of both scripts

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  GitHub Copilot - Claude Opus 4.6

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

[jscheffl]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

What do your mean with "embedding"? That the key is cloned in Airflow repo and COPY is made into docker or that wget is used in general to download? I mean the docker is anyway built dynamically... so what does "persist" mean? Git?

[wolfdn]

Better solution would be to embed GPG key in the image. We can probably automate prek to rertrieve GPG Keys and hard-code them in the image. This is not only faster but more secure . Those keys don't change - once published and we are using their IDs, they won't change.

I like the idea - do you mean something like this? #65408

[wolfdn]

Closing in favor of #65408