fix(server): install rustls default CryptoProvider at startup by moonming · Pull Request #34 · api7/aisix

moonming · 2026-04-24T02:47:47Z

Summary

aisix panics on first TLS handshake (etcd connect, reqwest /dp/register, etc.) with:

```
thread 'main' panicked at rustls/.../crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
```

rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (reqwest `rustls-tls`, etcd-client, tokio-rustls…), the runtime can't pick. Fix: call `aws_lc_rs::default_provider().install_default()` at the very top of `main`, before anything else.

`let _ =` because install is idempotent and the return value only signals "another crate got there first" — which doesn't affect correctness.

Why aws-lc-rs and not ring

It's the upstream rustls default as of 0.23, FIPS-capable out of the box, and already in the transitive dep graph through reqwest + etcd-client. Picking it avoids adding another RSA/EC crypto implementation to the binary.

How we caught it

AISIX-Cloud e2e stack exposed this: the DP container is `docker run` via the test harness, hits this panic within 100ms of startup, exits, and `docker run --rm` removes the container before the harness can query `docker port`. Container logs now dumped on failure.

Test plan

`cargo build --bin aisix`
`cargo fmt --check`
`cargo clippy --all-targets --locked -- -D warnings`
CI green
Re-publish `ghcr.io/moonming/ai-gateway:aisix-e2e` from main so AISIX-Cloud e2e picks it up

aisix panics on first TLS handshake (etcd connect, reqwest register call, etc.) with: thread 'main' panicked at rustls/.../crypto/mod.rs:249:14: Could not automatically determine the process-level CryptoProvider from Rustls crate features. rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (we have reqwest `rustls-tls`, etcd-client, tokio-rustls… all enabling one or the other), the runtime can't pick. The panic fires the first time any TLS operation touches the crypto layer. Call `aws_lc_rs::default_provider().install_default()` at the very top of main, before anything else loads. `let _ =` because install is idempotent and we don't care if another crate beat us to it.

Copilot

Pull request overview

Ensures aisix doesn’t panic on first TLS usage under rustls 0.23 by explicitly installing a process-wide rustls CryptoProvider at startup.

Changes:

Install the aws-lc-rs rustls CryptoProvider at the start of main() to avoid runtime provider auto-detection panics when multiple crypto backends are present via transitive dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-24T02:49:44Z

+ // already depends on transitively. Falls back to ring only if
+ // the process somehow has a provider installed already (idempotent).


The comment says this "falls back to ring" if a provider is already installed, but the code doesn't install ring or perform any fallback logic; it simply keeps whatever default provider was installed first. Consider rewording to avoid implying ring is involved (e.g., "if another provider is already installed, keep it").
Suggested change
// already depends on transitively. Falls back to ring only if
// the process somehow has a provider installed already (idempotent).
// already depends on transitively. If another provider is already
// installed for the process, `install_default()` keeps it unchanged.

…t tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides).

…40) * fix(aisix-etcd): make supervisor cache-write tests deterministic Two supervisor tests waited on the spawned cache write via `tokio::time::sleep(50ms)`. Under heavy CI load the spawn lost the race against the disk read that followed, surfacing as: resync_writes_to_disk_cache_then_restore_replays_it FAILED put_and_delete_keep_cache_in_sync FAILED Track the JoinHandle for each spawned write in a `pending_writes` Mutex<Vec<_>> on the Supervisor, and expose a test-only async `await_pending_cache_writes` that drains and awaits them. Both tests now wait on real completion instead of a wall clock. The new field is `#[cfg(test)]`-friendly via the awaiter — production code never reads it. If a handle is dropped during shutdown the underlying write either completed or was cancelled; the on-disk cache is best-effort, and the next live cycle re-publishes from etcd anyway. * fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock unit tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides). * ci: tolerate artifact upload quota errors temporarily Both `rust unit + coverage` and `build ui` jobs are currently failing on the upload-artifact step with: Failed to CreateArtifact: Artifact storage quota has been hit. Unable to upload any new artifacts. Usage is recalculated every 6-12 hours. Tests + clippy pass; only the artifact upload is blocked. Add `continue-on-error: true` to those two upload-artifact steps so the test-passing signal isn't masked by the quota issue. Downstream jobs that need ui-dist (build-bin → e2e) will fail at download-artifact when the upload was skipped; e2e is already `continue-on-error: true` at the job level, and coverage-gate is advisory. Revert this once the org-level storage usage refreshes (within 6-12h) or the quota is raised. * ci: soft-fail build-aisix while artifact storage quota persists build-aisix downloads ui-dist from build-ui. With build-ui's upload-artifact set to continue-on-error during the storage quota outage, the download fails and build-aisix errors. Since build-aisix only feeds the advisory e2e job, mark it continue-on-error too so the PR doesn't go red on a transitive dependency. Revert with the other two when storage usage refreshes.

moonming · 2026-04-24T02:47:47Z

Summary

aisix panics on first TLS handshake (etcd connect, reqwest /dp/register, etc.) with:

```
thread 'main' panicked at rustls/.../crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
```

rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (reqwest `rustls-tls`, etcd-client, tokio-rustls…), the runtime can't pick. Fix: call `aws_lc_rs::default_provider().install_default()` at the very top of `main`, before anything else.

`let _ =` because install is idempotent and the return value only signals "another crate got there first" — which doesn't affect correctness.

Why aws-lc-rs and not ring

It's the upstream rustls default as of 0.23, FIPS-capable out of the box, and already in the transitive dep graph through reqwest + etcd-client. Picking it avoids adding another RSA/EC crypto implementation to the binary.

How we caught it

AISIX-Cloud e2e stack exposed this: the DP container is `docker run` via the test harness, hits this panic within 100ms of startup, exits, and `docker run --rm` removes the container before the harness can query `docker port`. Container logs now dumped on failure.

Test plan

`cargo build --bin aisix`
`cargo fmt --check`
`cargo clippy --all-targets --locked -- -D warnings`
CI green
Re-publish `ghcr.io/moonming/ai-gateway:aisix-e2e` from main so AISIX-Cloud e2e picks it up

aisix panics on first TLS handshake (etcd connect, reqwest register call, etc.) with: thread 'main' panicked at rustls/.../crypto/mod.rs:249:14: Could not automatically determine the process-level CryptoProvider from Rustls crate features. rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (we have reqwest `rustls-tls`, etcd-client, tokio-rustls… all enabling one or the other), the runtime can't pick. The panic fires the first time any TLS operation touches the crypto layer. Call `aws_lc_rs::default_provider().install_default()` at the very top of main, before anything else loads. `let _ =` because install is idempotent and we don't care if another crate beat us to it.

Copilot

Pull request overview

Ensures aisix doesn’t panic on first TLS usage under rustls 0.23 by explicitly installing a process-wide rustls CryptoProvider at startup.

Changes:

Install the aws-lc-rs rustls CryptoProvider at the start of main() to avoid runtime provider auto-detection panics when multiple crypto backends are present via transitive dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-24T02:49:44Z

+ // already depends on transitively. Falls back to ring only if
+ // the process somehow has a provider installed already (idempotent).


The comment says this "falls back to ring" if a provider is already installed, but the code doesn't install ring or perform any fallback logic; it simply keeps whatever default provider was installed first. Consider rewording to avoid implying ring is involved (e.g., "if another provider is already installed, keep it").
Suggested change
// already depends on transitively. Falls back to ring only if
// the process somehow has a provider installed already (idempotent).
// already depends on transitively. If another provider is already
// installed for the process, `install_default()` keeps it unchanged.

…t tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides).

…40) * fix(aisix-etcd): make supervisor cache-write tests deterministic Two supervisor tests waited on the spawned cache write via `tokio::time::sleep(50ms)`. Under heavy CI load the spawn lost the race against the disk read that followed, surfacing as: resync_writes_to_disk_cache_then_restore_replays_it FAILED put_and_delete_keep_cache_in_sync FAILED Track the JoinHandle for each spawned write in a `pending_writes` Mutex<Vec<_>> on the Supervisor, and expose a test-only async `await_pending_cache_writes` that drains and awaits them. Both tests now wait on real completion instead of a wall clock. The new field is `#[cfg(test)]`-friendly via the awaiter — production code never reads it. If a handle is dropped during shutdown the underlying write either completed or was cancelled; the on-disk cache is best-effort, and the next live cycle re-publishes from etcd anyway. * fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock unit tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides). * ci: tolerate artifact upload quota errors temporarily Both `rust unit + coverage` and `build ui` jobs are currently failing on the upload-artifact step with: Failed to CreateArtifact: Artifact storage quota has been hit. Unable to upload any new artifacts. Usage is recalculated every 6-12 hours. Tests + clippy pass; only the artifact upload is blocked. Add `continue-on-error: true` to those two upload-artifact steps so the test-passing signal isn't masked by the quota issue. Downstream jobs that need ui-dist (build-bin → e2e) will fail at download-artifact when the upload was skipped; e2e is already `continue-on-error: true` at the job level, and coverage-gate is advisory. Revert this once the org-level storage usage refreshes (within 6-12h) or the quota is raised. * ci: soft-fail build-aisix while artifact storage quota persists build-aisix downloads ui-dist from build-ui. With build-ui's upload-artifact set to continue-on-error during the storage quota outage, the download fails and build-aisix errors. Since build-aisix only feeds the advisory e2e job, mark it continue-on-error too so the PR doesn't go red on a transitive dependency. Revert with the other two when storage usage refreshes.

moonming · 2026-04-24T02:47:47Z

Summary

aisix panics on first TLS handshake (etcd connect, reqwest /dp/register, etc.) with:

```
thread 'main' panicked at rustls/.../crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
```

rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (reqwest `rustls-tls`, etcd-client, tokio-rustls…), the runtime can't pick. Fix: call `aws_lc_rs::default_provider().install_default()` at the very top of `main`, before anything else.

`let _ =` because install is idempotent and the return value only signals "another crate got there first" — which doesn't affect correctness.

Why aws-lc-rs and not ring

It's the upstream rustls default as of 0.23, FIPS-capable out of the box, and already in the transitive dep graph through reqwest + etcd-client. Picking it avoids adding another RSA/EC crypto implementation to the binary.

How we caught it

AISIX-Cloud e2e stack exposed this: the DP container is `docker run` via the test harness, hits this panic within 100ms of startup, exits, and `docker run --rm` removes the container before the harness can query `docker port`. Container logs now dumped on failure.

Test plan

`cargo build --bin aisix`
`cargo fmt --check`
`cargo clippy --all-targets --locked -- -D warnings`
CI green
Re-publish `ghcr.io/moonming/ai-gateway:aisix-e2e` from main so AISIX-Cloud e2e picks it up

aisix panics on first TLS handshake (etcd connect, reqwest register call, etc.) with: thread 'main' panicked at rustls/.../crypto/mod.rs:249:14: Could not automatically determine the process-level CryptoProvider from Rustls crate features. rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (we have reqwest `rustls-tls`, etcd-client, tokio-rustls… all enabling one or the other), the runtime can't pick. The panic fires the first time any TLS operation touches the crypto layer. Call `aws_lc_rs::default_provider().install_default()` at the very top of main, before anything else loads. `let _ =` because install is idempotent and we don't care if another crate beat us to it.

Copilot

Pull request overview

Ensures aisix doesn’t panic on first TLS usage under rustls 0.23 by explicitly installing a process-wide rustls CryptoProvider at startup.

Changes:

Install the aws-lc-rs rustls CryptoProvider at the start of main() to avoid runtime provider auto-detection panics when multiple crypto backends are present via transitive dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-24T02:49:44Z

+ // already depends on transitively. Falls back to ring only if
+ // the process somehow has a provider installed already (idempotent).


The comment says this "falls back to ring" if a provider is already installed, but the code doesn't install ring or perform any fallback logic; it simply keeps whatever default provider was installed first. Consider rewording to avoid implying ring is involved (e.g., "if another provider is already installed, keep it").
Suggested change
// already depends on transitively. Falls back to ring only if
// the process somehow has a provider installed already (idempotent).
// already depends on transitively. If another provider is already
// installed for the process, `install_default()` keeps it unchanged.

…t tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides).

…40) * fix(aisix-etcd): make supervisor cache-write tests deterministic Two supervisor tests waited on the spawned cache write via `tokio::time::sleep(50ms)`. Under heavy CI load the spawn lost the race against the disk read that followed, surfacing as: resync_writes_to_disk_cache_then_restore_replays_it FAILED put_and_delete_keep_cache_in_sync FAILED Track the JoinHandle for each spawned write in a `pending_writes` Mutex<Vec<_>> on the Supervisor, and expose a test-only async `await_pending_cache_writes` that drains and awaits them. Both tests now wait on real completion instead of a wall clock. The new field is `#[cfg(test)]`-friendly via the awaiter — production code never reads it. If a handle is dropped during shutdown the underlying write either completed or was cancelled; the on-disk cache is best-effort, and the next live cycle re-publishes from etcd anyway. * fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock unit tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides). * ci: tolerate artifact upload quota errors temporarily Both `rust unit + coverage` and `build ui` jobs are currently failing on the upload-artifact step with: Failed to CreateArtifact: Artifact storage quota has been hit. Unable to upload any new artifacts. Usage is recalculated every 6-12 hours. Tests + clippy pass; only the artifact upload is blocked. Add `continue-on-error: true` to those two upload-artifact steps so the test-passing signal isn't masked by the quota issue. Downstream jobs that need ui-dist (build-bin → e2e) will fail at download-artifact when the upload was skipped; e2e is already `continue-on-error: true` at the job level, and coverage-gate is advisory. Revert this once the org-level storage usage refreshes (within 6-12h) or the quota is raised. * ci: soft-fail build-aisix while artifact storage quota persists build-aisix downloads ui-dist from build-ui. With build-ui's upload-artifact set to continue-on-error during the storage quota outage, the download fails and build-aisix errors. Since build-aisix only feeds the advisory e2e job, mark it continue-on-error too so the PR doesn't go red on a transitive dependency. Revert with the other two when storage usage refreshes.

moonming · 2026-04-24T02:47:47Z

Summary

aisix panics on first TLS handshake (etcd connect, reqwest /dp/register, etc.) with:

```
thread 'main' panicked at rustls/.../crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
```

rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (reqwest `rustls-tls`, etcd-client, tokio-rustls…), the runtime can't pick. Fix: call `aws_lc_rs::default_provider().install_default()` at the very top of `main`, before anything else.

`let _ =` because install is idempotent and the return value only signals "another crate got there first" — which doesn't affect correctness.

Why aws-lc-rs and not ring

It's the upstream rustls default as of 0.23, FIPS-capable out of the box, and already in the transitive dep graph through reqwest + etcd-client. Picking it avoids adding another RSA/EC crypto implementation to the binary.

How we caught it

AISIX-Cloud e2e stack exposed this: the DP container is `docker run` via the test harness, hits this panic within 100ms of startup, exits, and `docker run --rm` removes the container before the harness can query `docker port`. Container logs now dumped on failure.

Test plan

`cargo build --bin aisix`
`cargo fmt --check`
`cargo clippy --all-targets --locked -- -D warnings`
CI green
Re-publish `ghcr.io/moonming/ai-gateway:aisix-e2e` from main so AISIX-Cloud e2e picks it up

aisix panics on first TLS handshake (etcd connect, reqwest register call, etc.) with: thread 'main' panicked at rustls/.../crypto/mod.rs:249:14: Could not automatically determine the process-level CryptoProvider from Rustls crate features. rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (we have reqwest `rustls-tls`, etcd-client, tokio-rustls… all enabling one or the other), the runtime can't pick. The panic fires the first time any TLS operation touches the crypto layer. Call `aws_lc_rs::default_provider().install_default()` at the very top of main, before anything else loads. `let _ =` because install is idempotent and we don't care if another crate beat us to it.

Copilot

Pull request overview

Ensures aisix doesn’t panic on first TLS usage under rustls 0.23 by explicitly installing a process-wide rustls CryptoProvider at startup.

Changes:

Install the aws-lc-rs rustls CryptoProvider at the start of main() to avoid runtime provider auto-detection panics when multiple crypto backends are present via transitive dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-24T02:49:44Z

+ // already depends on transitively. Falls back to ring only if
+ // the process somehow has a provider installed already (idempotent).


The comment says this "falls back to ring" if a provider is already installed, but the code doesn't install ring or perform any fallback logic; it simply keeps whatever default provider was installed first. Consider rewording to avoid implying ring is involved (e.g., "if another provider is already installed, keep it").
Suggested change
// already depends on transitively. Falls back to ring only if
// the process somehow has a provider installed already (idempotent).
// already depends on transitively. If another provider is already
// installed for the process, `install_default()` keeps it unchanged.

…t tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides).

…40) * fix(aisix-etcd): make supervisor cache-write tests deterministic Two supervisor tests waited on the spawned cache write via `tokio::time::sleep(50ms)`. Under heavy CI load the spawn lost the race against the disk read that followed, surfacing as: resync_writes_to_disk_cache_then_restore_replays_it FAILED put_and_delete_keep_cache_in_sync FAILED Track the JoinHandle for each spawned write in a `pending_writes` Mutex<Vec<_>> on the Supervisor, and expose a test-only async `await_pending_cache_writes` that drains and awaits them. Both tests now wait on real completion instead of a wall clock. The new field is `#[cfg(test)]`-friendly via the awaiter — production code never reads it. If a handle is dropped during shutdown the underlying write either completed or was cancelled; the on-disk cache is best-effort, and the next live cycle re-publishes from etcd anyway. * fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock unit tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides). * ci: tolerate artifact upload quota errors temporarily Both `rust unit + coverage` and `build ui` jobs are currently failing on the upload-artifact step with: Failed to CreateArtifact: Artifact storage quota has been hit. Unable to upload any new artifacts. Usage is recalculated every 6-12 hours. Tests + clippy pass; only the artifact upload is blocked. Add `continue-on-error: true` to those two upload-artifact steps so the test-passing signal isn't masked by the quota issue. Downstream jobs that need ui-dist (build-bin → e2e) will fail at download-artifact when the upload was skipped; e2e is already `continue-on-error: true` at the job level, and coverage-gate is advisory. Revert this once the org-level storage usage refreshes (within 6-12h) or the quota is raised. * ci: soft-fail build-aisix while artifact storage quota persists build-aisix downloads ui-dist from build-ui. With build-ui's upload-artifact set to continue-on-error during the storage quota outage, the download fails and build-aisix errors. Since build-aisix only feeds the advisory e2e job, mark it continue-on-error too so the PR doesn't go red on a transitive dependency. Revert with the other two when storage usage refreshes.

moonming · 2026-04-24T02:47:47Z

Summary

aisix panics on first TLS handshake (etcd connect, reqwest /dp/register, etc.) with:

```
thread 'main' panicked at rustls/.../crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
```

rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (reqwest `rustls-tls`, etcd-client, tokio-rustls…), the runtime can't pick. Fix: call `aws_lc_rs::default_provider().install_default()` at the very top of `main`, before anything else.

`let _ =` because install is idempotent and the return value only signals "another crate got there first" — which doesn't affect correctness.

Why aws-lc-rs and not ring

It's the upstream rustls default as of 0.23, FIPS-capable out of the box, and already in the transitive dep graph through reqwest + etcd-client. Picking it avoids adding another RSA/EC crypto implementation to the binary.

How we caught it

AISIX-Cloud e2e stack exposed this: the DP container is `docker run` via the test harness, hits this panic within 100ms of startup, exits, and `docker run --rm` removes the container before the harness can query `docker port`. Container logs now dumped on failure.

Test plan

`cargo build --bin aisix`
`cargo fmt --check`
`cargo clippy --all-targets --locked -- -D warnings`
CI green
Re-publish `ghcr.io/moonming/ai-gateway:aisix-e2e` from main so AISIX-Cloud e2e picks it up

aisix panics on first TLS handshake (etcd connect, reqwest register call, etc.) with: thread 'main' panicked at rustls/.../crypto/mod.rs:249:14: Could not automatically determine the process-level CryptoProvider from Rustls crate features. rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (we have reqwest `rustls-tls`, etcd-client, tokio-rustls… all enabling one or the other), the runtime can't pick. The panic fires the first time any TLS operation touches the crypto layer. Call `aws_lc_rs::default_provider().install_default()` at the very top of main, before anything else loads. `let _ =` because install is idempotent and we don't care if another crate beat us to it.

Copilot

Pull request overview

Ensures aisix doesn’t panic on first TLS usage under rustls 0.23 by explicitly installing a process-wide rustls CryptoProvider at startup.

Changes:

Install the aws-lc-rs rustls CryptoProvider at the start of main() to avoid runtime provider auto-detection panics when multiple crypto backends are present via transitive dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-24T02:49:44Z

+ // already depends on transitively. Falls back to ring only if
+ // the process somehow has a provider installed already (idempotent).


The comment says this "falls back to ring" if a provider is already installed, but the code doesn't install ring or perform any fallback logic; it simply keeps whatever default provider was installed first. Consider rewording to avoid implying ring is involved (e.g., "if another provider is already installed, keep it").
Suggested change
// already depends on transitively. Falls back to ring only if
// the process somehow has a provider installed already (idempotent).
// already depends on transitively. If another provider is already
// installed for the process, `install_default()` keeps it unchanged.

…t tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides).

…40) * fix(aisix-etcd): make supervisor cache-write tests deterministic Two supervisor tests waited on the spawned cache write via `tokio::time::sleep(50ms)`. Under heavy CI load the spawn lost the race against the disk read that followed, surfacing as: resync_writes_to_disk_cache_then_restore_replays_it FAILED put_and_delete_keep_cache_in_sync FAILED Track the JoinHandle for each spawned write in a `pending_writes` Mutex<Vec<_>> on the Supervisor, and expose a test-only async `await_pending_cache_writes` that drains and awaits them. Both tests now wait on real completion instead of a wall clock. The new field is `#[cfg(test)]`-friendly via the awaiter — production code never reads it. If a handle is dropped during shutdown the underlying write either completed or was cancelled; the on-disk cache is best-effort, and the next live cycle re-publishes from etcd anyway. * fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock unit tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides). * ci: tolerate artifact upload quota errors temporarily Both `rust unit + coverage` and `build ui` jobs are currently failing on the upload-artifact step with: Failed to CreateArtifact: Artifact storage quota has been hit. Unable to upload any new artifacts. Usage is recalculated every 6-12 hours. Tests + clippy pass; only the artifact upload is blocked. Add `continue-on-error: true` to those two upload-artifact steps so the test-passing signal isn't masked by the quota issue. Downstream jobs that need ui-dist (build-bin → e2e) will fail at download-artifact when the upload was skipped; e2e is already `continue-on-error: true` at the job level, and coverage-gate is advisory. Revert this once the org-level storage usage refreshes (within 6-12h) or the quota is raised. * ci: soft-fail build-aisix while artifact storage quota persists build-aisix downloads ui-dist from build-ui. With build-ui's upload-artifact set to continue-on-error during the storage quota outage, the download fails and build-aisix errors. Since build-aisix only feeds the advisory e2e job, mark it continue-on-error too so the PR doesn't go red on a transitive dependency. Revert with the other two when storage usage refreshes.

moonming · 2026-04-24T02:47:47Z

Summary

aisix panics on first TLS handshake (etcd connect, reqwest /dp/register, etc.) with:

```
thread 'main' panicked at rustls/.../crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
```

rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (reqwest `rustls-tls`, etcd-client, tokio-rustls…), the runtime can't pick. Fix: call `aws_lc_rs::default_provider().install_default()` at the very top of `main`, before anything else.

`let _ =` because install is idempotent and the return value only signals "another crate got there first" — which doesn't affect correctness.

Why aws-lc-rs and not ring

It's the upstream rustls default as of 0.23, FIPS-capable out of the box, and already in the transitive dep graph through reqwest + etcd-client. Picking it avoids adding another RSA/EC crypto implementation to the binary.

How we caught it

AISIX-Cloud e2e stack exposed this: the DP container is `docker run` via the test harness, hits this panic within 100ms of startup, exits, and `docker run --rm` removes the container before the harness can query `docker port`. Container logs now dumped on failure.

Test plan

`cargo build --bin aisix`
`cargo fmt --check`
`cargo clippy --all-targets --locked -- -D warnings`
CI green
Re-publish `ghcr.io/moonming/ai-gateway:aisix-e2e` from main so AISIX-Cloud e2e picks it up

aisix panics on first TLS handshake (etcd connect, reqwest register call, etc.) with: thread 'main' panicked at rustls/.../crypto/mod.rs:249:14: Could not automatically determine the process-level CryptoProvider from Rustls crate features. rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (we have reqwest `rustls-tls`, etcd-client, tokio-rustls… all enabling one or the other), the runtime can't pick. The panic fires the first time any TLS operation touches the crypto layer. Call `aws_lc_rs::default_provider().install_default()` at the very top of main, before anything else loads. `let _ =` because install is idempotent and we don't care if another crate beat us to it.

Copilot

Pull request overview

Ensures aisix doesn’t panic on first TLS usage under rustls 0.23 by explicitly installing a process-wide rustls CryptoProvider at startup.

Changes:

Install the aws-lc-rs rustls CryptoProvider at the start of main() to avoid runtime provider auto-detection panics when multiple crypto backends are present via transitive dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-24T02:49:44Z

+ // already depends on transitively. Falls back to ring only if
+ // the process somehow has a provider installed already (idempotent).


The comment says this "falls back to ring" if a provider is already installed, but the code doesn't install ring or perform any fallback logic; it simply keeps whatever default provider was installed first. Consider rewording to avoid implying ring is involved (e.g., "if another provider is already installed, keep it").
Suggested change
// already depends on transitively. Falls back to ring only if
// the process somehow has a provider installed already (idempotent).
// already depends on transitively. If another provider is already
// installed for the process, `install_default()` keeps it unchanged.

…t tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides).

…40) * fix(aisix-etcd): make supervisor cache-write tests deterministic Two supervisor tests waited on the spawned cache write via `tokio::time::sleep(50ms)`. Under heavy CI load the spawn lost the race against the disk read that followed, surfacing as: resync_writes_to_disk_cache_then_restore_replays_it FAILED put_and_delete_keep_cache_in_sync FAILED Track the JoinHandle for each spawned write in a `pending_writes` Mutex<Vec<_>> on the Supervisor, and expose a test-only async `await_pending_cache_writes` that drains and awaits them. Both tests now wait on real completion instead of a wall clock. The new field is `#[cfg(test)]`-friendly via the awaiter — production code never reads it. If a handle is dropped during shutdown the underlying write either completed or was cancelled; the on-disk cache is best-effort, and the next live cycle re-publishes from etcd anyway. * fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock unit tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides). * ci: tolerate artifact upload quota errors temporarily Both `rust unit + coverage` and `build ui` jobs are currently failing on the upload-artifact step with: Failed to CreateArtifact: Artifact storage quota has been hit. Unable to upload any new artifacts. Usage is recalculated every 6-12 hours. Tests + clippy pass; only the artifact upload is blocked. Add `continue-on-error: true` to those two upload-artifact steps so the test-passing signal isn't masked by the quota issue. Downstream jobs that need ui-dist (build-bin → e2e) will fail at download-artifact when the upload was skipped; e2e is already `continue-on-error: true` at the job level, and coverage-gate is advisory. Revert this once the org-level storage usage refreshes (within 6-12h) or the quota is raised. * ci: soft-fail build-aisix while artifact storage quota persists build-aisix downloads ui-dist from build-ui. With build-ui's upload-artifact set to continue-on-error during the storage quota outage, the download fails and build-aisix errors. Since build-aisix only feeds the advisory e2e job, mark it continue-on-error too so the PR doesn't go red on a transitive dependency. Revert with the other two when storage usage refreshes.

moonming · 2026-04-24T02:47:47Z

Summary

aisix panics on first TLS handshake (etcd connect, reqwest /dp/register, etc.) with:

```
thread 'main' panicked at rustls/.../crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
```

rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (reqwest `rustls-tls`, etcd-client, tokio-rustls…), the runtime can't pick. Fix: call `aws_lc_rs::default_provider().install_default()` at the very top of `main`, before anything else.

`let _ =` because install is idempotent and the return value only signals "another crate got there first" — which doesn't affect correctness.

Why aws-lc-rs and not ring

It's the upstream rustls default as of 0.23, FIPS-capable out of the box, and already in the transitive dep graph through reqwest + etcd-client. Picking it avoids adding another RSA/EC crypto implementation to the binary.

How we caught it

AISIX-Cloud e2e stack exposed this: the DP container is `docker run` via the test harness, hits this panic within 100ms of startup, exits, and `docker run --rm` removes the container before the harness can query `docker port`. Container logs now dumped on failure.

Test plan

`cargo build --bin aisix`
`cargo fmt --check`
`cargo clippy --all-targets --locked -- -D warnings`
CI green
Re-publish `ghcr.io/moonming/ai-gateway:aisix-e2e` from main so AISIX-Cloud e2e picks it up

aisix panics on first TLS handshake (etcd connect, reqwest register call, etc.) with: thread 'main' panicked at rustls/.../crypto/mod.rs:249:14: Could not automatically determine the process-level CryptoProvider from Rustls crate features. rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (we have reqwest `rustls-tls`, etcd-client, tokio-rustls… all enabling one or the other), the runtime can't pick. The panic fires the first time any TLS operation touches the crypto layer. Call `aws_lc_rs::default_provider().install_default()` at the very top of main, before anything else loads. `let _ =` because install is idempotent and we don't care if another crate beat us to it.

Copilot

Pull request overview

Ensures aisix doesn’t panic on first TLS usage under rustls 0.23 by explicitly installing a process-wide rustls CryptoProvider at startup.

Changes:

Install the aws-lc-rs rustls CryptoProvider at the start of main() to avoid runtime provider auto-detection panics when multiple crypto backends are present via transitive dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-24T02:49:44Z

+ // already depends on transitively. Falls back to ring only if
+ // the process somehow has a provider installed already (idempotent).


The comment says this "falls back to ring" if a provider is already installed, but the code doesn't install ring or perform any fallback logic; it simply keeps whatever default provider was installed first. Consider rewording to avoid implying ring is involved (e.g., "if another provider is already installed, keep it").
Suggested change
// already depends on transitively. Falls back to ring only if
// the process somehow has a provider installed already (idempotent).
// already depends on transitively. If another provider is already
// installed for the process, `install_default()` keeps it unchanged.

…t tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides).

…40) * fix(aisix-etcd): make supervisor cache-write tests deterministic Two supervisor tests waited on the spawned cache write via `tokio::time::sleep(50ms)`. Under heavy CI load the spawn lost the race against the disk read that followed, surfacing as: resync_writes_to_disk_cache_then_restore_replays_it FAILED put_and_delete_keep_cache_in_sync FAILED Track the JoinHandle for each spawned write in a `pending_writes` Mutex<Vec<_>> on the Supervisor, and expose a test-only async `await_pending_cache_writes` that drains and awaits them. Both tests now wait on real completion instead of a wall clock. The new field is `#[cfg(test)]`-friendly via the awaiter — production code never reads it. If a handle is dropped during shutdown the underlying write either completed or was cancelled; the on-disk cache is best-effort, and the next live cycle re-publishes from etcd anyway. * fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock unit tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides). * ci: tolerate artifact upload quota errors temporarily Both `rust unit + coverage` and `build ui` jobs are currently failing on the upload-artifact step with: Failed to CreateArtifact: Artifact storage quota has been hit. Unable to upload any new artifacts. Usage is recalculated every 6-12 hours. Tests + clippy pass; only the artifact upload is blocked. Add `continue-on-error: true` to those two upload-artifact steps so the test-passing signal isn't masked by the quota issue. Downstream jobs that need ui-dist (build-bin → e2e) will fail at download-artifact when the upload was skipped; e2e is already `continue-on-error: true` at the job level, and coverage-gate is advisory. Revert this once the org-level storage usage refreshes (within 6-12h) or the quota is raised. * ci: soft-fail build-aisix while artifact storage quota persists build-aisix downloads ui-dist from build-ui. With build-ui's upload-artifact set to continue-on-error during the storage quota outage, the download fails and build-aisix errors. Since build-aisix only feeds the advisory e2e job, mark it continue-on-error too so the PR doesn't go red on a transitive dependency. Revert with the other two when storage usage refreshes.

moonming · 2026-04-24T02:47:47Z

Summary

aisix panics on first TLS handshake (etcd connect, reqwest /dp/register, etc.) with:

```
thread 'main' panicked at rustls/.../crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
```

rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (reqwest `rustls-tls`, etcd-client, tokio-rustls…), the runtime can't pick. Fix: call `aws_lc_rs::default_provider().install_default()` at the very top of `main`, before anything else.

`let _ =` because install is idempotent and the return value only signals "another crate got there first" — which doesn't affect correctness.

Why aws-lc-rs and not ring

It's the upstream rustls default as of 0.23, FIPS-capable out of the box, and already in the transitive dep graph through reqwest + etcd-client. Picking it avoids adding another RSA/EC crypto implementation to the binary.

How we caught it

AISIX-Cloud e2e stack exposed this: the DP container is `docker run` via the test harness, hits this panic within 100ms of startup, exits, and `docker run --rm` removes the container before the harness can query `docker port`. Container logs now dumped on failure.

Test plan

`cargo build --bin aisix`
`cargo fmt --check`
`cargo clippy --all-targets --locked -- -D warnings`
CI green
Re-publish `ghcr.io/moonming/ai-gateway:aisix-e2e` from main so AISIX-Cloud e2e picks it up

aisix panics on first TLS handshake (etcd connect, reqwest register call, etc.) with: thread 'main' panicked at rustls/.../crypto/mod.rs:249:14: Could not automatically determine the process-level CryptoProvider from Rustls crate features. rustls 0.23 dropped implicit provider selection — when both `aws-lc-rs` and `ring` are reachable through transitive deps (we have reqwest `rustls-tls`, etcd-client, tokio-rustls… all enabling one or the other), the runtime can't pick. The panic fires the first time any TLS operation touches the crypto layer. Call `aws_lc_rs::default_provider().install_default()` at the very top of main, before anything else loads. `let _ =` because install is idempotent and we don't care if another crate beat us to it.

Copilot

Pull request overview

Ensures aisix doesn’t panic on first TLS usage under rustls 0.23 by explicitly installing a process-wide rustls CryptoProvider at startup.

Changes:

Install the aws-lc-rs rustls CryptoProvider at the start of main() to avoid runtime provider auto-detection panics when multiple crypto backends are present via transitive dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-24T02:49:44Z

+ // already depends on transitively. Falls back to ring only if
+ // the process somehow has a provider installed already (idempotent).


The comment says this "falls back to ring" if a provider is already installed, but the code doesn't install ring or perform any fallback logic; it simply keeps whatever default provider was installed first. Consider rewording to avoid implying ring is involved (e.g., "if another provider is already installed, keep it").
Suggested change
// already depends on transitively. Falls back to ring only if
// the process somehow has a provider installed already (idempotent).
// already depends on transitively. If another provider is already
// installed for the process, `install_default()` keeps it unchanged.

…t tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides).

…40) * fix(aisix-etcd): make supervisor cache-write tests deterministic Two supervisor tests waited on the spawned cache write via `tokio::time::sleep(50ms)`. Under heavy CI load the spawn lost the race against the disk read that followed, surfacing as: resync_writes_to_disk_cache_then_restore_replays_it FAILED put_and_delete_keep_cache_in_sync FAILED Track the JoinHandle for each spawned write in a `pending_writes` Mutex<Vec<_>> on the Supervisor, and expose a test-only async `await_pending_cache_writes` that drains and awaits them. Both tests now wait on real completion instead of a wall clock. The new field is `#[cfg(test)]`-friendly via the awaiter — production code never reads it. If a handle is dropped during shutdown the underlying write either completed or was cancelled; the on-disk cache is best-effort, and the next live cycle re-publishes from etcd anyway. * fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock unit tests CI on main has been red since #34 because aisix-core's Config crate loader merges every AISIX_-prefixed env var into the root Config struct (config-rs Environment::with_prefix("AISIX")), and Config has #[serde(deny_unknown_fields)]. The redis integration test sets AISIX_REDIS_URL on the rust-unit job, which leaks into every Config::load_from_path call as `redis_url` and panics: Config("deserialize: unknown field `redis_url`, expected one of `etcd`, `proxy`, `admin`, `observability`, `cache`, `managed`") 8 of 9 aisix-core::config::tests fail (the one that doesn't is rejects_unknown_fields, which intentionally swallows the error). Rename the env var so it doesn't sit under the AISIX_ prefix at all. crates/aisix-cache/tests/redis_integration.rs reads CACHE_TEST_REDIS_URL; CI sets the same. docs/testing.md + crates/aisix-cache/src/redis.rs comment updated to match. Verified: with the rename, all 9 config tests pass even with CACHE_TEST_REDIS_URL set; reproducing with the old AISIX_REDIS_URL still fails as expected (so the loader behaviour is unchanged for real AISIX_-prefixed env overrides). * ci: tolerate artifact upload quota errors temporarily Both `rust unit + coverage` and `build ui` jobs are currently failing on the upload-artifact step with: Failed to CreateArtifact: Artifact storage quota has been hit. Unable to upload any new artifacts. Usage is recalculated every 6-12 hours. Tests + clippy pass; only the artifact upload is blocked. Add `continue-on-error: true` to those two upload-artifact steps so the test-passing signal isn't masked by the quota issue. Downstream jobs that need ui-dist (build-bin → e2e) will fail at download-artifact when the upload was skipped; e2e is already `continue-on-error: true` at the job level, and coverage-gate is advisory. Revert this once the org-level storage usage refreshes (within 6-12h) or the quota is raised. * ci: soft-fail build-aisix while artifact storage quota persists build-aisix downloads ui-dist from build-ui. With build-ui's upload-artifact set to continue-on-error during the storage quota outage, the download fails and build-aisix errors. Since build-aisix only feeds the advisory e2e job, mark it continue-on-error too so the PR doesn't go red on a transitive dependency. Revert with the other two when storage usage refreshes.

CopilotAI review requested due to automatic review settings April 24, 2026 02:47

Copilotstarted reviewing on behalf of moonming April 24, 2026 02:48 View session

CopilotAI reviewed Apr 24, 2026
View reviewed changes

moonming merged commit bc06934 into mainApr 24, 2026
8 of 10 checks passed

moonming mentioned this pull request Apr 25, 2026
fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock CI #39
Closed
3 tasks

jarvis9443 deleted the fix/rustls-default-provider branch June 25, 2026 06:26

CopilotAI review requested due to automatic review settings April 24, 2026 02:47

Copilotstarted reviewing on behalf of moonming April 24, 2026 02:48 View session

CopilotAI reviewed Apr 24, 2026
View reviewed changes

moonming merged commit bc06934 into mainApr 24, 2026
8 of 10 checks passed

moonming mentioned this pull request Apr 25, 2026
fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock CI #39
Closed
3 tasks

jarvis9443 deleted the fix/rustls-default-provider branch June 25, 2026 06:26

CopilotAI review requested due to automatic review settings April 24, 2026 02:47

Copilotstarted reviewing on behalf of moonming April 24, 2026 02:48 View session

CopilotAI reviewed Apr 24, 2026
View reviewed changes

moonming merged commit bc06934 into mainApr 24, 2026
8 of 10 checks passed

moonming mentioned this pull request Apr 25, 2026
fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock CI #39
Closed
3 tasks

jarvis9443 deleted the fix/rustls-default-provider branch June 25, 2026 06:26

CopilotAI review requested due to automatic review settings April 24, 2026 02:47

Copilotstarted reviewing on behalf of moonming April 24, 2026 02:48 View session

CopilotAI reviewed Apr 24, 2026
View reviewed changes

moonming merged commit bc06934 into mainApr 24, 2026
8 of 10 checks passed

moonming mentioned this pull request Apr 25, 2026
fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock CI #39
Closed
3 tasks

jarvis9443 deleted the fix/rustls-default-provider branch June 25, 2026 06:26

CopilotAI review requested due to automatic review settings April 24, 2026 02:47

Copilotstarted reviewing on behalf of moonming April 24, 2026 02:48 View session

CopilotAI reviewed Apr 24, 2026
View reviewed changes

moonming merged commit bc06934 into mainApr 24, 2026
8 of 10 checks passed

moonming mentioned this pull request Apr 25, 2026
fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock CI #39
Closed
3 tasks

jarvis9443 deleted the fix/rustls-default-provider branch June 25, 2026 06:26

CopilotAI review requested due to automatic review settings April 24, 2026 02:47

Copilotstarted reviewing on behalf of moonming April 24, 2026 02:48 View session

CopilotAI reviewed Apr 24, 2026
View reviewed changes

moonming merged commit bc06934 into mainApr 24, 2026
8 of 10 checks passed

moonming mentioned this pull request Apr 25, 2026
fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock CI #39
Closed
3 tasks

jarvis9443 deleted the fix/rustls-default-provider branch June 25, 2026 06:26

CopilotAI review requested due to automatic review settings April 24, 2026 02:47

Copilotstarted reviewing on behalf of moonming April 24, 2026 02:48 View session

CopilotAI reviewed Apr 24, 2026
View reviewed changes

moonming merged commit bc06934 into mainApr 24, 2026
8 of 10 checks passed

moonming mentioned this pull request Apr 25, 2026
fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock CI #39
Closed
3 tasks

jarvis9443 deleted the fix/rustls-default-provider branch June 25, 2026 06:26

CopilotAI review requested due to automatic review settings April 24, 2026 02:47

Copilotstarted reviewing on behalf of moonming April 24, 2026 02:48 View session

CopilotAI reviewed Apr 24, 2026
View reviewed changes

moonming merged commit bc06934 into mainApr 24, 2026
8 of 10 checks passed

moonming mentioned this pull request Apr 25, 2026
fix(ci): rename AISIX_REDIS_URL → CACHE_TEST_REDIS_URL to unblock CI #39
Closed
3 tasks

jarvis9443 deleted the fix/rustls-default-provider branch June 25, 2026 06:26

		// already depends on transitively. Falls back to ring only if
		// the process somehow has a provider installed already (idempotent).

Uh oh!

Conversation

moonming commented Apr 24, 2026

Summary

Why aws-lc-rs and not ring

How we caught it

Test plan

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

CopilotAIApr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

moonming commented Apr 24, 2026

Summary

Why aws-lc-rs and not ring

How we caught it

Test plan

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

CopilotAIApr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

moonming commented Apr 24, 2026

Summary

Why aws-lc-rs and not ring

How we caught it

Test plan

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

CopilotAIApr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

moonming commented Apr 24, 2026

Summary

Why aws-lc-rs and not ring

How we caught it

Test plan

Uh oh!