Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ https://github.com/flexaccessdev/iroh-common-architecture. Do not duplicate it i
this repo; update it there and link to it.

That shared layer's code — `RelayConfig` and the relay probe, endpoint
building and rebuild, the home-relay watchdog, the endpoint-bound auth
building and rebuild, the endpoint-bound auth
transcript — lives in the `flexaccess-iroh` crate (`../flexaccess-iroh`,
consumed by git tag). Fix it there, tag a release, and bump the tag here; never
re-implement or fork a copy of it in this repo. Only ezvpn-specific pieces (the
Expand Down
6 changes: 3 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "ezvpn"
version = "0.0.46"
version = "0.0.47"
edition = "2024"
description = "IP-over-QUIC VPN tunnel via iroh P2P"
readme = "README.md"
Expand Down Expand Up @@ -32,9 +32,9 @@ flexaccess-keys = { git = "https://github.com/flexaccessdev/flexaccess-keys", ta
futures = "0.3"
ipnet = { version = "2", features = ["serde"] }
# Shared iroh transport layer (RelayConfig + per-relay probe, endpoint
# build/rebuild, home-relay watchdog, endpoint-bound auth transcript). No
# build/rebuild, endpoint-bound auth transcript). No
# `mdns` feature: ezvpn runs no local-network discovery.
flexaccess-iroh = { git = "https://github.com/flexaccessdev/flexaccess-iroh", tag = "v0.0.3" }
flexaccess-iroh = { git = "https://github.com/flexaccessdev/flexaccess-iroh", tag = "v0.0.5" }
# Redirected to the fork below via `[patch.crates-io]`, which also covers the
# shared crate's own `iroh` dependency so the graph holds a single `iroh`.
iroh = "1.1.0"
Expand Down
11 changes: 4 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -721,13 +721,10 @@ The client uses a stable per-process `device_id`, so the server normally assigns
the same IP during reconnects. Reassignment is expected mainly after server
restart or allocation state changes.

A **server** with custom relays watches its own home-relay registration: if it
has no connected home relay for 60s it re-checks the network, and if that has
not helped by 180s it rebuilds its endpoint in place (same node id) — the
in-process equivalent of a restart, so relay-only clients (the mobile apps,
anything off the LAN) are not stranded with connect timeouts until someone
restarts the service. See
[`docs/Architecture.md`](docs/Architecture.md#relay-watchdog-server-custom-relays).
Servers rely on iroh 1.1.x for relay reconnects and keep the same endpoint
during relay outages. The former server watchdog has been removed; see the
[relay recovery history and workaround](https://github.com/flexaccessdev/iroh-common-architecture/blob/9eacd43b80d867a8a4a76e3051237b854b4b0cd5/home-relay-watchdog.md)
if permanent loss of relay registration recurs.

## Relay and Address Lookup

Expand Down
54 changes: 6 additions & 48 deletions docs/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -702,54 +702,12 @@ sequenceDiagram
- Jitter: 0-500ms added to prevent thundering herd
- Counter reset: Resets to 0 after successful tunnel operation

### Relay Watchdog (Server, Custom Relays)

Implemented in the shared
[flexaccess-iroh](https://github.com/flexaccessdev/flexaccess-iroh) crate
(`flexaccess_iroh::relay_watchdog`), driven by the serve loop in
`VpnServer::run`. A custom-relay server is dialable from off the LAN only while
it is **registered on its home relay** (n0 address lookup is off; clients dial
by relay hint, and a relay forwards QUIC Initials only to endpoints connected to
it). iroh has been observed to silently lose its home relay for good after a
routine relay reconnect (relays behind Cloudflare tunnels reset idle WebSockets
roughly hourly): no dial retries, no warnings, no registration on any relay —
the server stops being reachable through the relays until the process
restarts, while LAN clients that find it over mDNS keep working and hide the
outage. Relay-only clients see connect timeouts.

The watchdog observes `Endpoint::home_relay_status()` and escalates like the
client's reconnect loop:

1. no connected home relay for `RELAY_OUTAGE_NUDGE` (60s) → log a warning and
call `Endpoint::network_change()` (forces a fresh net report and relay
re-selection — enough when only the bookkeeping went stale);
2. still none at `RELAY_OUTAGE_REBUILD` (180s from the outage start) → the
serve loop closes the endpoint (bounded by `REBUILD_CLOSE_TIMEOUT`, 5s; a
slower close finishes in the background), binds a fresh one with the
**same identity** (`server_rebuild_factory`: no per-relay probe, online
wait tolerated failing), and accepts on it. The TUN device, address pools,
client registries, and status socket carry over; the old endpoint's
connections end with it and those clients reconnect on their own. The TUN
reader's self-encapsulation filter (the endpoint's local UDP ports) is
re-read for the fresh sockets. A failed rebuild is retried every
`REBUILD_RETRY` (30s).

A rebuild only helps when iroh's bookkeeping went stale; when the relay itself
is unreachable the fresh endpoint never registers either, and rebuilding again
every three minutes would keep dropping the LAN clients that still work. The
watchdog therefore reports whether the endpoint held a home relay at any point
(`RelayOutage::relay_seen`), and the serve loop doubles the rebuild deadline
for each consecutive endpoint that never did (`rebuild_deadline`: 180s, 6m,
12m, 24m, then capped at `REBUILD_DEADLINE_MAX`, 30m). An endpoint that
registers resets the escalation to the usual 180s. The 60s nudge is unaffected.

A reconnect at any point resets the outage clock. Non-home relays are connected
on demand and dropped after a minute idle, which is normal and never counts as
an outage. With the default relays the watchdog is not armed: reachability
there rests on n0 publishing/resolution, not on one relay registration.

The watchdog is shared with flextunnel through that crate: fix it there, tag a
release, and bump the tag here.
### Server relay recovery

Servers rely on iroh 1.1.x for relay reconnects and keep the same endpoint
during relay outages. The former server watchdog has been removed; see the
[relay recovery history and workaround](https://github.com/flexaccessdev/iroh-common-architecture/blob/9eacd43b80d867a8a4a76e3051237b854b4b0cd5/home-relay-watchdog.md)
if permanent loss of relay registration recurs.

### Client Network Consistency Check (Reconnect)

Expand Down
15 changes: 3 additions & 12 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ use ezvpn::config::file_config::{
};
use ezvpn::runtime::LockRole;
use ezvpn::transport::endpoint::{
create_client_endpoint, create_server_endpoint, load_secret, server_rebuild_factory,
create_client_endpoint, create_server_endpoint, load_secret,
};
use ezvpn::transport::{
CongestionConfig, CongestionControl, parse_congestion_initial_window, set_congestion_config,
Expand Down Expand Up @@ -953,18 +953,9 @@ async fn run_vpn_server(resolved: ResolvedVpnServerConfig) -> Result<()> {
// for VPN traffic; relays are only the automatic fallback when a direct
// connection fails. A single endpoint serves both relay modes; internet
// discovery follows the mode (on for default relays, off for custom).
let endpoint = create_server_endpoint(&resolved.relay_config, secret_key.clone())
let endpoint = create_server_endpoint(&resolved.relay_config, secret_key)
.await
.context("Failed to create iroh endpoint")?;
// The relay watchdog's remedy of last resort: a fresh endpoint with the
// same identity. Only a custom-relay server hangs its reachability on one
// home-relay registration (n0 discovery is off, clients dial by relay
// hint), so the watchdog is armed for custom relays only.
let rebuild = resolved
.relay_config
.is_custom()
.then(|| server_rebuild_factory(resolved.relay_config.clone(), secret_key));

log::info!("VPN Server Node ID: {}", endpoint.id());
log::info!(
"Clients connect with: ezvpn client start --server-node-id {} --auth-key-file <KEY FILE>",
Expand All @@ -977,7 +968,7 @@ async fn run_vpn_server(resolved: ResolvedVpnServerConfig) -> Result<()> {
.context("Failed to create VPN server")?;

server
.run(endpoint, rebuild)
.run(endpoint)
.await
.map_err(|e| anyhow::anyhow!("VPN server error: {}", e))
}
Expand Down
27 changes: 4 additions & 23 deletions src/transport/endpoint.rs
Original file line number Diff line number Diff line change
@@ -1,26 +1,24 @@
//! ezvpn's endpoints: what this program layers onto the shared
//! [`flexaccess_iroh::endpoint`] builder — the VPN ALPN, its QUIC transport
//! tuning, the client/server identity rules, the bounded connect, and the
//! server's secret-key file. Relay configuration, the per-relay startup probe,
//! and the creation-vs-rebuild policy come from the shared crate.
//! server's secret-key file. Relay configuration and startup validation
//! come from the shared crate.

use crate::error::{VpnError, VpnResult};
use crate::transport::build_quic_transport_config;
use crate::tunnel::signaling::VPN_ALPN;
use anyhow::{Context, Result};
use base64::{Engine, engine::general_purpose::STANDARD as BASE64};
use flexaccess_iroh::endpoint::{
EndpointOptions, create_endpoint, endpoint_builder, rebuild_endpoint,
EndpointOptions, create_endpoint, endpoint_builder,
};
use iroh::{
Endpoint, EndpointAddr, EndpointId, SecretKey,
endpoint::{Builder as EndpointBuilder, Connection},
};
use std::path::Path;
use std::sync::Arc;
use std::time::Duration;

pub use flexaccess_iroh::endpoint::EndpointFactory;
pub use flexaccess_iroh::relay::RelayConfig;

/// Deadline for establishing the QUIC connection to the VPN server.
Expand Down Expand Up @@ -96,9 +94,7 @@ fn base_builder(relay_config: &RelayConfig, publish_address: bool) -> Result<End
}

/// A server endpoint builder: persistent identity (published on the default
/// relays) and the VPN ALPN. Binding policy is the caller's —
/// [`create_server_endpoint`] and [`server_rebuild_factory`] each layer their
/// own.
/// relays) and the VPN ALPN.
fn server_builder(relay_config: &RelayConfig, secret: SecretKey) -> Result<EndpointBuilder> {
Ok(base_builder(relay_config, true)?
.alpns(vec![VPN_ALPN.to_vec()])
Expand All @@ -118,21 +114,6 @@ pub async fn create_server_endpoint(relay_config: &RelayConfig, secret: SecretKe
create_endpoint(relay_config, server_builder(relay_config, secret)?).await
}

/// The rebuild recipe for the server endpoint, used when the relay watchdog
/// (`flexaccess_iroh::relay_watchdog`) gives up on the current one. Same
/// identity as the original, so the server's node id — what clients dial —
/// never changes. Tolerant rebuild policy (see [`rebuild_endpoint`]): no
/// relay probe, and the online wait may fail — the watchdog trips again if the
/// relays stay unreachable, with a lengthening deadline so a dead relay does
/// not churn the endpoint every few minutes (see `VpnServer::run`).
pub fn server_rebuild_factory(relay_config: RelayConfig, secret: SecretKey) -> EndpointFactory {
Arc::new(move || {
let relay_config = relay_config.clone();
let secret = secret.clone();
Box::pin(async move { rebuild_endpoint(server_builder(&relay_config, secret)?).await })
})
}

/// Create a client endpoint: ephemeral identity, never published (the client
/// only dials out; its credential is the auth keypair, not the endpoint id).
/// Strict first-creation policy.
Expand Down
Loading
Loading