x15904151985/EasyTier
1
0 Fork 0
mirror of https://github.com/EasyTier/EasyTier.git synced 2026-10-15 00:56:15 -08:00
Code Issues Packages Projects Releases Wiki Activity

feat(vpn): multi-client WireGuard portal with attached peers (#2502)

Browse Source 
* feat(peer): support protocol-agnostic attached peers

Add locally attached peers backed by independent, peer-level portable
managers and authenticated in-process ring connections. Carry trusted
connection provenance through packet admission so attached relay
privileges cannot be forged through packet headers.

Let every peer manager own ACL loading, sanitized policy updates, route
refresh, and runtime cleanup. In Secure Mode, grant attached identities
ephemeral credentials instead of sharing administrator and group secrets.

* feat(vpn): add reusable attached-peer portal runtime

Add a protocol-neutral portal runtime that converts authenticated client
sessions into attached EasyTier peers. Own per-client generations,
status, packet forwarding, address translation, and peer cleanup without
knowing the transport protocol.

Add transactional IPv4 source and destination rewriting with correct
IPv4, TCP, UDP, ICMP, and quoted-packet checksum updates. Keep the old
production portal path temporarily active until the WireGuard adapter is
migrated in the next change.

* feat(wireguard): attach named clients through peer portal

Replace the monolithic WireGuard portal with a native adapter that owns
key derivation, UDP demultiplexing, reauthentication, roaming, and
bounded per-client packet queues. Hand authenticated sessions to the
generic portal runtime for peer lifecycle and IPv4 translation.

Move portal configuration into the core instance model, require a
dedicated server key, and preserve existing listener, CLI, and runtime
configuration behavior. Reject runtime address conflicts before
publishing shared configuration.

* feat(vpn): expose per-client portal status

Project configured clients and their runtime state through the portal
RPC, including generated client configuration, listener, peer identity,
endpoint, tunnel address, ACL groups, and errors. Keep private client
configuration out of the broad instance-info response and expose the
explicit RPC through the CLI and Tauri bridge.

* feat(vpn): add portal configuration to web clients

Expose WireGuard portal listener, key, client, ACL group, and runtime
status fields in the shared frontend library, Web dashboard, and Tauri
client. Preserve UUID and uint64 values across protobuf JSON
boundaries, keep dynamic client editor rows stable, and document the
portal workflow.

* test(vpn): cover multi-client and roaming WireGuard portals

Add two three-node integration tests for the WireGuard VPN portal.

The multi-client test connects two kernel WireGuard clients from
separate network namespaces, verifies per-client connectivity to mesh
nodes, and exercises cross-client traffic that runs the IPv4 source
and destination translation in both directions. A TCP echo exchange
through the portal additionally covers the TCP pseudo-header checksum
rewrite path that ICMP-only ping tests miss, and portal status
snapshots must report both clients online with distinct peer ids and
correctly learned tunnel addresses.

The roaming test swaps the client namespace address (delete the old
address, then add the new one) so the kernel WireGuard source cache is
invalidated and the client keeps sending under the same session from
the new source, exactly like a real network change. The portal must
update the client endpoint on the same peer id via the data path
(same generation, no re-handshake, no detach/reconnect) while
connectivity to mesh nodes is preserved.

Supporting changes: run_wireguard_client now takes an interface name,
and the shared namespace topology gains net_f (10.1.2.5) on the portal
bridge for the second client.
This commit is contained in:
KKRainbow authored and GitHub committed 2026-08-21 10:59:05 +08:00
1 parent 57eb6908f4
commit 62e4fd15e9
63 files changed
+7759 -1216
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

easytier/src/common/global_ctx.rs
-13
View File
@@ -88,7 +88,6 @@ pub struct GlobalCtx {
cached_ipv4: AtomicCell<Option<cidr::Ipv4Inet>>,
cached_ipv6: AtomicCell<Option<cidr::Ipv6Inet>>,
vpn_portal_cidr: AtomicCell<Option<cidr::Ipv4Cidr>>,
hostname: Mutex<String>,
tun_device_name: Mutex<Option<String>>,
@@ -188,13 +187,6 @@ impl GlobalCtx {
let ipv6 = runtime
.map(|runtime| Self::runtime_ipv6(&runtime.peer))
.unwrap_or_else(|| config_fs.get_ipv6());
let vpn_portal_cidr = runtime
.map(|runtime| runtime.peer.vpn_portal_cidr)
.unwrap_or_else(|| {
config_fs
.get_vpn_portal_config()
.map(|config| config.client_cidr)
});
if flags.enable_encryption && effective_encryption_uses_xor(&flags.encryption_algorithm) {
tracing::warn!("using insecure XOR because no AEAD encryption is configured");
}
@@ -210,7 +202,6 @@ impl GlobalCtx {
event_bus,
cached_ipv4: AtomicCell::new(ipv4),
cached_ipv6: AtomicCell::new(ipv6),
vpn_portal_cidr: AtomicCell::new(vpn_portal_cidr),
hostname: Mutex::new(hostname),
tun_device_name: Mutex::new(None),
@@ -326,10 +317,6 @@ impl GlobalCtx {
*self.hostname.lock().unwrap() = hostname;
}
pub fn get_vpn_portal_cidr(&self) -> Option<cidr::Ipv4Cidr> {
self.vpn_portal_cidr.load()
}
pub fn get_flags(&self) -> Flags {
self.flags.load().as_ref().clone()
}
easytier/src/core.rs
+263 -19
View File
@@ -4,8 +4,8 @@ use crate::{
config::{
ConfigFileControl, ConfigLoader, ConsoleLoggerConfig, EncryptionAlgorithm,
FileLoggerConfig, LoggingConfigLoader, NetworkIdentity, PeerConfig, PortForwardConfig,
TomlConfigLoader, VpnPortalConfig, add_proxy_network_to_config, load_config_from_file,
load_toml_config_from_path, parse_mapped_listener_urls,
TomlConfigLoader, VpnPortalClientConfig, VpnPortalConfig, add_proxy_network_to_config,
load_config_from_file, load_toml_config_from_path, parse_mapped_listener_urls,
},
constants::EASYTIER_VERSION,
log,
@@ -281,6 +281,29 @@ struct NetworkOptions {
)]
vpn_portal: Option<String>,
#[arg(
long,
env = "ET_VPN_PORTAL_PRIVATE_KEY",
help = t!("core_clap.vpn_portal_private_key").to_string()
)]
vpn_portal_private_key: Option<String>,
#[arg(
long = "vpn-portal-client",
env = "ET_VPN_PORTAL_CLIENT",
value_delimiter = ',',
help = t!("core_clap.vpn_portal_client").to_string()
)]
vpn_portal_clients: Vec<String>,
#[arg(
long = "vpn-portal-client-group",
env = "ET_VPN_PORTAL_CLIENT_GROUP",
value_delimiter = ',',
help = t!("core_clap.vpn_portal_client_group").to_string()
)]
vpn_portal_client_groups: Vec<String>,
#[arg(
long,
env = "ET_DEFAULT_PROTOCOL",
@@ -864,6 +887,77 @@ impl Cli {
}
impl NetworkOptions {
fn parse_vpn_portal_listener(value: &str) -> anyhow::Result<SocketAddr> {
let url: url::Url = value
.parse()
.with_context(|| format!("failed to parse vpn portal url: {value}"))?;
if url.scheme() != "wg" {
anyhow::bail!("vpn portal URL must use the wg scheme: {value}");
}
if !url.path().is_empty() {
anyhow::bail!(
"legacy VPN portal CIDR paths are no longer supported; use wg://host:port and configure --vpn-portal-client NAME=IP"
);
}
if !url.username().is_empty()
|| url.password().is_some()
|| url.query().is_some()
|| url.fragment().is_some()
{
anyhow::bail!("vpn portal URL must have the form wg://host:port");
}
let host: IpAddr = url
.host_str()
.ok_or_else(|| anyhow::anyhow!("vpn portal url missing host"))?
.parse()
.with_context(|| "vpn portal listener host must be an IP address")?;
let port = url
.port()
.ok_or_else(|| anyhow::anyhow!("vpn portal url missing port"))?;
Ok(SocketAddr::new(host, port))
}
fn parse_vpn_portal_clients(&self) -> anyhow::Result<Vec<VpnPortalClientConfig>> {
let mut clients = self
.vpn_portal_clients
.iter()
.map(|value| {
let (name, virtual_ip) = value.split_once('=').ok_or_else(|| {
anyhow::anyhow!("invalid vpn portal client {value:?}; expected NAME=IP")
})?;
if name.is_empty() {
anyhow::bail!("vpn portal client name cannot be empty");
}
Ok(VpnPortalClientConfig {
name: name.to_owned(),
virtual_ip: virtual_ip.parse().with_context(|| {
format!("invalid virtual IP for vpn portal client {name}: {virtual_ip}")
})?,
groups: Vec::new(),
})
})
.collect::<anyhow::Result<Vec<_>>>()?;
for value in &self.vpn_portal_client_groups {
let (name, group) = value.split_once('=').ok_or_else(|| {
anyhow::anyhow!("invalid vpn portal client group {value:?}; expected NAME=GROUP")
})?;
if name.is_empty() || group.is_empty() {
anyhow::bail!("vpn portal client group name and group cannot be empty");
}
let client = clients
.iter_mut()
.find(|client| client.name == name)
.ok_or_else(|| {
anyhow::anyhow!("vpn portal client group references unknown CLI client: {name}")
})?;
client.groups.push(group.to_owned());
}
Ok(clients)
}
fn can_merge(
&self,
cfg: &TomlConfigLoader,
@@ -999,23 +1093,41 @@ impl NetworkOptions {
cfg.set_inst_name(inst_name.clone());
}
if let Some(vpn_portal) = self.vpn_portal.as_ref() {
let url: url::Url = vpn_portal
.parse()
.with_context(|| format!("failed to parse vpn portal url: {}", vpn_portal))?;
let host = url
.host_str()
.ok_or_else(|| anyhow::anyhow!("vpn portal url missing host"))?;
let port = url
.port()
.ok_or_else(|| anyhow::anyhow!("vpn portal url missing port"))?;
let client_cidr = url.path()[1..].parse().with_context(|| {
format!("failed to parse vpn portal client cidr: {}", url.path())
})?;
let wireguard_listen: SocketAddr = format!("{}:{}", host, port).parse().unwrap();
let has_vpn_portal_overrides = self.vpn_portal.is_some()
|| self.vpn_portal_private_key.is_some()
|| !self.vpn_portal_clients.is_empty()
|| !self.vpn_portal_client_groups.is_empty();
if has_vpn_portal_overrides {
if self.vpn_portal_clients.is_empty() && !self.vpn_portal_client_groups.is_empty() {
anyhow::bail!(
"--vpn-portal-client-group requires at least one --vpn-portal-client"
);
}
let existing = cfg.get_vpn_portal_config();
let wireguard_listen = match self.vpn_portal.as_deref() {
Some(value) => Self::parse_vpn_portal_listener(value)?,
None => existing
.as_ref()
.map(|portal| portal.wireguard_listen)
.ok_or_else(|| {
anyhow::anyhow!("--vpn-portal is required when no vpn_portal_config exists")
})?,
};
let wireguard_private_key = self
.vpn_portal_private_key
.clone()
.or_else(|| existing.as_ref()?.wireguard_private_key.clone());
let clients = if self.vpn_portal_clients.is_empty() {
existing.map_or_else(Vec::new, |portal| portal.clients)
} else {
self.parse_vpn_portal_clients()?
};
cfg.set_vpn_portal_config(VpnPortalConfig {
wireguard_listen,
client_cidr,
wireguard_private_key,
clients,
});
}
@@ -1497,7 +1609,7 @@ async fn run_main(cli: Cli) -> anyhow::Result<()> {
",
config_file,
control.permission,
cfg.dump()
cfg.dump_redacted()
);
manager.run_network_instance(cfg, control)?;
}
@@ -1514,7 +1626,7 @@ async fn run_main(cli: Cli) -> anyhow::Result<()> {
{}\n\
-----------------------------------\n\
",
cfg.dump()
cfg.dump_redacted()
);
manager.run_network_instance(cfg, ConfigFileControl::STATIC_CONFIG)?;
}
@@ -1826,4 +1938,136 @@ tcp_stun_servers = ["tcp.example.com:3478"]
assert_eq!(cfg.get_stun_servers_v6(), Some(Vec::new()));
assert_eq!(cfg.get_tcp_stun_servers(), Some(Vec::new()));
}
#[test]
fn vpn_portal_cli_uses_named_clients_and_preserves_unset_fields() {
let cfg = TomlConfigLoader::new_from_str(
r#"
[vpn_portal_config]
wireguard_listen = "127.0.0.1:51820"
wireguard_private_key = "existing-key"
[[vpn_portal_config.clients]]
name = "existing"
virtual_ip = "10.144.144.9"
"#,
)
.unwrap();
NetworkOptions {
vpn_portal: Some("wg://0.0.0.0:51821".to_owned()),
..Default::default()
}
.merge_into(&cfg)
.unwrap();
let preserved = cfg.get_vpn_portal_config().unwrap();
assert_eq!(preserved.wireguard_listen, "0.0.0.0:51821".parse().unwrap());
assert_eq!(
preserved.wireguard_private_key.as_deref(),
Some("existing-key")
);
assert_eq!(preserved.clients[0].name, "existing");
NetworkOptions {
vpn_portal_private_key: Some("replacement-key".to_owned()),
vpn_portal_clients: vec![
"alice=10.144.144.10".to_owned(),
"bob=10.144.144.11".to_owned(),
],
vpn_portal_client_groups: vec!["alice=staff".to_owned(), "alice=dev".to_owned()],
..Default::default()
}
.merge_into(&cfg)
.unwrap();
let replaced = cfg.get_vpn_portal_config().unwrap();
assert_eq!(replaced.wireguard_listen, "0.0.0.0:51821".parse().unwrap());
assert_eq!(
replaced.wireguard_private_key.as_deref(),
Some("replacement-key")
);
assert_eq!(
replaced
.clients
.iter()
.map(|client| client.name.as_str())
.collect::<Vec<_>>(),
vec!["alice", "bob"]
);
assert_eq!(
replaced.clients[0].groups,
vec!["staff".to_owned(), "dev".to_owned()]
);
assert!(replaced.clients[1].groups.is_empty());
}
#[test]
fn vpn_portal_cli_rejects_legacy_path_and_invalid_group_mapping() {
let error = NetworkOptions::parse_vpn_portal_listener("wg://0.0.0.0:51820/10.14.14.0/24")
.unwrap_err()
.to_string();
assert!(error.contains("legacy VPN portal CIDR"), "{error}");
let cfg = TomlConfigLoader::default();
let missing_clients = NetworkOptions {
vpn_portal: Some("wg://0.0.0.0:51820".to_owned()),
vpn_portal_client_groups: vec!["alice=staff".to_owned()],
..Default::default()
}
.merge_into(&cfg)
.unwrap_err()
.to_string();
assert!(
missing_clients.contains("requires at least one --vpn-portal-client"),
"{missing_clients}"
);
let unknown_client = NetworkOptions {
vpn_portal: Some("wg://0.0.0.0:51820".to_owned()),
vpn_portal_clients: vec!["alice=10.144.144.10".to_owned()],
vpn_portal_client_groups: vec!["bob=staff".to_owned()],
..Default::default()
}
.merge_into(&TomlConfigLoader::default())
.unwrap_err()
.to_string();
assert!(
unknown_client.contains("unknown CLI client: bob"),
"{unknown_client}"
);
}
#[test]
fn vpn_portal_cli_repeat_flags_use_singular_names() {
let cli = Cli::try_parse_from([
"easytier-core",
"--vpn-portal",
"wg://0.0.0.0:51820",
"--vpn-portal-private-key",
"private-key",
"--vpn-portal-client",
"alice=10.144.144.10",
"--vpn-portal-client",
"bob=10.144.144.11",
"--vpn-portal-client-group",
"alice=staff",
])
.unwrap();
assert_eq!(
cli.network_options.vpn_portal_clients,
vec![
"alice=10.144.144.10".to_owned(),
"bob=10.144.144.11".to_owned()
]
);
assert_eq!(
cli.network_options.vpn_portal_client_groups,
vec!["alice=staff".to_owned()]
);
assert_eq!(
cli.network_options.vpn_portal_private_key.as_deref(),
Some("private-key")
);
}
}
easytier/src/easytier-cli.rs
+29 -9
View File
@@ -2513,15 +2513,35 @@ impl<'a> CommandHandler<'a> {
self.print_results(&results, |resp| {
println!("portal_name: {}", resp.vpn_type);
println!(
r#"
############### client_config_start ###############
{}
############### client_config_end ###############
"#,
resp.client_config
);
println!("connected_clients:\n{:#?}", resp.connected_clients);
if let Some(listener) = &resp.listener {
println!("listener: {listener}");
}
for client in &resp.clients {
let state = easytier_proto::api::instance::VpnPortalClientState::try_from(
client.state,
)
.map_or("UNKNOWN", |state| state.as_str_name());
println!(
"\nclient: {}\nvirtual_ip: {}\nstate: {}\npeer_id: {}\nendpoint: {}\ntunnel_ip: {}\ngroups: {}",
client.name,
client.virtual_ip,
state,
client
.peer_id
.map(|value| value.to_string())
.unwrap_or_else(|| "-".to_owned()),
client.endpoint.as_deref().unwrap_or("-"),
client.tunnel_ip.as_deref().unwrap_or("-"),
client.groups.join(", "),
);
println!(
"############### client_config_start ###############\n{}############### client_config_end ###############",
client.client_config
);
if let Some(error) = &client.error {
println!("error: {error}");
}
}
Ok(())
})
}
easytier/src/instance/composition.rs
+7 -8
View File
@@ -3,7 +3,7 @@ use std::sync::Arc;
#[cfg(feature = "wrapped-transport")]
use easytier_core::gateway::proxy::wrapped_transport::WrappedTransportEngines;
#[cfg(feature = "wireguard")]
use easytier_core::gateway::vpn_portal::VpnPortalHost;
use easytier_core::gateway::vpn_portal::PortalHost;
#[cfg(test)]
use easytier_core::host::packet::{HostPacket, PacketSink};
#[cfg(feature = "web-client")]
@@ -255,13 +255,12 @@ fn configure_runtime_core_host_adapters(
if host_config.vpn_portal_enabled {
use crate::common::config::ConfigLoader as _;
adapters.vpn_portal = Some(crate::vpn_portal::wireguard::WireGuardPortalHost::new(
global_ctx.clone(),
global_ctx
.config
.get_vpn_portal_config()
.map(|config| config.wireguard_listen),
) as Arc<dyn VpnPortalHost>);
if let Some(config) = global_ctx.config.get_vpn_portal_config() {
adapters.vpn_portal = Some(crate::vpn_portal::wireguard::WireGuardPortalHost::new(
global_ctx.clone(),
config,
) as Arc<dyn PortalHost>);
}
}
adapters
}
easytier/src/tests/three_node.rs
+359 -14
View File
@@ -7,6 +7,8 @@ use std::{
time::Duration,
};
#[cfg(feature = "wireguard")]
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64_STANDARD};
use easytier_core::{
connectivity::protocol::raw::TunnelDialer,
foundation::stats::{LabelSet, LabelType, MetricName, MetricSnapshot},
@@ -99,7 +101,13 @@ async fn set_foreign_network_refresh_interval(inst: &Instance, seconds: u64) {
}
#[cfg(feature = "wireguard")]
use crate::{common::config::VpnPortalConfig, vpn_portal::wireguard::get_wg_config_for_portal};
use crate::{
common::config::{VpnPortalClientConfig, VpnPortalConfig},
vpn_portal::wireguard::test_wireguard_keys,
};
#[cfg(feature = "wireguard")]
use easytier_core::gateway::vpn_portal::PortalClientState;
pub fn prepare_linux_namespaces() {
del_netns("net_a");
@@ -107,12 +115,14 @@ pub fn prepare_linux_namespaces() {
del_netns("net_c");
del_netns("net_d");
del_netns("net_e");
del_netns("net_f");
create_netns("net_a", "10.1.1.1/24", "fd11::1/64");
create_netns("net_b", "10.1.1.2/24", "fd11::2/64");
create_netns("net_c", "10.1.2.3/24", "fd12::3/64");
create_netns("net_d", "10.1.2.4/24", "fd12::4/64");
create_netns("net_e", "10.1.1.3/24", "fd11::3/64");
create_netns("net_f", "10.1.2.5/24", "fd12::5/64");
prepare_bridge("br_a");
prepare_bridge("br_b");
@@ -122,6 +132,7 @@ pub fn prepare_linux_namespaces() {
add_ns_to_bridge("br_a", "net_e");
add_ns_to_bridge("br_b", "net_c");
add_ns_to_bridge("br_b", "net_d");
add_ns_to_bridge("br_b", "net_f");
}
pub fn get_inst_config(
@@ -1676,7 +1687,17 @@ use defguard_wireguard_rs::{
InterfaceConfiguration, WGApi, WireguardInterfaceApi, host::Peer, key::Key, net::IpAddrMask,
};
fn wireguard_ifname(base: &str) -> String {
if cfg!(target_os = "linux") || cfg!(target_os = "freebsd") {
base.to_owned()
} else {
"utun3".into()
}
}
#[allow(clippy::too_many_arguments)]
fn run_wireguard_client(
ifname: &str,
endpoint: SocketAddr,
peer_public_key: Key,
client_private_key: Key,
@@ -1684,12 +1705,7 @@ fn run_wireguard_client(
client_ip: String,
) -> Result<(), Box<dyn std::error::Error>> {
// Create new API object for interface
let ifname: String = if cfg!(target_os = "linux") || cfg!(target_os = "freebsd") {
"wg0".into()
} else {
"utun3".into()
};
let wgapi = WGApi::new(ifname.clone(), false)?;
let wgapi = WGApi::new(ifname.to_owned(), false)?;
// create interface
wgapi.create_interface()?;
@@ -1707,7 +1723,7 @@ fn run_wireguard_client(
// interface configuration
let interface_config = InterfaceConfiguration {
name: ifname.clone(),
name: ifname.to_owned(),
prvkey: client_private_key.to_string(),
address: client_ip,
port: 12345,
@@ -1730,10 +1746,25 @@ pub async fn wireguard_vpn_portal(#[values(true, false)] test_v6: bool) {
let insts = init_three_node_ex(
"tcp",
|config| {
let identity = config.get_network_identity();
config.set_network_identity(NetworkIdentity::new(
identity.network_name,
"wireguard-portal-test".to_owned(),
));
if config.get_inst_name() == "inst1" {
config
.add_proxy_cidr("198.51.100.0/24".parse().unwrap(), None)
.unwrap();
}
if config.get_inst_name() == "inst3" {
config.set_vpn_portal_config(VpnPortalConfig {
wireguard_listen: "0.0.0.0:22121".parse().unwrap(),
client_cidr: "10.14.14.0/24".parse().unwrap(),
wireguard_private_key: Some(BASE64_STANDARD.encode([42u8; 32])),
clients: vec![VpnPortalClientConfig {
name: "test-client".to_owned(),
virtual_ip: "10.144.144.4".parse().unwrap(),
groups: Vec::new(),
}],
});
}
config
@@ -1756,13 +1787,31 @@ pub async fn wireguard_vpn_portal(#[values(true, false)] test_v6: bool) {
let net_ns = NetNS::new(Some("net_d".into()));
let _g = net_ns.guard();
let wg_cfg = get_wg_config_for_portal(&insts[2].get_global_ctx().get_network_identity());
let portal_config = insts[2]
.get_global_ctx()
.config
.get_vpn_portal_config()
.unwrap();
let portal_info = insts[2].get_core_instance().vpn_portal_info().await;
assert_eq!(portal_info.clients.len(), 1);
let client_info = portal_info
.clients
.iter()
.find(|client| client.name == "test-client")
.expect("configured client must be reported");
assert!(
client_info.client_config.contains("198.51.100.0/24"),
"client config must include remote proxy CIDRs"
);
let (server_public, client_private) =
test_wireguard_keys(&portal_config, "test-client").unwrap();
run_wireguard_client(
&wireguard_ifname("wg0"),
dst_socket_addr,
Key::try_from(wg_cfg.my_public_key()).unwrap(),
Key::try_from(wg_cfg.peer_secret_key()).unwrap(),
vec!["10.14.14.0/24".to_string(), "10.144.144.0/24".to_string()],
"10.14.14.2".to_string(),
Key::try_from(server_public.as_slice()).unwrap(),
Key::try_from(client_private.as_slice()).unwrap(),
vec!["10.144.144.0/24".to_string()],
"192.0.2.42".to_string(),
)
.unwrap();
@@ -1788,6 +1837,302 @@ pub async fn wireguard_vpn_portal(#[values(true, false)] test_v6: bool) {
drop_insts(insts).await;
}
#[cfg(feature = "wireguard")]
#[tokio::test]
#[serial_test::serial]
pub async fn wireguard_vpn_portal_multi_client() {
use rand::Rng as _;
use tokio::{
io::{AsyncReadExt, AsyncWriteExt},
net::{TcpListener, TcpStream},
};
let insts = init_three_node_ex(
"tcp",
|config| {
let identity = config.get_network_identity();
config.set_network_identity(NetworkIdentity::new(
identity.network_name,
"wireguard-portal-multi-client-test".to_owned(),
));
if config.get_inst_name() == "inst3" {
config.set_vpn_portal_config(VpnPortalConfig {
wireguard_listen: "0.0.0.0:22121".parse().unwrap(),
wireguard_private_key: Some(BASE64_STANDARD.encode([42u8; 32])),
clients: vec![
VpnPortalClientConfig {
name: "client-a".to_owned(),
virtual_ip: "10.144.144.4".parse().unwrap(),
groups: Vec::new(),
},
VpnPortalClientConfig {
name: "client-b".to_owned(),
virtual_ip: "10.144.144.5".parse().unwrap(),
groups: Vec::new(),
},
],
});
}
config
},
false,
)
.await;
let portal_config = insts[2]
.get_global_ctx()
.config
.get_vpn_portal_config()
.unwrap();
for (ns, client_name, tunnel_ip) in [
("net_d", "client-a", "192.0.2.42"),
("net_f", "client-b", "192.0.2.43"),
] {
let net_ns = NetNS::new(Some(ns.into()));
let _g = net_ns.guard();
let (server_public, client_private) =
test_wireguard_keys(&portal_config, client_name).unwrap();
run_wireguard_client(
&wireguard_ifname("wg0"),
"10.1.2.3:22121".parse().unwrap(),
Key::try_from(server_public.as_slice()).unwrap(),
Key::try_from(client_private.as_slice()).unwrap(),
vec!["10.144.144.0/24".to_string()],
tunnel_ip.to_string(),
)
.unwrap();
}
// 两个客户端各自 ping mesh 内节点
for ns in ["net_d", "net_f"] {
wait_for_condition(
|| async { ping_test(ns, "10.144.144.1", None).await },
Duration::from_secs(10),
)
.await;
wait_for_condition(
|| async { ping_test(ns, "10.144.144.2", None).await },
Duration::from_secs(10),
)
.await;
}
// 跨客户端互 ping 对方的虚拟 IP：一次流量同时覆盖源地址改写
// （tunnel_ip -> virtual_ip）与目的地址改写（virtual_ip -> tunnel_ip），
// 回程再反向各执行一遍
wait_for_condition(
|| async { ping_test("net_d", "10.144.144.5", None).await },
Duration::from_secs(10),
)
.await;
wait_for_condition(
|| async { ping_test("net_f", "10.144.144.4", None).await },
Duration::from_secs(10),
)
.await;
// TCP 数据面：node1 侧看到的连接源地址必须是 client-a 的虚拟 IP，
// 并做一段随机数据回环，覆盖 TCP 增量校验和改写路径
let mut buf = vec![0u8; 1024];
rand::thread_rng().fill(&mut buf[..]);
let expected = buf.clone();
let echo_task = tokio::spawn(async move {
let net_ns = NetNS::new(Some("net_a".into()));
let _g = net_ns.guard();
let socket = TcpListener::bind("0.0.0.0:22222").await.unwrap();
let (mut st, addr) = socket.accept().await.unwrap();
assert_eq!(addr.ip().to_string(), "10.144.144.4".to_string());
let mut rbuf = vec![0u8; 1024];
st.read_exact(&mut rbuf).await.unwrap();
assert_eq!(rbuf, expected);
st.write_all(&rbuf).await.unwrap();
});
{
let net_ns = NetNS::new(Some("net_d".into()));
let _g = net_ns.guard();
let mut stream = TcpStream::connect("10.144.144.1:22222").await.unwrap();
stream.write_all(&buf).await.unwrap();
let mut rbuf = vec![0u8; 1024];
stream.read_exact(&mut rbuf).await.unwrap();
assert_eq!(rbuf, buf);
}
echo_task.await.unwrap();
// portal 状态：两个客户端均在线，tunnel_ip 学习正确，peer_id 互不相同
let portal_info = insts[2].get_core_instance().vpn_portal_info().await;
assert_eq!(portal_info.clients.len(), 2);
let client_a = portal_info
.clients
.iter()
.find(|client| client.name == "client-a")
.expect("client-a must be reported");
let client_b = portal_info
.clients
.iter()
.find(|client| client.name == "client-b")
.expect("client-b must be reported");
for client in [client_a, client_b] {
assert_eq!(client.state, PortalClientState::Online);
assert!(client.peer_id.is_some());
}
assert_ne!(client_a.peer_id, client_b.peer_id);
assert_eq!(client_a.tunnel_ip, Some("192.0.2.42".parse().unwrap()));
assert_eq!(client_b.tunnel_ip, Some("192.0.2.43".parse().unwrap()));
drop_insts(insts).await;
}
#[cfg(feature = "wireguard")]
#[tokio::test]
#[serial_test::serial]
pub async fn wireguard_vpn_portal_client_roaming() {
let insts = init_three_node_ex(
"tcp",
|config| {
let identity = config.get_network_identity();
config.set_network_identity(NetworkIdentity::new(
identity.network_name,
"wireguard-portal-roaming-test".to_owned(),
));
if config.get_inst_name() == "inst3" {
config.set_vpn_portal_config(VpnPortalConfig {
wireguard_listen: "0.0.0.0:22121".parse().unwrap(),
wireguard_private_key: Some(BASE64_STANDARD.encode([42u8; 32])),
clients: vec![VpnPortalClientConfig {
name: "roaming-client".to_owned(),
virtual_ip: "10.144.144.4".parse().unwrap(),
groups: Vec::new(),
}],
});
}
config
},
false,
)
.await;
let portal_config = insts[2]
.get_global_ctx()
.config
.get_vpn_portal_config()
.unwrap();
{
let net_ns = NetNS::new(Some("net_d".into()));
let _g = net_ns.guard();
let (server_public, client_private) =
test_wireguard_keys(&portal_config, "roaming-client").unwrap();
run_wireguard_client(
&wireguard_ifname("wg0"),
"10.1.2.3:22121".parse().unwrap(),
Key::try_from(server_public.as_slice()).unwrap(),
Key::try_from(client_private.as_slice()).unwrap(),
vec!["10.144.144.0/24".to_string()],
"192.0.2.42".to_string(),
)
.unwrap();
}
// 客户端在 net_d（源地址 10.1.2.4）上线
wait_for_condition(
|| async { ping_test("net_d", "10.144.144.1", None).await },
Duration::from_secs(10),
)
.await;
let peer_id = {
let info = insts[2].get_core_instance().vpn_portal_info().await;
let client = info
.clients
.iter()
.find(|client| client.name == "roaming-client")
.expect("roaming client must be reported");
assert_eq!(client.state, PortalClientState::Online);
assert!(
client
.endpoint
.as_deref()
.is_some_and(|endpoint| endpoint.starts_with("10.1.2.4:")),
"unexpected endpoint before roaming: {:?}",
client.endpoint
);
client.peer_id
};
assert!(peer_id.is_some());
// 模拟客户端换网络：net_d 把地址从 10.1.2.4 换成 10.1.2.9。内核
// WireGuard 为 peer endpoint 缓存的源地址随旧地址一起失效，客户端
// 不重建 peer、不重新握手，直接用原 session 从新源继续发数据包，
// portal 应在数据路径上更新 endpoint
for args in [
vec![
"netns".to_owned(),
"exec".to_owned(),
"net_d".to_owned(),
"ip".to_owned(),
"addr".to_owned(),
"del".to_owned(),
"10.1.2.4/24".to_owned(),
"dev".to_owned(),
get_guest_veth_name("net_d").to_owned(),
],
vec![
"netns".to_owned(),
"exec".to_owned(),
"net_d".to_owned(),
"ip".to_owned(),
"addr".to_owned(),
"add".to_owned(),
"10.1.2.9/24".to_owned(),
"dev".to_owned(),
get_guest_veth_name("net_d").to_owned(),
],
] {
let ret = std::process::Command::new("ip")
.args(&args)
.output()
.unwrap();
assert!(
ret.status.success(),
"ip {args:?} failed: {}",
String::from_utf8_lossy(&ret.stderr)
);
}
// 驱动流量（ping 经内核 WireGuard 加密后从新源发出），portal 应在
// 同一个 peer 上更新 endpoint：peer_id 不变说明是同代漫游，客户端没有掉线重连
wait_for_condition(
|| async {
ping_test("net_d", "10.144.144.1", None).await;
let info = insts[2].get_core_instance().vpn_portal_info().await;
info.clients.iter().any(|client| {
client.name == "roaming-client"
&& client.state == PortalClientState::Online
&& client.peer_id == peer_id
&& client
.endpoint
.as_deref()
.is_some_and(|endpoint| endpoint.starts_with("10.1.2.9:"))
})
},
Duration::from_secs(20),
)
.await;
// 漫游后连通性保持
wait_for_condition(
|| async { ping_test("net_d", "10.144.144.1", None).await },
Duration::from_secs(10),
)
.await;
wait_for_condition(
|| async { ping_test("net_d", "10.144.144.3", None).await },
Duration::from_secs(10),
)
.await;
drop_insts(insts).await;
}
#[cfg(feature = "wireguard")]
#[rstest::rstest]
#[tokio::test]
easytier/src/vpn_portal/wireguard.rs
+367 -90
View File
@@ -1,129 +1,406 @@
//! Native WireGuard Adapter for the VPN portal.
//!
//! One UDP socket and one shared MAC/cookie limiter demultiplex all configured
//! public keys. Per-client slots keep rekeys and endpoint roaming within one
//! authenticated generation; Core receives a new session only after the prior
//! generation has expired and been detached atomically.
mod engine;
use std::{
fmt,
net::{Ipv6Addr, SocketAddr, SocketAddrV6},
sync::Arc,
};
use anyhow::Context;
use base64::{Engine, prelude::BASE64_STANDARD};
use anyhow::Context as _;
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64_STANDARD};
use boringtun::x25519::{PublicKey, StaticSecret};
use easytier_core::{
gateway::vpn_portal::{VpnPortalClientConfigPlan, VpnPortalHost, VpnPortalListener},
socket::SocketListener,
config::toml::VpnPortalConfig,
gateway::vpn_portal::{PortalClientConfigPlan, PortalHost, PortalListener, PortalSession},
socket::{
ListenerConnectionCounter, NetNamespace, SocketContext, SocketListener,
udp::{UdpBindOptions, VirtualUdpSocket, VirtualUdpSocketFactory},
},
};
use hkdf::Hkdf;
use sha2::Sha256;
use tokio::{sync::mpsc, task::JoinSet};
use crate::{
common::{config::NetworkIdentity, global_ctx::ArcGlobalCtx},
tunnel::wireguard::{WgConfig, WgTunnelListener},
common::global_ctx::ArcGlobalCtx,
host_runtime::{NativeHostRuntime, native_host_runtime},
socket::udp::RuntimeUdpSocket,
};
pub(crate) fn get_wg_config_for_portal(nid: &NetworkIdentity) -> WgConfig {
let key_seed = format!(
"{}{}",
nid.network_name,
nid.network_secret.as_ref().unwrap_or(&String::new())
);
WgConfig::new_for_portal(&key_seed, &key_seed)
use self::engine::{DerivedClient, PortalEngine};
struct WireGuardPortalListener {
url: url::Url,
sockets: Vec<Arc<RuntimeUdpSocket>>,
receiver: mpsc::UnboundedReceiver<PortalSession>,
engine: Arc<PortalEngine>,
tasks: JoinSet<anyhow::Result<()>>,
listened: bool,
}
fn listener_endpoint(listener_url: &url::Url) -> &str {
&listener_url[url::Position::BeforeHost..url::Position::AfterPort]
}
pub struct WireGuardPortalHost {
global_ctx: ArcGlobalCtx,
wg_config: WgConfig,
listener_addr: Option<SocketAddr>,
}
impl WireGuardPortalHost {
pub fn new(global_ctx: ArcGlobalCtx, listener_addr: Option<SocketAddr>) -> Arc<Self> {
Arc::new(Self {
wg_config: get_wg_config_for_portal(&global_ctx.get_network_identity()),
global_ctx,
listener_addr,
})
}
async fn start_listener(&self, listener_addr: SocketAddr) -> anyhow::Result<VpnPortalListener> {
let mut listener_url = url::Url::parse("wg://0.0.0.0:0").unwrap();
listener_url.set_port(Some(listener_addr.port())).unwrap();
listener_url.set_ip_host(listener_addr.ip()).unwrap();
let mut listener = WgTunnelListener::new(listener_url, self.wg_config.clone());
{
let _guard = self.global_ctx.net_ns.guard();
listener
.listen()
.await
.context("failed to start WireGuard VPN portal listener")?;
}
Ok(Box::new(listener))
impl fmt::Debug for WireGuardPortalListener {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("WireGuardPortalListener")
.field("url", &self.url)
.finish_non_exhaustive()
}
}
#[async_trait::async_trait]
impl VpnPortalHost for WireGuardPortalHost {
async fn start_listeners(&self) -> anyhow::Result<Vec<VpnPortalListener>> {
let listener_addr = self.listener_addr.context("VPN portal config is not set")?;
let mut listeners = vec![self.start_listener(listener_addr).await?];
if let SocketAddr::V4(v4) = listener_addr
&& v4.ip().is_unspecified()
&& let Ok(listener) = self
.start_listener(SocketAddr::V6(SocketAddrV6::new(
Ipv6Addr::UNSPECIFIED,
v4.port(),
0,
0,
)))
.await
{
listeners.push(listener);
impl SocketListener for WireGuardPortalListener {
type Accepted = PortalSession;
async fn listen(&mut self) -> anyhow::Result<()> {
if self.listened {
return Ok(());
}
Ok(listeners)
self.listened = true;
for socket in &self.sockets {
let socket = socket.clone();
let engine = self.engine.clone();
self.tasks.spawn(async move {
loop {
let datagram = socket
.recv_session_datagram()
.await
.context("WireGuard portal UDP receive failed")?;
engine
.handle_datagram(socket.clone(), datagram.remote_addr, &datagram.payload)
.await;
}
});
}
let engine = self.engine.clone();
self.tasks.spawn(async move {
engine.run_timers().await;
Ok(())
});
Ok(())
}
async fn accept(&mut self) -> anyhow::Result<Self::Accepted> {
let tasks_active = !self.tasks.is_empty();
tokio::select! {
biased;
task = self.tasks.join_next(), if tasks_active => {
let error = match task {
Some(Ok(Err(error))) => {
anyhow::anyhow!("WireGuard portal reader stopped: {error:#}")
}
Some(Ok(Ok(()))) => {
anyhow::anyhow!("WireGuard portal task stopped unexpectedly")
}
Some(Err(error)) => {
anyhow::anyhow!("WireGuard portal task failed: {error}")
}
None => anyhow::anyhow!("WireGuard portal listener stopped"),
};
self.engine.cancel();
self.tasks.abort_all();
Err(error)
}
session = self.receiver.recv() => {
session.ok_or_else(|| anyhow::anyhow!("WireGuard portal listener stopped"))
}
}
}
fn local_url(&self) -> url::Url {
self.url.clone()
}
fn connection_counter(&self) -> Arc<dyn ListenerConnectionCounter> {
Arc::new(PortalConnectionCounter(self.engine.clone()))
}
}
impl Drop for WireGuardPortalListener {
fn drop(&mut self) {
self.engine.cancel();
self.tasks.abort_all();
}
}
struct PortalConnectionCounter(Arc<PortalEngine>);
impl fmt::Debug for PortalConnectionCounter {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.debug_struct("PortalConnectionCounter").finish()
}
}
impl ListenerConnectionCounter for PortalConnectionCounter {
fn get(&self) -> Option<u32> {
Some(self.0.connection_count())
}
}
pub struct WireGuardPortalHost {
global_ctx: ArcGlobalCtx,
config: VpnPortalConfig,
setup: Result<WireGuardPortalSetup, String>,
}
struct WireGuardPortalSetup {
server_private: [u8; 32],
server_public: PublicKey,
clients: Vec<DerivedClient>,
}
impl WireGuardPortalHost {
pub fn new(global_ctx: ArcGlobalCtx, config: VpnPortalConfig) -> Arc<Self> {
let setup = (|| -> anyhow::Result<_> {
let (master, server_private) = portal_master_and_server_key(&config)?;
let server_public = PublicKey::from(&StaticSecret::from(server_private));
let mut clients = Vec::with_capacity(config.clients.len());
for client in &config.clients {
let wireguard_private =
derive_named_key(&master, b"wireguard-client", &client.name)?;
let identity_private_key =
derive_named_key(&master, b"attached-noise", &client.name)?;
clients.push(DerivedClient {
config: client.clone(),
wireguard_private,
wireguard_public: PublicKey::from(&StaticSecret::from(wireguard_private)),
identity_private_key,
});
}
Ok(WireGuardPortalSetup {
server_private,
server_public,
clients,
})
})()
.map_err(|error| error.to_string());
Arc::new(Self {
global_ctx,
config,
setup,
})
}
async fn bind_socket(
&self,
runtime: &NativeHostRuntime,
address: SocketAddr,
only_v6: bool,
) -> anyhow::Result<Arc<RuntimeUdpSocket>> {
let context = SocketContext::default()
.with_socket_mark(self.global_ctx.get_flags().socket_mark)
.with_netns(self.global_ctx.net_ns.name().map(NetNamespace::new));
runtime
.bind_udp(
UdpBindOptions::port_bound_listener(address)
.with_context(context)
.with_only_v6(only_v6),
)
.await
}
}
fn secondary_ipv6_bind_address(address: SocketAddr, primary_port: u16) -> Option<SocketAddr> {
let SocketAddr::V4(address) = address else {
return None;
};
address
.ip()
.is_unspecified()
.then(|| SocketAddr::V6(SocketAddrV6::new(Ipv6Addr::UNSPECIFIED, primary_port, 0, 0)))
}
#[async_trait::async_trait]
impl PortalHost for WireGuardPortalHost {
async fn start_listeners(&self) -> anyhow::Result<Vec<PortalListener>> {
let setup = self
.setup
.as_ref()
.map_err(|error| anyhow::anyhow!(error.clone()))?;
let runtime = native_host_runtime();
let primary = self
.bind_socket(&runtime, self.config.wireguard_listen, false)
.await
.context("failed to bind WireGuard VPN portal")?;
let local = primary.local_addr()?;
let mut sockets = vec![primary.clone()];
if let Some(v6_address) =
secondary_ipv6_bind_address(self.config.wireguard_listen, local.port())
&& let Ok(v6) = self.bind_socket(&runtime, v6_address, true).await
{
sockets.push(v6);
}
let url = url::Url::parse(&format!("wg://{local}"))?;
let (accepted, receiver) = mpsc::unbounded_channel();
let engine = PortalEngine::new(setup.server_private, setup.clients.clone(), accepted);
Ok(vec![Box::new(WireGuardPortalListener {
url,
sockets,
receiver,
engine,
tasks: JoinSet::new(),
listened: false,
})])
}
fn name(&self) -> String {
"wireguard".to_owned()
}
fn render_client_config(&self, plan: &VpnPortalClientConfigPlan) -> String {
let listener_addr = listener_endpoint(&plan.listener_url);
fn render_client_config(&self, plan: &PortalClientConfigPlan) -> String {
let client = self
.setup
.as_ref()
.expect("client config is rendered only after successful startup")
.clients
.iter()
.find(|client| client.config.name == plan.name)
.expect("Core only renders configured clients");
let endpoint = &plan.listener_url[url::Position::BeforeHost..url::Position::AfterPort];
format!(
r#"
[Interface]
PrivateKey = {peer_secret_key}
Address = {address} # should assign an ip from this cidr manually
[Peer]
PublicKey = {my_public_key}
AllowedIPs = {allowed_ips}
Endpoint = {listener_addr} # should be the public ip(or domain) of the vpn server
PersistentKeepalive = 25
"#,
peer_secret_key = BASE64_STANDARD.encode(self.wg_config.peer_secret_key()),
my_public_key = BASE64_STANDARD.encode(self.wg_config.my_public_key()),
listener_addr = listener_addr,
allowed_ips = plan.allowed_ips.join(","),
address = plan.client_cidr.first_address().to_string() + "/32",
"[Interface]\nPrivateKey = {}\nAddress = {}/32\n\n[Peer]\nPublicKey = {}\nAllowedIPs = {}\nEndpoint = {} # replace wildcard with the public address\nPersistentKeepalive = 25\n",
BASE64_STANDARD.encode(client.wireguard_private),
plan.address,
BASE64_STANDARD.encode(
self.setup
.as_ref()
.expect("client config is rendered only after successful startup")
.server_public
.as_bytes()
),
plan.allowed_ips.join(", "),
endpoint,
)
}
}
fn not_started_client_config(&self) -> String {
"ERROR: Wireguard VPN Portal Not Started".to_owned()
}
fn portal_master_and_server_key(config: &VpnPortalConfig) -> anyhow::Result<([u8; 32], [u8; 32])> {
let encoded = config
.wireguard_private_key
.as_deref()
.filter(|key| !key.is_empty())
.ok_or_else(|| anyhow::anyhow!("WireGuard portal requires a dedicated private key"))?;
let decoded = BASE64_STANDARD
.decode(encoded)
.context("invalid base64 WireGuard portal private key")?;
let key: [u8; 32] = decoded
.try_into()
.map_err(|_| anyhow::anyhow!("WireGuard portal private key must be 32 bytes"))?;
Ok((key, key))
}
fn derive_named_key(master: &[u8; 32], domain: &[u8], name: &str) -> anyhow::Result<[u8; 32]> {
let mut context = Vec::new();
append_label(&mut context, domain);
append_label(&mut context, name.as_bytes());
let hkdf = Hkdf::<Sha256>::new(Some(b"easytier/wireguard-portal/v1"), master);
let mut output = [0u8; 32];
hkdf.expand(&context, &mut output)
.map_err(|_| anyhow::anyhow!("failed to derive portal client key"))?;
Ok(output)
}
fn append_label(output: &mut Vec<u8>, label: &[u8]) {
output.extend_from_slice(&(label.len() as u32).to_be_bytes());
output.extend_from_slice(label);
}
#[cfg(test)]
pub(crate) fn test_wireguard_keys(
config: &VpnPortalConfig,
client_name: &str,
) -> anyhow::Result<([u8; 32], [u8; 32])> {
let (master, server_private) = portal_master_and_server_key(config)?;
let client_private = derive_named_key(&master, b"wireguard-client", client_name)?;
let server_public = *PublicKey::from(&StaticSecret::from(server_private)).as_bytes();
Ok((server_public, client_private))
}
#[cfg(test)]
mod tests {
use super::listener_endpoint;
use super::*;
#[test]
fn listener_endpoint_uses_the_active_listener_url() {
fn named_keys_are_domain_separated_and_stable() {
let master = [7; 32];
let client = derive_named_key(&master, b"wireguard-client", "laptop").unwrap();
assert_eq!(
listener_endpoint(&"wg://192.0.2.10:51820".parse().unwrap()),
"192.0.2.10:51820"
client,
derive_named_key(&master, b"wireguard-client", "laptop").unwrap()
);
assert_eq!(
listener_endpoint(&"wg://[2001:db8::10]:51820".parse().unwrap()),
"[2001:db8::10]:51820"
assert_ne!(
client,
derive_named_key(&master, b"attached-noise", "laptop").unwrap()
);
assert_ne!(
client,
derive_named_key(&master, b"wireguard-client", "phone").unwrap()
);
}
#[test]
fn explicit_server_key_is_the_derivation_master() {
let key = [9; 32];
let config = VpnPortalConfig {
wireguard_listen: "127.0.0.1:51820".parse().unwrap(),
wireguard_private_key: Some(BASE64_STANDARD.encode(key)),
clients: Vec::new(),
};
assert_eq!(portal_master_and_server_key(&config).unwrap(), (key, key));
}
#[test]
fn portal_key_has_no_network_secret_fallback() {
let config = VpnPortalConfig {
wireguard_listen: "127.0.0.1:51820".parse().unwrap(),
wireguard_private_key: None,
clients: Vec::new(),
};
assert!(
portal_master_and_server_key(&config)
.unwrap_err()
.to_string()
.contains("dedicated private key")
);
}
#[test]
fn ipv6_wildcard_reuses_primary_ephemeral_port() {
assert_eq!(
secondary_ipv6_bind_address("0.0.0.0:0".parse().unwrap(), 43123),
Some("[::]:43123".parse().unwrap())
);
assert_eq!(
secondary_ipv6_bind_address("127.0.0.1:0".parse().unwrap(), 43123),
None
);
}
#[tokio::test]
async fn reader_failure_terminates_listener_accept() {
let (accepted, receiver) = mpsc::unbounded_channel();
let engine = PortalEngine::new([11; 32], Vec::new(), accepted);
let mut tasks: JoinSet<anyhow::Result<()>> = JoinSet::new();
tasks.spawn(async { anyhow::bail!("UDP receive failed") });
let mut listener = WireGuardPortalListener {
url: "wg://127.0.0.1:51820".parse().unwrap(),
sockets: Vec::new(),
receiver,
engine,
tasks,
listened: true,
};
let error = tokio::time::timeout(std::time::Duration::from_secs(1), listener.accept())
.await
.expect("listener accept ignored reader failure")
.unwrap_err();
assert!(error.to_string().contains("UDP receive failed"));
}
}
easytier/src/vpn_portal/wireguard/engine.rs
+435
View File
@@ -0,0 +1,435 @@
//! Shared WireGuard packet engine for named portal clients.
use atomic_shim::AtomicU64;
use std::{
collections::HashMap,
net::SocketAddr,
sync::{Arc, atomic::Ordering},
time::Duration,
};
use boringtun::{
noise::{
Packet, Tunn, TunnResult, errors::WireGuardError, handshake::parse_handshake_anon,
rate_limiter::RateLimiter,
},
x25519::{PublicKey, StaticSecret},
};
use easytier_core::{
config::toml::VpnPortalClientConfig, gateway::vpn_portal::PortalSession,
socket::udp::VirtualUdpSocket,
};
use tokio::{
sync::{Mutex, mpsc, watch},
task::JoinSet,
};
use tokio_util::sync::CancellationToken;
use crate::socket::udp::RuntimeUdpSocket;
const MIN_WIREGUARD_PACKET_CAPACITY: usize = 148;
// We pre-verify through this shared limiter and BoringTun verifies again inside
// each Tunn. Doubling the threshold preserves the intended 100 datagrams/s
// transition to cookies while retaining the upstream security ordering.
const DOUBLE_VERIFY_HANDSHAKE_LIMIT: u64 = 200;
const TIMER_INTERVAL: Duration = Duration::from_millis(250);
const PORTAL_PACKET_CAPACITY: usize = 128;
#[derive(Clone)]
pub(super) struct DerivedClient {
pub(super) config: VpnPortalClientConfig,
pub(super) wireguard_private: [u8; 32],
pub(super) wireguard_public: PublicKey,
pub(super) identity_private_key: [u8; 32],
}
struct PortalChannels {
endpoint: watch::Receiver<String>,
from_client: mpsc::Receiver<Vec<u8>>,
to_client: mpsc::Sender<Vec<u8>>,
}
struct ClientSession {
generation: u64,
endpoint: Option<Endpoint>,
endpoint_updates: watch::Sender<String>,
tunnel: Tunn,
from_client: mpsc::Sender<Vec<u8>>,
portal_channels: Option<PortalChannels>,
drain_capacity: usize,
tasks: JoinSet<()>,
}
struct ClientSlot {
client: DerivedClient,
index: u32,
next_generation: AtomicU64,
session: Mutex<Option<ClientSession>>,
}
#[derive(Clone)]
struct Endpoint {
socket: Arc<RuntimeUdpSocket>,
remote: SocketAddr,
}
impl ClientSession {
fn update_endpoint(&mut self, socket: Arc<RuntimeUdpSocket>, remote: SocketAddr) {
let changed = self
.endpoint
.as_ref()
.is_none_or(|endpoint| endpoint.remote != remote);
self.endpoint = Some(Endpoint { socket, remote });
if changed {
self.endpoint_updates.send_replace(remote.to_string());
}
}
}
pub(super) struct PortalEngine {
server_private: StaticSecret,
server_public: PublicKey,
rate_limiter: Arc<RateLimiter>,
by_public_key: HashMap<[u8; 32], Arc<ClientSlot>>,
by_index: HashMap<u32, Arc<ClientSlot>>,
accepted: mpsc::UnboundedSender<PortalSession>,
cancel: CancellationToken,
}
impl PortalEngine {
pub(super) fn new(
server_private: [u8; 32],
clients: Vec<DerivedClient>,
accepted: mpsc::UnboundedSender<PortalSession>,
) -> Arc<Self> {
let server_private = StaticSecret::from(server_private);
let server_public = PublicKey::from(&server_private);
let mut by_public_key = HashMap::with_capacity(clients.len());
let mut by_index = HashMap::with_capacity(clients.len());
for (offset, client) in clients.into_iter().enumerate() {
let index = u32::try_from(offset + 1).expect("client limit is below u32");
let public = *client.wireguard_public.as_bytes();
let slot = Arc::new(ClientSlot {
client,
index,
next_generation: AtomicU64::new(1),
session: Mutex::new(None),
});
by_public_key.insert(public, slot.clone());
by_index.insert(index, slot);
}
Arc::new(Self {
server_private,
server_public,
rate_limiter: Arc::new(RateLimiter::new(
&server_public,
DOUBLE_VERIFY_HANDSHAKE_LIMIT,
)),
by_public_key,
by_index,
accepted,
cancel: CancellationToken::new(),
})
}
pub(super) fn cancel(&self) {
self.cancel.cancel();
}
pub(super) fn connection_count(&self) -> u32 {
self.by_index
.values()
.filter(|slot| {
slot.session.try_lock().is_ok_and(|guard| {
guard
.as_ref()
.is_some_and(|session| session.portal_channels.is_none())
})
})
.count() as u32
}
pub(super) async fn handle_datagram(
self: &Arc<Self>,
socket: Arc<RuntimeUdpSocket>,
remote: SocketAddr,
datagram: &[u8],
) {
let mut cookie = [0u8; 148];
let parsed = match self
.rate_limiter
.verify_packet(Some(remote.ip()), datagram, &mut cookie)
{
Ok(packet) => packet,
Err(TunnResult::WriteToNetwork(reply)) => {
let _ = socket.send_to(reply, remote).await;
return;
}
Err(_) => return,
};
let slot = match &parsed {
Packet::HandshakeInit(init) => {
parse_handshake_anon(&self.server_private, &self.server_public, init)
.ok()
.and_then(|handshake| {
self.by_public_key
.get(&handshake.peer_static_public)
.cloned()
})
}
Packet::HandshakeResponse(response) => self.slot_by_receiver(response.receiver_idx),
Packet::PacketCookieReply(reply) => self.slot_by_receiver(reply.receiver_idx),
Packet::PacketData(data) => self.slot_by_receiver(data.receiver_idx),
};
let Some(slot) = slot else { return };
let mut session = slot.session.lock().await;
if session.is_none() {
if !matches!(parsed, Packet::HandshakeInit(_)) {
return;
}
*session = Some(self.new_session(&slot, socket.clone(), remote));
}
let current = session.as_mut().expect("created above");
let is_data = matches!(&parsed, Packet::PacketData(_));
let is_handshake_response = matches!(&parsed, Packet::HandshakeResponse(_));
// The shared pre-verification establishes the correct upstream order.
// Tunn::decapsulate performs a second MAC/cookie check because the
// dependency's verified-dispatch method is not public. Size the first
// output to the datagram: unauthenticated transport packets must not
// amplify a tiny allocation into a full-size IP buffer.
let mut output = vec![0u8; datagram.len().max(MIN_WIREGUARD_PACKET_CAPACITY)];
let mut result = current
.tunnel
.decapsulate(Some(remote.ip()), datagram, &mut output);
let mut first_result = true;
loop {
match result {
TunnResult::Done => {
if is_data {
current.update_endpoint(socket.clone(), remote);
self.activate_client(&slot, current);
}
current.drain_capacity = MIN_WIREGUARD_PACKET_CAPACITY;
break;
}
TunnResult::Err(WireGuardError::ConnectionExpired) => {
let expired = session.take();
drop(session);
Self::retire_session(expired);
return;
}
TunnResult::Err(_) => break,
TunnResult::WriteToNetwork(packet) => {
if (first_result && is_handshake_response && is_transport_data_packet(packet))
|| is_handshake_response_packet(packet)
{
current.update_endpoint(socket.clone(), remote);
}
let _ = socket.send_to(packet, remote).await;
// BoringTun queues a Core packet while it establishes a
// session. Its contract requires empty decapsulate calls
// after every network write until Done releases that queue.
first_result = false;
if output.len() < current.drain_capacity {
output.resize(current.drain_capacity, 0);
}
result = current.tunnel.decapsulate(None, &[], &mut output);
}
TunnResult::WriteToTunnelV4(packet, _) => {
current.update_endpoint(socket.clone(), remote);
self.activate_client(&slot, current);
match current.from_client.try_send(packet.to_vec()) {
Ok(()) => {}
Err(mpsc::error::TrySendError::Full(_)) => {
tracing::debug!(
client = %slot.client.config.name,
"dropping WireGuard packet because the client queue is full"
);
}
Err(mpsc::error::TrySendError::Closed(_)) => {
let generation = current.generation;
drop(session);
self.expire_if_current(slot, generation).await;
return;
}
}
break;
}
TunnResult::WriteToTunnelV6(_, _) => {
// Portal traffic is deliberately IPv4-only.
break;
}
}
}
}
fn activate_client(&self, slot: &ClientSlot, session: &mut ClientSession) {
let Some(channels) = session.portal_channels.take() else {
return;
};
let _ = self.accepted.send(PortalSession {
client_name: slot.client.config.name.clone(),
endpoint: channels.endpoint,
identity_private_key: slot.client.identity_private_key,
from_client: channels.from_client,
to_client: channels.to_client,
});
}
fn slot_by_receiver(&self, receiver: u32) -> Option<Arc<ClientSlot>> {
self.by_index.get(&(receiver >> 8)).cloned()
}
fn new_session(
self: &Arc<Self>,
slot: &Arc<ClientSlot>,
socket: Arc<RuntimeUdpSocket>,
remote: SocketAddr,
) -> ClientSession {
let generation = slot.next_generation.fetch_add(1, Ordering::Relaxed);
let (from_client, portal_from_client) = mpsc::channel(PORTAL_PACKET_CAPACITY);
let (portal_to_client, mut to_client) = mpsc::channel::<Vec<u8>>(PORTAL_PACKET_CAPACITY);
let (endpoint_updates, portal_endpoint) = watch::channel(remote.to_string());
let engine = Arc::downgrade(self);
let slot_for_task = Arc::downgrade(slot);
let mut tasks = JoinSet::new();
tasks.spawn(async move {
while let Some(payload) = to_client.recv().await {
let Some(engine) = engine.upgrade() else {
return;
};
let Some(slot) = slot_for_task.upgrade() else {
return;
};
engine
.encapsulate_for_client(&slot, generation, &payload)
.await;
}
if let (Some(engine), Some(slot)) = (engine.upgrade(), slot_for_task.upgrade()) {
engine.expire_if_current(slot, generation).await;
}
});
ClientSession {
generation,
endpoint: Some(Endpoint { socket, remote }),
endpoint_updates,
tunnel: Tunn::new(
self.server_private.clone(),
slot.client.wireguard_public,
None,
None,
slot.index,
Some(self.rate_limiter.clone()),
),
from_client,
portal_channels: Some(PortalChannels {
endpoint: portal_endpoint,
from_client: portal_from_client,
to_client: portal_to_client,
}),
drain_capacity: MIN_WIREGUARD_PACKET_CAPACITY,
tasks,
}
}
async fn encapsulate_for_client(
self: &Arc<Self>,
slot: &Arc<ClientSlot>,
generation: u64,
payload: &[u8],
) {
let mut output = vec![0u8; payload.len().saturating_add(148).max(148)];
let mut guard = slot.session.lock().await;
let Some(session) = guard
.as_mut()
.filter(|session| session.generation == generation)
else {
return;
};
match session.tunnel.encapsulate(payload, &mut output) {
TunnResult::WriteToNetwork(packet) => {
if is_handshake_initiation(packet) {
session.drain_capacity =
session.drain_capacity.max(payload.len().saturating_add(32));
}
if let Some(endpoint) = session.endpoint.clone() {
let _ = endpoint.socket.send_to(packet, endpoint.remote).await;
}
}
TunnResult::Done => {
session.drain_capacity =
session.drain_capacity.max(payload.len().saturating_add(32));
}
TunnResult::Err(WireGuardError::ConnectionExpired) => {
drop(guard);
self.expire_if_current(slot.clone(), generation).await;
}
_ => {}
}
}
async fn expire_if_current(self: &Arc<Self>, slot: Arc<ClientSlot>, generation: u64) {
let expired = {
let mut guard = slot.session.lock().await;
if guard
.as_ref()
.is_some_and(|session| session.generation == generation)
{
guard.take()
} else {
None
}
};
Self::retire_session(expired);
}
fn retire_session(expired: Option<ClientSession>) {
if let Some(mut expired) = expired {
expired.tasks.abort_all();
// Dropping the ring sink atomically disconnects the matching Core
// generation. A newer generation, if any, owns a different ring.
}
}
pub(super) async fn run_timers(self: Arc<Self>) {
let mut interval = tokio::time::interval(TIMER_INTERVAL);
loop {
tokio::select! {
_ = self.cancel.cancelled() => return,
_ = interval.tick() => {}
}
self.rate_limiter.reset_count();
for slot in self.by_index.values() {
let mut output = [0u8; 148];
let mut guard = slot.session.lock().await;
let Some(session) = guard.as_mut() else {
continue;
};
match session.tunnel.update_timers(&mut output) {
TunnResult::WriteToNetwork(packet) => {
if let Some(endpoint) = session.endpoint.clone() {
let _ = endpoint.socket.send_to(packet, endpoint.remote).await;
}
}
TunnResult::Err(WireGuardError::ConnectionExpired) => {
let expired = guard.take();
drop(guard);
Self::retire_session(expired);
}
_ => {}
}
}
}
}
}
fn is_handshake_initiation(packet: &[u8]) -> bool {
packet.len() == 148 && packet.get(..4) == Some(&1u32.to_le_bytes())
}
fn is_handshake_response_packet(packet: &[u8]) -> bool {
packet.len() == 92 && packet.get(..4) == Some(&2u32.to_le_bytes())
}
fn is_transport_data_packet(packet: &[u8]) -> bool {
packet.len() >= 32 && packet.get(..4) == Some(&4u32.to_le_bytes())
}