feat: add shared virtual nic (#2352)

* Add shared virtual NIC core with per-member routing

Share a named TUN across instances while keeping unnamed devices
dedicated. Track member addresses, routes, and MTU, and reject
overlapping destinations except the common Magic DNS host route.

Dispatch packets between the physical TUN and member tunnels. Keep
IPv4 and ordinary IPv6 source translation for mobile wrong-source
traffic. Update platform configuration and runtime host integration,
with unit and network namespace coverage for routing and lifecycle.

Use smoltcp for packet parsing and checksum updates. Keep core CI
matrix jobs independent.

Reserve native inbound flows before source translations allocate their
wire tuples. Keep active reservations under table pressure and leave
forwarded subnet traffic outside the local flow table.

Complete interface claim updates on the shared runtime and retain member
registrations until the transaction commits. Keep address, route, and
MTU cleanup ordered after cancelled callers, including device
initialization.

Project mobile VPN addresses, routes, and DNS from admitted native
members. Prefer unchanged attached claims and report conflicting members
individually. Apply each member's complete initial claims before opening
the shared fd, so conflicts leave healthy members usable.

Cover inbound IPv4/IPv6 collisions, cancellation during native changes
and dispatcher acknowledgments, admission priority, and atomic mobile
claim rejection with regression tests.

Use scoped TestNetwork fixtures for shared TUN bridge setup and cleanup.
Resolve public IPv6 namespace aliases through the lab's actual names so
the new tests remain isolated under parallel upstream test runs.

* Configure one mobile VPN from running TUN instances

Use admitted native members to configure VpnService and attach its
single fd to the core shared TUN. Wait for attachment completion and
record the actual attached members before refreshing VPN state. Report
rejected members in the GUI and preserve the VPN for unchanged healthy
members.

Reconcile ordinary configuration notifications without invalidating an
arbitrary member. Invalidate attachments on instance lifecycle changes,
including restarts that reuse an instance ID. Forward public IPv6
changes alongside proxy route changes to refresh the platform VPN
configuration.

Exclude confirmed IPv4-less members from readiness gating while
retaining retries for unavailable status and pending addresses. Quick
Settings stops all running TUN members and restores that group on the
next start, skipping removed configurations and no_tun networks.

Keep the FFI set_tun_fd call scoped to one instance and report
attachment errors. Cover mobile configuration, member rejection and
replacement, attachment completion, collection notifications, IPv4-less
members, and tile stop/start restoration in GUI regression tests.

Validate with 36 mobile VPN tests, vue-tsc --noEmit, Android GUI cargo
check, workspace clippy, and shared TUN network namespace tests.

* fix(tun): preserve shared NIC ownership and failure invariants

Keep mobile address and route claims in dispatcher tables without
applying desktop interface configuration to synthetic TUN names.

Limit the Magic DNS overlap exemption to route claims and remove the
shared route retry that bypassed claim accounting after invalidation.

Invalidate and stop dispatchers on failed mobile TUN replacement or I/O.
Bound TUN writes so stalled devices release pending source updates and
member cleanup instead of retaining an unusable shared dispatcher.

Treat missing Darwin and Windows addresses and routes as successful
deletions while preserving permission and other configuration errors.

Add regressions for claim conflicts, mobile configuration, failed TUN
replacement, stalled writes, and platform deletion error handling.

* fix(tun): forward single-member shared NIC traffic unchanged

Deliver TUN packets directly to the sole attached member without
destination matching or creating source translations. Keep native flow
reservations so a later member cannot collide with existing connections,
and continue applying mappings created before other members leave.

Reuse flow preparation with the original source address. Preserve the
desktop subnet forwarding bypass, including when the flow table is full.
IPv4 fragments and IPv6 extension headers remain unchanged on flows that
need no translation.

Cover unknown sources and destinations, first and later IP fragments,
native flows established before another member joins, and retained
translations after returning to one member. Document the forwarding
behavior and retain rejection coverage for multiple attached members.

* fix(tun): preserve wildcard UDP replies on shared member ports

Mark flow reservations created by incoming traffic separately from
outgoing mappings. When a wildcard UDP socket replies with another
member's source address, reuse the incoming reservation after source
correction and preserve its service port and inbound destination.

Keep exact outgoing mappings and outgoing flow collision handling ahead
of this rule. Incoming UDP sessions take priority when packet tuples
cannot distinguish a reply from a new client using identical endpoints.

Cover IPv4 and IPv6 replies, checksum updates, repeated inbound traffic,
existing outgoing mappings, TCP collisions, and single-to-shared member
transitions with regression tests.

* test(tun): wait for IPv6 activation in mobile configuration test

Assert that the configured IPv6 address is registered before forwarding
starts, then wait up to five seconds for the kernel to make it usable.
Netlink acknowledges address creation before asynchronous IPv6
activation completes, so an immediate bind can fail under CI
parallelism.

Disable automatic link-local address generation in the test namespace to
keep startup IPv6 control traffic out of member-routing assertions.

Validation: the original test failed 40 of 320 runs at 32-way
parallelism. The updated test passed all 320 runs; workspace fmt and
clippy also pass.
This commit is contained in:
KKRainbow authored and GitHub committed 2026-10-13 21:00:03 +08:00
1 parent 99fce327d5
commit 32dc7f1f40
42 files changed
+10052 -702

No files matched your search

@@ -3,9 +3,9 @@ use easytier_core::instance::CorePacketPlane;
#[cfg(feature = "magic-dns")]
use tokio_util::{sync::CancellationToken, task::AbortOnDropHandle};
use crate::common::global_ctx::ArcGlobalCtx;
#[cfg(feature = "magic-dns")]
use crate::instance::dns_server::{MAGIC_DNS_FAKE_IP, runner::DnsRunner};
use crate::{common::global_ctx::ArcGlobalCtx, instance::virtual_nic::NicBackend};
#[derive(Default)]
pub(super) struct MagicDnsRuntime {
@@ -26,6 +26,7 @@ impl MagicDnsRuntime {
packet_plane: std::sync::Arc<CorePacketPlane>,
tun_dev: Option<String>,
tun_ip: Ipv4Inet,
shared_route_backend: Option<NicBackend>,
) -> Self {
let active = global_ctx.get_flags().accept_dns.then(|| {
let mut runner = DnsRunner::new(
@@ -34,7 +35,8 @@ impl MagicDnsRuntime {
tun_dev,
tun_ip,
MAGIC_DNS_FAKE_IP.parse().unwrap(),
);
)
.with_shared_route_backend(shared_route_backend);
let cancel = CancellationToken::new();
let task_cancel = cancel.clone();
let task = tokio::spawn(async move {
@@ -54,6 +56,7 @@ impl MagicDnsRuntime {
_packet_plane: std::sync::Arc<CorePacketPlane>,
_tun_dev: Option<String>,
_tun_ip: Ipv4Inet,
_shared_route_backend: Option<NicBackend>,
) -> Self {
Self::default()
}
@@ -3,11 +3,43 @@ use std::{
sync::{Arc, OnceLock},
};
use easytier_core::host::packet::HostPacketReceiver;
use easytier_core::{host::packet::HostPacketReceiver, instance::CorePacketPlane};
use tokio::{sync::Mutex, task::JoinSet};
use super::MagicDnsRuntime;
use crate::instance::virtual_nic::NicCtx;
use crate::{
common::{error::Error, global_ctx::ArcGlobalCtx},
instance::{shared_virtual_nic::ArcSharedVirtualNicRegistry, virtual_nic::NicCtx},
};
pub(super) async fn create_nic_ctx(
global_ctx: ArcGlobalCtx,
packet_plane: Arc<CorePacketPlane>,
receiver: Arc<Mutex<HostPacketReceiver>>,
close_notifier: Arc<tokio::sync::Notify>,
registry: ArcSharedVirtualNicRegistry,
) -> Result<NicCtx, Error> {
#[cfg(not(mobile))]
if global_ctx.get_flags().dev_name.is_empty() {
return Ok(NicCtx::new(
global_ctx,
packet_plane,
receiver,
close_notifier,
));
}
let member_id = global_ctx.get_id();
NicCtx::new_shared(
global_ctx,
packet_plane,
receiver,
close_notifier,
registry,
member_id,
)
.await
}
struct NicCtxContainer {
_nic_ctx: Option<Box<dyn Any + Send>>,
@@ -65,6 +97,15 @@ impl TunNicState {
drop(old);
}
#[cfg(mobile)]
pub(super) async fn mobile_tun_claims(
&self,
) -> Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims> {
let nic = self.nic_ctx.lock().await;
let nic = nic.as_ref()?._nic_ctx.as_ref()?.downcast_ref::<NicCtx>()?;
nic.current_mobile_tun_claims().await
}
pub(super) async fn drain(&self) {
self.stop().await;
let receiver = self.receiver();
@@ -14,28 +14,37 @@ use tokio::{
};
use tokio_util::sync::CancellationToken;
use super::{MagicDnsRuntime, tun_common::TunNicState};
use super::{
MagicDnsRuntime,
tun_common::{TunNicState, create_nic_ctx},
};
use crate::{
common::{
error::Error,
global_ctx::{ArcGlobalCtx, GlobalCtxEvent},
},
instance::virtual_nic::NicCtx,
instance::shared_virtual_nic::ArcSharedVirtualNicRegistry,
};
pub(super) struct NativeTunRuntime {
global_ctx: ArcGlobalCtx,
cancel: CancellationToken,
nic: TunNicState,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
static_ip_task: Mutex<Option<JoinHandle<()>>>,
}
impl NativeTunRuntime {
pub(super) fn new(global_ctx: ArcGlobalCtx, cancel: CancellationToken) -> Self {
pub(super) fn new(
global_ctx: ArcGlobalCtx,
cancel: CancellationToken,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
) -> Self {
Self {
global_ctx,
cancel,
nic: TunNicState::empty(),
shared_virtual_nic_registry,
static_ip_task: Mutex::new(None),
}
}
@@ -67,6 +76,7 @@ impl NativeTunRuntime {
let cancel = self.cancel.clone();
let global_ctx = self.global_ctx.clone();
let receiver = self.nic.receiver();
let shared_virtual_nic_registry = self.shared_virtual_nic_registry.clone();
let (output, first_round) = oneshot::channel();
let task = tokio::spawn(async move {
let mut output = Some(output);
@@ -76,12 +86,29 @@ impl NativeTunRuntime {
return;
}
let closed = Arc::new(Notify::new());
let mut nic = NicCtx::new(
let mut nic = match create_nic_ctx(
global_ctx.clone(),
packet_plane.clone(),
receiver.clone(),
closed.clone(),
);
shared_virtual_nic_registry.clone(),
)
.await
{
Ok(nic) => nic,
Err(error) => {
if let Some(output) = output.take() {
let _ = output.send(Err(error));
return;
}
tracing::error!(?error, "failed to create native interface context");
tokio::select! {
_ = cancel.cancelled() => return,
_ = tokio::time::sleep(Duration::from_secs(1)) => {}
}
continue;
}
};
let result = tokio::select! {
biased;
_ = cancel.cancelled() => {
@@ -104,11 +131,13 @@ impl NativeTunRuntime {
}
let magic_dns = if let Some(ip) = ipv4 {
let shared_route_backend = nic.shared_route_backend_for_dns();
MagicDnsRuntime::start(
global_ctx.clone(),
packet_plane.clone(),
nic.ifname().await,
ip,
shared_route_backend,
)
} else {
MagicDnsRuntime::default()
@@ -161,6 +190,7 @@ impl NativeTunRuntime {
nic: self.nic.clone(),
closed: Arc::new(Notify::new()),
packet_plane,
shared_virtual_nic_registry: self.shared_virtual_nic_registry.clone(),
})
}
}
@@ -172,6 +202,7 @@ struct NativeDhcpIpv4Host {
nic: TunNicState,
closed: Arc<Notify>,
packet_plane: Arc<CorePacketPlane>,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
}
impl NativeDhcpIpv4Host {
@@ -199,21 +230,25 @@ impl NativeDhcpIpv4Host {
return Ok(Some(ip));
}
let mut nic = NicCtx::new(
let mut nic = create_nic_ctx(
self.global_ctx.clone(),
self.packet_plane.clone(),
self.nic.receiver(),
self.closed.clone(),
);
self.shared_virtual_nic_registry.clone(),
)
.await?;
tokio::select! {
_ = self.cancel.cancelled() => anyhow::bail!("instance is closing; DHCP update cancelled"),
result = nic.run(Some(ip), self.global_ctx.get_ipv6()) => result?,
}
let shared_route_backend = nic.shared_route_backend_for_dns();
let magic_dns = MagicDnsRuntime::start(
self.global_ctx.clone(),
self.packet_plane.clone(),
nic.ifname().await,
ip,
shared_route_backend,
);
self.nic.install(nic, magic_dns).await;
self.global_ctx.set_ipv4(Some(ip));
+111 -24
View File
@@ -8,26 +8,41 @@ use easytier_core::{
instance::CorePacketPlane,
};
use futures::FutureExt as _;
use tokio::sync::{Mutex, Notify, mpsc};
use tokio::sync::{Mutex, Notify, mpsc, oneshot};
use tokio_util::sync::CancellationToken;
use super::{MagicDnsRuntime, tun_common::TunNicState};
use super::{
MagicDnsRuntime,
tun_common::{TunNicState, create_nic_ctx},
};
use crate::{
common::global_ctx::{ArcGlobalCtx, GlobalCtxEvent},
instance::virtual_nic::NicCtx,
instance::shared_virtual_nic::ArcSharedVirtualNicRegistry,
};
struct MobileTunAttachment {
fd: i32,
replace_tun_fd: bool,
completion: Option<oneshot::Sender<anyhow::Result<()>>>,
claims: Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims>,
}
pub(super) struct NativeTunRuntime {
global_ctx: ArcGlobalCtx,
cancel: CancellationToken,
nic: TunNicState,
tun_fd: mpsc::Sender<i32>,
tun_fd_receiver: Mutex<Option<mpsc::Receiver<i32>>>,
tun_fd: mpsc::Sender<MobileTunAttachment>,
tun_fd_receiver: Mutex<Option<mpsc::Receiver<MobileTunAttachment>>>,
task: Mutex<Option<tokio::task::JoinHandle<()>>>,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
}
impl NativeTunRuntime {
pub(super) fn new(global_ctx: ArcGlobalCtx, cancel: CancellationToken) -> Self {
pub(super) fn new(
global_ctx: ArcGlobalCtx,
cancel: CancellationToken,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
) -> Self {
let (tun_fd, tun_fd_receiver) = mpsc::channel(16);
Self {
global_ctx,
@@ -36,6 +51,7 @@ impl NativeTunRuntime {
tun_fd,
tun_fd_receiver: Mutex::new(Some(tun_fd_receiver)),
task: Mutex::new(None),
shared_virtual_nic_registry,
}
}
@@ -51,23 +67,39 @@ impl NativeTunRuntime {
global_ctx: ArcGlobalCtx,
packet_plane: Arc<CorePacketPlane>,
fd: i32,
replace_tun_fd: bool,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
claims: Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims>,
) -> anyhow::Result<()> {
nic_state.drain().await;
if fd <= 0 {
return Ok(());
}
let closed = Arc::new(Notify::new());
let mut nic = NicCtx::new(
let mut nic = create_nic_ctx(
global_ctx.clone(),
packet_plane.clone(),
nic_state.receiver(),
closed,
);
nic.run_for_mobile(fd).await.context("add ip failed")?;
let magic_dns = global_ctx
.get_ipv4()
.map(|ip| MagicDnsRuntime::start(global_ctx, packet_plane, None, ip))
.unwrap_or_default();
shared_virtual_nic_registry,
)
.await?;
let claims = match claims {
Some(claims) => claims,
None => {
crate::instance::virtual_nic::NicCtx::mobile_tun_claims(&global_ctx, &packet_plane)
.await
}
};
nic.run_for_mobile(fd, replace_tun_fd, claims)
.await
.context("attach mobile TUN failed")?;
let magic_dns = if let Some(ip) = global_ctx.get_ipv4() {
let shared_route_backend = nic.shared_route_backend_for_dns();
MagicDnsRuntime::start(global_ctx, packet_plane, None, ip, shared_route_backend)
} else {
MagicDnsRuntime::default()
};
nic_state.install(nic, magic_dns).await;
Ok(())
}
@@ -80,28 +112,44 @@ impl NativeTunRuntime {
let nic_state = self.nic.clone();
let global_ctx = self.global_ctx.clone();
let cancel = self.cancel.clone();
let shared_virtual_nic_registry = self.shared_virtual_nic_registry.clone();
self.task.lock().await.replace(tokio::spawn(async move {
loop {
let fd = tokio::select! {
let attachment = tokio::select! {
_ = cancel.cancelled() => return,
fd = tun_fds.recv() => match fd { Some(fd) => fd, None => return },
attachment = tun_fds.recv() => match attachment {
Some(attachment) => attachment,
None => return,
},
};
if let Err(error) = Self::install_mobile_tun(
nic_state.clone(),
global_ctx.clone(),
packet_plane.clone(),
fd,
)
.await
{
let result = if attachment.fd <= 0 {
nic_state.drain().await;
Ok(())
} else {
Self::install_mobile_tun(
nic_state.clone(),
global_ctx.clone(),
packet_plane.clone(),
attachment.fd,
attachment.replace_tun_fd,
shared_virtual_nic_registry.clone(),
attachment.claims,
)
.await
};
if let Err(error) = &result {
tracing::error!(?error, "failed to attach mobile TUN fd");
}
if let Some(completion) = attachment.completion {
let _ = completion.send(result);
}
}
}));
Ok(())
}
pub(super) async fn shutdown(&self) {
self.cancel.cancel();
if let Some(task) = self.task.lock().await.take() {
let _ = task.await;
}
@@ -110,10 +158,43 @@ impl NativeTunRuntime {
pub(super) fn attach_fd(&self, fd: i32) -> anyhow::Result<()> {
self.tun_fd
.try_send(fd)
.try_send(MobileTunAttachment {
fd,
replace_tun_fd: true,
completion: None,
claims: None,
})
.map_err(|error| anyhow::anyhow!("failed to send TUN fd: {error}"))
}
pub(super) async fn attach_mobile_fd(
&self,
fd: i32,
replace_tun_fd: bool,
claims: Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims>,
) -> anyhow::Result<()> {
if self.task.lock().await.is_none() {
anyhow::bail!("mobile TUN runtime is not running");
}
let (completion, result) = oneshot::channel();
tokio::select! {
_ = self.cancel.cancelled() => anyhow::bail!("instance is closing; TUN attachment cancelled"),
send_result = self.tun_fd.send(MobileTunAttachment {
fd,
replace_tun_fd,
completion: Some(completion),
claims,
}) => send_result.map_err(|error| anyhow::anyhow!("failed to send TUN fd: {error}"))?,
}
tokio::select! {
_ = self.cancel.cancelled() => anyhow::bail!("instance is closing; TUN attachment cancelled"),
result = result => result
.map_err(|_| anyhow::anyhow!("mobile TUN runtime stopped before attachment completed"))?,
}
}
pub(super) fn dhcp_host(
&self,
operation: Arc<Mutex<()>>,
@@ -127,6 +208,12 @@ impl NativeTunRuntime {
closed: Arc::new(Notify::new()),
})
}
pub(super) async fn current_mobile_tun_claims(
&self,
) -> Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims> {
self.nic.mobile_tun_claims().await
}
}
struct NativeDhcpIpv4Host {