feat: add shared virtual nic (#2352)

* Add shared virtual NIC core with per-member routing

Share a named TUN across instances while keeping unnamed devices
dedicated. Track member addresses, routes, and MTU, and reject
overlapping destinations except the common Magic DNS host route.

Dispatch packets between the physical TUN and member tunnels. Keep
IPv4 and ordinary IPv6 source translation for mobile wrong-source
traffic. Update platform configuration and runtime host integration,
with unit and network namespace coverage for routing and lifecycle.

Use smoltcp for packet parsing and checksum updates. Keep core CI
matrix jobs independent.

Reserve native inbound flows before source translations allocate their
wire tuples. Keep active reservations under table pressure and leave
forwarded subnet traffic outside the local flow table.

Complete interface claim updates on the shared runtime and retain member
registrations until the transaction commits. Keep address, route, and
MTU cleanup ordered after cancelled callers, including device
initialization.

Project mobile VPN addresses, routes, and DNS from admitted native
members. Prefer unchanged attached claims and report conflicting members
individually. Apply each member's complete initial claims before opening
the shared fd, so conflicts leave healthy members usable.

Cover inbound IPv4/IPv6 collisions, cancellation during native changes
and dispatcher acknowledgments, admission priority, and atomic mobile
claim rejection with regression tests.

Use scoped TestNetwork fixtures for shared TUN bridge setup and cleanup.
Resolve public IPv6 namespace aliases through the lab's actual names so
the new tests remain isolated under parallel upstream test runs.

* Configure one mobile VPN from running TUN instances

Use admitted native members to configure VpnService and attach its
single fd to the core shared TUN. Wait for attachment completion and
record the actual attached members before refreshing VPN state. Report
rejected members in the GUI and preserve the VPN for unchanged healthy
members.

Reconcile ordinary configuration notifications without invalidating an
arbitrary member. Invalidate attachments on instance lifecycle changes,
including restarts that reuse an instance ID. Forward public IPv6
changes alongside proxy route changes to refresh the platform VPN
configuration.

Exclude confirmed IPv4-less members from readiness gating while
retaining retries for unavailable status and pending addresses. Quick
Settings stops all running TUN members and restores that group on the
next start, skipping removed configurations and no_tun networks.

Keep the FFI set_tun_fd call scoped to one instance and report
attachment errors. Cover mobile configuration, member rejection and
replacement, attachment completion, collection notifications, IPv4-less
members, and tile stop/start restoration in GUI regression tests.

Validate with 36 mobile VPN tests, vue-tsc --noEmit, Android GUI cargo
check, workspace clippy, and shared TUN network namespace tests.

* fix(tun): preserve shared NIC ownership and failure invariants

Keep mobile address and route claims in dispatcher tables without
applying desktop interface configuration to synthetic TUN names.

Limit the Magic DNS overlap exemption to route claims and remove the
shared route retry that bypassed claim accounting after invalidation.

Invalidate and stop dispatchers on failed mobile TUN replacement or I/O.
Bound TUN writes so stalled devices release pending source updates and
member cleanup instead of retaining an unusable shared dispatcher.

Treat missing Darwin and Windows addresses and routes as successful
deletions while preserving permission and other configuration errors.

Add regressions for claim conflicts, mobile configuration, failed TUN
replacement, stalled writes, and platform deletion error handling.

* fix(tun): forward single-member shared NIC traffic unchanged

Deliver TUN packets directly to the sole attached member without
destination matching or creating source translations. Keep native flow
reservations so a later member cannot collide with existing connections,
and continue applying mappings created before other members leave.

Reuse flow preparation with the original source address. Preserve the
desktop subnet forwarding bypass, including when the flow table is full.
IPv4 fragments and IPv6 extension headers remain unchanged on flows that
need no translation.

Cover unknown sources and destinations, first and later IP fragments,
native flows established before another member joins, and retained
translations after returning to one member. Document the forwarding
behavior and retain rejection coverage for multiple attached members.

* fix(tun): preserve wildcard UDP replies on shared member ports

Mark flow reservations created by incoming traffic separately from
outgoing mappings. When a wildcard UDP socket replies with another
member's source address, reuse the incoming reservation after source
correction and preserve its service port and inbound destination.

Keep exact outgoing mappings and outgoing flow collision handling ahead
of this rule. Incoming UDP sessions take priority when packet tuples
cannot distinguish a reply from a new client using identical endpoints.

Cover IPv4 and IPv6 replies, checksum updates, repeated inbound traffic,
existing outgoing mappings, TCP collisions, and single-to-shared member
transitions with regression tests.

* test(tun): wait for IPv6 activation in mobile configuration test

Assert that the configured IPv6 address is registered before forwarding
starts, then wait up to five seconds for the kernel to make it usable.
Netlink acknowledges address creation before asynchronous IPv6
activation completes, so an immediate bind can fail under CI
parallelism.

Disable automatic link-local address generation in the test namespace to
keep startup IPv6 control traffic out of member-routing assertions.

Validation: the original test failed 40 of 320 runs at 32-way
parallelism. The updated test passed all 320 runs; workspace fmt and
clippy also pass.
This commit is contained in:
KKRainbow authored and GitHub committed 2026-10-13 21:00:03 +08:00
1 parent 99fce327d5
commit 32dc7f1f40
42 files changed
+10052 -702

No files matched your search

+9 -1
View File
@@ -31,6 +31,8 @@ use crate::{
instance::runtime_host::NativeInstanceRuntimeHost,
};
#[cfg(feature = "tun")]
use super::shared_virtual_nic::ArcSharedVirtualNicRegistry;
use super::{
host::{NativeInstanceHost, native_instance_host},
runtime_executor::NativeInstanceExecutor,
@@ -48,6 +50,7 @@ pub(crate) type NativeCoreInstance = CoreInstance<NativeInstanceHost>;
pub(super) fn compose_native_core_instance(
toml_config: TomlConfig,
process_runtime: Arc<CoreProcessRuntime>,
#[cfg(feature = "tun")] shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
compact_runtime: bool,
executor: Option<NativeInstanceExecutor>,
) -> anyhow::Result<Arc<NativeCoreInstance>> {
@@ -62,7 +65,12 @@ pub(super) fn compose_native_core_instance(
&normalized,
&host_config,
));
let runtime_host = NativeInstanceRuntimeHost::new_with_executor(global_ctx.clone(), executor);
let runtime_host = NativeInstanceRuntimeHost::new_with_executor(
global_ctx.clone(),
#[cfg(feature = "tun")]
shared_virtual_nic_registry,
executor,
);
let mut adapters = runtime_core_host_adapters_with_packet_egress_and_config(
global_ctx.clone(),
process_runtime,
+115 -39
View File
@@ -5,7 +5,7 @@ use std::{net::Ipv4Addr, sync::Arc, time::Duration};
use easytier_core::instance::CorePacketPlane;
use crate::common::global_ctx::ArcGlobalCtx;
use crate::{common::global_ctx::ArcGlobalCtx, instance::virtual_nic::NicBackend};
use super::{client_instance::MagicDnsClientInstance, server_instance::MagicDnsServerInstance};
@@ -17,6 +17,34 @@ pub struct DnsRunner {
tun_dev: Option<String>,
tun_inet: Ipv4Inet,
fake_ip: Ipv4Addr,
shared_route_backend: Option<NicBackend>,
active_fake_ip_route: Option<MagicDnsFakeIpRouteClaim>,
}
struct MagicDnsFakeIpRouteClaim {
fake_ip: Ipv4Addr,
route_backend: NicBackend,
}
impl MagicDnsFakeIpRouteClaim {
async fn add(&self) -> anyhow::Result<()> {
let cost = if cfg!(target_os = "windows") {
Some(4)
} else {
None
};
self.route_backend
.add_route_with_cost(self.fake_ip, 32, cost)
.await?;
Ok(())
}
async fn remove(&self) {
if let Err(err) = self.route_backend.remove_route(self.fake_ip, 32).await {
tracing::warn!(?err, fake_ip = ?self.fake_ip, "remove magic dns route failed");
}
}
}
impl DnsRunner {
@@ -35,29 +63,74 @@ impl DnsRunner {
tun_dev,
tun_inet,
fake_ip,
shared_route_backend: None,
active_fake_ip_route: None,
}
}
pub(crate) fn with_shared_route_backend(mut self, route_backend: Option<NicBackend>) -> Self {
self.shared_route_backend = route_backend;
self
}
async fn clean_env(&mut self) {
if let Some(claim) = self.active_fake_ip_route.take() {
claim.remove().await;
}
if let Some(server) = self.server.take() {
server.clean_env().await;
}
self.client.take();
}
async fn run_once(&mut self) -> anyhow::Result<()> {
fn should_manage_fake_ip_route_externally(&self) -> bool {
self.shared_route_backend.is_some() && !self.tun_inet.contains(&self.fake_ip)
}
fn fake_ip_route_claim(&self) -> Option<MagicDnsFakeIpRouteClaim> {
if !self.should_manage_fake_ip_route_externally() {
return None;
}
Some(MagicDnsFakeIpRouteClaim {
fake_ip: self.fake_ip,
route_backend: self.shared_route_backend.clone()?,
})
}
async fn run_once(&mut self, cancel_token: &CancellationToken) -> anyhow::Result<()> {
// try server first
match MagicDnsServerInstance::new(
self.packet_plane.clone(),
self.global_ctx.clone(),
self.tun_dev.clone(),
self.tun_inet,
self.fake_ip,
)
.await
{
let server_result = if self.should_manage_fake_ip_route_externally() {
MagicDnsServerInstance::new_with_external_fake_ip_route(
self.packet_plane.clone(),
self.global_ctx.clone(),
self.tun_dev.clone(),
self.tun_inet,
self.fake_ip,
)
.await
} else {
MagicDnsServerInstance::new(
self.packet_plane.clone(),
self.global_ctx.clone(),
self.tun_dev.clone(),
self.tun_inet,
self.fake_ip,
)
.await
};
match server_result {
Ok(server) => {
self.server = Some(server);
// Keep the initialized resolver owned by the runner before adding
// its route, so route failure still reaches async server cleanup.
self.active_fake_ip_route = self.fake_ip_route_claim();
if let Some(claim) = &self.active_fake_ip_route {
claim.add().await.map_err(|err| {
anyhow::anyhow!("failed to add magic dns fake-ip route: {err}")
})?;
}
tracing::info!("DnsRunner::run_once: server started");
}
Err(e) => {
@@ -65,34 +138,37 @@ impl DnsRunner {
}
}
// every runner must run a client
let client = MagicDnsClientInstance::new(self.packet_plane.clone()).await?;
self.client = Some(client);
self.client.as_mut().unwrap().run_and_wait().await;
Err(anyhow::anyhow!("Client instance exit"))
}
pub async fn run(&mut self, canel_token: CancellationToken) {
loop {
tracing::info!("DnsRunner::run: start");
tokio::select! {
_ = canel_token.cancelled() => {
self.clean_env().await;
tracing::info!("DnsRunner::run: cancelled");
return;
}
ret = self.run_once() => {
self.clean_env().await;
if let Err(e) = ret {
tracing::error!("DnsRunner::run: {:?}", e);
} else {
tracing::info!("DnsRunner::run: unexpected exit, server may be down");
}
tokio::time::sleep(Duration::from_millis(500)).await;
}
}
// Every runner must run a client. Only this phase is cancellable: server
// initialization and route updates must finish before async cleanup.
tokio::select! {
_ = cancel_token.cancelled() => Ok(()),
ret = async {
let client = MagicDnsClientInstance::new(self.packet_plane.clone()).await?;
self.client = Some(client);
self.client.as_mut().unwrap().run_and_wait().await;
Err(anyhow::anyhow!("Client instance exit"))
} => ret,
}
}
pub async fn run(&mut self, cancel_token: CancellationToken) {
while !cancel_token.is_cancelled() {
tracing::info!("DnsRunner::run: start");
let ret = self.run_once(&cancel_token).await;
self.clean_env().await;
if cancel_token.is_cancelled() {
break;
}
if let Err(e) = ret {
tracing::error!("DnsRunner::run: {:?}", e);
} else {
tracing::info!("DnsRunner::run: unexpected exit, server may be down");
}
tokio::select! {
_ = cancel_token.cancelled() => break,
_ = tokio::time::sleep(Duration::from_millis(500)) => {}
}
}
tracing::info!("DnsRunner::run: cancelled");
}
}
@@ -14,8 +14,10 @@ use super::{
};
use crate::{
common::{
error::Error as EtError,
global_ctx::ArcGlobalCtx,
ifcfg::{IfConfiger, IfConfiguerTrait},
netns::NetNS,
},
instance::dns_server::{
config::{Record, RecordBuilder, RecordType},
@@ -51,7 +53,9 @@ use std::{collections::BTreeMap, io, net::Ipv4Addr, str::FromStr, sync::Arc, tim
pub(super) struct MagicDnsServerInstanceData {
dns_server: Server,
tun_dev: Option<String>,
net_ns: NetNS,
fake_ip: Ipv4Addr,
manage_fake_ip_route: bool,
route_store: MagicDnsRecordStore,
record_apply: tokio::sync::Mutex<()>,
@@ -356,12 +360,66 @@ fn get_system_config(
}
impl MagicDnsServerInstance {
async fn add_fake_ip_route(
tun_dev_name: &str,
fake_ip: Ipv4Addr,
net_ns: &NetNS,
cost: Option<i32>,
) -> Result<(), anyhow::Error> {
let ifcfg = IfConfiger::default();
let _guard = net_ns.guard();
match ifcfg.add_ipv4_route(tun_dev_name, fake_ip, 32, cost).await {
Err(EtError::IOError(err)) if err.kind() == io::ErrorKind::AlreadyExists => {
ifcfg.remove_ipv4_route(tun_dev_name, fake_ip, 32).await?;
ifcfg
.add_ipv4_route(tun_dev_name, fake_ip, 32, cost)
.await?;
Ok(())
}
ret => ret.map_err(Into::into),
}
}
async fn remove_fake_ip_route(tun_dev_name: &str, fake_ip: Ipv4Addr, net_ns: &NetNS) {
let ifcfg = IfConfiger::default();
let _guard = net_ns.guard();
if let Err(err) = ifcfg.remove_ipv4_route(tun_dev_name, fake_ip, 32).await {
tracing::warn!(
?err,
?tun_dev_name,
?fake_ip,
"remove magic dns route failed"
);
}
}
pub(crate) async fn new(
packet_plane: Arc<CorePacketPlane>,
global_ctx: ArcGlobalCtx,
tun_dev: Option<String>,
tun_inet: Ipv4Inet,
fake_ip: Ipv4Addr,
) -> Result<Self, anyhow::Error> {
Self::new_inner(packet_plane, global_ctx, tun_dev, tun_inet, fake_ip, true).await
}
pub(crate) async fn new_with_external_fake_ip_route(
packet_plane: Arc<CorePacketPlane>,
global_ctx: ArcGlobalCtx,
tun_dev: Option<String>,
tun_inet: Ipv4Inet,
fake_ip: Ipv4Addr,
) -> Result<Self, anyhow::Error> {
Self::new_inner(packet_plane, global_ctx, tun_dev, tun_inet, fake_ip, false).await
}
async fn new_inner(
packet_plane: Arc<CorePacketPlane>,
global_ctx: ArcGlobalCtx,
tun_dev: Option<String>,
tun_inet: Ipv4Inet,
fake_ip: Ipv4Addr,
manage_fake_ip_route: bool,
) -> Result<Self, anyhow::Error> {
let tcp_listener = runtime_rpc_listener(MAGIC_DNS_INSTANCE_SOCKET_ADDR.parse()?);
let mut rpc_server = StandAloneServer::new(tcp_listener);
@@ -374,7 +432,8 @@ impl MagicDnsServerInstance {
let mut dns_server = Server::new(dns_config);
dns_server.run().await?;
if !tun_inet.contains(&fake_ip)
if manage_fake_ip_route
&& !tun_inet.contains(&fake_ip)
&& let Some(tun_dev_name) = &tun_dev
{
let cost = if cfg!(target_os = "windows") {
@@ -382,16 +441,15 @@ impl MagicDnsServerInstance {
} else {
None
};
let ifcfg = IfConfiger {};
ifcfg
.add_ipv4_route(tun_dev_name, fake_ip, 32, cost)
.await?;
Self::add_fake_ip_route(tun_dev_name, fake_ip, &global_ctx.net_ns, cost).await?;
}
let data = Arc::new(MagicDnsServerInstanceData {
dns_server,
tun_dev: tun_dev.clone(),
net_ns: global_ctx.net_ns.clone(),
fake_ip,
manage_fake_ip_route,
route_store: MagicDnsRecordStore::default(),
record_apply: tokio::sync::Mutex::new(()),
system_config: get_system_config(tun_dev.as_deref())?,
@@ -436,14 +494,13 @@ impl MagicDnsServerInstance {
if let Err(e) = ret {
tracing::error!("Failed to close system config: {:?}", e);
}
if !self.tun_inet.contains(&self.data.fake_ip)
&& let Some(tun_dev_name) = &self.data.tun_dev
{
let ifcfg = IfConfiger {};
let _ = ifcfg
.remove_ipv4_route(tun_dev_name, self.data.fake_ip, 32)
.await;
}
}
if self.data.manage_fake_ip_route
&& !self.tun_inet.contains(&self.data.fake_ip)
&& let Some(tun_dev_name) = &self.data.tun_dev
{
Self::remove_fake_ip_route(tun_dev_name, self.data.fake_ip, &self.data.net_ns).await;
}
self.packet_filter.close().await;
+18
View File
@@ -1,5 +1,8 @@
use std::sync::Arc;
#[cfg(feature = "tun")]
use tokio::sync::Mutex;
#[cfg(any(feature = "management-rpc", test))]
use easytier_core::instance::manager::InstanceManager;
#[cfg(feature = "management-rpc")]
@@ -12,12 +15,20 @@ use easytier_core::{
use crate::common::global_ctx::EventBusSubscriber;
#[cfg(feature = "tun")]
use super::shared_virtual_nic::{ArcSharedVirtualNicRegistry, SharedVirtualNicRegistry};
use super::{
composition::compose_native_core_instance, host::NativeInstanceHost,
runtime_executor::NativeInstanceExecutor, runtime_host::NativeInstanceRuntimeHost,
};
pub type NativeCoreInstance = CoreInstance<NativeInstanceHost>;
#[cfg(all(feature = "management-rpc", feature = "tun"))]
mod mobile_tun;
#[cfg(all(feature = "management-rpc", feature = "tun"))]
pub use mobile_tun::MobileTunConfig;
#[cfg(all(feature = "management-rpc", feature = "tun", mobile))]
pub use mobile_tun::{attach_mobile_tun_fd, get_mobile_tun_config};
#[cfg(feature = "management-rpc")]
pub type NativeInstanceManager = InstanceManager<NativeInstanceFactory>;
#[cfg(feature = "management")]
@@ -97,6 +108,8 @@ fn native_instance_manager_with_optional_runtime(
/// Native construction Adapter for the canonical core InstanceManager.
pub struct NativeInstanceFactory {
process_runtime: Arc<CoreProcessRuntime>,
#[cfg(feature = "tun")]
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
runtime_handle: Option<tokio::runtime::Handle>,
compact_runtime: bool,
#[cfg(feature = "logging")]
@@ -107,6 +120,8 @@ impl NativeInstanceFactory {
pub fn new(process_runtime: Arc<CoreProcessRuntime>) -> Self {
Self {
process_runtime,
#[cfg(feature = "tun")]
shared_virtual_nic_registry: Arc::new(Mutex::new(SharedVirtualNicRegistry::new())),
runtime_handle: None,
compact_runtime: false,
#[cfg(feature = "logging")]
@@ -126,6 +141,7 @@ impl NativeInstanceFactory {
self
}
#[cfg(feature = "management-rpc")]
fn with_compact_runtime(mut self) -> Self {
self.compact_runtime = true;
self
@@ -159,6 +175,8 @@ impl InstanceFactory for NativeInstanceFactory {
let instance = compose_native_core_instance(
config,
self.process_runtime.clone(),
#[cfg(feature = "tun")]
self.shared_virtual_nic_registry.clone(),
self.compact_runtime,
executor,
)?;
+393
View File
@@ -0,0 +1,393 @@
#[cfg(any(mobile, test))]
use std::collections::BTreeSet;
#[cfg(mobile)]
use super::{NativeInstanceManager, NativeInstanceRuntimeHost};
#[cfg(any(mobile, test))]
use crate::instance::shared_virtual_nic::{
SharedIfConfig, SharedIfConfigClaims, SharedIfConfigConflict, SharedIpv4Route,
};
#[cfg(mobile)]
use easytier_core::instance::CoreInstanceState;
#[derive(Clone, Debug, Default, serde::Serialize)]
pub struct MobileTunConfig {
pub instance_ids: Vec<uuid::Uuid>,
pub addresses: Vec<String>,
pub routes: Vec<String>,
pub dns: Option<String>,
pub rejected: Vec<MobileTunMemberError>,
}
#[derive(Clone, Debug, serde::Serialize)]
pub struct MobileTunMemberError {
pub instance_id: uuid::Uuid,
pub error: String,
}
#[cfg(any(mobile, test))]
struct MobileTunMember {
instance_id: uuid::Uuid,
claims: SharedIfConfigClaims,
current_claims: Option<SharedIfConfigClaims>,
dns: bool,
}
#[cfg(any(mobile, test))]
fn magic_dns_route() -> SharedIpv4Route {
SharedIpv4Route::new(
crate::instance::dns_server::MAGIC_DNS_FAKE_IP
.parse()
.unwrap(),
32,
Some(1),
)
}
#[cfg(any(mobile, test))]
fn comparable_claims(mut claims: SharedIfConfigClaims) -> SharedIfConfigClaims {
claims.mtu = None;
let magic_dns = magic_dns_route();
claims
.ipv4_routes
.retain(|route| route.address != magic_dns.address || route.prefix != magic_dns.prefix);
claims
}
#[cfg(any(mobile, test))]
fn select_members(
mut members: Vec<MobileTunMember>,
mut rejected: Vec<MobileTunMemberError>,
) -> (Vec<MobileTunMember>, MobileTunConfig) {
// Keep unchanged attached members ahead of new or reconfigured members.
members.sort_by_key(|member| {
!member.current_claims.as_ref().is_some_and(|current| {
comparable_claims(current.clone()) == comparable_claims(member.claims.clone())
})
});
let mut ifcfg = SharedIfConfig::default();
let mut accepted = Vec::new();
for member in members {
let mut claims = member.claims.clone();
if member.dns {
claims.ipv4_routes.insert(magic_dns_route());
}
if let Err(error) = ifcfg.ensure_disjoint(member.instance_id, &claims) {
rejected.push(MobileTunMemberError {
instance_id: member.instance_id,
error: error.to_string(),
});
continue;
}
ifcfg.apply_member_claims(member.instance_id, claims);
accepted.push(member);
}
let config = config_for_members(&accepted, rejected);
(accepted, config)
}
#[cfg(any(mobile, test))]
fn config_for_members(
members: &[MobileTunMember],
rejected: Vec<MobileTunMemberError>,
) -> MobileTunConfig {
let mut config = MobileTunConfig {
rejected,
..Default::default()
};
let mut addresses = BTreeSet::new();
let mut routes = BTreeSet::new();
for member in members {
config.instance_ids.push(member.instance_id);
for address in &member.claims.ipv4_addresses {
addresses.insert(format!(
"{}/{}",
address.address(),
address.network_length()
));
routes.insert(format!(
"{}/{}",
address.network().first_address(),
address.network_length()
));
}
for address in &member.claims.ipv6_addresses {
addresses.insert(format!(
"{}/{}",
address.address(),
address.network_length()
));
routes.insert(format!(
"{}/{}",
address.network().first_address(),
address.network_length()
));
}
for route in &member.claims.ipv4_routes {
if let Ok(route) = cidr::Ipv4Inet::new(route.address, route.prefix) {
routes.insert(format!(
"{}/{}",
route.network().first_address(),
route.network_length()
));
}
}
for route in &member.claims.ipv6_routes {
if let Ok(route) = cidr::Ipv6Inet::new(route.address, route.prefix) {
routes.insert(format!(
"{}/{}",
route.network().first_address(),
route.network_length()
));
}
}
if member.dns {
config.dns = Some(crate::instance::dns_server::MAGIC_DNS_FAKE_IP.to_owned());
routes.insert(format!(
"{}/32",
crate::instance::dns_server::MAGIC_DNS_FAKE_IP
));
}
}
config.instance_ids.sort();
config.addresses = addresses.into_iter().collect();
config.routes = routes.into_iter().collect();
config
}
#[cfg(mobile)]
async fn collect_members(
manager: &NativeInstanceManager,
instance_ids: &[uuid::Uuid],
) -> (Vec<MobileTunMember>, Vec<MobileTunMemberError>) {
let mut members = Vec::new();
let mut rejected = Vec::new();
let mut seen = BTreeSet::new();
for &instance_id in instance_ids {
if !seen.insert(instance_id) {
continue;
}
let instance = manager.instance(instance_id);
let runtime = instance
.as_ref()
.filter(|instance| instance.state() == CoreInstanceState::Running)
.and_then(|instance| instance.runtime_host::<NativeInstanceRuntimeHost>())
.filter(|runtime| runtime.tun_enabled());
let Some(runtime) = runtime else {
rejected.push(MobileTunMemberError {
instance_id,
error: "TUN-enabled instance is not running".to_owned(),
});
continue;
};
let (claims, current_claims, dns) = runtime
.mobile_tun_claims(&instance.as_ref().unwrap().packet_plane())
.await;
members.push(MobileTunMember {
instance_id,
claims,
current_claims,
dns,
});
}
(members, rejected)
}
#[cfg(mobile)]
pub async fn get_mobile_tun_config(
manager: &NativeInstanceManager,
instance_ids: &[uuid::Uuid],
) -> MobileTunConfig {
let (members, rejected) = collect_members(manager, instance_ids).await;
select_members(members, rejected).1
}
#[cfg(any(mobile, test))]
fn is_claim_conflict(error: &anyhow::Error) -> bool {
error
.chain()
.any(|error| error.is::<SharedIfConfigConflict>())
}
#[cfg(mobile)]
pub async fn attach_mobile_tun_fd(
manager: &NativeInstanceManager,
fd: i32,
instance_ids: Option<&[uuid::Uuid]>,
) -> anyhow::Result<MobileTunConfig> {
let all_ids = manager
.instances()
.into_iter()
.filter(|instance| {
instance.state() == CoreInstanceState::Running
&& instance
.runtime_host::<NativeInstanceRuntimeHost>()
.is_some_and(NativeInstanceRuntimeHost::tun_enabled)
})
.map(|instance| instance.instance_id())
.collect::<BTreeSet<_>>()
.into_iter()
.collect::<Vec<_>>();
if fd <= 0 {
let mut errors = Vec::new();
for id in all_ids {
if let Err(error) = attach_member(manager, id, 0, false, None).await {
errors.push(error.to_string());
}
}
anyhow::ensure!(
errors.is_empty(),
"failed to detach mobile TUN fd: {}",
errors.join("; ")
);
return Ok(MobileTunConfig::default());
}
let (members, rejected) = collect_members(manager, instance_ids.unwrap_or(&all_ids)).await;
let (members, mut config) = select_members(members, rejected);
let mut attached = Vec::new();
// Open once; a rejected member never replaces or tears down this shared fd.
for member in members {
match attach_member(
manager,
member.instance_id,
fd,
attached.is_empty(),
Some(member.claims.clone()),
)
.await
{
Ok(()) => attached.push(member),
Err(error) if is_claim_conflict(&error) => {
attach_member(manager, member.instance_id, 0, false, None).await?;
config.rejected.push(MobileTunMemberError {
instance_id: member.instance_id,
error: error.to_string(),
});
}
Err(error) => {
for id in &all_ids {
let _ = attach_member(manager, *id, 0, false, None).await;
}
return Err(error.context("failed to attach mobile TUN fd"));
}
}
}
Ok(config_for_members(&attached, config.rejected))
}
#[cfg(mobile)]
async fn attach_member(
manager: &NativeInstanceManager,
instance_id: uuid::Uuid,
fd: i32,
replace_tun_fd: bool,
claims: Option<SharedIfConfigClaims>,
) -> anyhow::Result<()> {
let instance = manager
.instance(instance_id)
.ok_or_else(|| anyhow::anyhow!("instance {instance_id} is unavailable"))?;
let runtime = instance
.runtime_host::<NativeInstanceRuntimeHost>()
.ok_or_else(|| anyhow::anyhow!("native runtime host is unavailable"))?;
runtime
.attach_mobile_tun_fd(fd, replace_tun_fd, claims)
.await
}
#[cfg(test)]
mod tests {
use super::*;
use crate::instance::shared_virtual_nic::SharedIpv6Route;
fn member(id: u128, address: &str) -> MobileTunMember {
MobileTunMember {
instance_id: uuid::Uuid::from_u128(id),
claims: SharedIfConfigClaims {
ipv4_addresses: BTreeSet::from([address.parse().unwrap()]),
..Default::default()
},
current_claims: None,
dns: false,
}
}
#[test]
fn keeps_attached_member_when_bootstrap_subnets_overlap() {
let mut existing = member(2, "10.126.126.2/24");
existing.current_claims = Some(existing.claims.clone());
let (_, config) = select_members(vec![member(1, "10.126.126.1/24"), existing], Vec::new());
assert_eq!(config.instance_ids, vec![uuid::Uuid::from_u128(2)]);
assert_eq!(config.addresses, vec!["10.126.126.2/24"]);
assert_eq!(config.routes, vec!["10.126.126.0/24"]);
assert_eq!(config.rejected[0].instance_id, uuid::Uuid::from_u128(1));
}
#[test]
fn reconfigured_member_does_not_displace_unchanged_member() {
let mut changed = member(1, "10.0.2.1/24");
changed.current_claims = Some(member(1, "10.0.1.1/24").claims);
let mut unchanged = member(2, "10.0.2.2/24");
unchanged.current_claims = Some(unchanged.claims.clone());
unchanged.current_claims.as_mut().unwrap().mtu = Some(1300);
let (_, config) = select_members(vec![changed, unchanged], Vec::new());
assert_eq!(config.instance_ids, vec![uuid::Uuid::from_u128(2)]);
assert_eq!(config.rejected[0].instance_id, uuid::Uuid::from_u128(1));
}
#[test]
fn projects_ipv6_and_shared_dns_and_rejects_overlapping_proxy_routes() {
let mut first = member(1, "10.0.1.1/24");
first.dns = true;
first
.claims
.ipv6_addresses
.insert("2001:db8::1/128".parse().unwrap());
first
.claims
.ipv6_routes
.insert(SharedIpv6Route::new("::".parse().unwrap(), 0, Some(5)));
let mut second = member(2, "10.0.2.1/24");
second.dns = true;
let (_, config) = select_members(
vec![first, second, member(3, "100.100.100.101/32")],
Vec::new(),
);
assert_eq!(config.instance_ids.len(), 2);
assert!(config.addresses.contains(&"2001:db8::1/128".to_owned()));
assert!(config.routes.contains(&"::/0".to_owned()));
assert_eq!(
config
.routes
.iter()
.filter(|route| *route == "100.100.100.101/32")
.count(),
1
);
assert_eq!(config.dns.as_deref(), Some("100.100.100.101"));
assert_eq!(config.rejected[0].instance_id, uuid::Uuid::from_u128(3));
let mut route_owner = member(2, "10.0.2.1/24");
route_owner.claims.ipv4_routes.insert(SharedIpv4Route::new(
"10.0.1.0".parse().unwrap(),
24,
None,
));
let (_, config) = select_members(vec![member(1, "10.0.1.1/24"), route_owner], Vec::new());
assert_eq!(config.instance_ids.len(), 1);
assert_eq!(config.rejected[0].instance_id, uuid::Uuid::from_u128(2));
}
#[test]
fn distinguishes_claim_conflicts_from_device_failures_through_context() {
let mut claims = SharedIfConfig::default();
claims.apply_member_claims(uuid::Uuid::from_u128(1), member(1, "10.0.1.1/24").claims);
let conflict = claims
.ensure_disjoint(uuid::Uuid::from_u128(2), &member(2, "10.0.1.2/24").claims)
.unwrap_err();
let error = anyhow::Error::new(crate::common::error::Error::from(conflict))
.context("attach mobile TUN failed");
assert!(is_claim_conflict(&error));
assert!(!is_claim_conflict(&anyhow::Error::new(
std::io::Error::from_raw_os_error(19)
)));
}
}
+3
View File
@@ -19,6 +19,9 @@ pub(crate) mod listeners;
#[cfg(feature = "public-ipv6-provider")]
pub(crate) mod public_ipv6_provider;
#[cfg(feature = "tun")]
pub mod shared_virtual_nic;
#[cfg(feature = "tun")]
pub mod virtual_nic;
+69 -8
View File
@@ -1,13 +1,16 @@
use std::sync::Arc;
#[cfg(feature = "web-client")]
use easytier_core::config::runtime::CoreInstanceRuntimeConfig;
use easytier_core::{
config::runtime::CoreInstanceRuntimeConfig, gateway::dhcp::DhcpIpv4Host,
host::packet::HostPacketReceiver, instance::CorePacketPlane,
gateway::dhcp::DhcpIpv4Host, host::packet::HostPacketReceiver, instance::CorePacketPlane,
};
use tokio::sync::Mutex;
use tokio_util::sync::CancellationToken;
use crate::common::global_ctx::ArcGlobalCtx;
#[cfg(feature = "tun")]
use crate::instance::shared_virtual_nic::ArcSharedVirtualNicRegistry;
mod event_journal;
mod implementation;
@@ -42,16 +45,30 @@ pub(crate) struct NativeInstanceRuntimeHost {
impl NativeInstanceRuntimeHost {
#[cfg(test)]
pub(crate) fn new(global_ctx: ArcGlobalCtx) -> Arc<Self> {
Self::new_with_executor(global_ctx, None)
pub(crate) fn new(
global_ctx: ArcGlobalCtx,
#[cfg(feature = "tun")] shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
) -> Arc<Self> {
Self::new_with_executor(
global_ctx,
#[cfg(feature = "tun")]
shared_virtual_nic_registry,
None,
)
}
pub(super) fn new_with_executor(
global_ctx: ArcGlobalCtx,
#[cfg(feature = "tun")] shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
executor: Option<NativeInstanceExecutor>,
) -> Arc<Self> {
let cancel = CancellationToken::new();
let tun = NativeTunRuntime::new(global_ctx.clone(), cancel.clone());
let tun = NativeTunRuntime::new(
global_ctx.clone(),
cancel.clone(),
#[cfg(feature = "tun")]
shared_virtual_nic_registry,
);
let event_journal = EventJournal::new(&global_ctx);
Arc::new(Self {
global_ctx,
@@ -128,10 +145,44 @@ impl NativeInstanceRuntimeHost {
self.global_ctx.subscribe()
}
#[cfg(all(feature = "tun", mobile))]
pub(crate) fn tun_enabled(&self) -> bool {
!self.global_ctx.get_flags().no_tun
}
fn attach_runtime_tun_fd(&self, fd: i32) -> anyhow::Result<()> {
self.tun.attach_fd(fd)
}
#[cfg(all(feature = "tun", mobile))]
pub(crate) async fn attach_mobile_tun_fd(
&self,
fd: i32,
replace_tun_fd: bool,
claims: Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims>,
) -> anyhow::Result<()> {
self.tun.attach_mobile_fd(fd, replace_tun_fd, claims).await
}
#[cfg(all(feature = "tun", mobile))]
pub(crate) async fn mobile_tun_claims(
&self,
packet_plane: &CorePacketPlane,
) -> (
crate::instance::shared_virtual_nic::SharedIfConfigClaims,
Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims>,
bool,
) {
(
crate::instance::virtual_nic::NicCtx::mobile_tun_claims(&self.global_ctx, packet_plane)
.await,
self.tun.current_mobile_tun_claims().await,
cfg!(feature = "magic-dns")
&& self.global_ctx.get_flags().accept_dns
&& self.global_ctx.get_ipv4().is_some(),
)
}
fn install_packet_receiver(&self, receiver: HostPacketReceiver) -> anyhow::Result<()> {
self.tun.install_packet_receiver(receiver)
}
@@ -145,6 +196,16 @@ mod tests {
global_ctx::{GlobalCtx, GlobalCtxEvent},
};
fn runtime_host(global_ctx: ArcGlobalCtx) -> Arc<NativeInstanceRuntimeHost> {
NativeInstanceRuntimeHost::new(
global_ctx,
#[cfg(feature = "tun")]
Arc::new(tokio::sync::Mutex::new(
crate::instance::shared_virtual_nic::SharedVirtualNicRegistry::new(),
)),
)
}
#[cfg(feature = "web-client")]
fn runtime_config(config: &TomlConfig) -> CoreInstanceRuntimeConfig {
let normalized = easytier_core::instance::CoreInstanceConfig::from_toml(config).unwrap();
@@ -157,7 +218,7 @@ mod tests {
#[test]
fn runtime_host_owns_event_subscription_context() {
let global_ctx = Arc::new(GlobalCtx::new(TomlConfig::default()));
let runtime_host = NativeInstanceRuntimeHost::new(global_ctx.clone());
let runtime_host = runtime_host(global_ctx.clone());
let mut events = runtime_host.subscribe_event();
global_ctx.issue_event(GlobalCtxEvent::CredentialChanged);
@@ -178,7 +239,7 @@ mod tests {
config.set_ipv4(Some("10.20.0.1/24".parse().unwrap()));
config.set_ipv6(Some("fd00::1/64".parse().unwrap()));
let global_ctx = Arc::new(GlobalCtx::new(config.clone()));
let runtime_host = NativeInstanceRuntimeHost::new(global_ctx.clone());
let runtime_host = runtime_host(global_ctx.clone());
assert_eq!(global_ctx.get_hostname(), "before");
assert_eq!(global_ctx.get_ipv4(), Some("10.20.0.1/24".parse().unwrap()));
@@ -220,7 +281,7 @@ mod tests {
let config = TomlConfig::default();
config.set_dhcp(true);
let global_ctx = Arc::new(GlobalCtx::new(config.clone()));
let runtime_host = NativeInstanceRuntimeHost::new(global_ctx.clone());
let runtime_host = runtime_host(global_ctx.clone());
let lease = "10.20.0.7/24".parse().unwrap();
global_ctx.set_ipv4(Some(lease));
@@ -3,9 +3,9 @@ use easytier_core::instance::CorePacketPlane;
#[cfg(feature = "magic-dns")]
use tokio_util::{sync::CancellationToken, task::AbortOnDropHandle};
use crate::common::global_ctx::ArcGlobalCtx;
#[cfg(feature = "magic-dns")]
use crate::instance::dns_server::{MAGIC_DNS_FAKE_IP, runner::DnsRunner};
use crate::{common::global_ctx::ArcGlobalCtx, instance::virtual_nic::NicBackend};
#[derive(Default)]
pub(super) struct MagicDnsRuntime {
@@ -26,6 +26,7 @@ impl MagicDnsRuntime {
packet_plane: std::sync::Arc<CorePacketPlane>,
tun_dev: Option<String>,
tun_ip: Ipv4Inet,
shared_route_backend: Option<NicBackend>,
) -> Self {
let active = global_ctx.get_flags().accept_dns.then(|| {
let mut runner = DnsRunner::new(
@@ -34,7 +35,8 @@ impl MagicDnsRuntime {
tun_dev,
tun_ip,
MAGIC_DNS_FAKE_IP.parse().unwrap(),
);
)
.with_shared_route_backend(shared_route_backend);
let cancel = CancellationToken::new();
let task_cancel = cancel.clone();
let task = tokio::spawn(async move {
@@ -54,6 +56,7 @@ impl MagicDnsRuntime {
_packet_plane: std::sync::Arc<CorePacketPlane>,
_tun_dev: Option<String>,
_tun_ip: Ipv4Inet,
_shared_route_backend: Option<NicBackend>,
) -> Self {
Self::default()
}
@@ -3,11 +3,43 @@ use std::{
sync::{Arc, OnceLock},
};
use easytier_core::host::packet::HostPacketReceiver;
use easytier_core::{host::packet::HostPacketReceiver, instance::CorePacketPlane};
use tokio::{sync::Mutex, task::JoinSet};
use super::MagicDnsRuntime;
use crate::instance::virtual_nic::NicCtx;
use crate::{
common::{error::Error, global_ctx::ArcGlobalCtx},
instance::{shared_virtual_nic::ArcSharedVirtualNicRegistry, virtual_nic::NicCtx},
};
pub(super) async fn create_nic_ctx(
global_ctx: ArcGlobalCtx,
packet_plane: Arc<CorePacketPlane>,
receiver: Arc<Mutex<HostPacketReceiver>>,
close_notifier: Arc<tokio::sync::Notify>,
registry: ArcSharedVirtualNicRegistry,
) -> Result<NicCtx, Error> {
#[cfg(not(mobile))]
if global_ctx.get_flags().dev_name.is_empty() {
return Ok(NicCtx::new(
global_ctx,
packet_plane,
receiver,
close_notifier,
));
}
let member_id = global_ctx.get_id();
NicCtx::new_shared(
global_ctx,
packet_plane,
receiver,
close_notifier,
registry,
member_id,
)
.await
}
struct NicCtxContainer {
_nic_ctx: Option<Box<dyn Any + Send>>,
@@ -65,6 +97,15 @@ impl TunNicState {
drop(old);
}
#[cfg(mobile)]
pub(super) async fn mobile_tun_claims(
&self,
) -> Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims> {
let nic = self.nic_ctx.lock().await;
let nic = nic.as_ref()?._nic_ctx.as_ref()?.downcast_ref::<NicCtx>()?;
nic.current_mobile_tun_claims().await
}
pub(super) async fn drain(&self) {
self.stop().await;
let receiver = self.receiver();
@@ -14,28 +14,37 @@ use tokio::{
};
use tokio_util::sync::CancellationToken;
use super::{MagicDnsRuntime, tun_common::TunNicState};
use super::{
MagicDnsRuntime,
tun_common::{TunNicState, create_nic_ctx},
};
use crate::{
common::{
error::Error,
global_ctx::{ArcGlobalCtx, GlobalCtxEvent},
},
instance::virtual_nic::NicCtx,
instance::shared_virtual_nic::ArcSharedVirtualNicRegistry,
};
pub(super) struct NativeTunRuntime {
global_ctx: ArcGlobalCtx,
cancel: CancellationToken,
nic: TunNicState,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
static_ip_task: Mutex<Option<JoinHandle<()>>>,
}
impl NativeTunRuntime {
pub(super) fn new(global_ctx: ArcGlobalCtx, cancel: CancellationToken) -> Self {
pub(super) fn new(
global_ctx: ArcGlobalCtx,
cancel: CancellationToken,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
) -> Self {
Self {
global_ctx,
cancel,
nic: TunNicState::empty(),
shared_virtual_nic_registry,
static_ip_task: Mutex::new(None),
}
}
@@ -67,6 +76,7 @@ impl NativeTunRuntime {
let cancel = self.cancel.clone();
let global_ctx = self.global_ctx.clone();
let receiver = self.nic.receiver();
let shared_virtual_nic_registry = self.shared_virtual_nic_registry.clone();
let (output, first_round) = oneshot::channel();
let task = tokio::spawn(async move {
let mut output = Some(output);
@@ -76,12 +86,29 @@ impl NativeTunRuntime {
return;
}
let closed = Arc::new(Notify::new());
let mut nic = NicCtx::new(
let mut nic = match create_nic_ctx(
global_ctx.clone(),
packet_plane.clone(),
receiver.clone(),
closed.clone(),
);
shared_virtual_nic_registry.clone(),
)
.await
{
Ok(nic) => nic,
Err(error) => {
if let Some(output) = output.take() {
let _ = output.send(Err(error));
return;
}
tracing::error!(?error, "failed to create native interface context");
tokio::select! {
_ = cancel.cancelled() => return,
_ = tokio::time::sleep(Duration::from_secs(1)) => {}
}
continue;
}
};
let result = tokio::select! {
biased;
_ = cancel.cancelled() => {
@@ -104,11 +131,13 @@ impl NativeTunRuntime {
}
let magic_dns = if let Some(ip) = ipv4 {
let shared_route_backend = nic.shared_route_backend_for_dns();
MagicDnsRuntime::start(
global_ctx.clone(),
packet_plane.clone(),
nic.ifname().await,
ip,
shared_route_backend,
)
} else {
MagicDnsRuntime::default()
@@ -161,6 +190,7 @@ impl NativeTunRuntime {
nic: self.nic.clone(),
closed: Arc::new(Notify::new()),
packet_plane,
shared_virtual_nic_registry: self.shared_virtual_nic_registry.clone(),
})
}
}
@@ -172,6 +202,7 @@ struct NativeDhcpIpv4Host {
nic: TunNicState,
closed: Arc<Notify>,
packet_plane: Arc<CorePacketPlane>,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
}
impl NativeDhcpIpv4Host {
@@ -199,21 +230,25 @@ impl NativeDhcpIpv4Host {
return Ok(Some(ip));
}
let mut nic = NicCtx::new(
let mut nic = create_nic_ctx(
self.global_ctx.clone(),
self.packet_plane.clone(),
self.nic.receiver(),
self.closed.clone(),
);
self.shared_virtual_nic_registry.clone(),
)
.await?;
tokio::select! {
_ = self.cancel.cancelled() => anyhow::bail!("instance is closing; DHCP update cancelled"),
result = nic.run(Some(ip), self.global_ctx.get_ipv6()) => result?,
}
let shared_route_backend = nic.shared_route_backend_for_dns();
let magic_dns = MagicDnsRuntime::start(
self.global_ctx.clone(),
self.packet_plane.clone(),
nic.ifname().await,
ip,
shared_route_backend,
);
self.nic.install(nic, magic_dns).await;
self.global_ctx.set_ipv4(Some(ip));
+111 -24
View File
@@ -8,26 +8,41 @@ use easytier_core::{
instance::CorePacketPlane,
};
use futures::FutureExt as _;
use tokio::sync::{Mutex, Notify, mpsc};
use tokio::sync::{Mutex, Notify, mpsc, oneshot};
use tokio_util::sync::CancellationToken;
use super::{MagicDnsRuntime, tun_common::TunNicState};
use super::{
MagicDnsRuntime,
tun_common::{TunNicState, create_nic_ctx},
};
use crate::{
common::global_ctx::{ArcGlobalCtx, GlobalCtxEvent},
instance::virtual_nic::NicCtx,
instance::shared_virtual_nic::ArcSharedVirtualNicRegistry,
};
struct MobileTunAttachment {
fd: i32,
replace_tun_fd: bool,
completion: Option<oneshot::Sender<anyhow::Result<()>>>,
claims: Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims>,
}
pub(super) struct NativeTunRuntime {
global_ctx: ArcGlobalCtx,
cancel: CancellationToken,
nic: TunNicState,
tun_fd: mpsc::Sender<i32>,
tun_fd_receiver: Mutex<Option<mpsc::Receiver<i32>>>,
tun_fd: mpsc::Sender<MobileTunAttachment>,
tun_fd_receiver: Mutex<Option<mpsc::Receiver<MobileTunAttachment>>>,
task: Mutex<Option<tokio::task::JoinHandle<()>>>,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
}
impl NativeTunRuntime {
pub(super) fn new(global_ctx: ArcGlobalCtx, cancel: CancellationToken) -> Self {
pub(super) fn new(
global_ctx: ArcGlobalCtx,
cancel: CancellationToken,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
) -> Self {
let (tun_fd, tun_fd_receiver) = mpsc::channel(16);
Self {
global_ctx,
@@ -36,6 +51,7 @@ impl NativeTunRuntime {
tun_fd,
tun_fd_receiver: Mutex::new(Some(tun_fd_receiver)),
task: Mutex::new(None),
shared_virtual_nic_registry,
}
}
@@ -51,23 +67,39 @@ impl NativeTunRuntime {
global_ctx: ArcGlobalCtx,
packet_plane: Arc<CorePacketPlane>,
fd: i32,
replace_tun_fd: bool,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
claims: Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims>,
) -> anyhow::Result<()> {
nic_state.drain().await;
if fd <= 0 {
return Ok(());
}
let closed = Arc::new(Notify::new());
let mut nic = NicCtx::new(
let mut nic = create_nic_ctx(
global_ctx.clone(),
packet_plane.clone(),
nic_state.receiver(),
closed,
);
nic.run_for_mobile(fd).await.context("add ip failed")?;
let magic_dns = global_ctx
.get_ipv4()
.map(|ip| MagicDnsRuntime::start(global_ctx, packet_plane, None, ip))
.unwrap_or_default();
shared_virtual_nic_registry,
)
.await?;
let claims = match claims {
Some(claims) => claims,
None => {
crate::instance::virtual_nic::NicCtx::mobile_tun_claims(&global_ctx, &packet_plane)
.await
}
};
nic.run_for_mobile(fd, replace_tun_fd, claims)
.await
.context("attach mobile TUN failed")?;
let magic_dns = if let Some(ip) = global_ctx.get_ipv4() {
let shared_route_backend = nic.shared_route_backend_for_dns();
MagicDnsRuntime::start(global_ctx, packet_plane, None, ip, shared_route_backend)
} else {
MagicDnsRuntime::default()
};
nic_state.install(nic, magic_dns).await;
Ok(())
}
@@ -80,28 +112,44 @@ impl NativeTunRuntime {
let nic_state = self.nic.clone();
let global_ctx = self.global_ctx.clone();
let cancel = self.cancel.clone();
let shared_virtual_nic_registry = self.shared_virtual_nic_registry.clone();
self.task.lock().await.replace(tokio::spawn(async move {
loop {
let fd = tokio::select! {
let attachment = tokio::select! {
_ = cancel.cancelled() => return,
fd = tun_fds.recv() => match fd { Some(fd) => fd, None => return },
attachment = tun_fds.recv() => match attachment {
Some(attachment) => attachment,
None => return,
},
};
if let Err(error) = Self::install_mobile_tun(
nic_state.clone(),
global_ctx.clone(),
packet_plane.clone(),
fd,
)
.await
{
let result = if attachment.fd <= 0 {
nic_state.drain().await;
Ok(())
} else {
Self::install_mobile_tun(
nic_state.clone(),
global_ctx.clone(),
packet_plane.clone(),
attachment.fd,
attachment.replace_tun_fd,
shared_virtual_nic_registry.clone(),
attachment.claims,
)
.await
};
if let Err(error) = &result {
tracing::error!(?error, "failed to attach mobile TUN fd");
}
if let Some(completion) = attachment.completion {
let _ = completion.send(result);
}
}
}));
Ok(())
}
pub(super) async fn shutdown(&self) {
self.cancel.cancel();
if let Some(task) = self.task.lock().await.take() {
let _ = task.await;
}
@@ -110,10 +158,43 @@ impl NativeTunRuntime {
pub(super) fn attach_fd(&self, fd: i32) -> anyhow::Result<()> {
self.tun_fd
.try_send(fd)
.try_send(MobileTunAttachment {
fd,
replace_tun_fd: true,
completion: None,
claims: None,
})
.map_err(|error| anyhow::anyhow!("failed to send TUN fd: {error}"))
}
pub(super) async fn attach_mobile_fd(
&self,
fd: i32,
replace_tun_fd: bool,
claims: Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims>,
) -> anyhow::Result<()> {
if self.task.lock().await.is_none() {
anyhow::bail!("mobile TUN runtime is not running");
}
let (completion, result) = oneshot::channel();
tokio::select! {
_ = self.cancel.cancelled() => anyhow::bail!("instance is closing; TUN attachment cancelled"),
send_result = self.tun_fd.send(MobileTunAttachment {
fd,
replace_tun_fd,
completion: Some(completion),
claims,
}) => send_result.map_err(|error| anyhow::anyhow!("failed to send TUN fd: {error}"))?,
}
tokio::select! {
_ = self.cancel.cancelled() => anyhow::bail!("instance is closing; TUN attachment cancelled"),
result = result => result
.map_err(|_| anyhow::anyhow!("mobile TUN runtime stopped before attachment completed"))?,
}
}
pub(super) fn dhcp_host(
&self,
operation: Arc<Mutex<()>>,
@@ -127,6 +208,12 @@ impl NativeTunRuntime {
closed: Arc::new(Notify::new()),
})
}
pub(super) async fn current_mobile_tun_claims(
&self,
) -> Option<crate::instance::shared_virtual_nic::SharedIfConfigClaims> {
self.nic.mobile_tun_claims().await
}
}
struct NativeDhcpIpv4Host {
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+40 -5
View File
@@ -7,6 +7,8 @@ use easytier_core::{
process_runtime::CoreProcessRuntime,
};
#[cfg(feature = "tun")]
use crate::instance::shared_virtual_nic::{ArcSharedVirtualNicRegistry, SharedVirtualNicRegistry};
use crate::{
common::global_ctx::{ArcGlobalCtx, GlobalCtx},
instance::{
@@ -15,6 +17,8 @@ use crate::{
},
socket::udp::RuntimeUdpSocket,
};
#[cfg(feature = "tun")]
use tokio::sync::Mutex;
pub(crate) struct TestInstance {
core: Arc<NativeCoreInstance>,
@@ -26,7 +30,27 @@ impl TestInstance {
config: TomlConfig,
process_runtime: Arc<CoreProcessRuntime>,
) -> Self {
Self::compose(config, process_runtime, |_| {})
Self::compose(
config,
process_runtime,
#[cfg(feature = "tun")]
Arc::new(Mutex::new(SharedVirtualNicRegistry::new())),
|_| {},
)
}
#[cfg(feature = "tun")]
pub fn new_with_process_runtime_and_shared_virtual_nic_registry(
config: TomlConfig,
process_runtime: Arc<CoreProcessRuntime>,
shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
) -> Self {
Self::compose(config, process_runtime, shared_virtual_nic_registry, |_| {})
}
#[cfg(feature = "tun")]
pub fn new_shared_virtual_nic_registry() -> ArcSharedVirtualNicRegistry {
Arc::new(Mutex::new(SharedVirtualNicRegistry::new()))
}
pub fn new_with_process_runtime_and_stun_provider(
@@ -35,14 +59,21 @@ impl TestInstance {
provider: Box<dyn StunSocketMapper<RuntimeUdpSocket>>,
) -> Self {
let provider: Arc<dyn StunSocketMapper<RuntimeUdpSocket>> = Arc::from(provider);
Self::compose(config, process_runtime, move |adapters| {
adapters.replace_stun_provider(provider);
})
Self::compose(
config,
process_runtime,
#[cfg(feature = "tun")]
Arc::new(Mutex::new(SharedVirtualNicRegistry::new())),
move |adapters| {
adapters.replace_stun_provider(provider);
},
)
}
fn compose(
config: TomlConfig,
process_runtime: Arc<CoreProcessRuntime>,
#[cfg(feature = "tun")] shared_virtual_nic_registry: ArcSharedVirtualNicRegistry,
customize: impl FnOnce(
&mut easytier_core::instance::CoreHostAdapters<
crate::instance::host::NativeInstanceHost,
@@ -50,7 +81,11 @@ impl TestInstance {
),
) -> Self {
let global_ctx = Arc::new(GlobalCtx::new(config.clone()));
let runtime_host = NativeInstanceRuntimeHost::new(global_ctx.clone());
let runtime_host = NativeInstanceRuntimeHost::new(
global_ctx.clone(),
#[cfg(feature = "tun")]
shared_virtual_nic_registry,
);
let mut adapters = runtime_core_host_adapters_with_packet_egress(
global_ctx.clone(),
process_runtime,
File diff suppressed because it is too large. Load diff