EasyTier/docs/core-architecture.md

# EasyTier Core Architecture

## Status and scope

This document describes the current architecture after the portable-core
refactor. It is the source of truth for ownership, dependency direction,
feature boundaries, and validation. It intentionally records the resulting
design rather than the migration history.

The refactor has three principal crate roles:

- `easytier-core` owns portable EasyTier configuration, protocol state,
  routing, peer state, connectivity orchestration, packet processing, and
  instance lifecycle.
- `easytier` is the native composition root. It owns operating-system
  resources, native protocol engines, process integration, CLI and native
  presentation.
- `easytier-proto` owns generated protobuf and RPC types, descriptor data, and
  the feature slices needed by core and presentation users.

`easytier-core` is designed to compile without direct operating-system network
access. It supports native hosts through Rust traits and has a target-only WASI
adapter and ABI implementation under `easytier-core/src/wasi`.

This architecture does not require compatibility with old internal module
paths. Wire compatibility, configuration compatibility, management semantics,
and externally used application behaviour remain compatibility requirements.

## Architectural vocabulary

The following terms have specific meanings in this document:

- **Module**: an interface and the implementation hidden behind it.
- **Host**: the process or runtime embedding core and owning platform
  resources.
- **Host capability**: an operation core may request but must not implement
  with direct OS calls.
- **Adapter**: a concrete implementation of a Host capability or protocol
  extension.
- **Composition root**: code that creates core configuration, Host Adapters,
  instances, and process-level services.
- **Runtime configuration**: the authoritative normalized state used after an
  instance starts.
- **Packet plane**: portable packet classification, routing, transformation,
  proxy/NAT state, and forwarding decisions.

New abstractions should pass a deletion test: deleting a useful deep Module
should force non-trivial policy or lifecycle logic to reappear in multiple
callers. A pass-through wrapper with no independent invariant is not an
architectural boundary.

## Crate dependency direction

The principal dependency direction is:

```text
easytier-proto <- easytier-core <- easytier
```

Presentation crates and platform integrations consume these crates. Portable
policy must not move outward merely because one current consumer is native.
Conversely, core must not absorb an OS mechanism or a protocol engine whose
dependencies cannot satisfy the core target contract.

### `easytier-proto`

The protobuf crate is split by public Cargo features:

- `core` provides the common wire messages, peer RPC messages, generated RPC
  runtime, and descriptor bytes needed by core.
- `api` adds management API messages.
- protocol-specific features add only their generated message modules.
- `json-rpc` enables the well-known protobuf JSON types used by the management
  plane.
- `full` is the compatibility aggregate used by complete products.

The core crate depends on `easytier-proto` with default features disabled and
enables only `core`, adding API or JSON-RPC types through its own management
features.

The main core/native path has no `prost-reflect` dependency. OSPF route
reflection uses the focused wire editor in
`peers/route/route_peer_wire.rs`. It retains the original encoded
`RoutePeerInfo`, replaces only the fields credential filtering is allowed to
change, and leaves all other top-level and nested fields intact. This is
required so unknown fields survive mixed-version, multi-hop propagation.
Generated Rust types remain responsible for normal message construction and
validation.

Descriptor sets are still generated and embedded by `easytier-proto`; removing
runtime reflection did not remove descriptor data used by configuration and
RPC tooling. The OHOS integration has its own schema service and dependency
policy and is outside this replacement.

### `easytier-core`

Core owns portable behaviour and exposes capability seams. Its normal
dependencies use Tokio runtime, time, synchronization, and I/O traits without
requiring the full Tokio feature set.

Core may depend on optional portable engines when their owning feature is
enabled. It does not create real native TCP/UDP sockets, alter routes, open a
TUN device, enter a network namespace, configure system DNS, manage a service,
or invoke UPnP/NAT-PMP directly.

### `easytier`

The native crate owns:

- process startup, shutdown, signals, service management, and allocators;
- filesystem configuration input and persistence;
- real TCP/UDP, DNS, TUN, raw-socket, route, interface, namespace, and socket
  option operations;
- UPnP and NAT-PMP operations;
- Unix and FakeTCP resources;
- WebSocket/WSS, QUIC, WireGuard, and KCP concrete engines;
- native Magic DNS serving and system DNS integration;
- CLI, web, GUI, FFI, and native management presentation.

Native code may translate values and assemble Adapters. It must not maintain a
second peer graph, reproduce core routing or hole-punch policy, or invent an
alternative instance lifecycle.

## Internal core layers

The physical module layout follows this downward order:

```text
foundation
  <- config / packet
  <- socket
  <- host
  <- tunnel
  <- listener / connectivity
  <- peers / rpc
  <- gateway
  <- instance
  <- management
```

`process_runtime` is a process- or module-scoped owner shared by instances.
`wasi` is target integration and is compiled only for tests or the WASI target;
it is not an additional portable domain layer.

### Foundation

`foundation/` contains task supervision, the time facade, rate limiting,
statistics primitives, and the domain-neutral external operation broker. The
broker owns asynchronous operation lifecycle and completion storage while the
calling domain owns operation kinds, outcomes, resources, and errors.
Foundation must not depend on a domain layer.

### Configuration and packets

`config/` owns:

- the complete `TomlConfig` model;
- parsing, serialization, and validation;
- OS-independent defaults;
- peer, encryption, gateway, and API input models;
- normalized runtime snapshots and the live runtime configuration store.

The Host supplies platform facts through `CoreInstanceHostConfig`. Core applies
the policy that combines those facts with TOML input. This is especially
important for a WASI build: the compile-time guest target cannot be used as a
proxy for the Host operating system.

`packet/` owns EasyTier packet structures, compression, STUN and hole-punch
wire codecs. It does not own socket I/O or connection policy.

### Socket and Host seams

`socket/` contains transport-neutral primitives:

- `SocketContext`, including IP-family policy, optional socket mark, and an
  opaque network-namespace token;
- virtual TCP socket, listener, and factory traits;
- virtual UDP socket and factory traits;
- UDP session multiplexing, classification, and lifecycle;
- in-process Ring sockets.

`host/` is the single home of Host capability seams:

- DNS and DNS record resolution;
- connector environment observations;
- packet ingress and egress;
- Host socket operation bridges and handle-based TCP/UDP/listener adapters;
- host-owned, message-preserving tunnel endpoints.

Core owns scheduling, backpressure, cancellation, UDP session state, and
protocol state even when each actual operation crosses a Host Adapter. A Host
Adapter owns the real resource and performs the OS operation.

For host-owned tunnels, ownership crosses the guest ABI only after a bounded
Host listener queue accepts the tunnel. The queue is registered through
`CoreHostAdapters` and consumed by the normal `CoreListenerRuntime`, so the
tunnel still reaches `PeerAcceptedTunnelHandler` and cannot bypass peer
handshake, admission, events, or routing policy. Each Host receive produces
one complete `DummyTunnel` payload; no stream framing is added. The Host owns
transport-specific message validation and maps a clean close to tunnel EOF.

Hosts without outbound sockets select `CoreConnectivityMode::InboundOnly` for
one instance. That startup plan retains listeners and the peer/router while
omitting STUN, outbound connectors, and hole punching. Cargo features only
compile Host Adapters; they do not change `CoreInstance` fields, lifecycle, or
management semantics.

The native `NativeHostRuntime` is process-wide and does not retain an instance
`GlobalCtx`, namespace guard, socket mark, or connectivity state. Differences
between instances travel in each request's `SocketContext`. A narrow
instance-host projection may expose listener and interface facts, but it does
not become another socket factory.

### Tunnel and listener

A socket is a raw communication endpoint. A Tunnel is an EasyTier connection
created by adding framing, metadata, handshakes, and protocol lifecycle.

Core owns:

- raw TCP framing and upgrade;
- UDP tunnel/session framing and classification;
- Ring Tunnel identity and registry state;
- encryption and secure-datagram policy that is portable;
- client/server protocol selection interfaces;
- listener planning, optional/required listener policy, retry, accept
  scheduling, running-listener registry, and orderly shutdown.

Native protocol Adapters own WebSocket/WSS, QUIC, WireGuard, and KCP engines.
Unix and FakeTCP are socket resources that feed a core protocol upgrader; they
are not independent owners of EasyTier peer state.

Each protocol registration must provide a coherent client/server Adapter.
Unavailable configured transports must be rejected during validation or
protocol selection in the standard runtime, rather than silently falling back
to another transport. A compact compatibility Host may instead retain the
desired value for management readback and omit it from normalized runtime
state; it must not advertise or partially activate the unavailable transport.

### Connectivity

`connectivity/` owns:

- manual connection and endpoint discovery policy;
- direct candidate selection;
- retry, backoff, blacklists, and listener reuse;
- STUN requests, responses, probing, NAT inference, and published endpoint
  state;
- TCP and UDP hole-punch state machines;
- UDP port-mapping policy and lease lifecycle;
- conversion of successful sockets into protocol-upgrade requests.

The Host owns DNS execution, socket syscalls, interface enumeration, bind
device/mark/namespace operations, and concrete UPnP/NAT-PMP calls. STUN-only
hole punching remains available when the Host does not supply a port-mapping
Adapter.

Some connectivity files intentionally implement peer-facing adapter traits for
`PeerManagerCore`. These are localized integration edges between adjacent
domains, not permission for lower socket or Host layers to depend on peers.

### Peers and RPC

`peers/` is the authoritative owner of:

- admission and connection sessions;
- peer maps and connection lifecycle;
- ACL and whitelist decisions;
- OSPF route calculation and graph algorithms;
- peer and credential RPC registration;
- foreign-network admission, identity, relay, and lifecycle;
- peer-center state and public IPv6 policy;
- traffic metrics and peer snapshots.

Submodules progress from kernel types and utilities, through ACL/context,
connection state, route state, manager services, and finally foreign-network
and peer-center composition. Callers consume the public surface declared by
the domain rather than reaching into a parallel native peer owner.

`rpc/` owns the peer-flavoured RPC transport, packet fragmentation, client and
server lifecycle, handler registry, and standalone listener/client lifecycle.
Generated service descriptors and message types remain in `easytier-proto`.

### Gateway

`gateway/` owns portable packet-plane features:

- proxy CIDR state and monitoring policy;
- packet parsing, reassembly, NAT/proxy state, and TCP/UDP/ICMP decisions;
- the smoltcp-backed portable dataplane selected by its feature;
- SOCKS5 framing, authentication, association, routing, and session state;
- wrapped-transport planning and session state used by KCP and QUIC Adapters;
- DHCP allocation policy;
- Magic DNS route and response policy;
- VPN portal client/session policy;
- UDP broadcast classification and rewrite policy.

Each VPN portal client is normalized to one attached-peer IPv4 CIDR. The
portable gateway owns that client address and prefix; the hosting network
manager's DHCP or static address is not a source of portal client routing
facts.

TUN, raw sockets, transparent-destination lookup, concrete protocol engines,
native DNS servers, namespace operations, and route application stay in native
Adapters.

Optional gateway capabilities are selected by cohesive Modules. Disabled
implementations retain stable lifecycle calls and report unsupported
configuration in the standard runtime. A compact compatibility Host may
silently normalize those settings to no-ops while preserving the desired TOML
model; disabled implementations do not duplicate portable policy.

The instance-scoped `DataPlaneSession` composes the foundation operation broker
under the same session lock as its resource and quota state. The broker owns
generic completion, cancellation, free, drain, and take transitions. The data
plane retains TCP/UDP resource ownership, operation metadata, route deadlines,
and error semantics.

The proposed restructuring of the smoltcp data plane, SOCKS5 and port-forward
Adapters, portable KCP engine, event-driven FFI/WASI completion model, and Go
Host integration is tracked in
[`data-plane-runtime-plan.md`](data-plane-runtime-plan.md). That document is a
future implementation plan; this document remains the source of truth for the
currently implemented architecture until the plan is completed.

### Instance and management

`CoreInstance::new(CoreInstanceConfig, CoreHostAdapters)` is the sole direct
construction path for a normalized instance. `CoreInstance::from_toml` uses
the same normalization and construction path. Core constructs the peer graph,
runtime store, STUN collector, connectivity managers, listener runtime, packet
plane, gateway runtimes, and lifecycle owners.

A core instance:

- owns all mutable portable state for one network;
- is one-shot after `stop`;
- exposes one complete `start` and one `stop` lifecycle interface;
- starts Modules in a fixed serial composition order without cross-Module
  started flags or staged activation;
- installs initial ACL, proxy CIDR, and manual-peer inputs before startup;
- serializes lifecycle operations with one instance-level operation lock;
- owns cooperative cancellation and component shutdown order;
- exposes `CorePacketPlane` as the narrow packet/route projection used by Host
  dataplane Adapters;
- treats its normalized runtime store as authoritative after construction.

`CoreHostAdapters` contains the required Host, DNS, packet sink, and
`CoreProcessRuntime`, plus optional protocol and platform capabilities. The
bundle carries capabilities, not preconstructed portable managers.

Each Module owns partial-start cleanup for its internal resources.
`CoreInstance` has one outer cancellation and recovery path for the complete
serial startup. `Running` therefore means the Host runtime and every enabled
portable Module have started successfully; there is no separate post-Host
activation state. Host packet tasks stop before PeerManager resources are
cleared.

`InstanceManager<F>` is the canonical UUID-indexed instance collection for one
Host composition. Its `InstanceFactory` constructs one complete record before
the manager performs an atomic uniqueness check. The manager owns collection
membership; it does not own startup order, persistence, daemon policy, cached
errors, ABI handles, or RPC projections.

`management/` consumes the canonical manager and instances. It owns:

- stable UUID/name selection;
- read-only instance and peer management RPC;
- full process mutation and configuration transactions when enabled;
- persistence and logger-control capability interfaces;
- management listener/client lifecycle and JSON-RPC presentation.

There is one process-level management entry. Instances and the manager do not
depend on management response projections.

## Process-scoped state

`CoreProcessRuntime` owns portable resources shared across instances in one
process or instantiated module:

- the Ring Tunnel registry and namespace;
- a reference-counted protected TCP-port registry.

The composition root creates and shares one runtime. Management listener ports
are protected before bind and held by leases after the concrete port is known.
Native and target adapters supply bound resources but do not implement a
second protected-port registry.

Process-global capability objects may contain stateless or shared platform
mechanisms. They must not contain instance-specific peer, route,
configuration, or connectivity state.

## Runtime configuration authority

`TomlConfig` is the authoritative desired configuration used for management
readback and patch transactions. Compact Hosts keep unsupported accepted values
there so controllers observe the configuration they submitted.

The separately typed, normalized core runtime store is authoritative for live
behavior:

- peer feature flags and routing policy;
- listeners and initial peers;
- ACL and whitelist inputs;
- manual and VPN portal CIDRs;
- gateway and connectivity settings;
- runtime configuration patches.

Host persistence is an effect following a successful core transaction. A Host
Adapter must not call back into an instance to obtain a hidden configuration
snapshot while core is applying an operation.

Non-serializable resources such as TUN descriptors, packet sinks, execution
domains, and native protocol engines are construction context, not TOML
fields.

## Logging

The main native runtime uses a small logger implemented in
`easytier/src/common/log`:

- `log` records and `tracing` events share console and file sinks;
- timestamps, compact formatting, optional terminal colours, `NO_COLOR`, and
  basic `RUST_LOG` target/level filters are implemented directly;
- file rotation uses the existing EasyTier rolling appender;
- management RPC can reload the file level;
- an atomic maximum-level gate rejects disabled events before target matching
  or file-filter locking;
- concurrent file-level reload serializes the filter and atomic-level update.

File logging and no-file logging are separate selected backends. The default
tracing backend records events and deliberately ignores span trees. The
optional `tracing` feature selects the tokio-console subscriber integration;
only that diagnostic profile pulls the main crate's `tracing-subscriber` and
`console-subscriber` dependencies.

Contrib applications and platform integrations may have independent logging
requirements and are not implicitly wired to the native process logger.

## Feature model

Features represent coherent capabilities, not arbitrary source fragments.
Important core feature relationships are:

- `management-rpc` enables generated management API types and read-only
  management services.
- `management` adds configuration writes, full management composition, rich
  errors, and JSON-RPC.
- `proxy-packet` enables portable packet parsing/proxy machinery and the
  required smoltcp packet features.
- `proxy-smoltcp-stack` adds the async TCP/UDP smoltcp stack.
- `dns-resolver` is the shared Hickory resolver leaf used by endpoint
  discovery and Magic DNS without coupling either capability to the other.
- `endpoint-discovery` adds HTTPS endpoint discovery dependencies.
- `magic-dns` enables its DNS server, management wire messages, and portable
  packet-query integration.
- `tcp-hole-punch` enables the TCP hole-punch runtime.
- `dhcp-ipv4`, `public-ipv6-provider`, `vpn-portal`,
  `wrapped-transport`, and `proxy-cidr-monitor` are independent gateway or
  platform-policy leaves.
- `extended-services` is the compatibility aggregate for those leaves.
- encryption and compression engines remain independently selectable.

The native crate maps product features to the core and protocol features it
actually consumes. A protocol feature must not accidentally enable unrelated
gateway or management capabilities.

Production feature and platform selection belongs at Module or Adapter
boundaries rather than inside shared implementations. The logger demonstrates
the intended pattern: file and tracing variants are complete backend modules
with one stable interface, so shared event processing contains no feature
branches.

## Module boundaries

The dependency directions in this document define the intended module
boundaries. Changes that require a new upward edge must first define a stable
lower-layer interface or explicitly revise this architecture.

Modules are `pub(crate)` by default. Each domain's `mod.rs` declares its
outward surface. Public visibility is used for real cross-crate Host,
configuration, management, packet-plane, or test-support interfaces.

## Architectural invariants

1. Portable EasyTier policy has one owner in `easytier-core`.
2. Core does not perform real OS socket, DNS, TUN, route, filesystem
   configuration, process, or service-manager operations.
3. Host-OS policy is runtime input; a WASI compile target is not Host policy.
4. Every real socket and DNS operation crosses a Host capability seam.
5. Core owns socket scheduling, backpressure, protocol state, and cancellation.
6. Dial, accept, and hole-punch paths produce sockets before protocol upgrade.
7. Peer admission consumes upgraded transports and does not create OS
   resources.
8. Each instance owns its mutable peer, route, connectivity, gateway, and
   runtime configuration state.
9. One Host composition has one canonical UUID-to-instance manager.
10. Process-level runtimes do not capture instance state.
11. `CoreInstance::new` is the sole normalized direct construction entry.
12. The manager owns membership, not lifecycle or presentation.
13. Management consumes the manager; the manager does not return management
    projections.
14. Unknown protobuf fields in reflected route information survive forwarding
    and credential filtering.
15. Feature selection is localized at cohesive Module/Adapter boundaries.
16. The standard runtime rejects unsupported configured capabilities. Compact
    compatibility Hosts may preserve them as runtime no-ops, but never change
    wire protocol, advertise them, or silently fall back to an unsafe mode.

## Validation

Changes to these boundaries should run, at minimum:

```text
cargo fmt --all -- --check
cargo check -p easytier-core -p easytier-proto -p easytier --features full
cargo test -p easytier-core --lib
```

Feature work should add focused checks for the changed no-default, isolated,
default, full, and cross-target profiles. Socket, TUN, namespace, protocol
engine, and multi-node changes require the relevant Docker integration tests.
WASI ABI or Adapter changes require a `wasm32-wasip1` build and target-side
tests. These compiler-resolved profiles are the authority for feature and
target boundaries.

CI path filters include `easytier-core`, `easytier-proto`, native, web, GUI
Tauri, and contrib. The archived Rust test suite contains both `easytier` and
`easytier-core`.

## Known limitations and debt

- Some production feature and platform gates still select fields or statements
  inside shared implementations. New code should prefer complete Module or
  Adapter variants, and existing cases should move only when their owning
  Module is changed.
- Connectivity retains localized Adapter implementations that name
  `PeerManagerCore`; further decoupling requires an interface extraction, not
  a visibility-only move.
- Native Linux namespace guards exist in paths that can cross async suspension.
  Because `setns` is thread-local, those operations should eventually be kept
  on one non-migrating execution context.
- QUIC session retirement after failed or exhausted accepted sessions remains
  separate native-engine correctness work; it must preserve multiple
  connections sharing one QUIC endpoint/session.

These limitations are not reasons to add fallback owners or parallel state.
Fixes should preserve the ownership rules above and address the responsible
Module directly.