Files
EasyTier/easytier-go
KKRainbow c96b6c1961 fix(web): harden managed config sync between console and clients (#2567)
* fix(web): fence managed config runtime reconciliation

Keep runtime reconciliation tied to the currently authorized session so
stale connections cannot mutate a replacement session runtime.

Accumulate only contiguous dirty IDs and load their latest SQLite state.
Require the applied revision to match the earliest Patch base and the
persisted revision to match the latest target. Otherwise, reconcile the
full desired state.

Use separate runtime-state and config-cache epochs. Managed updates can
reuse observed configs; direct mutations invalidate them. Update sync
documentation to match.

* fix(web): interrupt validation retry on state changes

Track meaningful validation state changes separately from periodic dirty signals. Applied revision changes wake a failed validation immediately, while heartbeat-driven revalidation retains the retry backoff.

Treat Notify as a wake-up hint and recheck the state-change epoch after every wake so stored permits and periodic heartbeats cannot cause retry storms.

* fix(web): retry unconfirmed connected webhooks

Retry node-connected webhook delivery on retryable errors with a
short 100ms/500ms backoff and give up immediately on non-retryable
errors. Re-check that the session still owns the connection before
every attempt and before recording the delivery, so a replaced
session can no longer record a stale connected binding.

* fix(web): fence disconnects by session ownership

Return whether session removal actually removed the current route owner, and emit disconnected only for that owner. Replaced sessions can no longer invalidate a newer connected route.

* fix(web): hot-patch managed hostnames

Include hostname changes in the hot-patch path instead of falling
back to a full restart. When a full overwrite run is required and
the desired config has no hostname, inherit the current runtime
hostname so an unmanaged value survives until it is explicitly
cleared.

Read back the runtime config after an overwrite run and verify it
converged instead of assuming the desired state was applied.

* fix(web): retry transient runtime reconciliation failures

Keep the per-session managed runtime reconciliation worker alive when a
single database round fails. Retry from the next heartbeat so persisted
managed revisions can still converge after restart-time contention.

Reserve terminal worker shutdown for destroyed session or storage state,
and cover recovery after a transient revision read failure.

* fix(web): accept omitted hostname after runtime apply

Release 2.6.4 omits hostname from config readback when it matches the device hostname. Trust a successful hostname mutation only when the returned field is absent, while continuing to verify every other field and rejecting explicit mismatches.

* fix(web): ignore unmanaged runtime device names

Windows release 2.6.4 generates a random interface name when the managed config leaves dev_name empty. Exclude that runtime-owned value from reconciliation unless the desired config explicitly sets a non-empty device name, preventing endless overwrite restarts.

* feat(web): report failed network instances to console

Expose stopped Core instances with startup errors in heartbeats.

Merge Core failures with direct managed-run RPC failures in easytier-web.

Send failed instance IDs during token validation without error text.

Prune local run failures when managed configs are deleted.

* fix(web): distinguish unknown runtime application state

Track whether the current session has observed its applied revision
separately from the optional revision value. Report this fact through
validate-token so Console can preserve application state across
receiver restarts while recognizing deliberate pending mutations.

* feat(web): configure heartbeat timing from server

Heartbeat responses now provide the interval and RPC timeout.

Legacy servers use local defaults and remote values are clamped.

Web configuration and session receive timeout follow the policy.

* fix(web): reject inactive control sessions

Route control RPCs by machine id only to sessions whose RPC manager
is still running, so a session that has been stopped or replaced
can no longer receive control traffic addressed to the device.

* fix(core): filter network info before collection

When a collect-network-info request names specific instances,
collect those instances only instead of collecting every instance
and filtering the result afterwards, so unrequested instances no
longer run per-collection work on every request.

* feat(web): enable focused runtime diagnostics

Enable easytier-web info logs by default while preserving explicit log configuration. Record startup settings, session lifecycle, failed instance changes, webhook queue and request latency, and managed runtime operation timings for production diagnosis.

* fix(web): preserve managed revision across reconnects

Keep one runtime identifier for each Core WebClient lifetime.

Reuse its managed runtime state after transport reconnects.

Retain applied revisions and reconcile hints while disconnected.

Preserve runtime epochs so stale work cannot mark a revision applied.

Reject stale sessions from reclaiming routes after reconnect.

Core or Web restarts and legacy clients still use unknown state.

Immediately revalidate a restored revision after authentication.

Document local management RPC drift as an accepted trade-off.

This lets Console converge without waiting for periodic validation.

* fix(web): satisfy clippy across managed config sync tests

Scope managed runtime guards to blocks in runtime revision tests so
no std MutexGuard is held across await points, return the applied
revision directly instead of through a let binding, and pass
WebhookValidationInput to request_heartbeat_validation instead of
expanding it into eight separate arguments.

* fix(core): stop reporting failed instances as running in heartbeats

A stopped instance with a startup error appeared in both
running_network_instances and failed_network_instances, so the
server treated it as running and never re-ran its managed config.
Exclude failed instance ids when building the running list so the
reconciler restarts them.

* fix(core): close missed-wakeup race in instance state changes

wait_for_change created the Notified future before reading the
generation but only registered it when awaited. A change landing in
between fired notify_waiters with no registered waiter and delayed
the heartbeat by a full interval. Enable the future before reading
the generation so every change wakes a waiting heartbeat.

* fix(web): address review findings

Fence webhook validation and connection transitions against stale
state, redact credentials from default-level logs, and stabilize
runtime reconciliation:

- Record connected bindings only while the session still owns the
  machine route, and skip disconnect compensation once a replacement
  owns the route so a stale disconnect cannot revoke it.
- Discard webhook validation results when the change epoch moved
  during the HTTP round, so a stale rejection cannot invalidate the
  current session.
- Drop user_token fields from info and warn logs that became
  visible with info-level defaults.
- Restore a hostname omitted by the 2.6.4 readback into the cached
  runtime config after a successful mutation, so later rounds stop
  re-sending the same hostname patch.
- Reconcile running web configs when no revision is tracked so
  legacy unrevisioned updates converge, and wake sessions for
  unrevisioned full updates instead of waiting for the next
  heartbeat.

* chore(go): regenerate web proto bindings for heartbeat fields

Add failed_network_instances, support_heartbeat_policy, and the
heartbeat policy response fields to the checked-in Go bindings.
Other proto packages are left as-is because their drift predates
this change.

* fix(web): redact user tokens from positional log arguments

Three runtime reconciliation info logs and the user lookup error
contexts printed user_token through format arguments, which the
earlier field-syntax redaction missed. The reconcile log now fires
every round for unrevisioned machines, so remove the token from
these messages as well.

* fix(web): fence stale validation and runtime reconcile rounds

Check webhook validation epochs while holding the session write lock,
so stale success and rejection responses cannot change session state.
Advance the runtime epoch for unrevisioned full config updates, and
exclude failed instances from heartbeat and RPC reconciliation lists
so stopped instances are restarted instead of repeatedly hot-patched.

Release test read guards before awaiting validation apply calls. Set
up the no-pending condition before asserting that an applied revision
is a no-op, and verify that its runtime epoch remains unchanged.

Validation: all 137 client_manager tests passed.

* test(credentials): cover P2P with active VPN portal

Model an admin and temporary credential peer connected as a foreign network through a public server with data relay disabled. Verify their direct connection can be replaced after a WireGuard portal client comes online.

* test(credentials): stabilize two-admins failover assertions

The two-admins non-reusable credential test could fail on slow
convergence: after dropping the winning peer it relied on a single
route sample passing a bare AND condition, then re-asserted the same
expectations through one-shot checks seconds later. A transient route
flap in that window (for example a briefly resurrected winner route
from stale conn info) turned a passing convergence into a hard assert
failure. This matches the 48.9s CI flake of
credential_non_reusable_across_two_admins_allows_only_one_peer
observed on 2026-08-12.

Changes:

- wait for bidirectional admin connectivity (AND) with a 20s budget
  before issuing the credential, instead of a one-directional OR
- replace the failover wait_for_condition with
  wait_stable_failover_visibility_on_admins, which requires three
  consecutive samples of loser-present and winner-absent on both
  admins within the same 60s budget and logs every sample
- enrich the stable-single-winner timeout message with per-admin
  visibility flags and elapsed time for triage

All existing contracts are preserved; only observation windows and
diagnostics change. Validated in the rust container: three passes at
normal speed (54.1s / 53.8s / 53.1s) plus one slow-convergence round
(172.7s) that would have raced the old one-shot sampling; it now
passes with failover samples logged. cargo fmt and clippy -D warnings
clean.
2026-09-13 01:13:28 +08:00
..

EasyTier Go

easytier-go runs the wasm32-wasip1 build of easytier-core in a pure-Go process through wazero. EasyTier remains the source and producer of the embedded WASM; this repository adapts Go host capabilities to the ABI exported and imported by that artifact.

import (
    "net/netip"

    corehost "github.com/easytier/easytier/easytier-go"
)

Public API

The public package owns wazero, standard WASI, the EasyTier host ABI, guest driving, completion notification, and resource shutdown. Applications create a host, build a typed instance configuration, and then use standard Go network interfaces:

host, err := corehost.New(ctx, corehost.Options{})
if err != nil {
    return err
}
defer host.Close(ctx)

config, err := corehost.NewInstanceConfigBuilder("office").
    NetworkSecret("secret").
    IPv4(netip.MustParsePrefix("10.144.0.10/24")).
    AddPeers("tcp://198.51.100.10:11010").
    Build()
if err != nil {
    return err
}

instance, err := host.CreateInstance(ctx, config)
if err != nil {
    return err
}
defer instance.Close(ctx)

if err := instance.Start(ctx); err != nil {
    return err
}
if err := instance.SendPacket(ctx, packet); err != nil {
    return err
}
received, err := instance.ReceivePacket(ctx)

listener, err := instance.Listen("tcp4", ":8080")
connection, err := instance.Dial(ctx, "tcp4", "10.144.0.2:8080")
packets, err := instance.ListenPacket("udp4", ":5353")

CreateInstanceTOML loads a native EasyTier TOML document. Pass an empty instanceID to allocate a UUID. Existing instance_id and instance_name keys in the document are replaced by the host.

instance, err := host.CreateInstanceTOML(ctx, "office", "", configTOML)

Instance.ShowNodeInfo returns this instance's virtual IPv4 address and advertised hostname.

Web Client management

A host can also connect to an EasyTier Web configuration server. The embedded Rust WebClient retains the config-server protocol, heartbeat, reconnect, and secure-tunnel behavior; Go owns the resulting process-level instances:

webClient, err := host.ConnectWebClient(ctx, corehost.WebClientOptions{
    Endpoint:   "udp://config.example.com:22020/team-token",
    MachineID:  "11111111-2222-4333-8444-555555555555",
    Hostname:   "edge-gateway",
    SecureMode: true,
})
if err != nil {
    return err
}
defer webClient.Close(ctx)

for _, instance := range host.Instances() {
    log.Printf("%s: %v", instance.ID(), instance.State())
}

MachineID must be a stable UUID persisted by the application. Endpoint accepts tcp://, udp://, or the same shorthand token understood by native EasyTier. WebSocket transports are not part of this initial host integration. One WebClient may run per Host.

Web-created instances support the complete WebClientService lifecycle and status surface. Instances created through Host.CreateInstance are included in heartbeats and status listings, but are reported as read-only and cannot be overwritten, retained away, or deleted by the Web server. Host.Instances returns both ownership classes; Web-created instances use the same Instance data-plane and management APIs as application-created instances.

Instance.ListPeer and Instance.ListRoute call the embedded core's existing instance-scoped management RPCs and return peer and route slices directly. Their element types reuse the generated EasyTier protobuf models, while the request and response envelopes stay internal to the host. Callers never construct wire bytes or a separate RPC client. Cancelling the context frees the pending guest operation.

InstanceConfigBuilder exposes the instance settings supported by this host: network identity, hostname, virtual IPv4 address, peers and listeners, IPv4 and IPv6 STUN servers, Core-owned TCP and UDP port forwards, P2P policy, hole-punching methods, encryption, and secure mode. Omitted optional settings retain the embedded core's defaults. Calling STUNServers() or STUNServersV6() with no arguments explicitly selects an empty list.

AddPortForwards accepts typed rules containing a PortForwardTCP or PortForwardUDP protocol and netip.AddrPort bind and destination addresses. The embedded core owns their listener, overlay-flow, reload, and shutdown lifecycle.

Secure mode can generate an X25519 key with SecureMode() or use a caller supplied raw 32-byte private key with SecureModeWithPrivateKey(key). The public key is always derived by the builder. Secure mode currently requires a non-empty shared network secret; credential-based networks are a separate future configuration path.

Dial returns net.Conn, Listen returns net.Listener, and ListenPacket returns net.PacketConn. ABI v2 currently supports tcp, tcp4, udp, and udp4; destinations must be IPv4 literals and listeners bind all overlay IPv4 addresses. These APIs are overlay-only: an absent EasyTier route is returned as a normal network error and never falls back to the host network.

See KNOWN_LIMITATIONS.md for current UDP and port-forward edge cases.

No public type exposes wazero runtimes, WebAssembly pointers, raw handles, submit/take operations, or the cooperative drive loop. Each host owns one wazero runtime, guest module, and host completion domain; each instance is represented by an EasyTier guest handle. Per-instance drivers serialize guest calls through the host. The engine continues driving EasyTier after Start returns, calls easytier_instance_notify_completions before driving a host completion, and drains bounded data-plane completion batches after each guest turn.

The host serializes the typed configuration to TOML internally, wraps it in EasyTier's version 14 create envelope, and adds the configured environment snapshot. TOML, schema versions, and JSON envelopes are not application-facing APIs.

Cross-platform TUN example

The TUN example joins an existing EasyTier network with a fixed virtual IPv4 address on Linux, macOS, or Windows. It creates and configures the native TUN interface itself, then forwards raw IPv4 packets through SendPacket and ReceivePacket:

cd examples/tun
sudo go run . \
  -p tcp://198.51.100.10:11010 \
  --network-name office \
  --network-secret secret \
  --ipv4 10.144.0.10/24

Repeat -p to configure more peers. The command creates et-goN on Linux and Windows or utunN on macOS, assigns the requested address, and sets an MTU of 1380. Run it as root or with CAP_NET_ADMIN on Linux, with sudo on macOS, or from an Administrator terminal on Windows. Closing the command removes the TUN interface. The example does not install a default route or enable GSO.

On Linux and macOS, send SIGUSR1 to print the current peer list or SIGUSR2 to print the current route list.

Repeat -port-forward to expose local TCP or UDP ports through the embedded core's port-forward manager:

sudo go run . \
  -p tcp://198.51.100.10:11010 \
  --network-name office \
  --network-secret secret \
  --ipv4 10.144.0.10/24 \
  -port-forward tcp://127.0.0.1:5202/10.144.0.20:5201 \
  -port-forward udp://127.0.0.1:5202/10.144.0.20:5201

For example, run iperf3 -c 127.0.0.1 -p 5202 for TCP or add -u -b 0 -l 1200 for UDP. iperf3's UDP mode still needs the TCP forward for its control connection. The TUN example only parses these rules into the instance configuration; the core owns the host listeners and per-client overlay flows.

Instance.Dial example

The Dial example is a small overlay client dedicated to the public Instance.Dial API. TCP mode bridges the connected stream to standard input and output until the remote side closes or the command is interrupted:

printf 'GET / HTTP/1.0\r\nHost: 10.144.0.20\r\n\r\n' |
  go run ./examples/dial \
    -p tcp://198.51.100.10:11010 \
    --network-name office \
    --network-secret secret \
    --ipv4 10.144.0.10/24 \
    --network tcp4 \
    --address 10.144.0.20:8080

With --network udp4, standard input is sent as one datagram and one response datagram is written to standard output. The command does not create a local listener or implement port-forward management. It waits up to 10 seconds for a matching overlay or proxy route before dialing; override that limit with --connect-timeout.

Web Client example

The Web Client example registers a Host with an EasyTier Web configuration server and lets the server create, delete, and inspect its instances:

go run ./examples/web-client \
  --web-endpoint tcp://config.example.com:22020/team-token \
  --web-machine-id 11111111-2222-4333-8444-555555555555 \
  --web-hostname edge-gateway \
  --web-secure

--web-machine-id must remain stable across restarts. --web-hostname defaults to the system hostname. This example manages Host instances but does not create or attach an operating-system TUN interface.

Performance compared with native EasyTier

In this A/B benchmark two nodes on one i7-14700KF host (Linux 6.11) each run in their own network namespace, joined by a veth pair. Node A always runs a native easytier-core build of EasyTier master (2.6.4-6a186167) with overlay address 10.144.0.1/24; node B runs either the same native binary or this Go host (commit 78889d12, embedded EasyTier af640d49) with 10.144.0.2/24. A master build is used as the native baseline because the 2.6.4 release predates several native data-plane throughput fixes (EasyTier #2451, #2452). The underlay tunnel between the nodes is either tcp:// or udp://. Encryption is enabled and the overlay MTU is 1360 on both ends. Node A and the iperf3 server are pinned to CPUs 0,2,4,6, node B to 8,10,12,14. Each iperf3 run lasts 15 seconds and excludes the first 3 seconds. Forward means node B sends to node A; reverse uses iperf3 -R. Measured 2026-07-28.

The forwarding rows deliberately compare native Core port forwarding with the Go host's benchmark-only cmd/dial-forward-bench, which carries traffic through the public Instance.Dial API.

TCP, one stream:

Scenario Direction tcp:// native tcp:// Go host udp:// native udp:// Go host
TUN forward 6.06 Gbit/s 2.12 Gbit/s 3.71 Gbit/s 1.57 Gbit/s
TUN reverse 6.03 Gbit/s 2.57 Gbit/s 3.69 Gbit/s 1.48 Gbit/s
Native port forward / Go Dial forward 1.25 Gbit/s 1.29 Gbit/s 1.19 Gbit/s 1.20 Gbit/s
Native port forward / Go Dial reverse 6.49 Gbit/s 1.95 Gbit/s 4.26 Gbit/s 1.43 Gbit/s

UDP native port forward / Go Dial, 1 Gbit/s offered with 1200-byte datagrams (received / lost):

Direction tcp:// native tcp:// Go host udp:// native udp:// Go host
forward 996 Mbit/s / 0.3% 546 Mbit/s / 45% 996 Mbit/s / 0.3% 699 Mbit/s / 30%
reverse 950 Mbit/s / 5.0% 490 Mbit/s / 51% 983 Mbit/s / 1.7% 350 Mbit/s / 65%

Reading the numbers:

  • On TUN the Go host reaches roughly 35-45% of native single-stream throughput. Both sides run the same EasyTier core logic, so the gap is the WASM/Go data-plane boundary rather than routing or cryptography.
  • TCP forwarding is a tie at about 1.2 Gbit/s: both paths are bounded by the virtual TCP send path inside the shared EasyTier core, not by the host.
  • TCP reverse forwarding favors native by about 3x (4.3-6.5 versus 1.4-2.0 Gbit/s); the Go host benchmark's per-operation receive path is the limit.
  • Native sustains the offered 1 Gbit/s UDP nearly loss-free in both directions, while the Go host saturates at 350-700 Mbit/s with significant loss, consistent with the one-operation-per-datagram data-plane ABI documented in PERFORMANCE.md.

Platform capabilities

The default platform implementation uses Go's standard net and net.Resolver packages. Applications that need netns, socket marks, device binding, reuse policy, or custom DNS can inject capabilities through platform.Services:

host, err := corehost.New(ctx, corehost.Options{
    Platform: platform.Services{
        Sockets:     socketFactory,
        DNS:         dnsResolver,
        Environment: connectorEnvironment,
        Snapshot:    environmentSnapshot,
    },
})

platform.SocketFactory owns only TCP connect, UDP bind, and TCP listen creation. Once a standard Go network resource is returned, the host runtime owns its reads, writes, accepts, cancellation, and close path. EasyTier retains all routing, peer admission, protocol, retry, and connection policy.

The implementation is split by responsibility:

  • platform defines public capability ports; platform/netstd implements their portable defaults.
  • proto contains generated Go bindings for the existing EasyTier management protobuf definitions.
  • internal/reactor owns typed asynchronous operations, resources, operation IDs, backpressure, and completion signals without depending on wazero.
  • internal/hostabi implements the custom easytier_host imports, guest memory copying, wire codecs, and ABI status translation.
  • internal/coreabi owns guest memory, the big-endian data-plane wire codec, ABI discovery, and typed easytier_instance_*, easytier_data_plane_*, and easytier_rpc_* export calls.
  • internal/engine composes standard WASI, both EasyTier ABI directions, the single-owner driver, operation cancellation, deadlines, standard Go network resources, and instance shutdown.
  • internal/artifact contains only the embedded core and its provenance.

Embedded artifact

The committed WASM lets downstream Go builds and tests run without a Rust toolchain. Refresh it from a clean EasyTier checkout whenever the guest ABI or core implementation changes:

EASYTIER_SOURCE=/path/to/EasyTier go generate ./...

Generation runs EasyTier's script/build-wasi-core.sh, which builds release easytier_core.wasm with the Go-host features and writes an optimized easytier_core_go_host.wasm with the pinned, SHA-256-verified Binaryen release. The generator supplies a fixed source path remap and source-date epoch, then records the EasyTier commit and optimized artifact SHA-256. Tracked EasyTier changes block generation; unrelated untracked files do not. corehost.CoreInfo() exposes that provenance without exposing the artifact bytes.

The same generator rebuilds the Go protobuf bindings from that exact clean EasyTier commit and records their source commit and schema SHA-256. Host creation rejects an artifact/binding commit mismatch. Generation requires protoc 35.1 and protoc-gen-go 1.36.11 on PATH.

The test-only socket probe is retained from EasyTier commit 6a3d15f; its full commit and checksum are recorded in testdata/wasi_socket_guest.source.

Run all reactor, ABI conformance, lifecycle, and two-instance network tests with:

go test -count=1 ./...