What "end to end" means for nfltr tcp
nfltr tcp 5432 publishes a port on your machine through the relay: the machine connects out, and another machine of yours reaches the port with nfltr tcp-connect or nfltr dial, so nothing needs an open port. Every hop to the relay is TLS. The question this post answers is a different one: can the relay itself read the bytes?
Until v2.1.52 the honest answer for a default nfltr tcp was "it depends on the client", and the agent's banner did not say so. From v2.1.52, with end-to-end encryption on (--e2ee, the default), the answer is no: the agent takes only sessions that pin its certificate and refuses the rest. This is a behaviour change. If you run nfltr tcp and connect without --pin, your sessions are now refused; what to change is below.
Two kinds of session
nfltr tcp with --e2ee holds its own TLS certificate and prints its SHA-256 fingerprint. A client that passes that fingerprint (--pin <fingerprint>, or pin: on an nfltr up connect: mapping) runs TLS with the agent itself, inside the relay's TLS, and checks the agent's certificate against the pin. A relay that swaps the key fails the handshake (exit 7, nothing delivered).
A client without a pin cannot run that TLS: it has nothing to check the agent's certificate against, so a relay could answer with its own key. Such a session was plain text inside the relay's TLS, which is what nfltr tcp-connect reported ("e2ee: off"), while the agent's banner said "e2ee: enabled".
| session | before v2.1.52 | from v2.1.52 | what the relay can read |
|---|---|---|---|
nfltr tcp (--e2ee), client with --pin |
TLS handshake, then closed with no data (the bug) | carries the port's bytes | nothing but sizes and timing |
nfltr tcp (--e2ee), client without a pin |
carried the bytes in plain text | refused: exit 4, reason ERROR_REASON_E2EE_REQUIRED |
nothing; no byte flows |
nfltr tcp --e2ee=false, any client |
carried the bytes in plain text | unchanged | the bytes (inside its own TLS on each hop) |
nfltr up service with agent: and e2ee: true |
pinned only (already refused unpinned) | unchanged | nothing but sizes and timing |
The agent refuses, not the relay, because only the agent knows it offers end-to-end encryption; it refuses at the open, before it dials the port. The refusal says how to pin and never names a fingerprint: that answer comes through the relay unauthenticated, so a fingerprint in it could be the relay's. Take the fingerprint from the agent's own output, or from nfltr up's /status for services under agent:.
We chose a hard cut over a warning or a trust-on-first-use pin. A warning keeps a session the relay can read under a flag that says it cannot. Trust on first use pins whatever key the relay presents the first time. --e2ee=false stays for ports whose protocol encrypts end to end itself.
The bug we found on the way
Before changing the default, we checked that pinned sessions to nfltr tcp work at the sizes people move. They did not. On one private relay on loopback, a pinned session of 1 MiB, 100 MiB or 1 GiB finished its TLS handshake and then delivered 0 bytes, closing after 10 s. The agent took any pinned session to a tunnel without a service table for an encrypted share's HTTP and waited for an HTTP request that never came. Every version from v2.1.6 had it. It failed closed: no bytes reached either side and nothing was readable by the relay. Services under nfltr up agent: were not affected; they pass the same sizes.
The fix sends a pinned session to nfltr tcp to its port after the agent ends the TLS. The relay's encrypted shares keep their HTTP path.
Evidence: pinned sessions to nfltr tcp, before and after
- Setup
- One private nfltr-relay on loopback, an
nfltr tcp --e2eeagent in front of a generator that writes a known stream (AES-128-CTR of zeros), clientsnfltr tcp-connect --pinandnfltr tcp-connectwithout a pin; SHA-256 of what each client received compared with the stream's. - Numbers
- Before: pinned 1 MiB, 100 MiB and 1 GiB all received 0 bytes, each closed after 10.0 s; unpinned received every byte (plain text inside the relay's TLS). After: pinned 1 MiB in 0.03 s, 100 MiB in 0.10 s, 1 GiB in 0.82 s and four 1 GiB sessions at once in about 2.0 s each, every SHA-256 matching. A guard test in the relay client SDK sends 8 MiB through a pinned session and checks its SHA-256: it fails on the old code and passes on the new.
- Reproduce
- (a) With the released binaries:
nfltr tcp 9000 --name genin front of any server that writes a file, thennfltr dial gen 9000 --pin <fingerprint> | sha256sumon another machine; without--pinthe dial exits 4. (b) Our check, in the closed nfltr repository:make test-connect-conformance(TestE2EETCPAgent: 64 MiB throughnfltr dial --pin, SHA-256 compared, and an unpinned dial expected to exit 4). - Limits
- Loopback, so the times measure the agent and relay, not a network. Unpinned sessions on v2.1.51 and earlier were readable by the relay; nothing here shows whether any were read.
Testing the refusal
The new rule is a precondition on opening one session, so it is tested where it lives and end to end. In the relay client SDK, a tunnel that offers end-to-end encryption refuses a plain open and a pinned open it cannot end TLS for, and never dials the target. In nfltr's conformance suite, the real binaries run against a private relay: an unpinned nfltr dial to an nfltr tcp agent must exit 4 with the reason and --pin in its message, and a pinned one must carry 64 MiB intact. Before the change the unpinned dial exited 0 and was served.
Evidence: an unpinned session is refused
- Setup
- nfltr's connect conformance suite: the real nfltr and nfltr-relay binaries against a private TLS relay on one machine; an
nfltr tcpagent with its default--e2ee. - Numbers
- Pass or fail, no timings: unpinned
nfltr dialexits 4 (refused) with reason ERROR_REASON_E2EE_REQUIRED and "--pin" in the message; pinnednfltr dialcarries 64 MiB with matching SHA-256. On the previous relay client the unpinned dial exited 0. - Reproduce
- (a) With the released binaries:
nfltr tcp 9000, thennfltr dial <agent> 9000without--pin(exit 4) and with the printed fingerprint (served). (b) Our check, in the closed nfltr repository:make test-connect-conformance. - Limits
- One host. The suite does not run
nfltr ssh-proxyor Docker-based smokes; scripts and compose files that opened unpinned sessions were changed to run their agent with--e2ee=falseor to pin.
What to change
If your sessions to nfltr tcp now exit 4 with ERROR_REASON_E2EE_REQUIRED, choose one:
-
Keep end-to-end encryption. Copy the fingerprint from the agent's banner and pin it:
nfltr tcp 5432 --name my-db # on the other machine nfltr tcp-connect alice.my-db 5432 --pin <fingerprint> --listen :5432or
pin: "<fingerprint>"on annfltr upconnect:mapping, ornfltr dial my-db 5432 --pin <fingerprint>from a script. -
The protocol encrypts itself. SSH does, so
nfltr tcp 22 --e2ee=falsekeeps working without a pin, and the relay sees only SSH's ciphertext. The same holds for a database that requires TLS from its clients. -
A tcp service of
nfltr upwithoutagent:is now plain (--e2ee=false) and needs no pin. For end-to-end encryption, put it underagent:withe2ee: true, as before.
Limits
- The pin travels by you: copy it from the agent's output to the client once. nfltr does not distribute or rotate it. A new certificate on the agent means a new pin on each client.
nfltr shelldoes not take this path: its pinned sessions keep the share path they used before.nfltr ssh-configwrites no pin fornfltr ssh-proxy, so annfltr tcp 22agent behind it needs--e2ee=false.- End-to-end encryption hides contents, not metadata: the relay still sees which agent you reach, when, and how many bytes.
- Run your own relay if the metadata must stay on your network too; Run the relay on your own network covers it.
Is it for you?
Fits if you publish a database, a cache or an internal service with nfltr tcp and reach it from your own machines, and the relay must not be able to read it: from this release a session either pins the agent's key or is refused. If you publish SSH or another protocol that encrypts itself, pass --e2ee=false and nothing else changes. If you only use HTTP shares or nfltr up services under agent:, nothing changes for you.
Not ready for hiding who talks to whom: the relay still sees which agent you reach, when and how many bytes. Pins are copied by hand, with no distribution or rotation, and nfltr ssh-proxy and nfltr shell do not pin nfltr tcp agents.
Try next nfltr update, then nfltr tcp on one machine and nfltr tcp-connect or nfltr dial from another with the printed --pin. The release notes list everything else in v2.1.52; Services with no open ports shows the same pinning on nfltr up mappings, and the nfltr dial reference has every exit code.