What "end to end" means for nfltr tcp

nfltr tcp 5432 publishes a port on your machine through the relay: the machine connects out, and another machine of yours reaches the port with nfltr tcp-connect or nfltr dial, so nothing needs an open port. Every hop to the relay is TLS. The question this post answers is a different one: can the relay itself read the bytes?

Until v2.1.52 the honest answer for a default nfltr tcp was "it depends on the client", and the agent's banner did not say so. From v2.1.52, with end-to-end encryption on (--e2ee, the default), the answer is no: the agent takes only sessions that pin its certificate and refuses the rest. This is a behaviour change. If you run nfltr tcp and connect without --pin, your sessions are now refused; what to change is below.

Two kinds of session

nfltr tcp with --e2ee holds its own TLS certificate and prints its SHA-256 fingerprint. A client that passes that fingerprint (--pin <fingerprint>, or pin: on an nfltr up connect: mapping) runs TLS with the agent itself, inside the relay's TLS, and checks the agent's certificate against the pin. A relay that swaps the key fails the handshake (exit 7, nothing delivered).

A client without a pin cannot run that TLS: it has nothing to check the agent's certificate against, so a relay could answer with its own key. Such a session was plain text inside the relay's TLS, which is what nfltr tcp-connect reported ("e2ee: off"), while the agent's banner said "e2ee: enabled".

session before v2.1.52 from v2.1.52 what the relay can read
nfltr tcp (--e2ee), client with --pin TLS handshake, then closed with no data (the bug) carries the port's bytes nothing but sizes and timing
nfltr tcp (--e2ee), client without a pin carried the bytes in plain text refused: exit 4, reason ERROR_REASON_E2EE_REQUIRED nothing; no byte flows
nfltr tcp --e2ee=false, any client carried the bytes in plain text unchanged the bytes (inside its own TLS on each hop)
nfltr up service with agent: and e2ee: true pinned only (already refused unpinned) unchanged nothing but sizes and timing

The agent refuses, not the relay, because only the agent knows it offers end-to-end encryption; it refuses at the open, before it dials the port. The refusal says how to pin and never names a fingerprint: that answer comes through the relay unauthenticated, so a fingerprint in it could be the relay's. Take the fingerprint from the agent's own output, or from nfltr up's /status for services under agent:.

We chose a hard cut over a warning or a trust-on-first-use pin. A warning keeps a session the relay can read under a flag that says it cannot. Trust on first use pins whatever key the relay presents the first time. --e2ee=false stays for ports whose protocol encrypts end to end itself.

The bug we found on the way

Before changing the default, we checked that pinned sessions to nfltr tcp work at the sizes people move. They did not. On one private relay on loopback, a pinned session of 1 MiB, 100 MiB or 1 GiB finished its TLS handshake and then delivered 0 bytes, closing after 10 s. The agent took any pinned session to a tunnel without a service table for an encrypted share's HTTP and waited for an HTTP request that never came. Every version from v2.1.6 had it. It failed closed: no bytes reached either side and nothing was readable by the relay. Services under nfltr up agent: were not affected; they pass the same sizes.

The fix sends a pinned session to nfltr tcp to its port after the agent ends the TLS. The relay's encrypted shares keep their HTTP path.

Evidence: pinned sessions to nfltr tcp, before and after

Setup
One private nfltr-relay on loopback, an nfltr tcp --e2ee agent in front of a generator that writes a known stream (AES-128-CTR of zeros), clients nfltr tcp-connect --pin and nfltr tcp-connect without a pin; SHA-256 of what each client received compared with the stream's.
Numbers
Before: pinned 1 MiB, 100 MiB and 1 GiB all received 0 bytes, each closed after 10.0 s; unpinned received every byte (plain text inside the relay's TLS). After: pinned 1 MiB in 0.03 s, 100 MiB in 0.10 s, 1 GiB in 0.82 s and four 1 GiB sessions at once in about 2.0 s each, every SHA-256 matching. A guard test in the relay client SDK sends 8 MiB through a pinned session and checks its SHA-256: it fails on the old code and passes on the new.
Reproduce
(a) With the released binaries: nfltr tcp 9000 --name gen in front of any server that writes a file, then nfltr dial gen 9000 --pin <fingerprint> | sha256sum on another machine; without --pin the dial exits 4. (b) Our check, in the closed nfltr repository: make test-connect-conformance (TestE2EETCPAgent: 64 MiB through nfltr dial --pin, SHA-256 compared, and an unpinned dial expected to exit 4).
Limits
Loopback, so the times measure the agent and relay, not a network. Unpinned sessions on v2.1.51 and earlier were readable by the relay; nothing here shows whether any were read.

Testing the refusal

The new rule is a precondition on opening one session, so it is tested where it lives and end to end. In the relay client SDK, a tunnel that offers end-to-end encryption refuses a plain open and a pinned open it cannot end TLS for, and never dials the target. In nfltr's conformance suite, the real binaries run against a private relay: an unpinned nfltr dial to an nfltr tcp agent must exit 4 with the reason and --pin in its message, and a pinned one must carry 64 MiB intact. Before the change the unpinned dial exited 0 and was served.

Evidence: an unpinned session is refused

Setup
nfltr's connect conformance suite: the real nfltr and nfltr-relay binaries against a private TLS relay on one machine; an nfltr tcp agent with its default --e2ee.
Numbers
Pass or fail, no timings: unpinned nfltr dial exits 4 (refused) with reason ERROR_REASON_E2EE_REQUIRED and "--pin" in the message; pinned nfltr dial carries 64 MiB with matching SHA-256. On the previous relay client the unpinned dial exited 0.
Reproduce
(a) With the released binaries: nfltr tcp 9000, then nfltr dial <agent> 9000 without --pin (exit 4) and with the printed fingerprint (served). (b) Our check, in the closed nfltr repository: make test-connect-conformance.
Limits
One host. The suite does not run nfltr ssh-proxy or Docker-based smokes; scripts and compose files that opened unpinned sessions were changed to run their agent with --e2ee=false or to pin.

What to change

If your sessions to nfltr tcp now exit 4 with ERROR_REASON_E2EE_REQUIRED, choose one:

  • Keep end-to-end encryption. Copy the fingerprint from the agent's banner and pin it:

    nfltr tcp 5432 --name my-db
    # on the other machine
    nfltr tcp-connect alice.my-db 5432 --pin <fingerprint> --listen :5432

    or pin: "<fingerprint>" on an nfltr up connect: mapping, or nfltr dial my-db 5432 --pin <fingerprint> from a script.

  • The protocol encrypts itself. SSH does, so nfltr tcp 22 --e2ee=false keeps working without a pin, and the relay sees only SSH's ciphertext. The same holds for a database that requires TLS from its clients.

  • A tcp service of nfltr up without agent: is now plain (--e2ee=false) and needs no pin. For end-to-end encryption, put it under agent: with e2ee: true, as before.

Limits

  • The pin travels by you: copy it from the agent's output to the client once. nfltr does not distribute or rotate it. A new certificate on the agent means a new pin on each client.
  • nfltr shell does not take this path: its pinned sessions keep the share path they used before.
  • nfltr ssh-config writes no pin for nfltr ssh-proxy, so an nfltr tcp 22 agent behind it needs --e2ee=false.
  • End-to-end encryption hides contents, not metadata: the relay still sees which agent you reach, when, and how many bytes.
  • Run your own relay if the metadata must stay on your network too; Run the relay on your own network covers it.

Is it for you?

Fits if you publish a database, a cache or an internal service with nfltr tcp and reach it from your own machines, and the relay must not be able to read it: from this release a session either pins the agent's key or is refused. If you publish SSH or another protocol that encrypts itself, pass --e2ee=false and nothing else changes. If you only use HTTP shares or nfltr up services under agent:, nothing changes for you.

Not ready for hiding who talks to whom: the relay still sees which agent you reach, when and how many bytes. Pins are copied by hand, with no distribution or rotation, and nfltr ssh-proxy and nfltr shell do not pin nfltr tcp agents.

Try next nfltr update, then nfltr tcp on one machine and nfltr tcp-connect or nfltr dial from another with the printed --pin. The release notes list everything else in v2.1.52; Services with no open ports shows the same pinning on nfltr up mappings, and the nfltr dial reference has every exit code.

← All posts