The Architecture of Trust - Proxies, Reverse Tunnels, and Zero-Trust Authorization at Scale

Helpful context:

• Networking - How Packets Find Their Way Across the Internet
• Load Balancing & Proxies - Distributing Work So No Single Machine Drowns
• Security Fundamentals - What Attackers Know That You Might Not
• Identity & OTP - Proving Who You Are Without Sharing a Password
• Microservices - Small Services, Large Coordination Problems

If you ask ten engineers how a request travels from a browser to a private database, you get ten different, half-complete answers. Most people can explain either the network path or the identity path, rarely both, and the place where they actually intersect - the edge, where a proxy hands a request across a trust boundary - is where the real architecture decisions live.

Networking (proxies and tunnels) builds the pathway data travels across network boundaries. Identity and authorization (workload certificates, short-lived tokens, permission graphs) decide who is allowed to use that pathway and for what. Treat them as separate concerns and you get systems that are either impossible to operate, because nothing can reach anything, or trivially compromised, because everything trusts everything once it’s inside.

This post works through both from first principles: how a stateful firewall actually decides which packets are allowed back in, why a service with zero open inbound ports can still serve public traffic, and what a mature zero-trust system does differently at the moment a request crosses from “outside” to “inside.” For the mechanics of authentication itself - passwords, OAuth, JWTs - see Security Fundamentals and Identity & OTP linked above; this picks up from there and asks what happens once identity has been established and a request needs to travel.

The Physics of a Network Socket

Before any proxy or token enters the picture, a request is just packets between two OS-level sockets, and the ground truth here shapes everything built on top of it.

An IP address identifies a host on the network - the equivalent of a street address. A port is a numbered channel (0 to 65535) that the OS kernel binds to a specific running process - the apartment number within that address. A single server at 198.51.100.1 might have a web server listening on port 443, an SSH daemon on port 22, and Postgres on port 5432, each an independent process with its own socket.

An inbound connection is one where a remote client initiates the TCP handshake toward a port your machine is listening on. An outbound connection is one your process initiates toward someone else’s listening port. This direction matters more than almost anything else in network security, because nearly every firewall in production enforces the same asymmetric rule: block unsolicited inbound connections, allow essentially all outbound ones.

That raises an obvious question: if inbound is blocked, how does a reply ever get back to you?

The answer is connection tracking (conntrack). When your OS opens an outbound socket, the firewall records a 4-tuple - source IP, source port, destination IP, destination port - in a state table the moment the SYN packet leaves. When the remote server replies with a SYN-ACK, the firewall checks it against that table. If the reply’s addresses and ports match an entry created by a connection you initiated, it’s let through, even though it is, technically, an inbound packet. Nothing about the packet itself is special; it’s permitted because the firewall remembers it asked for exactly this.

Discomfort check. The instinctive assumption is that a server sitting in a private subnet can never serve public web traffic unless you open an inbound port, typically 443, in the firewall protecting it. That’s false, and the reason is the same full-duplex property conntrack relies on. A TCP connection is bidirectional once established. If the private server is the one that dials out to a public edge proxy and keeps that connection open, the edge proxy can push HTTP requests down that existing stream indefinitely. The private server never opens a listening port to the internet; it just never hangs up the outbound connection it made. This is the entire trick behind reverse tunnels, and most engineers' mental model of “private” assumes a listening socket has to exist somewhere on the public side of the boundary. It doesn’t.

The Taxonomy of Proxies

“Proxy” is used loosely enough in casual conversation that it’s worth being precise. Four distinct architectures get called by the same name, and they protect different things and route traffic in different directions.

Forward proxies sit in front of clients. Corporate networks route employee traffic through one to enforce content filtering and data loss prevention, and to mask the client’s real IP - the destination server only ever sees the proxy.

Reverse proxies sit in front of backends. When you hit api.example.com, you’re talking to Nginx or Envoy, not the application server directly. It terminates TLS, load-balances across replicas, rate-limits, and hides the internal topology - the mechanics of exactly how are covered in Load Balancing & Proxies, linked above.

Inverting proxies, also called reverse tunnels, solve a problem neither of the above can: a backend with no public IP and no open inbound ports at all, like an on-prem machine behind corporate NAT or a private VPC with no ingress rules. Instead of a proxy dialing in, a local agent on the private machine dials out to a public edge proxy and holds that connection open, authenticated via mTLS or a client secret. When a user hits the public domain, the edge proxy verifies them, then forwards the request down that already-open outbound pipe:

Service mesh sidecars apply the reverse-proxy idea to east-west traffic between your own services rather than north-south traffic from the internet - every instance gets its own small Envoy process handling load balancing, retries, and mTLS for its outbound calls. Load Balancing & Proxies covers how the sidecar model and its control plane actually work; here it matters mainly as the fourth entry in this taxonomy.

Discomfort check. The instinctive worry is that a private machine holding a persistent outbound connection open to the public internet has effectively handed an attacker a permanent way in. That’s not what a reverse tunnel does, and the reason is what flows through the tunnel, not the tunnel’s existence. The private machine only ever acts as a TLS client connecting outward to a certificate it has verified; it never listens for inbound connections from arbitrary hosts. The edge proxy authenticates and authorizes every user request before anything is forwarded down the tunnel, and it signs the forwarded request with a private key that the private agent validates against a known public key before executing anything. The private machine’s external attack surface, the set of things an internet scanner can even attempt to connect to, stays at zero open inbound ports throughout. The tunnel is a pipe the private side controls the far end of, not a hole in its perimeter.

Zero Trust: The Edge Translation Pattern

Getting a request across the network boundary is only half the problem. The more consequential failure mode in distributed systems is what happens after the request is inside.

The common anti-pattern, sometimes called “hard shell, soft center,” authenticates the user once at the API gateway, strips the auth headers, and forwards plain, unauthenticated HTTP calls internally on the assumption that the internal network is inherently safe. It’s a reasonable-sounding assumption until any single container in the call path is compromised: the moment that happens, the attacker has unauthenticated lateral access to every service downstream, because none of them ever checked anything.

The fix used in mature systems is the edge translation pattern: external credentials, such as session cookies or third-party OAuth tokens from Okta or Azure AD, are validated once at the edge and never forwarded any further. The edge mints a new, short-lived internal token in their place, and every hop afterward re-verifies both the calling workload’s identity, typically via mTLS backed by SPIFFE/SPIRE-issued certificates, and the internal token, independently.

Nothing downstream of the gateway ever sees the user’s original OAuth token or session cookie again.

Discomfort check. Since every service can already verify a signed token locally in zero milliseconds, minting an entirely new internal token can look like unnecessary ceremony. It seems simpler to just let every microservice parse the user’s raw email or ID out of the original credential and write a plain equality check against it. But hardcoding an identity schema across every microservice is a coupling problem disguised as a shortcut. If services parse a numeric user ID directly, you cannot add SAML or a second enterprise identity provider without touching every service that ever checked identity. If services log request headers for debugging, which they will, eventually, somewhere, forwarding the original OAuth token risks a long-lived, replayable user credential ending up in a log aggregator. An opaque internal principal, something like principal://auth/workforce/okta/alice, decouples “who issued this identity” from “what does this service need to check,” which is what lets you swap or federate identity providers later without a rewrite.

Authorization at Scale: From ACL Tables to Relationship Graphs

Establishing who the caller is only answers half the question that matters before any mutation. The other half, what they’re allowed to touch, is where naive designs collapse first at scale.

The obvious starting point is a permissions table and a query:

-- The anti-pattern: relational ACL query
SELECT 1 FROM document_permissions
WHERE document_id = 456 AND user_id = 123 AND permission = 'EDIT';

This works until permissions stop being flat. If Alice’s access comes through being a member of the Eng team, which belongs to the Engineering org, which has access to a Projects folder, which contains the document, a single permission check now needs recursive joins across a hierarchy table. And a dashboard listing 100 files triggers 100 separate authorization queries against a primary transactional database, an N+1 problem that shows up in production exactly when you can least afford it, during a traffic spike.

Modern authorization engines, such as OpenFGA, Ory Keto, and Permify, replace the table with a relationship graph instead:

A permission check becomes a graph-reachability question: can you get from Alice to RecoveryPlan by following edges that compose into “editor”? It’s evaluated in memory with localized caching, typically in under 5 milliseconds regardless of how deep the hierarchy is. It’s the same reason graph traversal beats repeated SQL joins for any deeply nested relationship: the structure is native to the query instead of being reconstructed from flat tables every time.

Discomfort check. Stateless signed tokens are attractive because services verify them in zero milliseconds without a database round-trip. The uncomfortable corollary is that if someone is terminated right now, they still have working access until their token happens to expire. That’s exactly why identity and authorization have to be decoupled deliberately rather than treated as one solved problem. Internal tokens are kept deliberately short-lived, two to five minutes is typical, which bounds the exposure window on its own. Authorization is checked separately and freshly on every state-changing call, against an engine like OpenFGA, which reflects a group membership revocation the instant it happens rather than whenever a cached token expires. For genuinely urgent lockouts, the identity provider increments a per-user token_epoch; the edge gateway rejects any refresh request for a stale epoch immediately, which caps the damage to whatever the current short-lived token can still do until it expires on its own. The token proves who someone is; it was never meant to prove they still should be able to do something.

The Anatomy of a Production Request

Put every piece together and a single “delete record” click traces a path like this:

Every arrow in that diagram is a decision this post covered: which direction the TCP handshake goes, which kind of proxy is involved, whether a credential gets translated or forwarded as-is, and whether the callee re-checks authorization or just trusts the caller. Skipping any one of them is exactly how a system ends up hard-shelled and soft-centered without anyone deciding that on purpose.

Future Outlook

SPIFFE/SPIRE-issued workload identities are increasingly replacing static service credentials and shared secrets for service-to-service auth, the same way passkeys are replacing passwords for humans - both are converging on “short-lived cryptographic proof tied to a specific, attestable identity” as the default rather than the advanced option. On the authorization side, general-purpose policy engines like OPA and Cedar, and relationship-graph engines like OpenFGA and Ory Keto, are converging too: expect the line between “policy-as-code” and “ReBAC” to blur as both solve the same underlying need - fast, centrally-updated, real-time permission checks that don’t require redeploying a service to change a rule.

Summary