Choosing Gateway or Mesh for East-West Traffic

“Should this call go through the gateway or through the mesh?” is asked as an architecture question and is really a question about who is calling. The answer follows from one property of the caller — whether it is inside your trust boundary and running your identity infrastructure — and everything else follows from that. This page gives the decision, the three cases where it is genuinely ambiguous, and what it costs to get it wrong in each direction.

Prerequisite concepts

This builds on API gateway vs service mesh, which covers the capability overlap between the two layers, and assumes the trust-boundary framing from security boundaries and zero trust.

The question that decides it

One property of the caller settles it If the caller holds a workload identity issued by your own infrastructure, the mesh can authenticate it cryptographically and the call belongs east-west. If it holds only a bearer credential you issued to an external party, the gateway is the only layer that can validate it, and the call belongs north-south regardless of where the caller physically runs. does the caller hold a workload identity? yes no east-west — the mesh authenticates it cryptographically, per hop retries, timeouts and outlier ejection belong to the sidecar north-south — only the gateway can validate a bearer credential quotas, contracts and versioning belong to the gateway Physical location is irrelevant: a partner service running in your cluster is still a north-south caller.

The rule generalises well because it tracks what each layer can actually verify. A sidecar proves who a workload is with a certificate it issued; it has no opinion about an OAuth token. A gateway validates tokens and quotas; it has no visibility into a call that never leaves the internal network.

The three ambiguous cases

An internal service calling another internal service, but on behalf of an end user. The mesh authenticates the workload; it does not authorise the user. The correct pattern is both: mesh mTLS for the hop, plus a propagated end-user token the receiving service validates. Routing it through the gateway to get token validation adds a hop and a bottleneck for no gain.

A partner service that you host. It runs in your cluster, so it looks east-west, but it is operated by someone else and holds a credential you issued to them. It is north-south. Give it a gateway route with its own quota, and do not put it on the mesh’s trust domain.

A batch job pulling from an internal API. No user, no external party, and often no sidecar because it runs outside the mesh. This is where teams reach for the gateway as a convenience, which is fine — but then it needs a real API credential and a quota, not an allow-listed IP range.

What each mistake costs

The two mistakes are not symmetrical Sending internal traffic through the edge gateway costs an extra network hop, a shared failure domain and a scaling bottleneck. Letting external traffic bypass the gateway costs the ability to authenticate, meter and version that traffic at all — an availability problem versus a correctness and security problem. east-west traffic sent through the gateway an extra hop, a shared failure domain, and one component that must scale with internal traffic recoverable: it is an availability and cost problem, visible on a latency graph north-south traffic that bypasses the gateway no token validation, no quota, no version contract, and no record that the caller exists not recoverable by tuning: it is a correctness problem, and it is invisible until it is an incident When genuinely unsure, err toward the gateway — the cost is measurable rather than silent.

Running both without doubling the policy

The failure mode of adopting both layers is duplicated enforcement: retries configured at the gateway and at the sidecar, so a single upstream blip produces the product of the two rather than the sum. Assign each control to exactly one layer and write it down.

One control, one owner External identity, quotas and version routing are owned by the gateway. Workload identity, per-hop retries, timeouts and outlier ejection are owned by the mesh. Retries in particular must be owned by one layer only, because two layers each retrying twice produces nine attempts rather than five. gateway owns mesh owns external identity and token validation workload identity and mutual TLS per-consumer quotas and billing per-hop timeouts version routing and deprecation outlier ejection and load balancing payload transformation traffic shifting between service versions retries: pick ONE layer — 2 retries at each of two layers is 9 attempts, not 5 Write the assignment into the platform docs; it is the question every new service will ask.

Decision matrix

Caller Layer Credential
Public client or mobile app gateway OAuth token or API key
Partner service, wherever it runs gateway issued client credential, own quota
Internal service, own cluster mesh workload certificate
Internal service acting for a user mesh, with a propagated user token both
Batch job outside the mesh gateway a real credential, not an IP allow-list
Third-party SaaS webhook gateway signed request or mutual TLS

Gotchas and failure signals

Latency that grows with internal traffic on an edge gateway is the signature of east-west calls being routed through it. Look for upstream hosts in the access log that are internal service names.

Two layers retrying shows as upstream load rising superlinearly during a partial outage. The arithmetic is in tuning retry budgets to prevent thundering herd.

An internal service that “just needs a route” is usually a request to skip the identity work. Ask which credential it will present; if the answer is an IP range, the design is not finished.

Validation

  • Every route on the edge gateway has an external consumer with a named credential
  • No internal service name appears as an upstream on an edge gateway route
  • Retries are configured at exactly one layer per call path, documented
  • Partner workloads hosted in-cluster are on gateway routes, not the mesh trust domain
  • End-user identity propagation is separate from workload identity, and both are validated

FAQ

What is the actual rule for deciding?

Ask whether the caller holds a workload identity issued by your own infrastructure. If it does, the mesh can authenticate it cryptographically and the call belongs east-west. If it holds a bearer credential you issued to an external party, only the gateway can validate it and the call is north-south — regardless of whether the caller happens to run inside your cluster.

Is it ever right to send internal traffic through the edge gateway?

Occasionally, for a workload that has no sidecar and no workload identity, such as a batch job outside the mesh. Treat it as a real API consumer with a credential and a quota rather than an allow-listed IP range. What you should avoid is routing meshed service-to-service calls through the edge, which adds a hop and makes the gateway scale with internal traffic.

Which layer should own retries?

Exactly one of them, per call path, and it should be written down. Two retries at the gateway and two at the sidecar is nine attempts rather than five, and the resulting load amplification during a partial outage looks like an upstream problem rather than a configuration one. The mesh is the usual owner for per-hop retries because it sees each hop.

How do I handle an internal call made on behalf of an end user?

Use both layers for their own concern: mesh mutual TLS authenticates the calling workload, and a propagated end-user token carries the user identity for the receiving service to validate. Routing the call through the gateway to obtain token validation adds a bottleneck without adding a check the receiving service could not perform itself.

Parent: API Gateway vs Service Mesh