Proxying WebSockets Through Kong and Envoy

A WebSocket that works on a developer machine and fails behind the gateway almost always fails for one of four reasons: the upgrade was not permitted, a timeout closed it, a header was dropped, or something in the path buffered it. This page walks the upgrade through Kong 3.x and Envoy 1.32+ with the settings that matter at each step, and gives the diagnostic that distinguishes the four cases in about a minute.

Prerequisite concepts

This assumes the connection-accounting and policy-window model from streaming, WebSocket and realtime gateways, and the upgrade mechanics described in protocol translation patterns. You should also know which policies your route applies, because after the upgrade none of them run again.

What has to survive the handshake

An upgrade is an ordinary HTTP request whose headers carry the request to switch protocols. Any hop that drops, rewrites or normalises those headers breaks the handshake, and the resulting error usually blames the client.

Four headers, four different failures when one goes missing Connection upgrade, Upgrade websocket, the Sec-WebSocket-Key and the version header must all arrive unchanged. Losing the connection header produces a plain 200 with the wrong body, losing the upgrade header produces a 400, a rewritten key fails the client-side handshake check, and a version mismatch produces a 426. header if it does not arrive intact connection: Upgrade a normal 200 response and no socket at all upgrade: websocket 400 from the upstream framework sec-websocket-key client rejects the accept value and closes sec-websocket-version: 13 426 Upgrade Required A gateway that strips unknown headers as a hygiene measure will break all four — allow-list these explicitly.

The upstream answers 101 Switching Protocols, and from that instant the connection is opaque to every proxy in the path.

Envoy 1.32+

Envoy requires the upgrade to be enabled explicitly, per listener or per route. Nothing else about the route changes, but the timeouts do.

# Envoy 1.32+ — WebSocket on /ws, request-response on everything else
route_config:
  name: edge
  virtual_hosts:
    - name: app
      domains: ["app.example.com"]
      routes:
        - match: { prefix: "/ws" }
          route:
            cluster: realtime_upstream
            timeout: 0s                  # no overall limit — a stream is one request
            idle_timeout: 300s           # but reap sockets exchanging nothing
            upgrade_configs:
              - upgrade_type: websocket
                enabled: true
        - match: { prefix: "/" }
          route:
            cluster: api_upstream
            timeout: 15s                 # unchanged for ordinary traffic
# The connection manager side — the stream idle timer is the one that surprises people
typed_config:
  "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
  stat_prefix: edge
  stream_idle_timeout: 0s                # default is 5 min and applies to upgrades too
  common_http_protocol_options:
    idle_timeout: 3600s
    max_connection_duration: 14400s      # forces reconnection every four hours
  upgrade_configs:
    - upgrade_type: websocket

stream_idle_timeout defaults to five minutes and applies to the upgraded stream, which is why a WebSocket configured only at the route level dies at exactly five minutes of silence. Setting it to zero disables the stream-level timer while leaving the route-level idle_timeout in place as the real reaper.

Kong 3.x

Kong proxies upgrades without special configuration, inheriting NGINX’s behaviour, but the timeouts default to a minute and will close an idle socket without ceremony.

_format_version: "3.0"
services:
  - name: realtime
    url: http://realtime.internal:8080
    connect_timeout: 5000
    read_timeout: 3600000        # ms — must exceed the longest expected silence
    write_timeout: 3600000
    routes:
      - name: ws
        paths: ["/ws"]
        strip_path: false        # the upstream usually needs the full path
        protocols: ["http", "https"]
    plugins:
      - name: jwt                # runs once, at the handshake
      - name: rate-limiting
        config:
          minute: 12             # connection attempts, not messages
          policy: redis

strip_path: false matters more here than on ordinary routes: many WebSocket servers route on the path they receive, and a stripped prefix produces a 404 during the handshake that looks like an upgrade failure.

Diagnosing a failure in one pass

The four failure classes are distinguishable by what the handshake returns and when the connection dies.

Start from what the handshake returned If the handshake never returns 101, the upgrade was not permitted or a header was lost. If it returns 101 and then closes at a consistent interval, a timer is responsible and the interval identifies which hop. If it returns 101 and stays open but no messages arrive, something is buffering. did the handshake return 101? no yes upgrade not enabled on the route, or a required header was dropped compare the headers the upstream received against the ones the client sent does it close at a consistent interval? yes no a timer — the interval names the hop 60 s client, 350 s cloud LB, 300 s stream idle buffering — frames held somewhere check every proxy between client and app 
# Does the upgrade complete at all?
curl -sSv -o /dev/null \
  -H "Connection: Upgrade" -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Version: 13" \
  -H "Sec-WebSocket-Key: $(head -c16 /dev/urandom | base64)" \
  https://app.example.com/ws 2>&1 | grep -E '^< HTTP|^< (upgrade|connection|sec-)'
# expect: < HTTP/1.1 101 Switching Protocols

# How long does an idle socket survive? Run this and watch the clock.
websocat -v wss://app.example.com/ws

An interval that is suspiciously round — exactly 60, exactly 300, exactly 350 seconds — is a timer, and the value usually identifies the hop without further investigation.

Decision matrix

Symptom Most likely cause Fix
200 with an HTML body connection header stripped upstream of the gateway allow-list the upgrade headers
404 during handshake path stripped by the route strip_path: false
101 then close at 300 s Envoy stream_idle_timeout default set it to 0s, keep route idle_timeout
101 then close at 60 s client library idle timer shorten the heartbeat, not the timeout
Connects, no messages arrive buffering in an intermediary disable response buffering on every hop
Works, then fails at scale connection ceiling or descriptor limit set both explicitly and alert before them

Gotchas and failure signals

Access logs appear only at close, so an upgrade that succeeds is invisible until the socket ends. Export an active-connection gauge or you are blind during exactly the period you care about.

Retries are meaningless after an upgrade and can be harmful before one: retrying a failed handshake against a second upstream is fine, but a proxy that retries mid-stream will produce a duplicate session the application did not expect.

Compression extensions negotiate per connection. permessage-deflate is agreed during the handshake, and a proxy that rewrites the extension header can leave the two ends disagreeing about whether frames are compressed — which appears as corrupt messages rather than as a handshake error.

A load balancer health check that opens and closes a socket counts against connection-rate limits and pollutes the age distribution. Give probes a separate route.

What a deploy does to an established socket The connection is established and idle for hours. A rolling deploy marks the node draining, which stops new connections but does not close existing ones until the drain deadline, at which point the socket is closed and the client reconnects to another node. The gap the client experiences is its own backoff, not the drain. established, idle, heartbeats only draining re-established elsewhere close frame sent drain deadline reached A drain that is shorter than your longest expected session simply moves the disconnection earlier — it does not avoid it. Send a close frame with a normal status so clients back off rather than treating it as a network error. Stagger node drains: every socket on a node reconnects at once, and ten nodes draining together is a stampede.

Validation

  • Handshake returns 101 through the full production path, not just against the upstream directly
  • An idle socket survives longer than the heartbeat interval by a comfortable margin
  • stream_idle_timeout explicitly set on Envoy, read_timeout and write_timeout on Kong
  • Upstream receives the path it expects, verified from the upstream’s own logs
  • Active-connection gauge visible on a dashboard alongside request-rate panels
  • Connection-attempt rate limit exercised with a simulated reconnect storm

FAQ

Why does my WebSocket close after exactly five minutes on Envoy?

That is stream_idle_timeout, which defaults to five minutes and applies to upgraded streams as well as ordinary requests. A route-level timeout: 0s does not disable it. Set stream_idle_timeout: 0s on the connection manager and keep a route-level idle_timeout as the mechanism that actually reaps dead sockets.

Does Kong need special configuration for WebSockets?

Not for the upgrade itself, which it proxies by inheriting NGINX behaviour. It does need the timeouts raised: read_timeout and write_timeout default to sixty seconds and will close an idle socket without any error the client can interpret. Set strip_path: false as well, since most WebSocket servers route on the path they receive.

Can I apply authentication to a WebSocket route?

Yes, and it runs exactly once during the handshake. That is enough to reject an unauthenticated connection, and it is not enough to enforce anything afterwards: a token that expires mid-stream keeps working until the connection closes. Keep the maximum connection duration below the token lifetime if that matters, and give the application a way to close sessions.

How do I tell a timeout from a dropped header?

By when it fails. If the handshake never returns 101, a header was lost or the upgrade was not enabled on the route. If it returns 101 and then closes at a suspiciously round interval — sixty, three hundred, three hundred and fifty seconds — a timer is responsible, and the value usually identifies which hop owns it.

Parent: Streaming, WebSocket & Realtime Gateways