Migrating from Ingress-NGINX to Gateway API

Ingress-NGINX has been the default answer for exposing HTTP from Kubernetes for years, and the reason to move off it is rarely a single missing feature. It is the accumulation: a dozen annotations whose behaviour is documented in a controller’s source, a rewrite rule nobody dares touch, and no way to give an application team control of their own routing without also giving them the object that holds the public certificate. This page is a cutover procedure that keeps both controllers running, moves one hostname at a time, and can be reversed at any point with a DNS change.

Prerequisite concepts

This assumes the resource model covered in Kubernetes Ingress and Gateway API — GatewayClass, Gateway, HTTPRoute, and the two-sided attachment rules. It also assumes you can provision a second load balancer address, because running the two data planes behind one address is the variant of this migration that goes wrong.

The shape of the migration

Both controllers run simultaneously on separate addresses. DNS decides which one serves a given hostname, so every step is reversible in the time it takes a record to expire.

DNS is the switch, so rollback costs a TTL The existing Ingress-NGINX controller keeps its address and its Ingress objects. A new Gateway on its own address serves HTTPRoutes. A hostname moves by repointing its DNS record, and moves back the same way, which means no cutover step requires deleting anything. DNS: orders.api.example.com before after Ingress-NGINX — 203.0.113.10 Ingress objects, unchanged still serving every other hostname Gateway — 203.0.113.44 HTTPRoutes, translated from annotations one hostname at a time Nothing is deleted until the last hostname has been on the new address for a full business cycle.

Step 1 — Install the CRDs and a Gateway on a new address

The Gateway API CRDs are versioned independently of any controller, and installing a version older than your controller expects produces validation errors that read like syntax mistakes. Pin both in the same release.

# Envoy Gateway 1.2+ / Gateway API v1.2 CRDs installed separately
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: edge-next
  namespace: gateway-infra
spec:
  gatewayClassName: envoy-gateway
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      hostname: "*.api.example.com"
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: api-example-com-tls      # the same Secret the Ingress uses
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels: { gateway-access: "edge" }

Confirm it received its own address before going further:

kubectl get gateway edge-next -n gateway-infra \
  -o jsonpath='{.status.addresses[0].value}{"\n"}'
# 203.0.113.44   <- must differ from the Ingress-NGINX service address

Step 2 — Translate annotations, one Ingress at a time

Most of the work is mechanical. The table below covers the annotations that account for the overwhelming majority of real Ingress objects.

Sort the annotations before translating any of them Roughly three quarters of annotations in a typical deployment map one to one onto typed fields and can be translated in bulk. A smaller group becomes an implementation-specific policy resource. A handful, chiefly configuration snippets, have no equivalent and are findings that need a decision rather than a translation. annotations in a typical deployment, by translation effort one-to-one with a typed field — rewrite, weight, timeout, redirect translate in bulk becomes a policy resource — CORS, auth, rate limit per implementation configuration-snippet read every one by hand The narrow dashed bar is where the schedule goes. An arbitrary block of proxy configuration injected into a location has no typed equivalent by design, and about half of them turn out to belong in the application anyway.
Ingress-NGINX annotation Gateway API equivalent
nginx.ingress.kubernetes.io/rewrite-target URLRewrite filter with path.type: ReplacePrefixMatch
nginx.ingress.kubernetes.io/canary-weight second entry in backendRefs with weight
nginx.ingress.kubernetes.io/proxy-read-timeout timeouts.backendRequest
nginx.ingress.kubernetes.io/ssl-redirect an HTTP listener plus a RequestRedirect filter
nginx.ingress.kubernetes.io/configuration-snippet no equivalent — this is the one to review by hand
nginx.ingress.kubernetes.io/enable-cors implementation policy resource, not portable

The snippet annotation is where migrations stall, and it is worth treating as a finding rather than a translation task: an arbitrary block of NGINX configuration injected into a location has no typed equivalent by design. Read each one and decide whether it is a filter, a policy, or something that belonged in the application all along.

# Before: Ingress with a rewrite and a canary
#   rewrite-target: /$2
#   canary-weight: "10"
# After:
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: orders
  namespace: orders-prod
spec:
  parentRefs:
    - { name: edge-next, namespace: gateway-infra, sectionName: https }
  hostnames: ["orders.api.example.com"]
  rules:
    - matches:
        - path: { type: PathPrefix, value: /api/orders }
      filters:
        - type: URLRewrite
          urlRewrite:
            path:
              type: ReplacePrefixMatch
              replacePrefixMatch: /            # /api/orders/8891 -> /8891
      backendRefs:
        - { name: orders, port: 8080, weight: 90 }
        - { name: orders-canary, port: 8080, weight: 10 }
      timeouts: { request: 5s, backendRequest: 2s }

Step 3 — Verify against the old path before moving DNS

The new route is reachable on the new address immediately. Test it there, with the Host header set, while production traffic is still going to the old controller.

curl -sS -o /dev/null -w '%{http_code} %{time_total}s\n' \
  --resolve orders.api.example.com:443:203.0.113.44 \
  https://orders.api.example.com/api/orders/8891

# and confirm the route attached and resolved:
kubectl get httproute orders -n orders-prod \
  -o jsonpath='{.status.parents[0].conditions[*].type}{"\n"}'
# Accepted ResolvedRefs

Compare status codes and response bodies between the two addresses for a representative set of paths, including one that should 404 and one that should be rejected by policy. A rewrite that drops a path segment usually passes the happy-path check and fails the 404 check, because the shape of the rewritten path only becomes visible when the upstream cannot find it.

Step 4 — Move DNS, watch, then repeat

Lower the record’s TTL a day in advance, move one hostname, and watch four things for an hour: 4xx rate, 5xx rate, p99 latency, and request volume on both controllers. Volume is the one people forget — if the old controller’s volume for that hostname does not fall to zero, something is still resolving the old address, and the two halves of your traffic are now being served by two different configurations.

Watch both sides of the cutover, not just the new one As the DNS change propagates, volume on the old controller falls while volume on the new one rises. The pair should cross cleanly and the old line should reach zero within a few TTLs. A floor that never reaches zero means a hard-coded address or a stale resolver somewhere. req/s minutes after the change Ingress-NGINX — must reach zero Gateway A residual floor on the dashed line is a client with a pinned IP or a resolver ignoring the TTL — find it before you decommission anything, because that client is the one that will page you at the end of the migration.

Decision matrix

Situation Approach
Dozens of simple Ingress objects translate in bulk, cut over in hostname batches
Heavy use of configuration-snippet audit each snippet first; some become application changes
A single shared wildcard certificate reuse the same Secret from both controllers, no re-issue needed
Cannot provision a second address use a canary weight at the DNS or CDN tier instead; do not share one address
Regulated workload needing an audit trail migrate last, after the pattern is proven on lower-risk hostnames

Gotchas and failure signals

Both controllers claiming the same Ingress class silently produces two data planes programmed from one object. Set ingressClassName explicitly on every remaining Ingress before you start, rather than relying on a default.

The rewrite semantics are not identical. ReplacePrefixMatch replaces exactly the matched prefix; the NGINX capture-group style could rewrite arbitrarily. Any rule using a regular expression capture needs its output checked, not assumed.

Certificates renewed by cert-manager keep working, but only if the new Gateway references the same Secret — a second Issuer racing for the same hostname produces rate-limit failures at the certificate authority rather than an obvious error.

Watch for Accepted: False after a namespace label change. The label selector on the listener is evaluated continuously; removing the label from a namespace detaches every route in it at once, with no deploy involved.

Validation

  • The new Gateway reports a distinct address in status.addresses
  • Every translated route reports Accepted and ResolvedRefs true
  • Response codes and bodies match between old and new addresses for happy path, 404 and policy-rejection cases
  • Old-controller request volume for the migrated hostname reaches zero
  • Rollback tested once, in daylight, by moving one hostname back

FAQ

Can both controllers run on the same cluster during the migration?

Yes, and they should. They watch different resource types and program different data planes, so the only shared state is the Kubernetes API server and any TLS Secret you reference from both. The important constraint is that each gets its own load balancer address — sharing one address means DNS can no longer select between them, and rollback stops being a record change.

How do I translate a rewrite-target annotation?

Most cases become a URLRewrite filter with path.type: ReplacePrefixMatch, where the replacement is what should stand in for the matched prefix. Annotations that used a regular-expression capture group have no direct equivalent, because the typed filter replaces the matched prefix rather than applying a pattern. Test those individually, including a path that should return 404, since a wrong rewrite usually still returns 200 on the happy path.

What happens to my cert-manager certificates?

Nothing, provided the new Gateway references the same Secret the Ingress used. The certificate object and its renewal loop are unchanged. What you must avoid is creating a second Issuer or Certificate for the same hostname while the first still exists — the two will race and you will hit the certificate authority rate limit rather than see a clear error.

How do I know the cutover is finished?

Request volume for the migrated hostname on the old controller reaches zero and stays there through at least one full traffic cycle including any overnight batch jobs. A small persistent floor means a client with a hard-coded address or a resolver ignoring the TTL, and that client is the one that breaks when the old controller is finally removed.

Parent: Kubernetes Ingress & Gateway API