How to forward the client IP for whitelisting policies instead of the Linkerd sidecar IP in NGINX Ingress Controller (v4.12+)?

Linkerd General Discussion
1

Question:
I’m using the NGINX Ingress Controller with Linkerd as a service mesh. I need to enforce IP whitelisting policies based on the client’s IP address, but the NGINX Ingress Controller is currently using the Linkerd sidecar’s IP instead. This causes the whitelisting policy to fail for my VPN and other API users.

Current Setup

  • NGINX Ingress Controller: v4.11 (downgraded because v4.12+ disallows configuration-snippetand server-snippet annotations for security reasons).

  • Linkerd: Injected as a sidecar into the application Pods.

  • Whitelisting: Configured using the nginx.ingress.kubernetes.io/whitelist-source-rangeannotation.

  • I currently rely on the disallowed snippets. I knew this was suboptimal from the start but now nginx gives the security warning I definitely want to fix it.

  • When I whitelist the specific sidecar IP, it works, but this is not a scalable solution.

  • I’ve tried skipping inbound ports and marking them as opaque in the deployment, but the client IP is still not forwarded correctly.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress-api
  namespace: api-dev
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/whitelist-source-range: "1.2.3.4/16"
    linkerd.io/inject: enabled
    nginx.ingress.kubernetes.io/use-forwarded-headers: "true"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - dev.api.example.com
    secretName: example-tls
  rules:
  - host: dev.api.example.com
    http:
      paths:
      - pathType: ImplementationSpecific
        path: "/calculate_test"
        backend:
          service:
            name: test-api-service
            port:
              number: 8080

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gini-api-deployment
  namespace: api-dev
  labels:
    app: gini-api
spec:
  replicas: 2
  selector:
    matchLabels:
      app: gini-api
  template:
    metadata:
      annotations:
        linkerd.io/inject: enabled
        config.alpha.linkerd.io/proxy-enable-native-sidecar: "true"
        config.linkerd.io/shutdown-grace-period: "60"
      labels:
        app: gini-api
    spec:
      containers:
      - name: gini-api
        image: healtheworld.azurecr.io/gini_api_dev:latest
        ports:
        - containerPort: 8080

apiVersion: v1
kind: Service
metadata:
  name: test-api-service
  namespace: api-dev
spec:
  type: ClusterIP
  ports:
  - port: 8080
    targetPort: 8080
  selector:
    app: test-api

Install or upgrade the Nginx Ingress Controller

  helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx \
    --version 4.11.0 \
    --set controller.service.externalTrafficPolicy=Local \
    --set-string controller.extraArgs.update-status="false" \
    --set controller.podAnnotations."linkerd\.io/inject"=enabled \
    --set controller.podAnnotations."config\.linkerd\.io/skip-inbound-ports"="80\,443" \
    --set controller.podAnnotations."config\.linkerd\.io/skip-outbound-ports"="80\,443" \
    --set controller.config.use-forwarded-headers="true" \
    --set controller.config.compute-full-forwarded-for="true" \
    --set controller.config.use-proxy-protocol="false" \
    --set controller.config.real-ip-header="X-Forwarded-For" \
    --set controller.config.proxy-real-ip-cidr="0.0.0.0/0" \
    --set controller.config.allow-snippet-annotations="true" \
    --set controller.config.enable-snippet="true" \
    --set controller.config.proxy-body-size="100m" \
    --set controller.metrics.enabled=true \
    --wait

I have changed the names, urls, IPs and dropped env vars/secrets, so apologies if I made any mistakes. I did not changes the annotations and ports, which I suspect to be the issue.

Does anyone have an idea on this?

It seems like a basic issue to use whitelisting based on environments (for dev VPNs for example) while using LinkerD and Nginx, and I cannot imagine being the first one doing this, but I couldn’t find a clear example or documentation anywhere.

Thanks for helping out!

2

Hi @TheusesKristus

Can you ensure the client IP is correctly forwarded through the NGINX Ingress Controller and Linkerd setup.