# Proxy Injection Not Working

**URL:** <https://linkerd.buoyant.io/t/proxy-injection-not-working/507>\
**Category:** Linkerd General Discussion\
**Tags:** proxy\
**Created:** [June 28, 2024, 8:24pm UTC](https://linkerd.buoyant.io/t/proxy-injection-not-working/507 "2024-06-28T20:24:32Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![abarrios](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@abarrios](https://linkerd.buoyant.io/u/abarrios)\
**Post date:** [June 28, 2024, 8:24pm UTC](https://linkerd.buoyant.io/t/proxy-injection-not-working/507/1 "2024-06-28T20:24:32Z")

</div>

Hello everyone,

I am running into an issue injecting a proxy in any deployment. I followed [Linkerd Demo App](https://linkerd.io/2.15/getting-started/#step-4-install-the-demo-app) to confirm the problem was not with my own deployment configurations. Any help would be much appreciated!

Emojivoto Web Deployment:

```auto
~$ kubectl describe deploy -n emojivoto web
Name: web
Namespace: emojivoto
CreationTimestamp: Fri, 28 Jun 2024 20:13:45 +0000
Labels: app.kubernetes.io/name=web
                        app.kubernetes.io/part-of=emojivoto
                        app.kubernetes.io/version=v11
Annotations: deployment.kubernetes.io/revision: 2
Selector: app=web-svc,version=v11
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
  Labels: app=web-svc
                    version=v11
  Annotations: linkerd.io/inject: enabled
  Service Account: web
  Containers:
   web-svc:
    Image: docker.l5d.io/buoyantio/emojivoto-web:v11
    Port: 8080/TCP
    Host Port: 0/TCP
    Requests:
      cpu: 100m
    Environment:
      WEB_PORT: 8080
      EMOJISVC_HOST: emoji-svc.emojivoto:8080
      VOTINGSVC_HOST: voting-svc.emojivoto:8080
      INDEX_BUNDLE: dist/index_bundle.js
    Mounts: <none>
  Volumes: <none>
Conditions:
  Type Status Reason
  ---- ------ ------
  Available True MinimumReplicasAvailable
  Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: web-6c5b7bb685 (1/1 replicas created)
Events:
  Type Reason Age From Message
  ---- ------ ---- ---- -------
  Normal ScalingReplicaSet 61s deployment-controller Scaled up replica set web-555b75889b to 1
  Normal ScalingReplicaSet 46s deployment-controller Scaled up replica set web-6c5b7bb685 to 1
  Normal ScalingReplicaSet 44s deployment-controller Scaled down replica set web-555b75889b to 0 from 1

```

Linkerd Check Emojivoto Proxy:

```auto
linkerd-data-plane
------------------
√ data plane namespace exists
DEBU[0003] Retrying on error: no "linkerd-proxy" containers found in the "emojivoto" namespace
/ waiting for check to complete DEBU[0008] Retrying on error: no "linkerd-proxy" containers found in the "emojivoto" namespace
\ waiting for check to complete DEBU[0013] Retrying on error: no "linkerd-proxy" containers found in the "emojivoto" namespace
/ waiting for check to complete DEBU[0018] Retrying on error: no "linkerd-proxy" containers found in the "emojivoto" namespace

```

Linkerd Proxy Injection Logs:

```auto
time="2024-06-28T19:55:55Z" level=info msg="running version edge-24.3.5"
time="2024-06-28T19:55:55Z" level=info msg="starting admin server on :9995"
time="2024-06-28T19:55:55Z" level=info msg="waiting for caches to sync"
time="2024-06-28T19:55:55Z" level=info msg="listening at :8443"
time="2024-06-28T19:55:55Z" level=info msg="caches synced"

```

List of current running pods in emojivoto:

```auto
~$ kubectl get pods -n emojivoto
NAME READY STATUS RESTARTS AGE
emoji-79d94b5dcb-nz2nh 1/1 Running 0 9m20s
vote-bot-5dd8699f6f-58p4v 1/1 Running 0 9m20s
voting-99fd4c49f-gvg8g 1/1 Running 0 9m20s
web-6c5b7bb685-pgqg4 1/1 Running 0 9m20s

```

---

<div class="post-metadata">

**Author:** ![Alen](https://avatars.discourse-cdn.com/v4/letter/a/9dc877/32.png) [@Alen](https://linkerd.buoyant.io/u/Alen)\
**Post date:** [July 3, 2024, 2:31pm UTC](https://linkerd.buoyant.io/t/proxy-injection-not-working/507/2 "2024-07-03T14:31:29Z")

</div>

Hi @abarrios ,

Normally if injection is not working it could be due to a few things:

1. annotation not in place (not the case here)
2. webhook certs not being correct (doesn’t appear to be a problem - we’d see it in the proxy logs in the injector if this was the case)
3. something else blocking communication between injector and kubeapi.

To explore #3, can you dump the logs from the linkerd-injector container (not the linkerd-proxy container) on the linkerd-injector workload in the linkerd ns when you get a chance?

---

<div class="post-metadata">

**Author:** ![abarrios](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@abarrios](https://linkerd.buoyant.io/u/abarrios)\
**Post date:** [July 3, 2024, 3:53pm UTC](https://linkerd.buoyant.io/t/proxy-injection-not-working/507/3 "2024-07-03T15:53:58Z")

</div>

Sure can. I believe the logs above were from the proxy-injector container but will paste again.

Proxy Injector Container:

```auto
time="2024-07-03T15:01:33Z" level=info msg="running version edge-24.6.4"
time="2024-07-03T15:01:33Z" level=info msg="starting admin server on :9995"
time="2024-07-03T15:01:33Z" level=info msg="waiting for caches to sync"
time="2024-07-03T15:01:33Z" level=info msg="listening at :8443"
time="2024-07-03T15:01:33Z" level=info msg="caches synced"

```

---

<div class="post-metadata">

**Author:** ![Alen](https://avatars.discourse-cdn.com/v4/letter/a/9dc877/32.png) [@Alen](https://linkerd.buoyant.io/u/Alen)\
**Post date:** [July 3, 2024, 4:49pm UTC](https://linkerd.buoyant.io/t/proxy-injection-not-working/507/4 "2024-07-03T16:49:39Z")

</div>

Definitely odd. I assume you’ve cycled the workloads after adding the annotation at the pod’s spec.template.annotations level?

If so, then this suggests that there’s something going on between the kubeapi and the linkerd control plane where the injector is not getting the webhook calls for whatever reason. Do you have any network policies in place that may be in play?

---

<div class="post-metadata">

**Author:** ![abarrios](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@abarrios](https://linkerd.buoyant.io/u/abarrios)\
**Post date:** [July 10, 2024, 8:04pm UTC](https://linkerd.buoyant.io/t/proxy-injection-not-working/507/5 "2024-07-10T20:04:55Z")

</div>

Correct, I did do a restart of the deployment to see if it would trigger linkerd, but no luck.

I did wait while we finished setting up BGP, assuming maybe that was the culprit, but still having the same issues. Nothing outstanding on the networking side besides using HTTP/S Proxy and generic Calico v3.28.0 networking config with BGP configured. Using Ubuntu 22.04 and Kubernetes v1.30.2.

---

<div class="post-metadata">

**Author:** ![william](https://yyz1.discourse-cdn.com/flex029/user_avatar/linkerd.buoyant.io/william/32/5_2.png) [@william](https://linkerd.buoyant.io/u/william)\
**Post date:** [July 10, 2024, 11:27pm UTC](https://linkerd.buoyant.io/t/proxy-injection-not-working/507/6 "2024-07-10T23:27:13Z")

</div>

If you set the proxy injector logs to debug, does that provide any additional information? The proxy injector is a mutating admission webhook, so it’s called by Kubernetes. Either it is failing (which hopefully the debug logs will show) or it isn’t being called.

If the debug logs don’t show anything, there are some other tips for debugging webhooks like this in Kubernetes, e.g. setting the failure policy to fail. See [How do I debug a Kubernetes validating admission webhook? - Stack Overflow](https://stackoverflow.com/questions/50767495/how-do-i-debug-a-kubernetes-validating-admission-webhook) for more.
