All - I had so many issues trying to get things working with cert-manager that I resorted to just passing the cert values to linkerd via helm install. Since I’m currently trying to let linkerd manage all the certificates I don’t know why we would be having this issue.
I did look through this thread but we are not using cert-manager - [Cert-Manager] Webhook Certificates renewal Failure - Linkerd General Discussion - Linkerd Support Forum by Buoyant
We are using ArgoCD as well so this was helpful for passing certificate values as helm parameters - Using GitOps with Linkerd with Argo CD | Linkerd
So our implementation looks like so
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
annotations:
argocd.argoproj.io/sync-wave: "-8"
name: mycluster-dev-004
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
destination:
namespace: linkerd
server: https://mycluster-dev-004.privatelink.eastus.azmk8s.io:443
sources:
- repoURL: myacr001.azurecr.io
chart: helm/linkerd-control-plane
targetRevision: 2024.10.2
helm:
releaseName: linkerd
parameters:
- name: identityTrustAnchorsPEM
value: |
-----BEGIN CERTIFICATE-----
MIIBjTCCATOgAwIBAgIQSRCv2i/DOw2RXPMkIEBbhDAKBggqhkjOPQQDAjAlMSMw
IQYDVQQDExpyb290LmxpbmtlcmQuY2x1c3Rlci5sb2NhbDAeFw0yNDExMDEwMTAx
MDFaFw0zNDExMDMxODI0MDlaMCUxIzAhBgNVBAMTGnJvb3QubGlua2VyZC5jbHVz
dGVyLmxvY2FsMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE7gxaUJz3hdrhVEl0
zZ5XIEHr8b8cPqcDurXY9wlu0Ztyn4rUrEjPyJVo5WkHNglo0GS1RMpvPygvzhV8
pg0OAaNFMEMwDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB/wQIMAYBAf8CAQEwHQYD
VR0OBBYEFATMqWGnVKeXeeoHCQkmT3Cb54j8MAoGCCqGSM49BAMCA0gAMEUCIQDY
T5pSK9hCwngViL0UN+xD9W5fQZANsNIANA1wXA99KwIgBN+vma45OujvP28e9n5d
6Cdw+4AFtVC+jWgtMk6sSvM=
-----END CERTIFICATE-----
- name: identity.issuer.tls.crtPEM
value: |
-----BEGIN CERTIFICATE-----
MIIBszCCAVmgAwIBAgIRAK7LHACXrzu6ojvPZoPu7PIwCgYIKoZIzj0EAwIwJTEj
MCEGA1UEAxMacm9vdC5saW5rZXJkLmNsdXN0ZXIubG9jYWwwHhcNMjMxMTAxMDEw
MTAxWhcNMzQxMTAzMTgyNTMzWjApMScwJQYDVQQDEx5pZGVudGl0eS5saW5rZXJk
LmNsdXN0ZXIubG9jYWwwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAAS8AW9txe3Y
7drTQz2NIolAWXLwsBGiexB6ZZpgOca/keyNpBGvdsqTsSAxIqH8seY2iGJOyw40
DXj9vIf/JX6yo2YwZDAOBgNVHQ8BAf8EBAMCAQYwEgYDVR0TAQH/BAgwBgEB/wIB
ADAdBgNVHQ4EFgQU+5J7RwNyMYWK6DnreL+3ELpdRIowHwYDVR0jBBgwFoAUBMyp
YadUp5d56gcJCSZPcJvniPwwCgYIKoZIzj0EAwIDSAAwRQIgJW+wyIy5fyh2p2JX
7QPgr1iCKc9j/TjQ4Uy/bGUbtAwCIQCVavQenEuHnc1a5DRUFT4p93Qf99VN0rsp
MLfaWSPXzA==
-----END CERTIFICATE-----
- name: identity.issuer.tls.keyPEM
value: |
-----BEGIN EC PRIVATE KEY-----
blahblahprivatekey
-----END EC PRIVATE KEY-----
- name: proxyInit.runAsRoot
value: "true"
- name: policyController.resources.cpu.limit
value: "150m"
- name: policyController.resources.cpu.request
value: "100m"
- name: policyController.resources.memory.limit
value: "100Mi"
- name: policyController.resources.memory.request
value: "20Mi"
- name: proxyInit.priviledged
value: "true" # TODO: This is false in default chart values
- name: proxyInit.runAsRoot
value: "true" # TODO: This is false in default chart values
# TODO: Move to secret / keyvault value
values: |
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: In
values:
- linux
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: my.nodepool.type
operator: In
values:
- management
- key: my.nodepool.type
operator: NotIn
values:
- system
project: mycluster-dev-004-project
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=false
ignoreDifferences:
- group: admissionregistration.k8s.io
kind: ValidatingWebhookConfiguration
jsonPointers:
- /webhooks
- group: admissionregistration.k8s.io
kind: MutatingWebhookConfiguration
jsonPointers:
- /webhooks
Now here is the error. This seems to show up pretty consistently after a few days (several times now after reinstalling/deleting things then rolling out/running on a fresh instance)
linkerd-webhooks-and-apisvc-tls
-------------------------------
× proxy-injector webhook has valid cert
cert is not issued by the trust anchor: x509: certificate signed by unknown authority (possibly because of "x509: invalid signature: parent certificate cannot sign this kind of certificate" while trying to verify candidate authority certificate "linkerd-proxy-injector.linkerd.svc")
see https://linkerd.io/2/checks/#l5d-proxy-injector-webhook-cert-valid for hints