Linkerd + Cluster Bootstrapping

I initially posted this on the linkerd slack - it was recommended that I post my questions here instead.

Is there a best-practice regarding bootstrapping a cluster with linkerd? it seems like there will always be workloads that need to exist prior to deploying linkerd and the only way to add these workloads to linkerd’s service mesh is to redeploy them after linkerd has been deployed.

An example:

  • I’m using the linkerd-control-plane helm chart to render my manifests.
  • Because i want to use GitOps, I’m going to use cert-manager and trust-manager custom resources to create the identity trust anchors - and so i’ll need to deploy both of these alongside linkerd-control-plane.
  • Eventually, I want to connect everything to Prometheus - so i’ll set podMonitor.enabled=true for the linkerd-control-plane helm chart.
  • Because this depends on custom resources provided from the Prometheus Operator - i need to deploy the Prometheus Operator alongside linkerd.
  • However, I’m using kube-prometheus-stack to deploy the Prometheus Operator. Consequently, I’m also deploying Prometheus, Grafana and Alertmanager alongside linkerd.
  • Unfortunately, i want to set up persistence for Prometheus - so I’ll want to deploy a CSI before deploying kube-prometheus-stack - I’ll end up deploying OpenEBS’ dynamic-localpv-provisioner alongside linkerd.

In my case, deploying linkerd requires deploying several pre-requisites - and no matter how you tackle deployments (all-at-once with eventual consistenty vs. gated in dependency order) - you’ll always end up with a set of workloads that need to be restarted once linkerd is deployed.

Any ideas for a ‘feels right’ solution to this problem? Could/should linkerd ship with some sort of job that can restart workloads that are annotated with automatic proxy injection that aren’t currently proxy injected?