Linkerd + Cluster Bootstrapping

Linkerd General Discussion
1

I initially posted this on the linkerd slack - it was recommended that I post my questions here instead.

Is there a best-practice regarding bootstrapping a cluster with linkerd? it seems like there will always be workloads that need to exist prior to deploying linkerd and the only way to add these workloads to linkerd’s service mesh is to redeploy them after linkerd has been deployed.

An example:

  • I’m using the linkerd-control-plane helm chart to render my manifests.
  • Because i want to use GitOps, I’m going to use cert-manager and trust-manager custom resources to create the identity trust anchors - and so i’ll need to deploy both of these alongside linkerd-control-plane.
  • Eventually, I want to connect everything to Prometheus - so i’ll set podMonitor.enabled=true for the linkerd-control-plane helm chart.
  • Because this depends on custom resources provided from the Prometheus Operator - i need to deploy the Prometheus Operator alongside linkerd.
  • However, I’m using kube-prometheus-stack to deploy the Prometheus Operator. Consequently, I’m also deploying Prometheus, Grafana and Alertmanager alongside linkerd.
  • Unfortunately, i want to set up persistence for Prometheus - so I’ll want to deploy a CSI before deploying kube-prometheus-stack - I’ll end up deploying OpenEBS’ dynamic-localpv-provisioner alongside linkerd.

In my case, deploying linkerd requires deploying several pre-requisites - and no matter how you tackle deployments (all-at-once with eventual consistenty vs. gated in dependency order) - you’ll always end up with a set of workloads that need to be restarted once linkerd is deployed.

Any ideas for a ‘feels right’ solution to this problem? Could/should linkerd ship with some sort of job that can restart workloads that are annotated with automatic proxy injection that aren’t currently proxy injected?

2

I’m probably the one who suggested posting here, so apologies for the crickets.

Restarting the pods as part of a bootstrap script doesn’t strike me as the end of the world. It’s not super elegant but it’s one of the consequences of Kubernetes’s immutable pod model and not the worst such consequence.

The closest to an automated solution that we have for this is on the Linkerd Enterprise side where we have a dataplane operator that can restart workloads to insert the proxy. But I don’t know if it really simplifies things for you since you have to create the CRs pointing it to the relevant workloads after Linkerd installation anyways, so it’s just another step you’d have to take. I supposed we could make this look at an annotation rather than a custom resource…