Using linkerd for multi cluster use with firewall

Linkerd General Discussion
, ,
1

In our setup, there are two separate kubernetes clusters in virtual private clouds, High and Low. These clusters run the same services, but with different data. No connections can be established from Low to High. All traffic leaving the High cluster must go through a firewall proxy inspecting the contents; connections that cannot be decrypted are dropped. Services in High need to access services in Low. Using normal TLS connections from High to Low is no issue, as long as the proxy’s certificate is trusted in both High and Low.

We’d like to join the two separate clusters into a single service mesh, like linkerd, as documented in Multi-cluster communication | Linkerd, ideally as a flat network. However, we are unsure if this is possible because of the firewall and if so, how linkerd should be configured.

Currently we see these two options:

  • Disable mTLS altogether for all services in both clusters.
  • Use two separate services meshes in High and in Low, essentially removing all benefits from having a single service mesh.

Are there any more? Do you have any other suggestions? If the required features would be covered by Linkerd Enterprise (I am unsure when reading the documentation on Security policy automation | Buoyant Enterprise for Linkerd), that would be fine.

2

Hi @kevfoerster ! Thanks for the interesting question here an apologies for the delay in responding. I’m a little confused by your statement that “connections that cannot be decrypted are dropped” but also “using normal TLS connections from High to Low is no issue”. If the connections are TLS’d, they won’t be able to be decrypted by the firewall. Or am I missing something?

FWIW, Linkerd’s model for multicluster is federation, so there’s not really “a single service mesh” vs “multiple service meshes”. Both clusters will run their own control plane, and the only questions are a) whether they share a TLS trust root, and b) whether they are connected in pod-to-pod (flat network) mode or through a gateway. And also FWIW, disabling TLS in Linkerd is not really possible.

3

Either way, it may be worth a deeper conversation here. We have some upcoming egress features in 2.17 that may be relevant to what you’re trying to accomplish.