Hi,
Environment: Running kube-proxy free Kubernetes with Cilium CNI (full eBPF…almost)
Since the edge-23.3.2 came out and then with Linkerd stable release 2.13, I had been trying to get the header based routing - Dynamic Request Routing to work however ran into an unexpected behaviour.
I could not get Linkerd to process HTTPRoutes CRs at all or atleast from the looks of it. For one, the traffic forwarding specifically header-based routing did not seem to work i.e not being forwarded to correct backend service based on header match but also, I could not find any logs whatsoever where Linkerd (debugs enabled) would show something about processing those CRs.
Was wrecking my brain with my own application first but then tried this exact example:
with same behaviour. From all the checks, everything seemed to be in order but I just could not get B backend from the above example. Even the linkerd diagnostic policy showed correct parameters and upon header value match, it should have forwarded traffic to configured backend service.
Basically it almost seemed like the whole HTTPRoute is just being ignored even though it did create that policy for the resource or Linkerd proxy was just not intercepting traffic at all.
My first suspicion was that it has to do something with iptables NAT rules and the fact that I am using CIlium with kube-proxy-replacement=strict so required backend rules are not being created or not translating into what Cilium would understand OR its just not letting Linkerd proxy to intercept traffic.
And indeed, when I changed my environment to run with kube-proxy, I got the B backend (from the example) as expected.
Could not find any specific documentation on either Cilium or Linkerd side but did find a small hint from one of the github issues (plus few comments hinting towards that kube-proxy free shouldnt be an issue) which then lead me to Cilium’s documentation regarding Istio integration:
and there I found the following little note:
If Cilium is deployed with the kube-proxy replacement (this is not the case by default) you need to pass
--config bpf-lb-sock-hostns-only=trueto cilium CLI orsocketLB.hostNamespaceOnlyoption with Helm. Without this option, when Cilium does service resolution via socket load balancing, Istio sidecar will be bypassed, resulting in loss of Istio features including encryption and telemetry.
with that config parameter passed to Cilium…voila! I got the B backend…