I’d like to submit a feature request (let me know if there is a better place to submit this!) for best-effort request affinity. Please let me know if this is already a way to do this, and I have just overlooked it.
Our use case is that several of our gRPC services keep a per-user in-memory cache. We’d like to be able to create a load balancing rule, so that all requests with the same UserId are sent to the same pod, in order to maximize cache hit rate (on a best effort basis). I did some research and it appears that Istio supports this via a loadBalancer.consistentHash policy, which provides soft affinity based on configured headers.
Please let me know if I can provide any more detail about our use case for this request.
Hey, thanks for speaking up. We’ve definitely gotten this a few times, and are trying to figure out how and when we might be able to tackle it. At the moment we’re looking to GEP-1619: Session Persistence | Gateway API as the path we’d likely follow, so I’m curious what you think of that…
Hi @Flynn ! Thank you for the reply, I wasn’t familiar with GEP-1619.
I read through GEP-1619 and I think that it is slightly different from what we are looking for. If I’m understanding it correctly, it looks like it distinguishes between the concepts of “session persistence” and “session affinity”:
It sounds like it proposes to achieve “session persistence” by having the backend or load balancer return a header value, which clients can set in subsequent requests to ensure that the same backend fulfills subsequent requests from that specific client.
What we want is what it refers to as “session affinity”, which GEP-1619 notes here that it intends to address this in a separate GEP. We have many clients and we want to allow clients to set a header value so that all requests with that same header value hit the same pod (on a best effort basis), to maximize cache hit rate. I’m not sure if I understand 100% of GEP-1619 but it’s not clear to me that it’ll allow us to achieve that.