# Memory leak issue on AWS EKS Node with aarch64 architecture after Edge-23.2.3

**URL:** <https://linkerd.buoyant.io/t/memory-leak-issue-on-aws-eks-node-with-aarch64-architecture-after-edge-23-2-3/142>\
**Category:** Linkerd General Discussion\
**Tags:** metrics\
**Created:** [May 29, 2023, 10:45am UTC](https://linkerd.buoyant.io/t/memory-leak-issue-on-aws-eks-node-with-aarch64-architecture-after-edge-23-2-3/142 "2023-05-29T10:45:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ericlin](https://avatars.discourse-cdn.com/v4/letter/e/c37758/32.png) [@ericlin](https://linkerd.buoyant.io/u/ericlin)\
**Post date:** [May 29, 2023, 10:45am UTC](https://linkerd.buoyant.io/t/memory-leak-issue-on-aws-eks-node-with-aarch64-architecture-after-edge-23-2-3/142/1 "2023-05-29T10:45:50Z")

</div>

Hi all,

We use Linkerd for gRPC load balancing on AWS EKS. After upgrading to Linkerd stable-2.13.0, our k8s services sometimes encountered gRPC `14 UNAVAILABLE: connections to all backends failing` errors.

We discovered it was because of the memory leak issue on the `system.mem.slab` memory of AWS EKS nodes, and seems only happened on AWS nodes with arrch64 (arm64) CPU.

 ![Screenshot 2023-05-29 at 6.27.24 PM](https://canada1.discourse-cdn.com/flex029/uploads/buoyant1/original/1X/ad8e8c06f8fbbf7dbe1510f29b68cc8617d506d4.png)

And we confirmed this issue started happening since `Edge-23.2.3`, we suspect this issue was caused by bumping the version of [slab](https://github.com/tokio-rs/slab) dependency to `0.4.8` ([PR](https://github.com/linkerd/linkerd2/pull/10362))

- Linkerd version: Edge-23.2.3
- Platform
  - Kernel Name: Linux
  - Kernel Release: 5.10.167-147.601.amzn2.aarch64
  - Kernel Version: #1 SMP Tue Feb 14 21:50:23 UTC 2023
  - Processor: aarch64

Would it be possible to request your assistance in checking on this matter? I sincerely appreciate your help.

---

<div class="post-metadata">

**Author:** ![william](https://yyz1.discourse-cdn.com/flex029/user_avatar/linkerd.buoyant.io/william/32/5_2.png) [@william](https://linkerd.buoyant.io/u/william)\
**Post date:** [May 31, 2023, 10:13pm UTC](https://linkerd.buoyant.io/t/memory-leak-issue-on-aws-eks-node-with-aarch64-architecture-after-edge-23-2-3/142/2 "2023-05-31T22:13:55Z")

</div>

This kind of thing is hard to investigate from the outside, but as a starting point could you try a more recent edge release, or 2.13.3?

---

<div class="post-metadata">

**Author:** ![ericlin](https://avatars.discourse-cdn.com/v4/letter/e/c37758/32.png) [@ericlin](https://linkerd.buoyant.io/u/ericlin)\
**Post date:** [June 1, 2023, 2:37am UTC](https://linkerd.buoyant.io/t/memory-leak-issue-on-aws-eks-node-with-aarch64-architecture-after-edge-23-2-3/142/3 "2023-06-01T02:37:52Z")

</div>

Thanks for the suggestion, I will try the latest edge release [Edge-23.5.3](https://github.com/linkerd/linkerd2/releases/tag/edge-23.5.3) and update the result here.

---

<div class="post-metadata">

**Author:** ![ericlin](https://avatars.discourse-cdn.com/v4/letter/e/c37758/32.png) [@ericlin](https://linkerd.buoyant.io/u/ericlin)\
**Post date:** [June 8, 2023, 9:25am UTC](https://linkerd.buoyant.io/t/memory-leak-issue-on-aws-eks-node-with-aarch64-architecture-after-edge-23-2-3/142/4 "2023-06-08T09:25:18Z")

</div>

Hi @william we tried the latest stable release `2.13.4` and seems the memory leak issue for AWS nodes with `aarch64 (ARM64)` CPU still exists.

 ![Screenshot 2023-06-08 at 5.21.34 PM](https://canada1.discourse-cdn.com/flex029/uploads/buoyant1/original/1X/7bb02c90144a085aaf86f5d8f2e21f15930ad57c.png)

---

<div class="post-metadata">

**Author:** ![william](https://yyz1.discourse-cdn.com/flex029/user_avatar/linkerd.buoyant.io/william/32/5_2.png) [@william](https://linkerd.buoyant.io/u/william)\
**Post date:** [June 13, 2023, 10:41pm UTC](https://linkerd.buoyant.io/t/memory-leak-issue-on-aws-eks-node-with-aarch64-architecture-after-edge-23-2-3/142/5 "2023-06-13T22:41:06Z")

</div>

Sorry for the delay. I think this warrants filing a [GitHub issue](https://github.com/linkerd/linkerd2/issues). Please provide as much information as you can about how to reproduce this… I am not sure if we have access to an aarch64 AWS cluster but if you have a minimalist repro case that will be helpful.
