Service per-route Route Metrics numbers do not show up

Linkerd General Discussion
1

Hi all,

I am learning how to configure to see Per-Route Metrics by following the documentations.

I am using the given “booksapp” example.

After following all the steps and applied “Service Profiles” for all books, authors and webapp services, I got “Live Call” views with running requests and numbers in the Viz.

As for “Route Metrics” I can see the list of routes as configured in “Service Profiles” yml files.

But there is no “Success Rate” and “RPS” nor other Latency numbers showed up.

Results as follows:

$ linkerd viz routes deploy/books
ROUTE                     SERVICE   SUCCESS   RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99
DELETE /books/{id}.json     books         -     -             -             -             -
GET /books.json             books         -     -             -             -             -
GET /books/{id}.json        books         -     -             -             -             -
POST /books.json            books         -     -             -             -             -
PUT /books/{id}.json        books         -     -             -             -             -
[DEFAULT]                   books         -     -             -             -             -

After checking with Tap, I found that the request logged does not contain
rt_route, thus :path is used for Regex matching in Service Profile.

Example Tap res as follows.

req id=1:22 proxy=in  src=10.1.0.103:56104 dst=10.1.0.104:7002 tls=true :method=GET :authority=books:7002 :path=/books.json src_client_id=webapp.default.serviceaccount.identity.linkerd.cluster.local src_control_plane_ns=linkerd src_deployment=webapp src_namespace=default src_pod=webapp-77bb8d7ff5-zt47r src_pod_template_hash=77bb8d7ff5 src_serviceaccount=webapp src_tls=true dst_authz_group= dst_authz_kind=default dst_authz_name=all-unauthenticated dst_control_plane_ns=linkerd dst_deployment=books dst_namespace=default dst_pod=books-6f595989ff-s24vf dst_pod_template_hash=6f595989ff dst_route_group= dst_route_kind=default dst_route_name=default dst_serviceaccount=books dst_srv_group= dst_srv_kind=default dst_srv_name=all-unauthenticated dst_tls=loopback

req id=1:23 proxy=in  src=10.1.0.103:56104 dst=10.1.0.104:7002 tls=true :method=GET :authority=books:7002 :path=/books.json src_client_id=webapp.default.serviceaccount.identity.linkerd.cluster.local src_control_plane_ns=linkerd src_deployment=webapp src_namespace=default src_pod=webapp-77bb8d7ff5-zt47r src_pod_template_hash=77bb8d7ff5 src_serviceaccount=webapp src_tls=true dst_authz_group= dst_authz_kind=default dst_authz_name=all-unauthenticated dst_control_plane_ns=linkerd dst_deployment=books dst_namespace=default dst_pod=books-6f595989ff-s24vf dst_pod_template_hash=6f595989ff dst_route_group= dst_route_kind=default dst_route_name=default dst_serviceaccount=books dst_srv_group= dst_srv_kind=default dst_srv_name=all-unauthenticated dst_tls=loopback

Did I miss anything or does it requires special specific configuration to make the details collected?

Thanks,
Thar

2

Hey @tharhtet, can you share your Linkerd version?

3

This appears to be a known issue:

Please try bumping the traffic pod, and see if that helps.

4

Hello Jason,

I am using stable-2.13.3

$ linkerd version
Client version: stable-2.13.3
Server version: stable-2.13.3

Thank you!

5

Thanks @alpeb I will try that.

6

Looks like, I am confirming the the following bug again.

I setup the Service Profile before injecting linkerd to existing pods.

Then the per routes metric numbers appear. All OK.

$ linkerd viz -n booksapp routes svc/books
ROUTE                     SERVICE   SUCCESS      RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99
DELETE /books/{id}.json     books   100.00%   1.1rps           8ms          17ms          19ms
GET /books.json             books   100.00%   1.7rps           9ms          19ms          20ms
GET /books/{id}.json        books   100.00%   2.1rps           8ms          10ms          10ms
POST /books.json            books    50.00%   2.3rps          20ms          29ms          30ms
PUT /books/{id}.json        books    56.67%   1.0rps          27ms          93ms          99ms
[DEFAULT]                   books         -        -             -             -             -

So it is due to known issue.

Thank you!