Istio Rate Limiting Secrets: How Sidecar Outshine DestinationRule

The problem

In my previous blogs, I discussed the Istio destination rule and its application of rate limiting on both the client side (with the UO flag) and the server side (with the URX flag). This dual implementation can lead to confusion regarding where the rate limit is enforced, making it more challenging to troubleshoot issues or effectively plan our rate-limiting strategy.

Optimizing Traffic Management in Kubernetes with Istio’s Destination Rules

The first solution that comes to mind is to use a workload selector applied exclusively to the server service, effectively implementing rate limiting only on the server side.

Istio Rate Limiting: Using Workload Selectors

Unfortunately, when attempting this approach, you may find that the configuration is not applied to the server side as expected. This issue has been documented and discussed in the Istio community (https://github.com/istio/istio/issues/41235).

When I check inbound cluster configuration I see connection pool settings are not applied

When I remove workloadSelector from destination rule everything works as expected

The solution

In Istio 1.21, the team added similar functionality from the destination rule to the sidecar resource, allowing for better control over traffic management and rate limiting.

Added connection pool settings to the Sidecar API to enable configuring the inbound connection pool for sidecars in the mesh. Previously, the DestinationRule’s connection pool settings applied to both client and server sidecars. Using the updated Sidecar API, it’s now possible to configure the server’s connection pool separately from the clients’ in the mesh. (reference) (Issue #32130),(Issue #41235)

Let’s create a sidecar resource and compare it to a DestinationRule to understand the differences.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: app-b-destination-rule
spec:
  host: app-b
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 10
      http:
        http1MaxPendingRequests: 5

Will become this:

apiVersion: networking.istio.io/v1beta1
kind: Sidecar
metadata:
  name: app-b-sidecar
spec:
  workloadSelector:
    labels:
      app: app-b
  inboundConnectionPool:
    tcp:
      maxConnections: 10
    http:
      http1MaxPendingRequests: 5

NOTE 1: Each namespace can have only one Sidecar configuration without any workloadSelector that specifies the default for all pods in that namespace. It is recommended to use the name default for the namespace-wide sidecar. The behavior of the system is undefined if more than one selector-less Sidecar configurations exist in a given namespace. The behavior of the system is undefined if two or more Sidecar configurations with a workloadSelector select the same workload instance.

NOTE 2: A Sidecar configuration in the MeshConfig root namespace will be applied by default to all namespaces without a Sidecar configuration. This global default Sidecar configuration should not have any workloadSelector.

NOTE 3: A Sidecar is not applicable to gateways, even though gateways are istio-proxies.

As you can see, the trafficPolicy.connectionPool in the DestinationRule has been mirrored as inboundConnectionPool in the sidecar resource, while the other settings remain the same as in connectionPool.

To apply the sidecar configuration to a specific service, we need to use a workloadSelector instead of host.

In Grafana we can see that app-a is rate limited to app-b only with URX flag.

Conclusion

In conclusion, understanding the nuances between configuring rate limiting in Istio DestinationRules and sidecar resources are crucial for effective traffic management. By utilizing workloadSelector the sidecar resource, we gain more precise control over where and how these policies are applied, ensuring that our configurations align with the desired service behavior. This approach not only simplifies troubleshooting but also enhances the flexibility and reliability of our service mesh.

ref

https://istio.io/latest/news/releases/1.21.x/announcing-1.21/change-notes/
https://istio.io/latest/docs/reference/config/networking/sidecar/