ThrottlingTroll 6.1.0-beta1

ThrottlingTroll

Rate limiting/throttling middleware for ASP.NET and Azure Functions.

<img alt="Nuget" src="https://img.shields.io/nuget/v/ThrottlingTroll?label=current%20version">

<img alt="Nuget" src="https://img.shields.io/nuget/dt/ThrottlingTroll?label=ThrottlingTroll%20downloads"> <img alt="Nuget" src="https://img.shields.io/nuget/dt/ThrottlingTroll.AzureFunctions?label=ThrottlingTroll.AzureFunctions%20downloads"> <img alt="Nuget" src="https://img.shields.io/nuget/dt/ThrottlingTroll.AzureFunctionsAspNet?label=ThrottlingTroll.AzureFunctionsAspNet%20downloads">

Install from Nuget:

Features

• Supports ASP.NET, Azure Functions (.NET Isolated) and Azure Functions with ASP.NET Core Integration.

• Ingress throttling, aka let your service automatically respond with 429 TooManyRequests to some obtrusive clients.

      sequenceDiagram
          Client->>+YourService: #127760;HTTP
          alt limit exceeded?
              YourService-->>Client:❌ 429 TooManyRequests
          else
              YourService-->>-Client:✅ 200 OK
          end
  

  Implemented as an ASP.NET Core Middleware (for ASP.NET) and as an Azure Functions Middleware (for Azure Functions).

• Egress throttling, aka limit the number of calls your code is making against some external endpoint.

      sequenceDiagram
          YourService->>+HttpClient: SendAsync()
          alt limit exceeded?
              HttpClient-->>YourService:❌ 429 TooManyRequests
          else
              HttpClient->>+TheirService: #127760;HTTP
              TheirService-->>-HttpClient:✅ 200 OK
              HttpClient-->>-YourService:✅ 200 OK
          end
  

  Implemented as an HttpClient DelegatingHandler, which produces 429 TooManyRequests response (without making the actual call) when a limit is exceeded.

• Propagating 429 TooManyRequests from egress to ingress, aka when your service internally makes an HTTP request which results in 429 TooManyRequests, your service can automatically respond with same 429 TooManyRequests to its calling client.

      sequenceDiagram
          Client->>+YourService: #127760;HTTP
          YourService->>+TheirService: #127760;HTTP
          TheirService-->>-YourService:❌ 429 TooManyRequests
          YourService-->>-Client:❌ 429 TooManyRequests
  

• Custom response fabrics. For ingress it gives full control on what to return when a request is being throttled, and also allows to implement delayed responses (instead of just returning 429 TooManyRequests):

      sequenceDiagram
          Client->>+YourService: #127760;HTTP
          alt limit exceeded?
              YourService-->>YourService: await Task.Delay(RetryAfter)
              YourService-->>Client:✅ 200 OK
          else
              YourService-->>-Client:✅ 200 OK
          end
  

  For egress it also allows ThrottlingTroll to do automatic retries for you:

      sequenceDiagram
          YourService->>+HttpClient: SendAsync()
  
          loop while 429 TooManyRequests
              HttpClient->>+TheirService: #127760;HTTP
              TheirService-->>-HttpClient:❌ 429 TooManyRequests
              HttpClient-->>HttpClient: await Task.Delay(RetryAfter)
          end
  
          HttpClient->>+TheirService: #127760;HTTP
          TheirService-->>-HttpClient:✅ 200 OK
          HttpClient-->>-YourService:✅ 200 OK
  

• Storing rate counters in a distributed cache, making your rate limiting policy consistent across all your computing instances. Supported distributed counter stores are:

  • ThrottlingTroll.CounterStores.Redis
  • ThrottlingTroll.CounterStores.AzureTable
  • ThrottlingTroll.CounterStores.DistributedCache

  And you can implement your own.

• Dynamically configuring rate limits, so that those limits can be adjusted on-the-go, without restarting the service.

• IdentityIdExtractors, that allow you to limit clients individually, based on their IP-addresses, api-keys, tokens, headers, query strings, claims etc. etc.

• CostExtractors, that you can use to assign custom costs to different requests. Default cost is 1, but if some of your requests are heavier than the others, you can assign higher costs to them. Allows for even greater rate limiting flexibility.

Supported rate limiting algorithms

• FixedWindow. No more than PermitLimit requests are allowed in IntervalInSeconds. Here is an illustration for the case of no more than 2 requests per each 8 seconds:

  <img src="https://github.com/ThrottlingTroll/ThrottlingTroll/assets/5447190/ffb0bdc8-736b-4c6f-9eb4-db54ce72e034" height="300px"/>

  The typical drawback of FixedWindow algorithm is that you'd get request rate bursts at the end of each window. So specifically to cope that we have

• SlidingWindow. No more than PermitLimit requests are allowed in IntervalInSeconds, but that interval is split into NumOfBuckets. The main benefit of this algorithm over FixedWindow is that if a client constantly exceedes PermitLimit, it will never get any valid response and will always get 429 TooManyRequests. Here is an illustration for the case of no more than 2 requests per each 8 seconds with 2 buckets:

  <img src="https://github.com/ThrottlingTroll/ThrottlingTroll/assets/5447190/e18abb9c-d1dd-4b64-a007-220605ed03e9" height="300px"/>

  In other words, with SlidingWindow your service gets a smoother request rate.

• Semaphore aka Concurrency Limiter. No more than PermitLimit requests are allowed to be executed concurrently. Here is an illustration for the case of no more than 3 concurrent requests:

  <img src="https://github.com/ThrottlingTroll/ThrottlingTroll/assets/5447190/0beeac73-5d35-482a-a790-a3fe9ea6e38b" height="300px"/>

  If you set Semaphore's PermitLimit to 1 and use RedisCounterStore, then ThrottlingTroll will act as a distributed lock. If you add an IdentityIdExtractor (identifying requests by e.g. a query string parameter), then it will turn into named distributed locks.

How to configure and use

Configuration and usage with ASP.NET and Azure Functions is very similar yet slightly different:

Samples

Sample projects that demonstrate all the above concepts:

Contributing

Is very much welcomed.