Rate Limiting pattern

[iluwatar]

Description:
The Rate Limiting design pattern is crucial in a microservices architecture to prevent abuse and ensure fair usage of resources. This pattern restricts the number of requests a service can handle within a specified timeframe, thereby enhancing reliability and performance by preventing overloading. This implementation will involve creating a mechanism to count and limit the number of incoming requests to a service, and take appropriate actions when the limit is exceeded.

Main Elements of the Pattern:

1. Rate Limiter: A component responsible for tracking the number of requests and enforcing the rate limit.
2. Policy Configuration: Defines the rate limit rules, such as the number of requests allowed per time unit and what actions to take when the limit is exceeded.
3. Client Identification: Mechanism to identify clients (e.g., IP address, API key) to apply rate limits individually.
4. Throttling: Delaying or rejecting requests when the rate limit is exceeded.
5. Monitoring and Logging: Tools to monitor and log rate-limiting activities for analysis and tuning.

References:

1. Rate Limiting Pattern - Microservices.io
2. API Rate Limiting - Amazon API Gateway
3. Rate Limiting Strategies and Techniques
4. Project Contribution Guidelines

Acceptance Criteria:

1. Implement a rate limiter that can be configured with different policies.
2. Provide configuration options to define rate limit rules (e.g., requests per second).
3. Ensure the rate limiter can identify different clients and apply rate limits accordingly.
4. Implement mechanisms to throttle or reject requests exceeding the limit.
5. Include logging to capture rate-limiting events for monitoring purposes.
6. Provide comprehensive documentation and examples demonstrating how to use the rate limiter.
7. Write unit and integration tests to ensure the rate limiter works as expected under various scenarios.

[flow6979]

Hey @iluwatar, Thank you for detail explanation and i really like to share my knowledge (gain some more) regarding rate limiting pattern.

I want to give my time to this, can you please tell me how can i proceed further in right/good direction.
Thank you!!

[iluwatar]

Hey @flow6979, maybe a good starting point would be to research how rate limiting has been implemented in popular open source API management solutions like these:

1. Kong - A scalable, open-source API gateway with a rich plugin ecosystem. Source Code

2. Tyk - An open-source API gateway and management platform with extensive features. Source Code

3. WSO2 API Manager - A comprehensive open-source API management tool for full lifecycle API management. Source Code

4. Gravitee.io - A flexible and modular API management platform with powerful access control and analytics. Source Code

5. API Umbrella - An open-source API management tool providing gateway functionalities and analytics. Source Code

6. KrakenD - A high-performance, open-source API gateway designed for scalability and flexibility. Source Code

7. Express Gateway - An API gateway built entirely on Express.js, focused on extensibility and ease of use. Source Code

8. DreamFactory - An open-source platform for generating APIs from various data sources with powerful management features. Source Code

9. Apache APISIX - A dynamic, high-performance open-source API gateway with a rich plugin ecosystem. Source Code

[ylcn91]

@iluwatar

We can try to implement different algorithms, which user can select on different choices?

public enum AlgorithmType {
FIXED_WINDOW,
SLIDING_WINDOW,
TOKEN_BUCKET,
LEAKY_BUCKET
}

and also different Throttling Strategies:

public enum ThrottlingStrategyType {
DELAY,
REJECT,
EXPONENTIAL_BACKOFF
}

that would be beneficial?
Maybe we can also add a simple Circuit Breaker for failure with state of open, half open, closed etc?

which user can select from configuration:

--Algorithm Types by Endpoint
--Format: endpoint=ALGORITHM_TYPE
ratelimiter.algorithmByEndpoint.endpoint1=FIXED_WINDOW
ratelimiter.algorithmByEndpoint.endpoint2=SLIDING_WINDOW
ratelimiter.algorithmByEndpoint.endpoint3=TOKEN_BUCKET
ratelimiter.algorithmByEndpoint.endpoint4=LEAKY_BUCKET

and for Throttling

--Throttling Strategies by Endpoint and Client Type
--Format: endpoint.CLIENT_TYPE=THROTTLING_STRATEGY_TYPE
ratelimiter.throttlingStrategiesByEndpointAndClientType.endpoint1.STANDARD=DELAY
ratelimiter.throttlingStrategiesByEndpointAndClientType.endpoint1.PREMIUM=REJECT
ratelimiter.throttlingStrategiesByEndpointAndClientType.endpoint2.STANDARD=EXPONENTIAL_BACKOFF
ratelimiter.throttlingStrategiesByEndpointAndClientType.endpoint2.PREMIUM=DELAY

I can work on that module.

[iluwatar]

Sounds good @ylcn91, but according to my books @flow6979 is already working on this

[ylcn91]

@flow6979 let me know if any help needed, we can work together if you wish.

[flow6979]

Hello @ylcn91 yes sure, lets work together. Can we have a google meet to discuss our findings/knowledge this sunday (or any other day you are comfortable with).

[Edamijueda]

@ylcn91 @flow6979 I will like to train pls. I am interested. I have began reading up on the rate limiting pattern.

[kavitha186]

@iluwatar should we also include client Rate Limiting? I can build a Distributed Rate limiting for the api with Lua scripting

[ylcn91]

I have added some code to my fork. you can check it @flow6979 you can understand what I try to do, if not we can have some meeting.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. The issue will be unassigned if no further activity occurs. Thank you for your contributions.

[manishrw]

@iluwatar I can take it up.