> ## Documentation Index
> Fetch the complete documentation index at: https://choked.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Token bucket
# Token Bucket
The token bucket classes that power Choked's dual rate limiting functionality.
## Overview
Choked uses two different token bucket implementations depending on your backend configuration:
* **RedisTokenBucket**: For distributed rate limiting using Redis
* **ProxyTokenBucket**: For managed rate limiting via the Choked proxy service
The appropriate implementation is automatically selected based on your `Choked` class configuration. You typically don't need to use these classes directly.
## RedisTokenBucket
Used when a `Choked` instance is created with `redis_url`. Provides distributed coordination using Redis with atomic Lua scripts.
### Constructor
```python theme={null}
RedisTokenBucket(
key: str,
request_capacity: int,
request_refill_rate: float,
token_capacity: int,
token_refill_rate: float,
redis_url: Optional[str] = None
)
```
Unique identifier for the rate limit bucket
Maximum number of request tokens in the bucket (0 = no request limiting)
Rate at which request tokens are added to the bucket (tokens per second, 0.0 = no request limiting)
Maximum number of content tokens in the bucket (0 = no token limiting)
Rate at which content tokens are added to the bucket (tokens per second, 0.0 = no token limiting)
Redis connection URL. If not provided, uses the redis\_url from the parent Choked instance.
### Methods
#### acquire()
```python theme={null}
async def acquire(requests_needed: int = 1, tokens_needed: int = 0) -> bool
```
Attempts to acquire the specified number of request and content tokens from their respective buckets.
**Parameters:**
* `requests_needed` (int): Number of request tokens to acquire (typically 1)
* `tokens_needed` (int): Number of content tokens to acquire (estimated from function args)
**Returns:** `True` if both types of tokens were successfully acquired, `False` otherwise.
**Behavior:**
* Both request and token buckets are checked atomically
* If either bucket lacks sufficient tokens, no tokens are consumed
* Buckets are refilled based on elapsed time since last access
* Uses Redis Lua scripts for atomic operations
### Configuration
Requires Redis connection via the parent `Choked` instance:
```python theme={null}
choke = Choked(redis_url="redis://localhost:6379/0")
# RedisTokenBucket automatically uses this redis_url
```
### Redis Storage
The Redis implementation stores:
* Request bucket state: `{key}:requests` (current tokens, last refill time)
* Token bucket state: `{key}:tokens` (current tokens, last refill time)
* Uses Redis Hash data type for efficient storage
* Atomic operations via Lua scripts prevent race conditions
## ProxyTokenBucket
Used when a `Choked` instance is created with `api_token`. Provides managed rate limiting through the Choked proxy service.
### Constructor
```python theme={null}
ProxyTokenBucket(
api_token: str,
key: str,
request_capacity: int,
request_refill_rate: float,
token_capacity: int,
token_refill_rate: float
)
```
API token for the Choked proxy service
Unique identifier for the rate limit bucket
Maximum number of request tokens in the bucket (0 = no request limiting)
Rate at which request tokens are added to the bucket (tokens per second, 0.0 = no request limiting)
Maximum number of content tokens in the bucket (0 = no token limiting)
Rate at which content tokens are added to the bucket (tokens per second, 0.0 = no token limiting)
### Methods
#### acquire()
```python theme={null}
async def acquire(requests_needed: int = 1, tokens_needed: int = 0) -> bool
```
Attempts to acquire tokens via the proxy service.
**Parameters:**
* `requests_needed` (int): Number of request tokens to acquire
* `tokens_needed` (int): Number of content tokens to acquire
**Returns:** `True` if tokens were successfully acquired, `False` otherwise.
**Behavior:**
* Makes HTTP request to proxy service
* Service handles bucket logic server-side
* Automatic retry on network failures
* No local state storage required
### Configuration
Requires API token for the managed service. Contact us for access.
### Network Behavior
* HTTP-based communication with proxy service
* Automatic retries on network failures
* Respects standard HTTP timeouts
* Falls back gracefully on service unavailability
## Usage Notes
### Automatic Instantiation
Token buckets are automatically created by the `Choked` decorator:
```python theme={null}
from choked import Choked
choke = Choked(redis_url="redis://localhost:6379/0")
@choke(key="my_api", request_limit="10/s", token_limit="1000/m", token_estimator="openai")
def my_function():
# RedisTokenBucket automatically created with:
# - key="my_api"
# - request_capacity=10, request_refill_rate=10.0
# - token_capacity=1000, token_refill_rate=16.67
pass
```
### Rate Limit Conversion
The `Choked` class automatically converts rate limit strings to bucket parameters:
```python theme={null}
# "10/s" becomes: capacity=10, refill_rate=10.0 tokens/second
# "100/m" becomes: capacity=100, refill_rate=1.67 tokens/second (100/60)
# None becomes: capacity=0, refill_rate=0.0 (no limiting)
```
### Dual Bucket Coordination
Both implementations coordinate request and token buckets:
```python theme={null}
# Request-only limiting
await bucket.acquire(requests_needed=1, tokens_needed=0)
# Token-only limiting
await bucket.acquire(requests_needed=0, tokens_needed=estimated_tokens)
# Dual limiting
await bucket.acquire(requests_needed=1, tokens_needed=estimated_tokens)
```
### Error Handling
Both implementations handle errors gracefully:
* **Network failures**: Automatic retry with exponential backoff
* **Redis connection issues**: Connection pooling and retry logic
* **Service unavailability**: Graceful degradation (may allow requests through)
* **Invalid parameters**: Immediate validation errors
## Performance Characteristics
### RedisTokenBucket Performance
* **Latency**: \~1-2ms per acquire() call (local Redis)
* **Throughput**: 10,000+ operations/second per bucket
* **Memory**: \~100 bytes per bucket in Redis
* **Network**: 1 round-trip per acquire() call
* **Scalability**: Horizontal scaling via Redis clustering
### ProxyTokenBucket Performance
* **Latency**: \~10-50ms per acquire() call (network dependent)
* **Throughput**: 1,000+ operations/second per bucket
* **Memory**: Zero local memory usage
* **Network**: 1 HTTP request per acquire() call
* **Scalability**: Managed by proxy service infrastructure
### Performance Optimization
* Both implementations cache connection objects
* Redis implementation uses connection pooling
* Proxy implementation reuses HTTP connections
* No performance penalty for unused buckets (request\_capacity=0 or token\_capacity=0)
## Advanced Usage
### Direct Instantiation (Not Recommended)
While possible, direct instantiation bypasses the `Choked` class conveniences:
```python theme={null}
from choked.token_bucket import RedisTokenBucket
# Manual instantiation (not recommended)
bucket = RedisTokenBucket(
key="manual_bucket",
request_capacity=10,
request_refill_rate=10.0,
token_capacity=1000,
token_refill_rate=16.67,
redis_url="redis://localhost:6379/0"
)
# Manual usage
success = await bucket.acquire(requests_needed=1, tokens_needed=50)
if success:
# Proceed with rate-limited operation
pass
```
### Custom Integration
For custom rate limiting logic:
```python theme={null}
import asyncio
from choked.token_bucket import RedisTokenBucket
async def custom_rate_limiter():
bucket = RedisTokenBucket("custom", 5, 1.0, 0, 0.0) # 5 requests/5 seconds
while True:
if await bucket.acquire(1, 0):
print("Request allowed")
else:
print("Rate limited, waiting...")
await asyncio.sleep(1)
```
### Monitoring and Observability
Both implementations support monitoring:
```python theme={null}
# Redis backend: Monitor via Redis commands
# HGETALL bucket_key:requests # Current request bucket state
# HGETALL bucket_key:tokens # Current token bucket state
# Proxy backend: Monitor via service dashboard
# (Contact us for monitoring access)
```
## Thread Safety
* Both implementations are fully thread-safe and async-safe
* Redis implementation uses atomic Lua scripts
* Proxy implementation uses thread-safe HTTP clients
* Safe for concurrent use across multiple threads/processes
* No race conditions in bucket state management
## Migration Between Backends
Switching between Redis and proxy backends:
```python theme={null}
# From Redis to Proxy
# OLD: choke = Choked(redis_url="redis://localhost:6379/0")
# NEW: choke = Choked(api_token="your-token")
# From Proxy to Redis
# OLD: choke = Choked(api_token="your-token")
# NEW: choke = Choked(redis_url="redis://localhost:6379/0")
# Bucket state is not preserved across backend changes
# Rate limiting behavior remains identical
```
The token bucket implementations provide the foundation for Choked's powerful dual rate limiting system, handling the complex coordination between request and token limits while maintaining high performance and reliability.