You've just spent hours optimizing a database query, only to realize your function calls the same expensive operation repeatedly with identical inputs. Your code works, but it's leaving performance on the table. Caching isn't just for distributed systems — it's a tool every Python developer should have in their toolkit, and using it wrong can cause more problems than it solves.
What you'll learn
- When caching actually helps (and when it makes things worse)
- How to use
functools.lru_cacheeffectively in your codebase - Building custom cache decorators for specific use cases
- Common caching pitfalls that cause bugs in production
Why caching matters now
Modern applications handle more data than ever, and users expect sub-second responses. Database queries, API calls, and complex computations all consume resources that scale poorly with traffic. Caching lets you reuse expensive results across requests, reducing load on downstream systems and improving response times. The Python standard library includes powerful caching tools that require zero dependencies, yet many developers reach for external packages before understanding what's built-in.
Understanding the caching fundamentals
Caching stores the result of an expensive operation so subsequent calls with the same inputs return instantly. The trade-off is memory — every cached result occupies space that could be used elsewhere. Good caching strategies balance speed against memory usage, and they handle cache invalidation gracefully. In Python, the most common approach is memoization: caching function return values based on their arguments.
The simplest caching is unlimited — store everything forever. This works for small datasets but causes memory leaks with large ones or unbounded input spaces. Better strategies evict old entries when the cache reaches a size limit, using policies like Least Recently Used (LRU) or First-In-First-Out (FIFO).
Using functools.lru_cache
Python's functools.lru_cache decorator implements memoization with an LRU eviction policy. It's thread-safe, works with hashable arguments, and requires minimal setup. This makes it perfect for pure functions — those that always return the same output for the same input and have no side effects.
from functools import lru_cache
import time
@lru_cache(maxsize=128) # Cache up to 128 most recent calls
def expensive_computation(n: int) -> int:
"""Simulate an expensive calculation."""
time.sleep(0.1) # Pretend this takes 100ms
return n * n
# First call: takes 100ms, computes and caches result
result1 = expensive_computation(5)
# Subsequent calls with same argument: ~0ms, returns cached result
result2 = expensive_computation(5)
print(f"Cache info: {expensive_computation.cache_info()}")
# Output: CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)
The maxsize parameter controls how many distinct argument combinations are stored. Setting it to None creates an unbounded cache — use with caution. The cache_info() method reveals hit/miss statistics, helping you tune the size. When you need to clear the cache manually, call expensive_computation.cache_clear().
Real-world tip: Use lru_cache for recursive algorithms like Fibonacci calculations. Without caching, the naive recursive implementation has exponential time complexity. With caching, it becomes linear.
@lru_cache(maxsize=None)
def fibonacci(n: int) -> int:
if n < 2:
return n
return fibonacci(n - 1) + fibonacci(n - 2)
# fibonacci(100) completes instantly; without cache, it would hang
Building custom cache decorators
Sometimes lru_cache doesn't fit your needs. You might want time-based expiration, size limits based on memory usage, or custom eviction logic. Building your own decorator gives you full control.
from functools import wraps
import time
from typing import Callable, Any, Dict
def timed_cache(seconds: int = 60):
"""Cache results for a specified time period."""
def decorator(func: Callable) -> Callable:
cache: Dict[Any, tuple] = {} # Maps args to (result, timestamp)
@wraps(func)
def wrapper(*args, **kwargs) -> Any:
# Create a cache key from function arguments
key = (args, frozenset(kwargs.items()))
now = time.time()
if key in cache:
result, timestamp = cache[key]
if now - timestamp < seconds:
return result # Return cached result if still valid
# Compute and store new result
result = func(*args, **kwargs)
cache[key] = (result, now)
return result
wrapper.cache_clear = lambda: cache.clear() # Manual clear support
return wrapper
return decorator
@timed_cache(seconds=30)
def fetch_user_data(user_id: int) -> dict:
"""Simulate an API call that should be cached for 30 seconds."""
# In real code, this would be an actual API request
return {"id": user_id, "name": f"User {user_id}", "timestamp": time.time()}
This decorator expires cached entries after a fixed duration, useful for data that changes periodically but not on every request. The key generation handles both positional and keyword arguments using frozenset for deterministic hashing.
Trade-off: Custom decorators give you flexibility but require careful implementation. Thread safety, memory management, and key collision handling are all on you. Start with lru_cache and only build custom when you've proven it's necessary.
Caching with external storage
In-memory caching works for single-process applications, but distributed systems need shared storage. Redis is the go-to solution here — it's fast, supports expiration natively, and works across multiple servers.
import json
import redis
from functools import wraps
# Connect to Redis (adjust host/port for your setup)
redis_client = redis.Redis(host='localhost', port=6379, db=0, decode_responses=True)
def redis_cache(ttl: int = 300):
"""Cache function results in Redis with TTL in seconds."""
def decorator(func: Callable) -> Callable:
@wraps(func)
def wrapper(*args, **kwargs) -> Any:
# Generate unique cache key
key = f"{func.__name__}:{args}:{frozenset(kwargs.items())}"
# Try to get cached result
cached = redis_client.get(key)
if cached:
return json.loads(cached)
# Compute, cache, and return result
result = func(*args, **kwargs)
redis_client.setex(key, ttl, json.dumps(result))
return result
return wrapper
return decorator
@redis_cache(ttl=60)
def external_api_call(endpoint: str) -> dict:
"""Simulate an expensive external API call."""
# Real implementation would use requests or httpx
return {"endpoint": endpoint, "data": "expensive result"}
The setex command sets the value with an expiration time, so Redis automatically evicts stale entries. This is safer than manual expiration handling and prevents memory bloat.
Common pitfalls
Caching mutable objects
Storing lists, dictionaries, or other mutable objects in a cache can lead to subtle bugs. If the caller modifies the returned object, the cached value changes too, affecting future calls. Always return copies or use immutable data structures.
# DON'T: Returning mutable cached objects
@lru_cache()
def get_config() -> dict:
return {"setting": "value"}
config = get_config()
config["setting"] = "modified" # Corrupts the cache!
# DO: Return copies or use immutable types
from copy import deepcopy
@lru_cache()
def get_config_safe() -> dict:
return deepcopy({"setting": "value"})
Forgetting cache invalidation
Cached data eventually becomes stale. If you cache user permissions but don't invalidate them when roles change, users might retain access they shouldn't have. Design your cache keys with invalidation in mind — include version numbers or timestamps, and provide clear methods to clear related entries.
Over-caching
Not everything needs caching. Simple operations might take longer to retrieve from cache than to compute. Profile your code before adding caching, and measure the impact. A cache miss that triggers the original computation plus cache overhead is slower than no cache at all.
Wrap-up
Effective caching starts with understanding what's expensive in your code and whether reusing results is safe. Start with functools.lru_cache for pure functions, move to time-based expiration for data that changes periodically, and use Redis for distributed systems. Profile before and after — caching should improve performance measurably, not just theoretically.
Next steps:
- Run
python -m cProfileon your slow functions to identify caching candidates - Check the
functoolsdocumentation for@cache(Python 3.9+) and@lru_cacheparameters - Explore Redis patterns for your specific use case if you're building distributed systems
