Benchmarked flagsmith 5.0.1 vs 3.8.0 against the same environment (edge API, enable_local_evaluation=True, 262 features, single-threaded, after 200-iter warmup).
p50 per call
| Operation | v3.8.0 | v5.0.1 | Regression |
|---|
get_identity_flags | 0.39 ms | 0.82 ms | 2.1× |
get_environment_flags | 0.11 ms | 0.79 ms | 7.4× |
is_feature_enabled (cached Flags) | 0.08 µs | 0.08 µs | — |
Throughput drops from ~2,570 → ~1,215 identity evals/sec per core. Latency variance also grew (stdev ~10 µs → ~220 µs).
Repro
fromtimeimportperf_counterfromflagsmithimportFlagsmithclient=Flagsmith(
environment_key="<server-side-key>",
api_url="https://edge.api.flagsmith.com/api/v1/",
enable_local_evaluation=True,
environment_refresh_interval_seconds=3600,
)
client.update_environment()
# warmupfor_inrange(200):
client.get_identity_flags(identifier="anonymous", traits={"venue_id": "12345"})
fornin (100, 1_000, 10_000):
t0=perf_counter()
for_inrange(n):
client.get_identity_flags(identifier="anonymous", traits={"venue_id": "12345"})
elapsed=perf_counter() -t0print(f"{n:>6} iters: {elapsed*1000:8.1f} ms ({elapsed/n*1e6:7.2f} µs/call)")Environment: flag-engine 10.0.1, Python 3.12, macOS.
Benchmarked
flagsmith5.0.1 vs 3.8.0 against the same environment (edge API,enable_local_evaluation=True, 262 features, single-threaded, after 200-iter warmup).p50 per call
get_identity_flagsget_environment_flagsis_feature_enabled(cachedFlags)Throughput drops from ~2,570 → ~1,215 identity evals/sec per core. Latency variance also grew (stdev ~10 µs → ~220 µs).
Repro
Environment: flag-engine 10.0.1, Python 3.12, macOS.