Skip to content

test: Isolate local feature flag polling assertions - #868

Merged
marandaneto merged 1 commit into
mainfrom
fix/feature-flag-cohort-test-race
Aug 13, 2026
Merged

test: Isolate local feature flag polling assertions#868
marandaneto merged 1 commit into
mainfrom
fix/feature-flag-cohort-test-race

Conversation

@marandaneto

Copy link
Copy Markdown
Member

💡 Motivation and Context

Several local feature flag evaluation tests patched the module-level posthog.client.get function and asserted that it was never called. A background poller from another client could call the same mock while the test was running. This caused an intermittent CI failure even though the client under test evaluated its flag locally.

The affected tests now patch _load_feature_flags on their own client instance. The assertions still verify that local evaluation does not load flag definitions, but requests from unrelated clients can no longer change the call count.

💚 How did you test it?

  • Ran the four affected tests 30 times: all passed.
  • Ran posthog/test/test_feature_flags.py: 173 passed with 13 subtests.
  • Ran Ruff formatting and lint checks.
  • Ran mypy for the changed test file.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran sampo add to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with Pi under human direction after a flaky Python 3.13 CI failure on #867. Pi autoreview reported no actionable findings for commit 428fd8ca52ee0ccf48b361dfb3d85a823e0a7db4.

@marandanetomarandaneto self-assigned this Aug 12, 2026
@marandaneto
marandaneto marked this pull request as ready for review August 12, 2026 09:11
@marandaneto
marandaneto requested a review from a team as a code ownerAugust 12, 2026 09:11
@greptile-apps

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "test: isolate local flag polling asserti..." | Re-trigger Greptile

@github-actions

github-actionsBot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

posthog-python Compliance Report

Date: 2026-08-12 09:14:55 UTC
Duration: 256365ms

✅ All Tests Passed!

111/111 tests passed


Capture_V1 Tests

94/94 tests passed

View Details
TestStatusDuration
Endpoint And Method.Targets V1 Endpoint517ms
Endpoint And Method.Does Not Use Legacy Endpoints510ms
Required Headers.Has Authorization Bearer Header511ms
Required Headers.Has Content Type Json510ms
Required Headers.Has Posthog Sdk Info Format511ms
Required Headers.Has Posthog Attempt Header510ms
Required Headers.Has Posthog Request Id510ms
Required Headers.Has Posthog Request Timestamp510ms
Required Headers.Has User Agent510ms
Body Format.Body Has Created At And Batch509ms
Body Format.No Api Key In Body510ms
Body Format.No Sent At In Body510ms
Event Format.Event Has Required Root Fields510ms
Event Format.Event Uuid Is Valid510ms
Event Format.Event Timestamp Is Rfc3339511ms
Event Format.Distinct Id Is String510ms
Event Format.Distinct Id At Root Not Properties510ms
Event Format.Custom Properties Preserved510ms
Event Format.Set Properties Preserved510ms
Event Format.Set Once Properties Preserved509ms
Event Format.Groups Properties Preserved511ms
Event Format.Sdk Generates Uuid If Not Provided510ms
Event Format.Event Has Required Root Fields Batch513ms
Event Format.Event Uuid Is Valid Batch514ms
Event Format.Event Timestamp Is Rfc3339 Batch513ms
Event Format.Distinct Id Is String Batch514ms
Event Format.Distinct Id At Root Not Properties Batch514ms
Event Format.Custom Properties Preserved Batch513ms
Event Format.Set Properties Preserved Batch513ms
Event Format.Set Once Properties Preserved Batch514ms
Event Format.Groups Properties Preserved Batch513ms
Event Format.Sdk Generates Uuid If Not Provided Batch514ms
Batch Behavior.Multiple Events In Single Batch517ms
Batch Behavior.Batch Envelope Smoke516ms
Batch Behavior.Flush With No Events Sends Nothing506ms
Batch Behavior.Flush At Triggers Batch1012ms
Batch Behavior.Created At Reflects Batch Creation Time511ms
Deduplication.Generates Unique Uuids517ms
Deduplication.Different Events Same Content Different Uuids512ms
Deduplication.Preserves Uuid On Retry6514ms
Deduplication.Preserves Timestamp On Retry6519ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry6524ms
Deduplication.No Duplicate Events In Batch518ms
Header Behavior On Retry.Attempt Header Starts At One511ms
Header Behavior On Retry.Attempt Header Increments On Retry13523ms
Header Behavior On Retry.Request Id Preserved On Retry6519ms
Header Behavior On Retry.Different Requests Have Different Request Ids3018ms
Header Behavior On Retry.Request Timestamp Changes On Retry6518ms
Response Format Validation.Success Response Has Uuid Keyed Results511ms
Response Format Validation.Success Response Has Ok For Each Event514ms
Response Format Validation.Success No Retry After When All Ok512ms
Response Format Validation.Success Retry After Present When Retry Events1515ms
Response Format Validation.Success No Retry After When Drop Only512ms
Response Format Validation.Response Echoes Request Id510ms
Retry Behavior.Retries On 4086519ms
Retry Behavior.Retries On 5006515ms
Retry Behavior.Retries On 5038522ms
Retry Behavior.Retries On 5046519ms
Retry Behavior.Retryable Errors Have Retry After3517ms
Retry Behavior.Respects Retry After On Retryable Error11523ms
Retry Behavior.Does Not Retry On 4002514ms
Retry Behavior.Does Not Retry On 4012514ms
Retry Behavior.Does Not Retry On 4022512ms
Retry Behavior.Does Not Retry On 4132513ms
Retry Behavior.Does Not Retry On 4152512ms
Retry Behavior.Non Retryable Errors Have No Retry After2511ms
Retry Behavior.Implements Backoff22536ms
Retry Behavior.Max Retries Respected22525ms
Partial Batch Handling.Handles 200 Full Success2513ms
Partial Batch Handling.Handles 200 With All Ok3516ms
Partial Batch Handling.Does Not Retry Dropped Events3513ms
Partial Batch Handling.Does Not Retry Limited Events3513ms
Partial Batch Handling.Prunes Ok Events On Partial Retry6516ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry6517ms
Partial Batch Handling.Retries Only Retry Events From Partial6523ms
Partial Batch Handling.Partial Retry Preserves Uuids6521ms
Partial Batch Handling.Partial Retry Attempt Header Increments6516ms
Partial Batch Handling.Partial Retry Request Id Preserved6522ms
Partial Batch Handling.Respects Retry After On Partial8519ms
Partial Batch Handling.Unknown Result Treated As Terminal3516ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry3520ms
Compression.Sends Gzip Content Encoding510ms
Compression.No Content Encoding When Disabled509ms
Compression.Compressed Body Is Decompressible510ms
Error Handling.Does Not Retry On Unknown 4Xx2511ms
Event Options.Cookieless Mode Override510ms
Event Options.Disable Skew Correction Override510ms
Event Options.Process Person Profile Override510ms
Event Options.Product Tour Id Override509ms
Event Options.Unset Options Omitted510ms
Event Options.Options Override In Batch513ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties509ms
Geoip And Historical Migration.Historical Migration Set In Body511ms
Geoip And Historical Migration.Historical Migration Absent By Default510ms

Feature_Flags Tests

17/17 tests passed

View Details
TestStatusDuration
Request Payload.Request With Person Properties Device Id11ms
Request Payload.Flags Request Uses V2 Query Param9ms
Request Payload.Flags Request Hits Flags Path Not Decide9ms
Request Payload.Flags Request Omits Authorization Header9ms
Request Payload.Token In Flags Body Matches Init9ms
Request Payload.Groups Round Trip9ms
Request Payload.Groups Default To Empty Object8ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False9ms
Request Payload.Disable Geoip Omitted Defaults To False9ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key9ms
Request Lifecycle.No Flags Request On Init Alone3ms
Request Lifecycle.No Flags Request On Normal Capture509ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests14ms
Request Lifecycle.Mock Response Value Is Returned To Caller9ms
Retry Behavior.Retries Flags On 502312ms
Retry Behavior.Retries Flags On 504313ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event511ms

@arnohillenarnohillen left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[arno's agent] Reviewed at head 428fd8c: change verified against head code, CI classified, no blocking findings.

@marandaneto
marandaneto merged commit 8f3a568 into mainAug 13, 2026
39 checks passed
@marandaneto
marandaneto deleted the fix/feature-flag-cohort-test-race branch August 13, 2026 15:16
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@marandaneto@arnohillen