Skip to content

[in_app_purchase_storekit] Address flaky tests - #11270

Merged
stuartmorgan-g merged 7 commits into
flutter:mainfrom
LouiseHsu:storekit-test-flakes
Mar 25, 2026
Merged

[in_app_purchase_storekit] Address flaky tests#11270
stuartmorgan-g merged 7 commits into
flutter:mainfrom
LouiseHsu:storekit-test-flakes

Conversation

@LouiseHsu

@LouiseHsuLouiseHsu commented Mar 16, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Fixesflutter/flutter#182845
Probably fixesflutter/flutter#183318

Pre-Review Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools.
  • I read the [Tree Hygiene] page, which explains my responsibilities.
  • I read and followed the [relevant style guides] and ran [the auto-formatter].
  • I signed the [CLA].
  • The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
  • I [linked to at least one issue that this PR fixes] in the description above.
  • I followed [the version and CHANGELOG instructions], using [semantic versioning] and the [repository CHANGELOG style], or I have commented below to indicate which documented exception this PR falls under[^1].
  • I updated/added any relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or I have commented below to indicate which [test exemption] this PR falls under[^1].
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on [Discord].

@LouiseHsuLouiseHsu added the CICD Run CI/CD label Mar 16, 2026
@LouiseHsuLouiseHsu added CICD Run CI/CD and removed CICD Run CI/CD labels Mar 16, 2026
@LouiseHsuLouiseHsu added CICD Run CI/CD and removed CICD Run CI/CD labels Mar 17, 2026
@LouiseHsuLouiseHsu added CICD Run CI/CD and removed CICD Run CI/CD labels Mar 17, 2026
@LouiseHsuLouiseHsu changed the title test flakes[in_app_purchase_storekit] Address flaky testsMar 17, 2026
@LouiseHsuLouiseHsu added CICD Run CI/CD and removed CICD Run CI/CD labels Mar 17, 2026
@LouiseHsu
LouiseHsu marked this pull request as ready for review March 17, 2026 23:25
gemini-code-assist[bot]
gemini-code-assistBot reviewed Mar 17, 2026
View reviewed changes

@gemini-code-assistgemini-code-assistBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses flaky tests in InAppPurchaseStoreKit2PluginTests.swift by skipping certain tests on a specific OS version known to have a StoreKit bug, and by increasing the timeout for two other tests. My review focuses on improving the maintainability of these changes. I've suggested extracting duplicated test-skipping logic into a helper function and replacing a magic number for a timeout with a named constant. These changes will make the test suite cleaner and easier to manage.

Comment thread.../in_app_purchase_storekit/example/shared/RunnerTests/InAppPurchaseStoreKit2PluginTests.swift
Comment thread.../in_app_purchase_storekit/example/shared/RunnerTests/InAppPurchaseStoreKit2PluginTests.swift
@LouiseHsuLouiseHsu removed the CICD Run CI/CD label Mar 17, 2026
@LouiseHsuLouiseHsu added the CICD Run CI/CD label Mar 17, 2026
LouiseHsu
LouiseHsu commented Mar 18, 2026
View reviewed changes
Comment thread.../in_app_purchase_storekit/example/shared/RunnerTests/InAppPurchaseStoreKit2PluginTests.swift
Comment on lines +184 to +188
let osVersion = ProcessInfo.processInfo.operatingSystemVersion
try XCTSkipIf(
osVersion.majorVersion == 23 && osVersion.minorVersion == 2,
"Known StoreKitTest bug on Xcode 23.2 with setSimulatedError() when used on .loadProducts API"
)

@LouiseHsuLouiseHsuMar 18, 2026

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://developer.apple.com/forums/thread/808030
.setSimulatedError doesnt work on specifically 26.2, but apples availability annotation doesnt support donut hole versioning.

@jmagmanjmagmanMar 19, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add the URL into a comment somewhere? Definitely also in this PR description.

@stuartmorgan-gstuartmorgan-g added CICD Run CI/CD and removed CICD Run CI/CD labels Mar 18, 2026
@LouiseHsuLouiseHsu added CICD Run CI/CD and removed CICD Run CI/CD labels Mar 19, 2026
@LouiseHsuLouiseHsu added the autosubmit Merge PR when tree becomes green via auto submit App label Mar 19, 2026
@stuartmorgan-g
stuartmorgan-g merged commit 309a35a into flutter:mainMar 25, 2026
81 checks passed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Mar 25, 2026
@engine-flutter-autoroll
309a35a40 [in_app_purchase_storekit] Address flaky tests (flutter/pac…
734c82a
…kages#11270)
@engine-flutter-autorollengine-flutter-autoroll mentioned this pull request Mar 25, 2026
Merged
github-merge-queueBot pushed a commit to flutter/flutter that referenced this pull request Mar 25, 2026
@engine-flutter-autoroll
Roll Packages from 8dcfd1186ef9 to 5909bdde6090 (13 revisions) (#184123)
dcf3549
flutter/packages@8dcfd11...5909bdd
2026-03-25 47866232+chunhtai@users.noreply.github.com [ci] add more
permissions for create-pull-request (flutter/packages#11302)
2026-03-25 stuartmorgan@google.com [various] Add
`unintended_html_in_doc_comment` to analysis options
(flutter/packages#11303)
2026-03-25 matt.boetger@gmail.com Use deprecated dependency until legacy
renderer is removed (flutter/packages#11185)
2026-03-25 louisehsu@google.com [in_app_purchase_storekit] Address flaky
tests (flutter/packages#11270)
2026-03-25 stuartmorgan@google.com [google_maps_flutter] Fix A2A iOS
builds (flutter/packages#11290)
2026-03-25 stuartmorgan@google.com [mustache_template] Fix broken README
link (flutter/packages#11306)
2026-03-25 stuartmorgan@google.com [ci] Add a workflow to auto-remove
CICD label (flutter/packages#11301)
2026-03-25 stuartmorgan@google.com [various] Convert plugin builds to
Kotlin gradle (flutter/packages#11172)
2026-03-25 spkhalad@gmail.com [webview_flutter_platform_interface] Add
support for getting cookie (flutter/packages#11037)
2026-03-25 stuartmorgan@google.com [cupertino_icons] Remove empty Dart
file (flutter/packages#11308)
2026-03-25 stuartmorgan@google.com [camera] Regenerate iOS example with
Swift (flutter/packages#11283)
2026-03-25 tarrinneal@gmail.com [pigeon] Optimize data class equality
and hashing in Dart, Kotlin, java, and Swift, adds equality in other
languages (flutter/packages#11140)
2026-03-25 katelovett@google.com [two_dimensional_scrollables] Fix span
border decorations in flipped cross axes (flutter/packages#11334)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC flutter-ecosystem@google.com on the revert to ensure that a
human
is aware of the problem.
To file a bug in Flutter:
https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
okorohelijah pushed a commit to okorohelijah/packages that referenced this pull request Mar 26, 2026
@LouiseHsu@okorohelijah
[in_app_purchase_storekit] Address flaky tests (flutter#11270)
8a8073e
Fixesflutter/flutter#182845
Probably fixesflutter/flutter#183318
## Pre-Review Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [x] I read the [Tree Hygiene] page, which explains my
responsibilities.
- [x] I read and followed the [relevant style guides] and ran [the
auto-formatter].
- [x] I signed the [CLA].
- [x] The title of the PR starts with the name of the package surrounded
by square brackets, e.g. `[shared_preferences]`
- [x] I [linked to at least one issue that this PR fixes] in the
description above.
- [x] I followed [the version and CHANGELOG instructions], using
[semantic versioning] and the [repository CHANGELOG style], or I have
commented below to indicate which documented exception this PR falls
under[^1].
- [x] I updated/added any relevant documentation (doc comments with
`///`).
- [x] I added new tests to check the change I am making, or I have
commented below to indicate which [test exemption] this PR falls
under[^1].
- [x] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
mboetger pushed a commit to mboetger/flutter that referenced this pull request Mar 26, 2026
@engine-flutter-autoroll@mboetger
Roll Packages from 8dcfd1186ef9 to 5909bdde6090 (13 revisions) (flutt…
8b835c2
…er#184123)
flutter/packages@8dcfd11...5909bdd
2026-03-25 47866232+chunhtai@users.noreply.github.com [ci] add more
permissions for create-pull-request (flutter/packages#11302)
2026-03-25 stuartmorgan@google.com [various] Add
`unintended_html_in_doc_comment` to analysis options
(flutter/packages#11303)
2026-03-25 matt.boetger@gmail.com Use deprecated dependency until legacy
renderer is removed (flutter/packages#11185)
2026-03-25 louisehsu@google.com [in_app_purchase_storekit] Address flaky
tests (flutter/packages#11270)
2026-03-25 stuartmorgan@google.com [google_maps_flutter] Fix A2A iOS
builds (flutter/packages#11290)
2026-03-25 stuartmorgan@google.com [mustache_template] Fix broken README
link (flutter/packages#11306)
2026-03-25 stuartmorgan@google.com [ci] Add a workflow to auto-remove
CICD label (flutter/packages#11301)
2026-03-25 stuartmorgan@google.com [various] Convert plugin builds to
Kotlin gradle (flutter/packages#11172)
2026-03-25 spkhalad@gmail.com [webview_flutter_platform_interface] Add
support for getting cookie (flutter/packages#11037)
2026-03-25 stuartmorgan@google.com [cupertino_icons] Remove empty Dart
file (flutter/packages#11308)
2026-03-25 stuartmorgan@google.com [camera] Regenerate iOS example with
Swift (flutter/packages#11283)
2026-03-25 tarrinneal@gmail.com [pigeon] Optimize data class equality
and hashing in Dart, Kotlin, java, and Swift, adds equality in other
languages (flutter/packages#11140)
2026-03-25 katelovett@google.com [two_dimensional_scrollables] Fix span
border decorations in flipped cross axes (flutter/packages#11334)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC flutter-ecosystem@google.com on the revert to ensure that a
human
is aware of the problem.
To file a bug in Flutter:
https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
ahmedsameha1 pushed a commit to ahmedsameha1/flutter that referenced this pull request Apr 14, 2026
@engine-flutter-autoroll@ahmedsameha1
Roll Packages from 8dcfd1186ef9 to 5909bdde6090 (13 revisions) (flutt…
86d6002
…er#184123)
flutter/packages@8dcfd11...5909bdd
2026-03-25 47866232+chunhtai@users.noreply.github.com [ci] add more
permissions for create-pull-request (flutter/packages#11302)
2026-03-25 stuartmorgan@google.com [various] Add
`unintended_html_in_doc_comment` to analysis options
(flutter/packages#11303)
2026-03-25 matt.boetger@gmail.com Use deprecated dependency until legacy
renderer is removed (flutter/packages#11185)
2026-03-25 louisehsu@google.com [in_app_purchase_storekit] Address flaky
tests (flutter/packages#11270)
2026-03-25 stuartmorgan@google.com [google_maps_flutter] Fix A2A iOS
builds (flutter/packages#11290)
2026-03-25 stuartmorgan@google.com [mustache_template] Fix broken README
link (flutter/packages#11306)
2026-03-25 stuartmorgan@google.com [ci] Add a workflow to auto-remove
CICD label (flutter/packages#11301)
2026-03-25 stuartmorgan@google.com [various] Convert plugin builds to
Kotlin gradle (flutter/packages#11172)
2026-03-25 spkhalad@gmail.com [webview_flutter_platform_interface] Add
support for getting cookie (flutter/packages#11037)
2026-03-25 stuartmorgan@google.com [cupertino_icons] Remove empty Dart
file (flutter/packages#11308)
2026-03-25 stuartmorgan@google.com [camera] Regenerate iOS example with
Swift (flutter/packages#11283)
2026-03-25 tarrinneal@gmail.com [pigeon] Optimize data class equality
and hashing in Dart, Kotlin, java, and Swift, adds equality in other
languages (flutter/packages#11140)
2026-03-25 katelovett@google.com [two_dimensional_scrollables] Fix span
border decorations in flipped cross axes (flutter/packages#11334)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC flutter-ecosystem@google.com on the revert to ensure that a
human
is aware of the problem.
To file a bug in Flutter:
https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
creatorpiyush pushed a commit to creatorpiyush/packages that referenced this pull request Jun 10, 2026
@LouiseHsu
[in_app_purchase_storekit] Address flaky tests (flutter#11270)
3a9c411
Fixesflutter/flutter#182845
Probably fixesflutter/flutter#183318
## Pre-Review Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [x] I read the [Tree Hygiene] page, which explains my
responsibilities.
- [x] I read and followed the [relevant style guides] and ran [the
auto-formatter].
- [x] I signed the [CLA].
- [x] The title of the PR starts with the name of the package surrounded
by square brackets, e.g. `[shared_preferences]`
- [x] I [linked to at least one issue that this PR fixes] in the
description above.
- [x] I followed [the version and CHANGELOG instructions], using
[semantic versioning] and the [repository CHANGELOG style], or I have
commented below to indicate which documented exception this PR falls
under[^1].
- [x] I updated/added any relevant documentation (doc comments with
`///`).
- [x] I added new tests to check the change I am making, or I have
commented below to indicate which [test exemption] this PR falls
under[^1].
- [x] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jmagmanjmagmanjmagman left review comments
@LongCatIsLooongLongCatIsLooongLongCatIsLooong approved these changes
+1 more reviewer
@gemini-code-assistgemini-code-assist[bot]gemini-code-assist[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

autosubmitMerge PR when tree becomes green via auto submit AppCICDRun CI/CDp: in_app_purchaseplatform-iosplatform-macos

Projects

None yet

Milestone

No milestone

4 participants

@LouiseHsu@jmagman@LongCatIsLooong@stuartmorgan-g