[ci] Run web tests in wasm (unit + integration). by ditman · Pull Request #8111 · flutter/packages

ditman · 2024-11-16T00:35:23Z

Adds CI configuration to run web integration tests (in the master channel) compiled to Wasm.

It also removes the build-examples step from web integration tests, in some isolated testing:

platform tests shard	With build-examples	Without build-examples
1	30m	21m
2	13m	11m
3	17m	10m

Issues

FixesAdd packages repo CI job to run web integration tests compiled to WASM flutter#151664

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

ditman

Changes are required, do not merge as is.

ditman · 2024-12-20T03:51:56Z

Talk to @eyebrowsoffire about this new delay.

We discussed this, the skwasm renderer is now async, so it makes sense that there's some timing differences compared to the JS one. Needing this additional delay is probably caused by the tester not being aware of this asynchronicity, and declaring a frame "pumped" before it's really been fully rendered.

…ion to test CI before it propagates.

… wasm in all_apps

… different entrypoint each time; I'm not sure prebuilding helps with anything other than pre-fetching deps (?)

ditman · 2024-12-24T02:54:15Z

[0:00] Running for packages/animations...
No version change.
Found NEXT; validating next version in the CHANGELOG.
No CHANGELOG change found.
If this PR needs an exemption from the standard policy of listing all changes in the CHANGELOG,
comment in the PR to explain why the PR is exempt, and add (or ask your reviewer to add) the
"override: no changelog needed" label.
Otherwise, please add a NEXT entry in the CHANGELOG as described in the contributing guide.

The only change to packages/animations is the .gitignore, which is having a flutter fix that I introduced in May 2022 (here). No need to version anything, it's purely repo related.

stuartmorgan-g · 2025-01-06T19:43:31Z

My only question is if we want to run the tests both on master and stable or master only is fine.

My rule of thumb for expensive tests is to ask: is someone likely to make a change that only breaks on the combination of stable and the new config? Or can we get almost as much confidence that if some PR passes some configuration on stable, and also passes both configurations on master, that it'll be fine on the other configuration on stable?

If we don't have that confidence, then we can add (ideally post-submit-only) stable versions, otherwise I tend to lean toward reducing CI load.

stuartmorgan-g

LGTM

ditman · 2025-01-07T01:56:52Z

[...] otherwise I tend to lean toward reducing CI load.

@stuartmorgan I agree, let's go with just master for now. I'm not sure what can be introduced into wasm compilation that would break this stuff too badly only on stable (but we're very smart people, I'm sure we can find a way :P)

Anyway, it's easy enough to add the stable shards down the line, so let's go with this!

ditman · 2025-01-07T18:40:35Z

The following packages had errors:
packages/animations:
Missing version change

Adding label to skip version change in packages/animations. The only change is in the .gitignore file (see message above)

ditman · 2025-01-07T19:18:11Z

One last update to give flakes a chance to reconsider their failing stance :P

…kages#8111)

flutter/packages@3fc6b7a...6554751 2025-01-09 stanleycocoa@gmail.com [webview_flutter_wkwebview] Add a listener for the canGoBack property change on the iOS platform. (flutter/packages#8203) 2025-01-09 39571861+memishood@users.noreply.github.com [flutter_svg] Implement errorBuilder callback (flutter/packages#8364) 2025-01-08 10687576+bparrishMines@users.noreply.github.com [interactive_media_ads] Adds remaining methods for internal wrapper of the iOS native `IMAAdDisplayContainer`. (flutter/packages#8153) 2025-01-08 jhy03261997@gmail.com [go_router_builder] Add a fallback for a not null List or Set param (flutter/packages#8349) 2025-01-08 engine-flutter-autoroll@skia.org Manual roll Flutter (stable) from 8495dee to 17025dd (2 revisions) (flutter/packages#8396) 2025-01-08 10687576+bparrishMines@users.noreply.github.com [ci][video_player_android][camera_android_camerax] Manual roll and fix tests (flutter/packages#8391) 2025-01-07 ditman@gmail.com [ci] Run web tests in wasm (unit + integration). (flutter/packages#8111) 2025-01-07 liff@iki.fi [camera_platform_interface] Add API support query for image streaming (flutter/packages#8307) 2025-01-07 10687576+bparrishMines@users.noreply.github.com [camera_android_camerax][video_player_android] Suppresses deprecation of the `SurfaceProducer.Callback.onSurfaceDestroyed` method (flutter/packages#8388) 2025-01-07 jessiewong401@gmail.com Bump Remaining Plugin Example Apps to targetSdkVersion >= 34 and compileSdkVersion >= 34 (flutter/packages#8361) 2025-01-07 matanlurey@users.noreply.github.com Replace `onSurfaceCreated` with `onSurfaceAvailable`. (flutter/packages#8384) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages-flutter-autoroll Please CC flutter-ecosystem@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

This PR moves the new Wasm shards out of bringup, so they become part of the normal testing flow. Here's some information about the last ~90 builds in the staging builder: * Unit tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_2%20master?limit=200) * Integration tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_2%20master?limit=200) * [Shard 3](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_3%20master?limit=200) ## Issues * Continuation of #8111 * Completes flutter/flutter#151664 * Closesflutter/flutter#127730

Adds CI configuration to run web integration tests (in the master channel) compiled to Wasm. It also removes the `build-examples` step from web integration tests, in some isolated testing: | platform tests shard | With build-examples | Without build-examples | |---|-----|-----| | 1 | 30m | 21m | | 2 | 13m | 11m | | 3 | 17m | 10m | ## Issues * Fixesflutter/flutter#151664

This PR moves the new Wasm shards out of bringup, so they become part of the normal testing flow. Here's some information about the last ~90 builds in the staging builder: * Unit tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_2%20master?limit=200) * Integration tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_2%20master?limit=200) * [Shard 3](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_3%20master?limit=200) ## Issues * Continuation of flutter#8111 * Completes flutter/flutter#151664 * Closesflutter/flutter#127730

Adds CI configuration to run web integration tests (in the master channel) compiled to Wasm. It also removes the `build-examples` step from web integration tests, in some isolated testing: | platform tests shard | With build-examples | Without build-examples | |---|-----|-----| | 1 | 30m | 21m | | 2 | 13m | 11m | | 3 | 17m | 10m | ## Issues * Fixesflutter/flutter#151664

This PR moves the new Wasm shards out of bringup, so they become part of the normal testing flow. Here's some information about the last ~90 builds in the staging builder: * Unit tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_2%20master?limit=200) * Integration tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_2%20master?limit=200) * [Shard 3](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_3%20master?limit=200) ## Issues * Continuation of flutter#8111 * Completes flutter/flutter#151664 * Closesflutter/flutter#127730

Adds CI configuration to run web integration tests (in the master channel) compiled to Wasm. It also removes the `build-examples` step from web integration tests, in some isolated testing: | platform tests shard | With build-examples | Without build-examples | |---|-----|-----| | 1 | 30m | 21m | | 2 | 13m | 11m | | 3 | 17m | 10m | ## Issues * Fixesflutter/flutter#151664

This PR moves the new Wasm shards out of bringup, so they become part of the normal testing flow. Here's some information about the last ~90 builds in the staging builder: * Unit tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_2%20master?limit=200) * Integration tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_2%20master?limit=200) * [Shard 3](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_3%20master?limit=200) ## Issues * Continuation of flutter#8111 * Completes flutter/flutter#151664 * Closesflutter/flutter#127730

Adds CI configuration to run web integration tests (in the master channel) compiled to Wasm. It also removes the `build-examples` step from web integration tests, in some isolated testing: | platform tests shard | With build-examples | Without build-examples | |---|-----|-----| | 1 | 30m | 21m | | 2 | 13m | 11m | | 3 | 17m | 10m | ## Issues * Fixesflutter/flutter#151664

This PR moves the new Wasm shards out of bringup, so they become part of the normal testing flow. Here's some information about the last ~90 builds in the staging builder: * Unit tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_2%20master?limit=200) * Integration tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_2%20master?limit=200) * [Shard 3](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_3%20master?limit=200) ## Issues * Continuation of flutter#8111 * Completes flutter/flutter#151664 * Closesflutter/flutter#127730

ditman force-pushed the run-web-integration-wasm branch from 8a28941 to d6bdc4fCompare December 17, 2024 23:35

github-actionsBot added p: pointer_interceptor platform-web p: video_player p: url_launcher p: google_maps_flutter labels Dec 20, 2024

ditman commented Dec 20, 2024
View reviewed changes

github-actionsBot added the p: animations label Dec 20, 2024

ditman added 16 commits December 23, 2024 15:28

Add a --wasm test shard in master.
a1d0b38

Add web_platform_tests_wasm config. Copy that config to non-wasm vers…
9f4cd1f
…ion to test CI before it propagates.

Remove extra comma
b518877

url_launcher_web tests now pass with --wasm
0f80fcb

video_player tests now pass with --wasm
1542405

pointer_interceptor_web wasm tests now pass.
1685463

Exclude file_selector, since it does not have integration tests.
b56ac8d

Attempt to pass multiple exclusion files, similarly to how we exclude…
c510654
… wasm in all_apps

google_maps_flutter_web tests now pass in wasm
bcd7f6f

dart format .
01a9321

Remove unnecessary import.
6b01a26

Clarify note
be17b43

Shard wasm tests in 3 shards.
9c10258

Try prebuilding the examples.
8bb9e64

Do not prebuild apps. Integration tests on the web work by building a…
18674b3
… different entrypoint each time; I'm not sure prebuilding helps with anything other than pre-fetching deps (?)

Update animations/example gitignore file.
598ed80

ditman force-pushed the run-web-integration-wasm branch from 420958b to 598ed80Compare December 23, 2024 23:28

ditman added the override: no changelog needed Override the check requiring CHANGELOG updates for most changes label Dec 24, 2024

ditman added 3 commits December 23, 2024 19:41

Merge branch 'main' into run-web-integration-wasm
630bfe6

Merge branch 'flutter:main' into run-web-integration-wasm
dc40e4d

google_adsense Fix experimental test so it passes in wasm.
67d7897

stuartmorgan-g approved these changes Jan 6, 2025
View reviewed changes

Comment threadpackages/animations/example/.gitignore Outdated

Remove unneeded comment from gitignore
6660cfb

ditman added the override: no versioning needed Override the check requiring version bumps for most changes label Jan 7, 2025

ditman changed the title ~~[ci][web] Run web tests in wasm (unit + integration).~~[ci] Run web tests in wasm (unit + integration).Jan 7, 2025

Merge branch 'main' into run-web-integration-wasm
fcaff2a

ditman added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 7, 2025

auto-submitBot merged commit 11a9fa8 into flutter:mainJan 7, 2025

engine-flutter-autoroll mentioned this pull request Jan 8, 2025
Roll Packages from 3fc6b7ace3ff to 11a9fa8bf6e5 (5 revisions) flutter/flutter#161315
Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jan 8, 2025
11a9fa8[ci] Run web tests in wasm (unit + integration). (flutter/pac…
e61c372
…kages#8111)

engine-flutter-autoroll mentioned this pull request Jan 8, 2025
Roll Packages from 3fc6b7ace3ff to 11a9fa8bf6e5 (5 revisions) flutter/flutter#161325
Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jan 8, 2025
11a9fa8[ci] Run web tests in wasm (unit + integration). (flutter/pac…
888b5ea
…kages#8111)

engine-flutter-autoroll mentioned this pull request Jan 9, 2025
Roll Packages from 3fc6b7ace3ff to 65547511c004 (11 revisions) flutter/flutter#161379
Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jan 9, 2025
11a9fa8[ci] Run web tests in wasm (unit + integration). (flutter/pac…
e71b756
…kages#8111)

ditman mentioned this pull request Jan 16, 2025
[google_maps_flutter] Support for Ground Overlay #8432
Merged
11 tasks

This was referenced Feb 5, 2025
[ci] Move Wasm test shards out of bringup. #8572
Merged
[integration_test] Flutter Web WASM support flutter/flutter#127730
Closed

Navigation Menu

Uh oh!

[ci] Run web tests in wasm (unit + integration). - #8111

[ci] Run web tests in wasm (unit + integration).#8111
auto-submit[bot] merged 32 commits into
flutter:mainfrom
ditman:run-web-integration-wasm

ditman commented Nov 16, 2024•
edited
Loading

Uh oh!

ditman left a comment

Uh oh!

Uh oh!

Uh oh!

ditmanDec 20, 2024

Uh oh!

ditmanDec 20, 2024

Uh oh!

Uh oh!

Uh oh!

ditman commented Dec 24, 2024

Uh oh!

stuartmorgan-g commented Jan 6, 2025

Uh oh!

stuartmorgan-g left a comment

Uh oh!

Uh oh!

ditman commented Jan 7, 2025

Uh oh!

ditman commented Jan 7, 2025•
edited
Loading

Uh oh!

ditman commented Jan 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

ditman commented Nov 16, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issues

Pre-launch Checklist

Uh oh!

ditman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ditmanDec 20, 2024

Choose a reason for hiding this comment

Uh oh!

ditmanDec 20, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ditman commented Dec 24, 2024

Uh oh!

stuartmorgan-g commented Jan 6, 2025

Uh oh!

stuartmorgan-g left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ditman commented Jan 7, 2025

Uh oh!

ditman commented Jan 7, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ditman commented Jan 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ditman commented Nov 16, 2024•
edited
Loading

ditman commented Jan 7, 2025•
edited
Loading