#1509 Migrate to Manifest V3

[sosnovsky]

This PR migrates extension code to Manifest V3

close#1509

---

Tests(delete all except exactly one):

• Tests added or updated

---

To be filled by reviewers

I have reviewed that this PR... (tick whichever items you personally focused on during this review):

• addresses the issue it closes (if any)
• code is readable and understandable
• is accompanied with tests, or tests are not needed
• is free of vulnerabilities
• is documented clearly and usefully, or doesn't need documentation

[sosnovsky]

@ioanmo226 here is my current changes for manifest v3 migration, you can continue working on this code, feel free to ask questions:

• the biggest change in manifest v3 is usage of service workers instead of background pages - https://developer.chrome.com/docs/extensions/develop/migrate/to-service-workers. Previously background pages were constantly running in background, while service workers are alive only when we call them from extension.
• it breaks our ExpirationCache, as with current implementation it's not possible to use it for storing cache values for predefined period of time. I renamed it to SimpleExpirationCache, and tried to use chrome.storage.session for storing cache values
• also window. is not available in service worker, it led to many window is undefined errors. I replaced window.location.href with location.href, window.document.body with document.body etc and those errors are gone, but not sure if it breaks this code functionality
• I also copied code from extension/js/background_page to extension/js/service_worker with some changes to make it work with manifest v3

Per updated Google timeline - in June 2024 they'll start disabling manifest v2 extensions in pre-release versions of Chrome, so by that time we should have finished and tested migration to manifest v3.
It'll be great to have fully functional build in the beginning of April, so we can test it to find any issues during April and release final manifest v3 build in the beginning of May.

If some functionality can't be fully migrated to manifest v3 - we can simplify it for now and work on full functionality later. Google regularly adds new features to manifest v3, maybe some new feature will be helpful for us and make implementation easier.

P.S. Also Firefox has a different implementation for manifest v3 - https://extensionworkshop.com/documentation/develop/manifest-v3-migration-guide/. They continue support background pages, but they are non-persistent in manifest v3, need to adapt code for them too.

[ioanatflowcrypt]

@sosnovsky Thank you for your detailed response and code update.

I have now finished checking your code and manfest v3 update guide.

Could you list TODO list (to update to v3) you had in mind? It would help me complete task faster.

PS: I lost 2FA code for my previuos github account and submitted recovery action. Might be recovered in 1-2 days.

[sosnovsky]

Could you list TODO list (to update to v3) you had in mind? It would help me complete task faster.

For now we should start with these issues, as implementing them will make possible to run UI tests which will show broken functionality, and then it'll be easier to understand what needs to be fixed:

• insert "Secure compose" button on each gmail.com page load (currently it's inserted only on first load, I think, when service worker is initialized)
• implement replacement for current ExpirationCache, as with manifest v3 it's not possible to store something in memory for predefined amount in time

Initially I thought it'll be possible to create manifest v2 and v3 builds from the same codebase, then we could merge manifest v3 changes to master one by one, but it seems to be quite complicated, so we'll need to have separate branch for manifest v3 until it's ready to be released.

[sosnovsky]

@sosnovsky Could you please check if Setup Flowrypt initial screen works fine in your side?
When I tap Sign in with google and then select account and click next, it just do nothing and console displays Uncaught (in promise) Error: No tab with id: 1745224200.

It probably doesn't work, as I've tested only already configured extension and didn't go through setup process.

Also Secure compose button is not even inserted on first load in my side.
Could you check if you pushed latest changes?

Yes, it's my latest changes, probably one of my last changes broke compose button insert even on the first load :)
Maybe it's related to inMemoryStorage, need to check if token is successfully fetched there

[sosnovsky]

Got it. Thank you for your answer/

Feel free to message me if you'll have any questions about manifest v3 migration, as it's quite complicated issue and probably second opinion will be useful for making some implementation decisions.

[sosnovsky]

Which second option do you mean?

I meant second opinion there :) So if you're not sure how some functionality should be converted to manifest v3, we can discuss implementation details here to find the best solution

[sosnovsky]

Given the necessity of setInterval or setTimeout in various parts of the extension, I recommend utilizing chrome.alarms instead.
I have implemented this solution in the latest commit. I welcome your feedback and am open to any suggestions you may have. Additionally, please inform me if you consent to adding the alarms permission.

Yes, we'll need to use alarms permission, as it's required for running timers in service worker (https://developer.chrome.com/docs/extensions/develop/migrate/to-service-workers#alarms), so you can add it to manifest.json. As I remember, adding alarms permission shouldn't show additional permissions alert for users.

By the way, when working on manifest v3 update I found tool for testing extension update - https://github.com/GoogleChromeLabs/extension-update-testing-tool. It'll show how update to the new version will look for users, so you can check if any additional permission alerts will be shown (we should try to migrate without such additional alerts, as some users can decline them and then some functionality will be broken))

PS: I thought there was logic in Catch.setHandledTimeout but I can see that it does nothing.
Maybe we need to implement setHandledTimeout and setHandledInterval with chrome.alarms api?

I commented initial logic and temporary replaced it with return ms, as extension failed to build because of window is undefined error. We'll need to reimplement it using chrome.alarms API, you're right

[sosnovsky]

From mentioned options I would prefer using webpack, as we already use it for packing @openpgp/web-stream-tools for content script.

But probably it's possible to just remove openpgp and node-forge from service worker?
Some time ago we worked on moving openpgp.js from background page to content script (#4906), and as it's now included in content script maybe we can now remove it from service worker and move remaining functionality to content script?
Same for node-forge - can try to move it's calls to content script.

[sosnovsky]

Yeah, looks like it won't be easy to remove openpgp from service worker (probably can work on it after finishing manifest v3 migration), but it seems to work well with your current code.
And what about node-forge, can it be removed from service worker?

[sosnovsky]

I think the only option we have is use webpack to pack libraries into background.js

Ok, let's go with webpack solution for now

[sosnovsky]

Do we have to fix this issue in forge and submit PR?

It seems node-forge isn't actively developed now, last release was a couple years ago.
Probably we can apply fixes to node-forge when building extension, like we currently do for openpgp and web-stream-tools, what do you think?

flowcrypt-browser/scripts/build.sh

Line 124 in 2bbe516

apply_regex_replace $STREAMS_REGEX$STREAMS_FILES

[sosnovsky]

Could you take a look at this issue? Would be much appreciated. I'll also look at this issue too though.

For sure, I'll check it later today or tomorrow

[sosnovsky]

@sosnovsky Have you checked?

Yes, I'm checking it - tried to build without webpack and noticed that forge.js isn't included in lib folder for service worker (only openpgp and streams files are there). I think that's why building with webpack doesn't help, as importing forge doesn't correctly references forge.js, checking it further now

[sosnovsky]

Maybe you didn't remove this comment in tsconfig.json?

Yeah, you're right, after removing this comment forge.js appeared in service worker, thanks for the tip!

[sosnovsky]

@sosnovsky I tried to build forge mjs file from forge repository with rollup bundler. It seems to work fine. What do you think about my solution?

For me it doesn't work - secure compose button isn't added to Gmail page and there are errors in console:

Does it work for you?

[sosnovsky]

@ioanmo226 did you try to use offscreen document for fixing openpgp and forge imports? Here is another article - https://developer.chrome.com/blog/Offscreen-Documents-in-Manifest-v3
Not sure if it can be used for this, but can be worth checking.