Multiple projects storage foundation - #1327
Open
microbit-matt-hillsdon wants to merge 6 commits into
Open
Conversation
Groundwork for a project library: opening a project will swap the FileSystem's backing storage rather than replace files one by one. switchStorage waits for any in-flight initialisation, makes the new storage the record, refills the hex file system from it and bumps the version of every file it holds so an editor showing a same-named file reloads it. Versions increment rather than reset because the editor keys on name plus version; a reset to 1 could collide with the old project's version and leave stale content on screen.
The editor's file system is now mirrored to IndexedDB rather than session storage, as the first step towards a library of projects. Behaviour is unchanged for the user: one project per tab, and a new tab gets a new project. - projects-db.ts: the library. Two stores, project metadata and files keyed by [projectId, name], so listing is cheap and a change writes only the files it touched. The database name includes the base path because production, beta and review builds share an origin. Opening asserts the stores exist so an old build fails clearly against a database a newer one created. - indexeddb-storage.ts: an FSStorage for one project, used as the SplitStrategyStorage secondary. Keystroke writes are coalesced per file and flushed in one transaction after a short delay and when the page is hidden. A failed flush is reported, not thrown, and its changes dropped: the in-memory primary still has them and retrying forever against a full quota helps nobody. - current-project.ts: which project a tab opens. The tab's current project id lives in session storage. A session-storage file system from before this change is migrated into the library on first load, so a reload after deploy lands in the user's work; only the file system keys are removed, since session settings live there too. Without IndexedDB, or with an incompatible database, the editor falls back to session storage as before. - SplitStrategyStorage accepts a promise of its secondary, since opening IndexedDB is asynchronous and the Host API is not. The shared FSStorage tests move to storage-tests.ts so the new storage runs them too, against fake-indexeddb.
Review builds are internal, and a project made on one branch is useful on the next, so they share a database rather than each having their own by base path. Production and beta keep separate libraries. The cost is that a schema change can leave the shared library incompatible with an older build. On non-public stages that now shows a "Breaking change to stored data" page offering to clear the library and reload, as ml-trainer does, instead of quietly falling back to session storage. Public stages keep the quiet fallback. The outcome of opening the library is held in a small external store, since storage opens at module load before React mounts.
|
Preview build will be at |
…prompt The dirty flag means "changed since the last hex save" and exists only to warn before work is lost: the before-unload prompt and the replace-project confirmation are its only readers. Neither applies to a project in the database, which outlives the tab, so the flag is no longer stored there and IndexedDBFSStorage always reports not dirty. Session storage keeps it and FileSystem still tracks it for the tab, so the replace confirmation works for edits made in this tab until the replace flows go. openCurrentProjectStorage now reports through storage-status when the projects database is active, and BeforeUnloadDirtyCheck registers nothing in that case. The session-storage fallback and iframe mode keep the prompt. The edits e2e spec now asserts the prompt is absent and that text typed just before a reload survives via the pagehide flush. The old helper proved nothing: Playwright accepts an unlistened beforeunload dialog itself, so the new one listens for it.
…er mode The three storage modes now behave differently and only the projects database path was exercised in the browser. storage-errors.test.ts uses a new noIndexedDB fixture option, an init script that hides indexedDB before the app loads, and checks the fallback keeps the before-unload prompt and survives a reload. iframe.test.ts embeds the editor with controller=1 and checks the workspacesync, workspaceloaded, workspacesave and importproject messages and the prompt. The host page is an HTML string in the spec, served by intercepting a request for it, so it is never part of the build.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note: targets new home-page integration branch as part of a multi-step plan.
The file system is now mirrored to IndexedDB instead of session storage.
projects-db.ts: two stores, project metadata and files keyed by[projectId, name], so listing is cheap and a change writes only the files it touched. Opening asserts the stores exist and throwsVersionErrorotherwise.indexeddb-storage.ts: anFSStoragefor one project, used as theSplitStrategyStoragesecondary. Keystroke writes coalesce per file and flush as one transaction after 300ms and when the page is hidden. A failed flush is reported, not thrown, and its changes dropped: the in-memory primary still has them and retrying against a full quota helps nobody.current-project.ts: which project a tab opens. The current id lives in session storage, so a project is still per tab. A session-storage project is migrated on first load so a reload after deploy lands in the user's work; only thefs/keys are removed, since session settings share that storage. Without IndexedDB, or with an incompatible database on a public stage, the editor falls back to session storage as before.SplitStrategyStorageaccepts a promise for its secondary, since opening IndexedDB is asynchronous and the Host API is not.Production and beta get separate databases by base path (
python-editor/v/3,python-editor/v/beta). Review builds sharepython-editor-review: they are internal and a project from one branch is useful on the next. If/when we hit a schema change, non-public stages show a "Breaking change to stored data" page with clear-and-reload, which is useful during development.Adds
FileSystem.switchStorage. Opening a project will swap the file system's backing storage rather than replace files one by one. Waits for any in-flight initialise, makes the new storage the record, refills the hex file system, bumps every file's version so a same-named file reloads in the editor, and notifies. Not yet called outside tests.User-visible behaviour on this branch
The difference is that a project now survives the tab closing. Nothing lists the projects yet. This is obviously weird hence the integration branch, but a step in the right direction!
Known issues, to be resolved by later plan steps
#project:link (microbit.org "open in Python Editor") still replaces the current project's files, as it does today. Later work will make it create a new project.Logging.error. The toast comes with the pages.