Skip to content

Fine grained filesystem setup - #31265

Merged
PVince81 merged 25 commits into
masterfrom
fs-limited-setup
Mar 24, 2022
Merged

Fine grained filesystem setup#31265
PVince81 merged 25 commits into
masterfrom
fs-limited-setup

Conversation

@icewind1991

@icewind1991icewind1991 commented Feb 18, 2022•
edited
Loading

Copy link
Copy Markdown
Member

Instead of always setting up the entire filesystem, it tries to only setup the bits needed for the path being accessed.

this is done by looking at the cached mounts in the oc_mounts table to find the one matching for the path (and optionally also any mounts in subfolders), getting the mountpoints from these mounts and only setting up the mounts from those providers.

Various systems are in place to try and ensure that this doesn't lead to an outdated setup if the mounts accessible to the user has changed

  • The setup logic listens to various core events that might cause mount changes (sharing, group membership changes, etc) and performs a full setup afterwards
  • A new event has been added that apps can use to trigger a full setup the next time for cases not covered by the above events
  • every 5min a full setup is trigger to ensure that even if something is missed the information isn't outdated for long

@icewind1991icewind1991 added the 2. developing Work in progress label Feb 18, 2022
@icewind1991icewind1991 added this to the Nextcloud 24 milestone Feb 18, 2022
This was referenced Feb 18, 2022
Merged
Merged
@icewind1991
icewind1991force-pushed the fs-limited-setup branch 4 times, most recently from f6106b3 to a924e07CompareMarch 2, 2022 16:22
@icewind1991
icewind1991force-pushed the fs-limited-setup branch 5 times, most recently from e6bd6c5 to 1842956CompareMarch 3, 2022 17:35
@icewind1991icewind1991 mentioned this pull request Mar 3, 2022
Merged
1 task
@icewind1991
icewind1991force-pushed the fs-limited-setup branch 5 times, most recently from e6c59b2 to 190fa2fCompareMarch 10, 2022 18:31
@icewind1991
icewind1991force-pushed the fs-limited-setup branch 3 times, most recently from 92e8fa3 to 22daf38CompareMarch 17, 2022 17:41
@icewind1991icewind1991 changed the title 🚧 Fine grained filesystem setup 🚧Fine grained filesystem setupMar 17, 2022
@icewind1991
icewind1991force-pushed the fs-limited-setup branch 2 times, most recently from 781a13b to 3c57be1CompareMarch 17, 2022 18:35
@icewind1991

icewind1991 commented Mar 17, 2022

Copy link
Copy Markdown
MemberAuthor

Ok, I think this is done for review/testing

Particular attention should be given to situations that change what mounts are available for a user (adding/removing shares/external storages/etc) and making sure file listings reflect these changes

@icewind1991icewind1991 added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Mar 17, 2022
@icewind1991
fix method name
63ad99b
Signed-off-by: Robin Appelman <robin@icewind.nl>
artonge
artonge reviewed Mar 24, 2022
View reviewed changes
Comment threadapps/files_external/lib/Service/StoragesService.php Outdated
Comment threadapps/files_external/lib/Service/StoragesService.php
Comment threadlib/private/Files/Config/MountProviderCollection.php Outdated
Comment threadlib/private/Files/Config/MountProviderCollection.php Outdated
Comment threadapps/files_sharing/lib/SharedMount.php
PVince81
PVince81 reviewed Mar 24, 2022
View reviewed changes
Comment threadlib/private/Files/SetupManager.php Outdated
@icewind1991
make mount cache duration configurable
b7a7425
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
invalidate mount cache after share rename
db2418b
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
add comment for getMountForPath loop
1179873
Signed-off-by: Robin Appelman <robin@icewind.nl>
artonge
artonge requested changes Mar 24, 2022
View reviewed changes
Comment threadlib/private/Files/Config/MountProviderCollection.php Outdated
Comment threadlib/private/Files/Config/MountProviderCollection.php Outdated
artonge
artonge reviewed Mar 24, 2022
View reviewed changes
Comment threadlib/private/Files/Config/MountProviderCollection.php Outdated
@icewind1991
Apply suggestions from code review
881e107
Co-authored-by: Louis <6653109+artonge@users.noreply.github.com>
Co-authored-by: Côme Chilliet <91878298+come-nc@users.noreply.github.com>
Signed-off-by: Robin Appelman <robin@icewind.nl>
PVince81
PVince81 approved these changes Mar 24, 2022
View reviewed changes

@PVince81PVince81 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@blizzzblizzz added 4. to release Ready to be released and/or waiting for tests to finish and removed 3. to review Waiting for reviews labels Mar 24, 2022
@PVince81

PVince81 commented Mar 24, 2022

Copy link
Copy Markdown
Member

@icewind1991 phpcs unhappy, can you fix ?

@blizzz

blizzz commented Mar 24, 2022

Copy link
Copy Markdown
Member

also s3 tests don't pass

@PVince81
Make phpcs happy in MountProviderCollection
1bc86a9
Signed-off-by: Vincent Petry <vincent@nextcloud.com>
@PVince81
Add missing event dispatcher in test constructor
d92c7bd
Signed-off-by: Vincent Petry <vincent@nextcloud.com>
@PVince81
Add missing PHPDoc in InvalidateMountCacheEvent
91ab4e1
Signed-off-by: Vincent Petry <vincent@nextcloud.com>
@PVince81

PVince81 commented Mar 24, 2022

Copy link
Copy Markdown
Member

all necessary tests passed successfully in previous builds and my recent changes are not related to those

the last commit was to satisfy "checkers" and that one passes now

merging...

@PVince81
PVince81 merged commit a7e778b into masterMar 24, 2022
@PVince81
PVince81 deleted the fs-limited-setup branch March 24, 2022 20:08
@nickvergessennickvergessen mentioned this pull request Jul 20, 2022
Closed
@PVince81PVince81 mentioned this pull request Aug 4, 2022
Open
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@come-nccome-nccome-nc left review comments
@PVince81PVince81PVince81 approved these changes
@artongeartongeartonge approved these changes
@PytalPytalPytal approved these changes
@CarlSchwanCarlSchwanAwaiting requested review from CarlSchwan
@ArtificialOwlArtificialOwlAwaiting requested review from ArtificialOwl

Assignees

No one assigned

Labels

4. to releaseReady to be released and/or waiting for tests to finishenhancementperformance 🚀

Projects

None yet

Milestone

Nextcloud 24

Development

Successfully merging this pull request may close these issues.

6 participants

@icewind1991@PVince81@blizzz@artonge@Pytal@come-nc