Skip to content

s3 external storage fixes - #29220

Merged
skjnldsv merged 11 commits into
masterfrom
s3-external-list
Oct 22, 2021
Merged

s3 external storage fixes#29220
skjnldsv merged 11 commits into
masterfrom
s3-external-list

Conversation

@icewind1991

@icewind1991icewind1991 commented Oct 13, 2021•
edited
Loading

Copy link
Copy Markdown
Member
  • Fix "delete markers" causing ghost folders
  • More efficient file scanning
  • Make directory copy a bit more reliable
  • More reliable detection of changed folder contents
  • CI against 2 different "s3 compatible" implementations and with versioning enabled and disabled

@icewind1991icewind1991 added the 2. developing Work in progress label Oct 13, 2021
@icewind1991
icewind1991 marked this pull request as draft October 13, 2021 17:43
@icewind1991
icewind1991force-pushed the s3-external-list branch 10 times, most recently from a851448 to 3eed59dCompareOctober 14, 2021 18:00
@icewind1991icewind1991 changed the title s3 external storage listing reworks3 external storage fixesOct 15, 2021
@icewind1991
s3 external storage listing rework
09ffac5
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
doesDirectoryExist fixes
5e3c8b3
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
more reliable directory copy
294af42
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
ci
294b218
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
optimize filetype for s3 directories a bit
d3bd0b5
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
remove old migration method
3463769
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
more reliable hasUpdated for s3
4bd08af
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
more reliable return value for Watcher::checkUpdate
55346b5
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
always set Key field in headObject
247e12d
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991

icewind1991 commented Oct 15, 2021

Copy link
Copy Markdown
MemberAuthor

Ok, this should be good now.

It grew a bit in scope outside the "fix ghost folders" in order to have ci be happy in all cases but those changes should be good to have anyway

@icewind1991
icewind1991 marked this pull request as ready for review October 15, 2021 15:19
@icewind1991icewind1991 added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Oct 15, 2021
@icewind1991icewind1991 added this to the Nextcloud 23 milestone Oct 15, 2021
@PVince81PVince81 mentioned this pull request Oct 18, 2021
Closed
@icewind1991
minor directory detect improvements
eb6e6e3
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991
cache versioning enabled status
a1ca901
Signed-off-by: Robin Appelman <robin@icewind.nl>
PVince81
PVince81 reviewed Oct 21, 2021
View reviewed changes
Comment threadapps/files_external/lib/Lib/Storage/AmazonS3.php
/** @var bool|null */
private $versioningEnabled = null;

/** @var IMemcache */

@PVince81PVince81Oct 21, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make that ICache ?

PVince81
PVince81 approved these changes Oct 21, 2021
View reviewed changes

@PVince81PVince81 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

mind having a look at the code scanning issues before merging ?

@skjnldsvskjnldsv added 4. to release Ready to be released and/or waiting for tests to finish bug and removed 3. to review Waiting for reviews labels Oct 22, 2021
@skjnldsv
skjnldsv merged commit 2be0eda into masterOct 22, 2021
@skjnldsv
skjnldsv deleted the s3-external-list branch October 22, 2021 10:06
@PVince81

PVince81 commented Oct 22, 2021

Copy link
Copy Markdown
Member

should we limit backport of this to stable22 ?
it's a rather big impactful change

@skjnldsv

skjnldsv commented Oct 22, 2021

Copy link
Copy Markdown
Member

Ok! 👍

@skjnldsv

skjnldsv commented Oct 22, 2021

Copy link
Copy Markdown
Member

/backport to stable22

@backportbot-nextcloudbackportbot-nextcloudBot mentioned this pull request Oct 22, 2021
Merged
@skjnldsvskjnldsv mentioned this pull request Oct 25, 2021
Merged
25 tasks
@icewind1991

icewind1991 commented Oct 26, 2021

Copy link
Copy Markdown
MemberAuthor

/backport to stable21

@backportbot-nextcloudbackportbot-nextcloudBot mentioned this pull request Oct 26, 2021
Merged
@CarlSchwanCarlSchwan mentioned this pull request Oct 29, 2021
Closed
This was referenced Nov 2, 2021
Closed
Closed
@PVince81PVince81 mentioned this pull request Feb 14, 2022
Closed
joshtrichards added a commit that referenced this pull request Mar 1, 2026
@joshtrichards
refactor(files_external/S3): drop unused methods
08b465d
getContentLength() and getLastModified() are dead code. Originally added in PR #11518 and made defunct in PR #29220.
Signed-off-by: Josh <josh.t.richards@gmail.com>
@joshtrichardsjoshtrichards mentioned this pull request Mar 1, 2026
Merged
9 tasks
chandrika1993 added a commit to chandrika1993/server that referenced this pull request Mar 10, 2026
@chandrika1993
refactor(files_external/S3): drop unused methods
64aff5e
getContentLength() and getLastModified() are dead code. Originally added in PR nextcloud#11518 and made defunct in PR nextcloud#29220.
Signed-off-by: Josh <josh.t.richards@gmail.com>
Signed-off-by: Chandrika Mohan <chandrikalov@gmail.com>
@dwsteeledwsteele mentioned this pull request Jun 24, 2026
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PVince81PVince81PVince81 approved these changes
@skjnldsvskjnldsvskjnldsv approved these changes

Assignees

No one assigned

Labels

4. to releaseReady to be released and/or waiting for tests to finishbug

Projects

None yet

Milestone

Nextcloud 23

Development

Successfully merging this pull request may close these issues.

3 participants

@icewind1991@PVince81@skjnldsv