Skip to content

ngclient: simplify storing a downloaded file - #1799

Merged
lukpueh merged 1 commit into
theupdateframework:developfrom
MVrachev:rm-persist_temp_file
Jan 27, 2022
Merged

ngclient: simplify storing a downloaded file#1799
lukpueh merged 1 commit into
theupdateframework:developfrom
MVrachev:rm-persist_temp_file

Conversation

@MVrachev

@MVrachevMVrachev commented Jan 26, 2022

Copy link
Copy Markdown
Collaborator

Fixes#1761

Description of the changes being introduced by the pull request:

Replace the usage of securesystemslib.util.persist_temp_file() with
shutil.copyfileobj() as file system abstraction is not used in the
client.
This way we prevent securesystemslib.exception.StorageError from
leaking through client API calls.

Note: with those changes, we are no longer do fsync.

Signed-off-by: Martin Vrachev mvrachev@vmware.com

Please verify and check that the pull request fulfills the following
requirements
:

  • The code follows the Code Style Guidelines
  • Tests have been added for the bug fix or new feature
  • Docs have been added for the bug fix or new feature

@coveralls

coveralls commented Jan 26, 2022

Copy link
Copy Markdown

Pull Request Test Coverage Report for Build 1756649124

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 4 of 4(100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.9%) to 98.608%

TotalsCoverage Status
Change from base Build 1756435191:0.9%
Covered Lines:3938
Relevant Lines:3965

💛 - Coveralls

jku
jku approved these changes Jan 26, 2022

@jkujku left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks correct to me.

To make it more obvious you could mention in the commit message that this means we no longer do fsync. It believe that's fine in our use case but could be worth mentioning.

@MVrachev
MVrachevforce-pushed the rm-persist_temp_file branch from ab1aabd to fec5bf1CompareJanuary 26, 2022 17:11
@MVrachevMVrachev mentioned this pull request Jan 27, 2022
3 tasks
@MVrachev
MVrachevforce-pushed the rm-persist_temp_file branch from fec5bf1 to fda0b1bCompareJanuary 27, 2022 12:50
@MVrachev

Copy link
Copy Markdown
CollaboratorAuthor

@jku I updated pr, so it changes the exceptions docstring inside updater.download_target().

Additionally, I made it dependent on #1787 to make sure we end up with the correct exception docstring for updater.download_target().

@jkujku left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is correct, left a comment about the docstring

Comment threadtuf/ngclient/updater.py Outdated
RepositoryError: Downloaded target failed to be verified in some way
exceptions.StorageError: Downloaded target could not be written
to disk
OSError: Cannot open "filepath" for writing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strictly speaking there are quite a few reasons for OSError here, not just failing to open file (like not enough space as an example).

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the message.
Do you think that now is better?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure there was anything wrong with the original message? Alternatively "Failed to write target to file" could work.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understood you had some reservations about it.
I updated the pr with your suggestion.

@MVrachev
MVrachevforce-pushed the rm-persist_temp_file branch from fda0b1b to 8c4b171CompareJanuary 27, 2022 13:39
@MVrachev
MVrachev requested a review from jkuJanuary 27, 2022 13:40
@MVrachev
MVrachevforce-pushed the rm-persist_temp_file branch from 8c4b171 to 88e9afeCompareJanuary 27, 2022 14:15
Replace the usage of securesystemslib.util.persist_temp_file() with
shutil.copyfileobj() as file system abstraction is not used in the
client.
This way we prevent securesystemslib.exception.StorageError from
leaking through client API calls.
Note: with those changes we are no longer do fsync.
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
@MVrachev
MVrachevforce-pushed the rm-persist_temp_file branch from 88e9afe to 3fa0668CompareJanuary 27, 2022 14:15
jku
jku approved these changes Jan 27, 2022

@jkujku left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks.

If another maintainer wants to chime in on the issue of not using persist_temp_file() (and as a result, not fsyncing) that would be welcome: I've tried to think the failure scenarios through and this seems like the simplest code that does what we want.

@lukpueh
lukpueh merged commit 4de5617 into theupdateframework:developJan 27, 2022
@MVrachev
MVrachev deleted the rm-persist_temp_file branch January 27, 2022 16:07
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ngclient: throws securesystemslib errors

4 participants

@MVrachev@coveralls@jku@lukpueh