Experimental client: use bundle

[jku]

Use the MetadataBundle in the new Updater:

• Remove MetadataWrapper
• Update and expand tests
• Stop using tuf.settings, store the constants in updater_rework.py
• Use MetadataBundle to verify current set of metadata, remove all verification code from Updater
• load local metadata from disk
• persist new verified metadata to disk

Fixes#1304
Fixes#1320
Fixes#1319

[jku]

The constants could use some thought: We definitely don't want to keep using tuf.settings, but I guess the question is, should these be instance variables in Updater instead of module level ones:

MAX_ROOT_ROTATIONS = 32
MAX_DELEGATIONS = 32
DEFAULT_ROOT_MAX_LENGTH = 512000 # bytes
DEFAULT_TIMESTAMP_MAX_LENGTH = 16384 # bytes
DEFAULT_SNAPSHOT_MAX_LENGTH = 2000000 # bytes
DEFAULT_TARGETS_MAX_LENGTH = 5000000 # bytes

Maybe it does not hurt if they were instance variables instead. They don't even need to be optional constructor arguments (as initialization does not use any of them: caller can initialize the Updater and then modify the values).
I guess the fanciest option is that constructor takes an optional argument dataclass UpdaterOptions with those attributes ...

[MVrachev]

After I made consistent_snapshot optional for root here, I realized that the MetadataBundle was created before that.
Can we check that if consistent_snapshot is not provided everything will work with the update process?

[sechkova]

The constants could use some thought: We definitely don't want to keep using tuf.settings, but I guess the question is, should these be instance variables in Updater instead of module level ones:

MAX_ROOT_ROTATIONS = 32
MAX_DELEGATIONS = 32
DEFAULT_ROOT_MAX_LENGTH = 512000 # bytes
DEFAULT_TIMESTAMP_MAX_LENGTH = 16384 # bytes
DEFAULT_SNAPSHOT_MAX_LENGTH = 2000000 # bytes
DEFAULT_TARGETS_MAX_LENGTH = 5000000 # bytes

Maybe it does not hurt if they were instance variables instead. They don't even need to be optional constructor arguments (as initialization does not use any of them: caller can initialize the Updater and then modify the values).
I guess the fanciest option is that constructor takes an optional argument dataclass UpdaterOptions with those attributes ...

Given the proposed structure in #1397 (comment) where the idea is to expose only the Updater class in the client __init__ , probably the constants need to be inside the class (or the fancier options). However this can happen when solving #1397.

I left some minor comments but I in general I like how Updater looks now.

[sechkova]

This trick of mine with logging needs to be reverted now that we agreed on %s-strings in the logging calls.
But not in this PR. I'll note it for myself.

[sechkova]

Captured in #1400

[sechkova]

Using self._bundle.update_delegated_targets here leaves MetadataBundle.update_targets() unused.
Not a big deal but mentioning it.

[jku]

Addressed the resolved discussions. Also checked that consistent_snapshots=None seems to work fine (using a falsy test)

[joshuagl]

This looks very concise, nice. It took me a couple of reads to figure out why the fetching of metadata only happened in the exception handler of a load. That might want some attention in terms of comments and/or function names? 🤔

I also noticed there are some unhandled exceptions which I am not sure whether it is reasonable to leave those to a caller to handle or not. Would welcome thoughts on that.

Overall, very solid. LGTM.

[joshuagl]

Here, and other _load_*() functions, the exceptions raised by MetadataBundle.update_snapshot and the open and write calls in _persist_metadata become errors the user must handle. The former seems reasonable enough, I'm less certain about the latter. Especially when get_one_valid_targetinfo() -> _preorder_depth_first_walk() could result in a myriad of errors (download, file open/write, ???).

Probably a topic for #1312 ?

[jku]

yes I vote for handling in 1312. I think the three classes of exceptions (repository / download / file IO) are logical and also the granularity that I could see a serious client wanting to handle (or not handle as might be reasonable with e.g. file IO ).