New API: Add Key and Role classes by MVrachev · Pull Request #1360 · theupdateframework/python-tuf

MVrachev · 2021-04-21T15:34:49Z

Addresses, but doesn't fix: #1139

Description of the changes being introduced by the pull request:

In the top level metadata classes, there are complex attributes such as
meta in Targets and Snapshot, key and roles in Root etc.
We want to represent those complex attributes with a class to allow
easier verification and support for metadata with unrecognized fields.
For more context read ADR 0004 and ADR 0008 in the docs/adr folder.

The changes in this pr include:

addition of a new Key class integrated into Root
addition of a new Roles class integrated into
added support for unrecognized fields
added tests for unrecognized fields in Key and Roles

Please verify and check that the pull request fulfills the following
requirements:

The code follows the Code Style Guidelines
Tests have been added for the bug fix or new feature
Docs have been added for the bug fix or new feature

MVrachev · 2021-04-21T15:43:04Z

There is an interesting reason for the CI failure:
/home/runner/work/tuf/tuf/tuf/api/metadata.py:532:4: R0914: Too many local variables (18/15) (too-many-locals)

I used many local variables to make it clear what their purpose is.
Any suggestions? Should I disable this warning with an inline comment or?

mnm678 · 2021-04-21T21:51:29Z

There is an interesting reason for the CI failure:
/home/runner/work/tuf/tuf/tuf/api/metadata.py:532:4: R0914: Too many local variables (18/15) (too-many-locals)
I used many local variables to make it clear what their purpose is.
Any suggestions? Should I disable this warning with an inline comment or?

This is interesting. I think local variables generally make code more readable, so I'm in favor of disabling this warning.

jku · 2021-04-22T05:34:06Z

I used many local variables to make it clear what their purpose is.
Any suggestions? Should I disable this warning with an inline comment or?

I don't think the function looks horrible but ... you could make Role.from_dict() and Key.from_dict() -- this would fit the existing style and would probably remove half the variables.

jku

Thanks Martin. I have one potential bug (the keyids type issue which I'm not yet sure how to handle), the rest are just nits/suggestions/questions

jku · 2021-04-22T05:57:34Z

There are currently no Key/Role specific tests: We probably should have at least very basic ones if we now have objects?

One thing that might be nice to check for is invalid data -- what does the failure look like if the metadata is not valid (say a missing required field). I don't think this needs to be exhaustively tested but would be nice to see at least an example: I assume with this code we mostly get KeyErrors?

jku

Just one real issue: the keyids uniqueness checks and creation of the set still have a smell around them: I think the error isn't currently raised (when using from_dict()).

jku · 2021-04-26T13:34:47Z

+ # Add unrecognized fields to all metadata sub (helper) classes.
+ if metadata == "root":
+ for keyid in dict1["signed"]["keys"].keys():
+ dict1["signed"]["keys"][keyid]["d"] = "c"
+ for role_str in dict1["signed"]["roles"].keys():
+ dict1["signed"]["roles"][role_str]["e"] = "g"


this looks like it will become hard to maintain when more tests are added but I'm fine with it for now: can you make an issue about doing this in a more structured way (like automate the dictionary poisoning so that ever dictionary is injected, not just ones we list here)?

Yes, you are right. We should automate this.
In a discussion, you showed a prototype automating this process.
Will appreciate it if you propose these changes and we can discuss them, but I don't think this is in the scope of this pr.

MVrachev · 2021-04-27T10:03:29Z

Updated the pr by:

changing the type of keyds to a list inside the Role.__init__
don't make create a set in Role.from_dict and left the Role.__init__ to make that check
added a unit test case for creating a Role instance with duplicates in keyid.

In the top level metadata classes, there are complex attributes such as "meta" in Targets and Snapshot, "key" and "roles" in Root etc. We want to represent those complex attributes with a class to allow easier verification and support for metadata with unrecognized fields. For more context read ADR 0004 and ADR 0008 in the docs/adr folder. Signed-off-by: Martin Vrachev <mvrachev@vmware.com>

The from_dict() method simplifies the object creation. Signed-off-by: Martin Vrachev <mvrachev@vmware.com>

MVrachev · 2021-04-27T12:20:23Z

Rebased after #1363 was merged.

From the specification: "Clients MUST ensure that for any KEYID represented in this key list and in other files, only one unique key has that KEYID." The “only one unique key has that KEYID” is a requirement which can’t be achieved if two keyids are the same. So, in order to mandate that requirement it makes sense to use a set which will guarantee us the keyid’s uniqueness. Signed-off-by: Martin Vrachev <mvrachev@vmware.com>

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>

jku

I think this looks good. I did notice one unneeded "if" that I had missed earlier (adding duplicates to a set is safe).

jku · 2021-04-27T12:54:44Z

+ if keyid not in self.roles[role].keyids:
+ self.roles[role].keyids.add(keyid)


only realized this now but the check here is not needed with a set

Verify that adding an already existing key to keyid for a particular role in Root won't create duplicate key. Signed-off-by: Martin Vrachev <mvrachev@vmware.com>

MVrachev changed the title ~~Key role classes~~New API: Add Key and Role classesApr 21, 2021

MVrachev mentioned this pull request Apr 21, 2021
New metadata API: add MetaFile and TargetFile classes #1329
Merged
3 tasks

jku reviewed Apr 22, 2021
View reviewed changes

Comment threadtuf/api/metadata.py Outdated
Comment threadtuf/api/metadata.py Outdated
Comment threadtuf/api/metadata.py Outdated
Comment threadtuf/api/metadata.py Outdated
Comment threadtuf/api/metadata.py Outdated
Comment threadtuf/api/metadata.py Outdated
Comment threadtuf/api/metadata.py

MVrachev mentioned this pull request Apr 22, 2021
New metadata API: add support for ADR 0008 #1345
Merged
3 tasks

MVrachev force-pushed the key-role-classes branch from 26982f7 to 4f17315Compare April 26, 2021 11:50

jku requested changes Apr 26, 2021
View reviewed changes

MVrachev force-pushed the key-role-classes branch from 4f17315 to 7d6dc65Compare April 27, 2021 10:00

Martin Vrachev added 2 commits April 27, 2021 15:09

MVrachev force-pushed the key-role-classes branch from 7d6dc65 to e604c8bCompare April 27, 2021 12:14

Add from_dict methods in Key and Role classes
ef71c2d
The from_dict() method simplifies the object creation. Signed-off-by: Martin Vrachev <mvrachev@vmware.com>

MVrachev force-pushed the key-role-classes branch from e604c8b to ded92fcCompare April 27, 2021 12:19

Martin Vrachev added 3 commits April 27, 2021 15:26

Check the format of keyval in Root
0c9dc0e
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>

New API: Add Key/Role specific tests
54a535e
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>

MVrachev force-pushed the key-role-classes branch from ded92fc to 54a535eCompare April 27, 2021 12:27

jku approved these changes Apr 27, 2021
View reviewed changes

keyid: verify adding an existing key is ignored
1ce94b9
Verify that adding an already existing key to keyid for a particular role in Root won't create duplicate key. Signed-off-by: Martin Vrachev <mvrachev@vmware.com>

MVrachev force-pushed the key-role-classes branch from 68b76d3 to 1ce94b9Compare April 29, 2021 09:34

jku merged commit 765caf5 into theupdateframework:developApr 29, 2021

MVrachev deleted the key-role-classes branch April 29, 2021 13:02

		if keyid not in self.roles[role].keyids:
		self.roles[role].keyids.add(keyid)

Uh oh!

Conversation

MVrachev commented Apr 21, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MVrachev commented Apr 21, 2021

Uh oh!

mnm678 commented Apr 21, 2021

Uh oh!

jku commented Apr 22, 2021

Uh oh!

jku left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jku commented Apr 22, 2021

Uh oh!

jku left a comment

Choose a reason for hiding this comment

Uh oh!

jkuApr 26, 2021

Choose a reason for hiding this comment

Uh oh!

MVrachevApr 27, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MVrachev commented Apr 27, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MVrachev commented Apr 27, 2021

Uh oh!

jku left a comment

Choose a reason for hiding this comment

Uh oh!

jkuApr 27, 2021

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MVrachev commented Apr 21, 2021•
edited
Loading

MVrachev commented Apr 27, 2021•
edited
Loading