Store severity scores by sbs2001 · Pull Request #290 · aboutcode-org/vulnerablecode

sbs2001 · 2020-11-29T10:42:36Z

Implementation of #157 (comment)

Fixes#157

Add data structures to store severity scores

pombredanne

Thanks!
See my comments in line for your consideration.

pombredanne

See my suggested changes for the models.

pombredanne · 2020-12-17T19:33:58Z

@sbs2001 you wrote:

I thought about #290 (comment) , and I think storing the ScoringSystems in the db would be a better approach

It can but then how scoring systems will we have? At most a couple per data sources, and in reality only a handful. And this is purely static data, and not really usable as data for some query except for the identifier so I am not sure this needs the added ceremony and complexity when stored in the DB. (e.g. you then need fixtures to bootstrap etc.). I see this instead as an enhanced but still very small list of value choices for a single field.

pombredanne

Thanks!
See my comments inline

* Correct typo in severity_systems.py * Use typo for scoring fields instead of list in models.py Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* Correct typo in severity_systems.py * Use typo for scoring fields instead of list in models.py * Handle absence of bugzilla and RHSA better in redhat.py Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

pombredanne

Almost there ! see my comments inline for your consideration.

A VulnerabilitySeverity models is added in models.py to store severity of vulnerability. Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

A dataclass `VulnerabilitySeverity` is added to enable to transport of severity scores. The logic in importer_runner.py is modified to store, update severity scores and link it to reference and vulnerability Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

scoring_system_identifier is changed to scoring_system Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* Correct typo in severity_systems.py * Use typo for scoring fields instead of list in models.py * Handle absence of bugzilla and RHSA better in redhat.py Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

pombredanne · 2021-02-09T12:41:26Z

+ bugzilla = advisory_data.get("bugzilla")
+ if bugzilla:
+ url = "https://bugzilla.redhat.com/show_bug.cgi?id={}".format(bugzilla)
+ bugzilla_data = requests.get(f"https://bugzilla.redhat.com/rest/bug/{bugzilla}").json()


Just as a side note, this is the kind of JSON we would likely need to store forever as back auditable evidence when we will do this later ... which likely calls for a central place where we fetch things from

pombredanne · 2021-02-09T12:43:29Z

+ max_length=50,
+ choices=scoring_system_choices,
+ help_text="Identifier for the scoring system used. Available choices are: {} ".format(
+ ", ".join(


For readability, an intermediate variable would be better for this

pombredanne · 2021-02-09T12:45:11Z

 namespace="redhat",
 name="bash",
 version="4.1.2-48.el6",
- qualifiers=OrderedDict(),


Feel free to remove OrderedDict from the whole codebase BTW since the dicts are always ordered now.

pombredanne

LGTM! let's merge.

* Correct typo in severity_systems.py * Use typo for scoring fields instead of list in models.py * Handle absence of bugzilla and RHSA better in redhat.py Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

sbs2001 requested a review from pombredanne November 29, 2020 10:42

sbs2001 changed the title ~~Store severity scores~~ [WIP]Store severity scores Nov 29, 2020

pombredanne requested changes Dec 1, 2020
View reviewed changes

pombredanne reviewed Dec 2, 2020
View reviewed changes

Comment threadvulnerabilities/.vscode/settings.json Outdated

pombredanne reviewed Dec 11, 2020
View reviewed changes

Comment threadvulnerabilities/data_source.py Outdated
Comment threadvulnerabilities/data_source.py

pombredanne reviewed Dec 11, 2020
View reviewed changes

Comment threadvulnerabilities/models.py Outdated

sbs2001 force-pushed the store_severity branch from efcf6df to f6371c6Compare December 18, 2020 09:23

sbs2001 mentioned this pull request Dec 19, 2020
REST API: Bulk requests for packages and vulnerabilities #284
Closed

sbs2001 force-pushed the store_severity branch 3 times, most recently from 0118d9e to b24c934Compare December 24, 2020 09:50

sbs2001 changed the title ~~[WIP]Store severity scores~~ Store severity scores Dec 24, 2020

sbs2001 force-pushed the store_severity branch 3 times, most recently from d6b2bee to 9a72e52Compare January 24, 2021 13:03

sbs2001 requested a review from pombredanne January 26, 2021 04:20

pombredanne requested changes Jan 27, 2021
View reviewed changes

sbs2001 mentioned this pull request Jan 29, 2021
Make RedHat CVE import more robust #319
Merged

sbs2001 force-pushed the store_severity branch from 9a72e52 to b0aeef5Compare January 30, 2021 11:14

sbs2001 force-pushed the store_severity branch from 2656e76 to 783292bCompare January 31, 2021 11:09

pombredanne reviewed Feb 2, 2021
View reviewed changes

Comment threadvulnerabilities/import_runner.py

pombredanne requested changes Feb 2, 2021
View reviewed changes

sbs2001 added 5 commits February 3, 2021 14:19

Add db models to store severity scores of vulnerabilities
4fc018f
A VulnerabilitySeverity models is added in models.py to store severity of vulnerability. Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Store severity scores given by NVD.
221bc12
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Store severity scores given by RedHat
52724a5
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Remove VulnerabilityReferenceInserter class
98e4082
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

sbs2001 added 16 commits February 3, 2021 14:19

Make code review changes
9fc7ec8
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Avoid unnnecessary init of default args in test_nvd.py
b0ae6a8
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Delete vscode config and add it to gitignore
5d22856
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Rename scores to severities in the advisory dataclass
190440b
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Improve order of fields in Reference dataclass
ee7e509
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Store severity scores given by NVD.
9ca5558
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Store severity scores given by RedHat
0887e83
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Misc Data Structure changes
86f2e4f
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Refactor tests and importers to use new data structure field names
e8e6166
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Rebase and resolve conflicts
c9df2f9
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Create and use ScoringSystem objects for handling severity
0312cfb
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Cast score value to string while inserting it into db
641b457
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Change field naming in VulnerabilitySeverity model
be24df0
scoring_system_identifier is changed to scoring_system Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Make PR review changes for PRaboutcode-org#290
5428b37
* Correct typo in severity_systems.py * Use typo for scoring fields instead of list in models.py * Handle absence of bugzilla and RHSA better in redhat.py Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Add scoring system for vectors
d78cd0f
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

Fix tests
8d19463
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

sbs2001 force-pushed the store_severity branch from c737b5a to 8d19463Compare February 3, 2021 08:50

sbs2001 requested a review from pombredanne February 3, 2021 11:29

pombredanne reviewed Feb 9, 2021
View reviewed changes

pombredanne approved these changes Feb 9, 2021
View reviewed changes

Merge branch 'main' into store_severity
bea7f16

sbs2001 merged commit 1d7b64e into aboutcode-org:mainFeb 9, 2021

sbs2001 mentioned this pull request Feb 14, 2021
Change API to adapt to new models. #351
Closed

sbs2001 mentioned this pull request Apr 26, 2021
Essential modifications required in Models. #22
Closed

Uh oh!

Uh oh!

Conversation

sbs2001 commented Nov 29, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pombredanne commented Dec 17, 2020

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pombredanneFeb 9, 2021

Choose a reason for hiding this comment

Uh oh!

pombredanneFeb 9, 2021

Choose a reason for hiding this comment

Uh oh!

pombredanneFeb 9, 2021

Choose a reason for hiding this comment

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sbs2001 commented Nov 29, 2020•
edited
Loading