New/exception classes - #3

Merged
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes
Aug 15, 2019
Merged

New/exception classes#3
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes

Conversation

@Script-Nomad

Copy link
Copy Markdown
Collaborator

I had a need for this in order to implement my own parser for nessus's xml output and ran into some trouble that prompted me to implement a few changes to the library. The most significant change was the creation of a new exceptions class file in the base of the library (because it should be accessible from a base import).

The purpose of the exception class file is to introduce custom exceptions for the library which should be created for edge-case scenarios such as the one I encountered while parsing a nessus file. Specifically the error was caused by a configuration change in the scan for nessus to not scan the local host. This apparently creates a unique pluginID that is missing most of the ordinarily common and essential attributes that libnessus requires.

Aside from this tweak, it's mostly just adding some new functionality (like getting the CVE as a string) and some minor spelling mistakes I encountered while casually making changes.

Thanks in advance for reviewing this and your work on this library. Extremely useful stuff. 👍

It's always preferable to create custom exceptions for weird edge-cases
like missing attributes from the Report object. This is esspecially
helpful in cases where certain nessus plugins don't return all of the
attributes that are necessary, and gives users a straight-forward way to
handle and pass the exception accordingly.
implemented exception handling for missing attributes
plus fixed some minor spelling errors.
Comment threadlibnessus/parser.py Outdated
@bmx0r

Copy link
Copy Markdown
Owner

Thanks a lot for you contribution to this project.
do you mind doing two things:

  • check my comment as i believe you made a typo :)
  • change travis.yml
-- pip install pep8 --use-mirrors
-- pip install pyflakes --use-mirrors
+- pip install pep8
+- pip install pyflakes

we should then have a partial green build
And then i will probably merge :)

@Script-Nomad

Script-Nomad commented Nov 20, 2018

Copy link
Copy Markdown
CollaboratorAuthor

Alright. Changes are pushed. Thanks for catching that typo. Travis doesn't seem to like the pep8 . check though. 🤷‍♂️

There is another issue I encountered with a separate Nessus file that resulted in similar issues that cause the parser to fail. This is because it can't find all the necessary attributes, even though it would normally be handled by for report_item in root.findall("ReportItem") here.
https://github.com/bmx0r/python-libnessus/blob/master/libnessus/parser.py#L75

The problem is that this area of the code is meant to be transparent to the user, or at least I assume so based on the number of private method calls. Raising the error here won't really have any useful effect for the user if they create a NessusReport object, which calls the parser...which parses the hosts...which parses the items, and so on.

The issue is that there is at least one "Info" plugin that is missing these required attributes by design, and there may be more in the future. Ultimately, I think it would be more ideal to accept all ReportItems regardless of what attributes are missing and let the user decide whether or not to accept them or throw them away accordingly by raising the MissingAttribute exception to the top when the user tries to access an item property that is empty.

This is my temporary workaround just so that I can parse the file and throw away the invalid ReportItems:

 for report_item in root.findall("ReportItem"):
try:
_new_item = cls.parse_reportitem(report_item)
except MissingAttribute:
continue
_vuln_list.append(_new_item)

Obviously not a permanent solution. I'm willing to help change the implementation, but I'll wait to know how you would want to handle the issue.

  1. Would you rather accept bad/empty attributes, or throw away bad ReportItems and risk missing details?
  2. Would you want to warn the user in either case and how would you want to handle that?

I've noticed that in every case, the problematic plugin is the "You didn't set Credentials" Info finding which is a bit annoying. It's actually completely empty, except for the name and description. So far, it looks like this is the only problematic plugin.

annoying_nessus_plugin

Curse you Tenable -.-

@bmx0r

Copy link
Copy Markdown
Owner

Hey,
Sorry for this late answer...
You can solve this has it better suit you, but it would be nice to put at least a warning message.
Would you like to do this in the same PR? or will you open another one?
Regards

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Hey, no worries. Sorry that I'm a bit late too. I can work on this and should be able to solve the issues of encountering empty plugins. I think the best solution is to catch any exceptions due to incomplete plugins and warn the user. I have a feeling that if I push this PR now that it will likely break things down the line, so I will fix things here when I have the time.

Implemented pythonic log parser
Added debug logging + PEP 257 Docstring changes
Implementing libnessus logging
Edited to PEP 257 compliant docstrings
Implemented "Strict" parsing option and log warnings to alert users when invalid report items are encountered
@Script-Nomad

Script-Nomad commented Jun 4, 2019

Copy link
Copy Markdown
CollaboratorAuthor

Hi there @bmx0r Sorry this sat for so long, but I finally managed to gather some time to implement some changes that I think are a reasonable approach to dealing with the Bad/MissingAttributes situation. In order to prevent existing codebases that depend on libnessus from breaking, I implemented an optional "strict" boolean argument to all of the parser functions which get passed down to the private functions. By default, it is set to false. As long as strict parsing is false, it will simply warn users when it encounters an invalid ReportItem object due to missing attributes. Users may explicitly set strict parsing to True, which will throw an exception and force the user to deal with it as they please.

So far my testing of this has not introduced any other problems and works as intended. Unfortunately, I am not at all familiar with Travis-CI and cannot get it to behave. As far as I can tell, the code meets all the reasonable PEP8 standards, and works on Python 3.3+. I have not tested any versions prior to that. If it is explicitly required that all of the Travis-CI build checks pass, let me know, but otherwise, I believe this is good to go. If any specific changes are needed, mark them and I'll be happy to address.

Cheers 🍻

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Fixed the Travis-CI build config.
As it turns out, pyflakes no longer works with python<=2.6 or python<=3.3, so I had to remove them from the config.
Pep8 checks still cause the build to fail because of issues in some of the imported modules, rather than the actual libnessus code, so they have been omitted.
The libnessus code itself is still pep8 compliant. 👍

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

@bmx0r Just wanted to drop a reminder in here in case this hasn't been seen yet. If changes are needed, let me know. Would be nice if this could get merged into pypi so that a local fork isn't needed anymore. 😆

@bmx0r
bmx0r merged commit 88bdf57 into bmx0r:masterAug 15, 2019
@bmx0r

Copy link
Copy Markdown
Owner

and finnaly: https://pypi.org/project/python-libnessus/
Sorry for the delay ;)
A thanks you a lot for the contribution :)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Script-Nomad@bmx0r
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

New/exception classes - #3

Merged
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes
Aug 15, 2019
Merged

New/exception classes#3
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes

Conversation

@Script-Nomad

Copy link
Copy Markdown
Collaborator

I had a need for this in order to implement my own parser for nessus's xml output and ran into some trouble that prompted me to implement a few changes to the library. The most significant change was the creation of a new exceptions class file in the base of the library (because it should be accessible from a base import).

The purpose of the exception class file is to introduce custom exceptions for the library which should be created for edge-case scenarios such as the one I encountered while parsing a nessus file. Specifically the error was caused by a configuration change in the scan for nessus to not scan the local host. This apparently creates a unique pluginID that is missing most of the ordinarily common and essential attributes that libnessus requires.

Aside from this tweak, it's mostly just adding some new functionality (like getting the CVE as a string) and some minor spelling mistakes I encountered while casually making changes.

Thanks in advance for reviewing this and your work on this library. Extremely useful stuff. 👍

It's always preferable to create custom exceptions for weird edge-cases
like missing attributes from the Report object. This is esspecially
helpful in cases where certain nessus plugins don't return all of the
attributes that are necessary, and gives users a straight-forward way to
handle and pass the exception accordingly.
implemented exception handling for missing attributes
plus fixed some minor spelling errors.
Comment threadlibnessus/parser.py Outdated
@bmx0r

Copy link
Copy Markdown
Owner

Thanks a lot for you contribution to this project.
do you mind doing two things:

  • check my comment as i believe you made a typo :)
  • change travis.yml
-- pip install pep8 --use-mirrors
-- pip install pyflakes --use-mirrors
+- pip install pep8
+- pip install pyflakes

we should then have a partial green build
And then i will probably merge :)

@Script-Nomad

Script-Nomad commented Nov 20, 2018

Copy link
Copy Markdown
CollaboratorAuthor

Alright. Changes are pushed. Thanks for catching that typo. Travis doesn't seem to like the pep8 . check though. 🤷‍♂️

There is another issue I encountered with a separate Nessus file that resulted in similar issues that cause the parser to fail. This is because it can't find all the necessary attributes, even though it would normally be handled by for report_item in root.findall("ReportItem") here.
https://github.com/bmx0r/python-libnessus/blob/master/libnessus/parser.py#L75

The problem is that this area of the code is meant to be transparent to the user, or at least I assume so based on the number of private method calls. Raising the error here won't really have any useful effect for the user if they create a NessusReport object, which calls the parser...which parses the hosts...which parses the items, and so on.

The issue is that there is at least one "Info" plugin that is missing these required attributes by design, and there may be more in the future. Ultimately, I think it would be more ideal to accept all ReportItems regardless of what attributes are missing and let the user decide whether or not to accept them or throw them away accordingly by raising the MissingAttribute exception to the top when the user tries to access an item property that is empty.

This is my temporary workaround just so that I can parse the file and throw away the invalid ReportItems:

 for report_item in root.findall("ReportItem"):
try:
_new_item = cls.parse_reportitem(report_item)
except MissingAttribute:
continue
_vuln_list.append(_new_item)

Obviously not a permanent solution. I'm willing to help change the implementation, but I'll wait to know how you would want to handle the issue.

  1. Would you rather accept bad/empty attributes, or throw away bad ReportItems and risk missing details?
  2. Would you want to warn the user in either case and how would you want to handle that?

I've noticed that in every case, the problematic plugin is the "You didn't set Credentials" Info finding which is a bit annoying. It's actually completely empty, except for the name and description. So far, it looks like this is the only problematic plugin.

annoying_nessus_plugin

Curse you Tenable -.-

@bmx0r

Copy link
Copy Markdown
Owner

Hey,
Sorry for this late answer...
You can solve this has it better suit you, but it would be nice to put at least a warning message.
Would you like to do this in the same PR? or will you open another one?
Regards

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Hey, no worries. Sorry that I'm a bit late too. I can work on this and should be able to solve the issues of encountering empty plugins. I think the best solution is to catch any exceptions due to incomplete plugins and warn the user. I have a feeling that if I push this PR now that it will likely break things down the line, so I will fix things here when I have the time.

Implemented pythonic log parser
Added debug logging + PEP 257 Docstring changes
Implementing libnessus logging
Edited to PEP 257 compliant docstrings
Implemented "Strict" parsing option and log warnings to alert users when invalid report items are encountered
@Script-Nomad

Script-Nomad commented Jun 4, 2019

Copy link
Copy Markdown
CollaboratorAuthor

Hi there @bmx0r Sorry this sat for so long, but I finally managed to gather some time to implement some changes that I think are a reasonable approach to dealing with the Bad/MissingAttributes situation. In order to prevent existing codebases that depend on libnessus from breaking, I implemented an optional "strict" boolean argument to all of the parser functions which get passed down to the private functions. By default, it is set to false. As long as strict parsing is false, it will simply warn users when it encounters an invalid ReportItem object due to missing attributes. Users may explicitly set strict parsing to True, which will throw an exception and force the user to deal with it as they please.

So far my testing of this has not introduced any other problems and works as intended. Unfortunately, I am not at all familiar with Travis-CI and cannot get it to behave. As far as I can tell, the code meets all the reasonable PEP8 standards, and works on Python 3.3+. I have not tested any versions prior to that. If it is explicitly required that all of the Travis-CI build checks pass, let me know, but otherwise, I believe this is good to go. If any specific changes are needed, mark them and I'll be happy to address.

Cheers 🍻

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Fixed the Travis-CI build config.
As it turns out, pyflakes no longer works with python<=2.6 or python<=3.3, so I had to remove them from the config.
Pep8 checks still cause the build to fail because of issues in some of the imported modules, rather than the actual libnessus code, so they have been omitted.
The libnessus code itself is still pep8 compliant. 👍

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

@bmx0r Just wanted to drop a reminder in here in case this hasn't been seen yet. If changes are needed, let me know. Would be nice if this could get merged into pypi so that a local fork isn't needed anymore. 😆

@bmx0r
bmx0r merged commit 88bdf57 into bmx0r:masterAug 15, 2019
@bmx0r

Copy link
Copy Markdown
Owner

and finnaly: https://pypi.org/project/python-libnessus/
Sorry for the delay ;)
A thanks you a lot for the contribution :)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Script-Nomad@bmx0r
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

New/exception classes - #3

Merged
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes
Aug 15, 2019
Merged

New/exception classes#3
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes

Conversation

@Script-Nomad

Copy link
Copy Markdown
Collaborator

I had a need for this in order to implement my own parser for nessus's xml output and ran into some trouble that prompted me to implement a few changes to the library. The most significant change was the creation of a new exceptions class file in the base of the library (because it should be accessible from a base import).

The purpose of the exception class file is to introduce custom exceptions for the library which should be created for edge-case scenarios such as the one I encountered while parsing a nessus file. Specifically the error was caused by a configuration change in the scan for nessus to not scan the local host. This apparently creates a unique pluginID that is missing most of the ordinarily common and essential attributes that libnessus requires.

Aside from this tweak, it's mostly just adding some new functionality (like getting the CVE as a string) and some minor spelling mistakes I encountered while casually making changes.

Thanks in advance for reviewing this and your work on this library. Extremely useful stuff. 👍

It's always preferable to create custom exceptions for weird edge-cases
like missing attributes from the Report object. This is esspecially
helpful in cases where certain nessus plugins don't return all of the
attributes that are necessary, and gives users a straight-forward way to
handle and pass the exception accordingly.
implemented exception handling for missing attributes
plus fixed some minor spelling errors.
Comment threadlibnessus/parser.py Outdated
@bmx0r

Copy link
Copy Markdown
Owner

Thanks a lot for you contribution to this project.
do you mind doing two things:

  • check my comment as i believe you made a typo :)
  • change travis.yml
-- pip install pep8 --use-mirrors
-- pip install pyflakes --use-mirrors
+- pip install pep8
+- pip install pyflakes

we should then have a partial green build
And then i will probably merge :)

@Script-Nomad

Script-Nomad commented Nov 20, 2018

Copy link
Copy Markdown
CollaboratorAuthor

Alright. Changes are pushed. Thanks for catching that typo. Travis doesn't seem to like the pep8 . check though. 🤷‍♂️

There is another issue I encountered with a separate Nessus file that resulted in similar issues that cause the parser to fail. This is because it can't find all the necessary attributes, even though it would normally be handled by for report_item in root.findall("ReportItem") here.
https://github.com/bmx0r/python-libnessus/blob/master/libnessus/parser.py#L75

The problem is that this area of the code is meant to be transparent to the user, or at least I assume so based on the number of private method calls. Raising the error here won't really have any useful effect for the user if they create a NessusReport object, which calls the parser...which parses the hosts...which parses the items, and so on.

The issue is that there is at least one "Info" plugin that is missing these required attributes by design, and there may be more in the future. Ultimately, I think it would be more ideal to accept all ReportItems regardless of what attributes are missing and let the user decide whether or not to accept them or throw them away accordingly by raising the MissingAttribute exception to the top when the user tries to access an item property that is empty.

This is my temporary workaround just so that I can parse the file and throw away the invalid ReportItems:

 for report_item in root.findall("ReportItem"):
try:
_new_item = cls.parse_reportitem(report_item)
except MissingAttribute:
continue
_vuln_list.append(_new_item)

Obviously not a permanent solution. I'm willing to help change the implementation, but I'll wait to know how you would want to handle the issue.

  1. Would you rather accept bad/empty attributes, or throw away bad ReportItems and risk missing details?
  2. Would you want to warn the user in either case and how would you want to handle that?

I've noticed that in every case, the problematic plugin is the "You didn't set Credentials" Info finding which is a bit annoying. It's actually completely empty, except for the name and description. So far, it looks like this is the only problematic plugin.

annoying_nessus_plugin

Curse you Tenable -.-

@bmx0r

Copy link
Copy Markdown
Owner

Hey,
Sorry for this late answer...
You can solve this has it better suit you, but it would be nice to put at least a warning message.
Would you like to do this in the same PR? or will you open another one?
Regards

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Hey, no worries. Sorry that I'm a bit late too. I can work on this and should be able to solve the issues of encountering empty plugins. I think the best solution is to catch any exceptions due to incomplete plugins and warn the user. I have a feeling that if I push this PR now that it will likely break things down the line, so I will fix things here when I have the time.

Implemented pythonic log parser
Added debug logging + PEP 257 Docstring changes
Implementing libnessus logging
Edited to PEP 257 compliant docstrings
Implemented "Strict" parsing option and log warnings to alert users when invalid report items are encountered
@Script-Nomad

Script-Nomad commented Jun 4, 2019

Copy link
Copy Markdown
CollaboratorAuthor

Hi there @bmx0r Sorry this sat for so long, but I finally managed to gather some time to implement some changes that I think are a reasonable approach to dealing with the Bad/MissingAttributes situation. In order to prevent existing codebases that depend on libnessus from breaking, I implemented an optional "strict" boolean argument to all of the parser functions which get passed down to the private functions. By default, it is set to false. As long as strict parsing is false, it will simply warn users when it encounters an invalid ReportItem object due to missing attributes. Users may explicitly set strict parsing to True, which will throw an exception and force the user to deal with it as they please.

So far my testing of this has not introduced any other problems and works as intended. Unfortunately, I am not at all familiar with Travis-CI and cannot get it to behave. As far as I can tell, the code meets all the reasonable PEP8 standards, and works on Python 3.3+. I have not tested any versions prior to that. If it is explicitly required that all of the Travis-CI build checks pass, let me know, but otherwise, I believe this is good to go. If any specific changes are needed, mark them and I'll be happy to address.

Cheers 🍻

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Fixed the Travis-CI build config.
As it turns out, pyflakes no longer works with python<=2.6 or python<=3.3, so I had to remove them from the config.
Pep8 checks still cause the build to fail because of issues in some of the imported modules, rather than the actual libnessus code, so they have been omitted.
The libnessus code itself is still pep8 compliant. 👍

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

@bmx0r Just wanted to drop a reminder in here in case this hasn't been seen yet. If changes are needed, let me know. Would be nice if this could get merged into pypi so that a local fork isn't needed anymore. 😆

@bmx0r
bmx0r merged commit 88bdf57 into bmx0r:masterAug 15, 2019
@bmx0r

Copy link
Copy Markdown
Owner

and finnaly: https://pypi.org/project/python-libnessus/
Sorry for the delay ;)
A thanks you a lot for the contribution :)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Script-Nomad@bmx0r
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

New/exception classes - #3

Merged
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes
Aug 15, 2019
Merged

New/exception classes#3
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes

Conversation

@Script-Nomad

Copy link
Copy Markdown
Collaborator

I had a need for this in order to implement my own parser for nessus's xml output and ran into some trouble that prompted me to implement a few changes to the library. The most significant change was the creation of a new exceptions class file in the base of the library (because it should be accessible from a base import).

The purpose of the exception class file is to introduce custom exceptions for the library which should be created for edge-case scenarios such as the one I encountered while parsing a nessus file. Specifically the error was caused by a configuration change in the scan for nessus to not scan the local host. This apparently creates a unique pluginID that is missing most of the ordinarily common and essential attributes that libnessus requires.

Aside from this tweak, it's mostly just adding some new functionality (like getting the CVE as a string) and some minor spelling mistakes I encountered while casually making changes.

Thanks in advance for reviewing this and your work on this library. Extremely useful stuff. 👍

It's always preferable to create custom exceptions for weird edge-cases
like missing attributes from the Report object. This is esspecially
helpful in cases where certain nessus plugins don't return all of the
attributes that are necessary, and gives users a straight-forward way to
handle and pass the exception accordingly.
implemented exception handling for missing attributes
plus fixed some minor spelling errors.
Comment threadlibnessus/parser.py Outdated
@bmx0r

Copy link
Copy Markdown
Owner

Thanks a lot for you contribution to this project.
do you mind doing two things:

  • check my comment as i believe you made a typo :)
  • change travis.yml
-- pip install pep8 --use-mirrors
-- pip install pyflakes --use-mirrors
+- pip install pep8
+- pip install pyflakes

we should then have a partial green build
And then i will probably merge :)

@Script-Nomad

Script-Nomad commented Nov 20, 2018

Copy link
Copy Markdown
CollaboratorAuthor

Alright. Changes are pushed. Thanks for catching that typo. Travis doesn't seem to like the pep8 . check though. 🤷‍♂️

There is another issue I encountered with a separate Nessus file that resulted in similar issues that cause the parser to fail. This is because it can't find all the necessary attributes, even though it would normally be handled by for report_item in root.findall("ReportItem") here.
https://github.com/bmx0r/python-libnessus/blob/master/libnessus/parser.py#L75

The problem is that this area of the code is meant to be transparent to the user, or at least I assume so based on the number of private method calls. Raising the error here won't really have any useful effect for the user if they create a NessusReport object, which calls the parser...which parses the hosts...which parses the items, and so on.

The issue is that there is at least one "Info" plugin that is missing these required attributes by design, and there may be more in the future. Ultimately, I think it would be more ideal to accept all ReportItems regardless of what attributes are missing and let the user decide whether or not to accept them or throw them away accordingly by raising the MissingAttribute exception to the top when the user tries to access an item property that is empty.

This is my temporary workaround just so that I can parse the file and throw away the invalid ReportItems:

 for report_item in root.findall("ReportItem"):
try:
_new_item = cls.parse_reportitem(report_item)
except MissingAttribute:
continue
_vuln_list.append(_new_item)

Obviously not a permanent solution. I'm willing to help change the implementation, but I'll wait to know how you would want to handle the issue.

  1. Would you rather accept bad/empty attributes, or throw away bad ReportItems and risk missing details?
  2. Would you want to warn the user in either case and how would you want to handle that?

I've noticed that in every case, the problematic plugin is the "You didn't set Credentials" Info finding which is a bit annoying. It's actually completely empty, except for the name and description. So far, it looks like this is the only problematic plugin.

annoying_nessus_plugin

Curse you Tenable -.-

@bmx0r

Copy link
Copy Markdown
Owner

Hey,
Sorry for this late answer...
You can solve this has it better suit you, but it would be nice to put at least a warning message.
Would you like to do this in the same PR? or will you open another one?
Regards

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Hey, no worries. Sorry that I'm a bit late too. I can work on this and should be able to solve the issues of encountering empty plugins. I think the best solution is to catch any exceptions due to incomplete plugins and warn the user. I have a feeling that if I push this PR now that it will likely break things down the line, so I will fix things here when I have the time.

Implemented pythonic log parser
Added debug logging + PEP 257 Docstring changes
Implementing libnessus logging
Edited to PEP 257 compliant docstrings
Implemented "Strict" parsing option and log warnings to alert users when invalid report items are encountered
@Script-Nomad

Script-Nomad commented Jun 4, 2019

Copy link
Copy Markdown
CollaboratorAuthor

Hi there @bmx0r Sorry this sat for so long, but I finally managed to gather some time to implement some changes that I think are a reasonable approach to dealing with the Bad/MissingAttributes situation. In order to prevent existing codebases that depend on libnessus from breaking, I implemented an optional "strict" boolean argument to all of the parser functions which get passed down to the private functions. By default, it is set to false. As long as strict parsing is false, it will simply warn users when it encounters an invalid ReportItem object due to missing attributes. Users may explicitly set strict parsing to True, which will throw an exception and force the user to deal with it as they please.

So far my testing of this has not introduced any other problems and works as intended. Unfortunately, I am not at all familiar with Travis-CI and cannot get it to behave. As far as I can tell, the code meets all the reasonable PEP8 standards, and works on Python 3.3+. I have not tested any versions prior to that. If it is explicitly required that all of the Travis-CI build checks pass, let me know, but otherwise, I believe this is good to go. If any specific changes are needed, mark them and I'll be happy to address.

Cheers 🍻

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Fixed the Travis-CI build config.
As it turns out, pyflakes no longer works with python<=2.6 or python<=3.3, so I had to remove them from the config.
Pep8 checks still cause the build to fail because of issues in some of the imported modules, rather than the actual libnessus code, so they have been omitted.
The libnessus code itself is still pep8 compliant. 👍

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

@bmx0r Just wanted to drop a reminder in here in case this hasn't been seen yet. If changes are needed, let me know. Would be nice if this could get merged into pypi so that a local fork isn't needed anymore. 😆

@bmx0r
bmx0r merged commit 88bdf57 into bmx0r:masterAug 15, 2019
@bmx0r

Copy link
Copy Markdown
Owner

and finnaly: https://pypi.org/project/python-libnessus/
Sorry for the delay ;)
A thanks you a lot for the contribution :)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Script-Nomad@bmx0r
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

New/exception classes - #3

Merged
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes
Aug 15, 2019
Merged

New/exception classes#3
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes

Conversation

@Script-Nomad

Copy link
Copy Markdown
Collaborator

I had a need for this in order to implement my own parser for nessus's xml output and ran into some trouble that prompted me to implement a few changes to the library. The most significant change was the creation of a new exceptions class file in the base of the library (because it should be accessible from a base import).

The purpose of the exception class file is to introduce custom exceptions for the library which should be created for edge-case scenarios such as the one I encountered while parsing a nessus file. Specifically the error was caused by a configuration change in the scan for nessus to not scan the local host. This apparently creates a unique pluginID that is missing most of the ordinarily common and essential attributes that libnessus requires.

Aside from this tweak, it's mostly just adding some new functionality (like getting the CVE as a string) and some minor spelling mistakes I encountered while casually making changes.

Thanks in advance for reviewing this and your work on this library. Extremely useful stuff. 👍

It's always preferable to create custom exceptions for weird edge-cases
like missing attributes from the Report object. This is esspecially
helpful in cases where certain nessus plugins don't return all of the
attributes that are necessary, and gives users a straight-forward way to
handle and pass the exception accordingly.
implemented exception handling for missing attributes
plus fixed some minor spelling errors.
Comment threadlibnessus/parser.py Outdated
@bmx0r

Copy link
Copy Markdown
Owner

Thanks a lot for you contribution to this project.
do you mind doing two things:

  • check my comment as i believe you made a typo :)
  • change travis.yml
-- pip install pep8 --use-mirrors
-- pip install pyflakes --use-mirrors
+- pip install pep8
+- pip install pyflakes

we should then have a partial green build
And then i will probably merge :)

@Script-Nomad

Script-Nomad commented Nov 20, 2018

Copy link
Copy Markdown
CollaboratorAuthor

Alright. Changes are pushed. Thanks for catching that typo. Travis doesn't seem to like the pep8 . check though. 🤷‍♂️

There is another issue I encountered with a separate Nessus file that resulted in similar issues that cause the parser to fail. This is because it can't find all the necessary attributes, even though it would normally be handled by for report_item in root.findall("ReportItem") here.
https://github.com/bmx0r/python-libnessus/blob/master/libnessus/parser.py#L75

The problem is that this area of the code is meant to be transparent to the user, or at least I assume so based on the number of private method calls. Raising the error here won't really have any useful effect for the user if they create a NessusReport object, which calls the parser...which parses the hosts...which parses the items, and so on.

The issue is that there is at least one "Info" plugin that is missing these required attributes by design, and there may be more in the future. Ultimately, I think it would be more ideal to accept all ReportItems regardless of what attributes are missing and let the user decide whether or not to accept them or throw them away accordingly by raising the MissingAttribute exception to the top when the user tries to access an item property that is empty.

This is my temporary workaround just so that I can parse the file and throw away the invalid ReportItems:

 for report_item in root.findall("ReportItem"):
try:
_new_item = cls.parse_reportitem(report_item)
except MissingAttribute:
continue
_vuln_list.append(_new_item)

Obviously not a permanent solution. I'm willing to help change the implementation, but I'll wait to know how you would want to handle the issue.

  1. Would you rather accept bad/empty attributes, or throw away bad ReportItems and risk missing details?
  2. Would you want to warn the user in either case and how would you want to handle that?

I've noticed that in every case, the problematic plugin is the "You didn't set Credentials" Info finding which is a bit annoying. It's actually completely empty, except for the name and description. So far, it looks like this is the only problematic plugin.

annoying_nessus_plugin

Curse you Tenable -.-

@bmx0r

Copy link
Copy Markdown
Owner

Hey,
Sorry for this late answer...
You can solve this has it better suit you, but it would be nice to put at least a warning message.
Would you like to do this in the same PR? or will you open another one?
Regards

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Hey, no worries. Sorry that I'm a bit late too. I can work on this and should be able to solve the issues of encountering empty plugins. I think the best solution is to catch any exceptions due to incomplete plugins and warn the user. I have a feeling that if I push this PR now that it will likely break things down the line, so I will fix things here when I have the time.

Implemented pythonic log parser
Added debug logging + PEP 257 Docstring changes
Implementing libnessus logging
Edited to PEP 257 compliant docstrings
Implemented "Strict" parsing option and log warnings to alert users when invalid report items are encountered
@Script-Nomad

Script-Nomad commented Jun 4, 2019

Copy link
Copy Markdown
CollaboratorAuthor

Hi there @bmx0r Sorry this sat for so long, but I finally managed to gather some time to implement some changes that I think are a reasonable approach to dealing with the Bad/MissingAttributes situation. In order to prevent existing codebases that depend on libnessus from breaking, I implemented an optional "strict" boolean argument to all of the parser functions which get passed down to the private functions. By default, it is set to false. As long as strict parsing is false, it will simply warn users when it encounters an invalid ReportItem object due to missing attributes. Users may explicitly set strict parsing to True, which will throw an exception and force the user to deal with it as they please.

So far my testing of this has not introduced any other problems and works as intended. Unfortunately, I am not at all familiar with Travis-CI and cannot get it to behave. As far as I can tell, the code meets all the reasonable PEP8 standards, and works on Python 3.3+. I have not tested any versions prior to that. If it is explicitly required that all of the Travis-CI build checks pass, let me know, but otherwise, I believe this is good to go. If any specific changes are needed, mark them and I'll be happy to address.

Cheers 🍻

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Fixed the Travis-CI build config.
As it turns out, pyflakes no longer works with python<=2.6 or python<=3.3, so I had to remove them from the config.
Pep8 checks still cause the build to fail because of issues in some of the imported modules, rather than the actual libnessus code, so they have been omitted.
The libnessus code itself is still pep8 compliant. 👍

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

@bmx0r Just wanted to drop a reminder in here in case this hasn't been seen yet. If changes are needed, let me know. Would be nice if this could get merged into pypi so that a local fork isn't needed anymore. 😆

@bmx0r
bmx0r merged commit 88bdf57 into bmx0r:masterAug 15, 2019
@bmx0r

Copy link
Copy Markdown
Owner

and finnaly: https://pypi.org/project/python-libnessus/
Sorry for the delay ;)
A thanks you a lot for the contribution :)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Script-Nomad@bmx0r
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

New/exception classes - #3

Merged
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes
Aug 15, 2019
Merged

New/exception classes#3
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes

Conversation

@Script-Nomad

Copy link
Copy Markdown
Collaborator

I had a need for this in order to implement my own parser for nessus's xml output and ran into some trouble that prompted me to implement a few changes to the library. The most significant change was the creation of a new exceptions class file in the base of the library (because it should be accessible from a base import).

The purpose of the exception class file is to introduce custom exceptions for the library which should be created for edge-case scenarios such as the one I encountered while parsing a nessus file. Specifically the error was caused by a configuration change in the scan for nessus to not scan the local host. This apparently creates a unique pluginID that is missing most of the ordinarily common and essential attributes that libnessus requires.

Aside from this tweak, it's mostly just adding some new functionality (like getting the CVE as a string) and some minor spelling mistakes I encountered while casually making changes.

Thanks in advance for reviewing this and your work on this library. Extremely useful stuff. 👍

It's always preferable to create custom exceptions for weird edge-cases
like missing attributes from the Report object. This is esspecially
helpful in cases where certain nessus plugins don't return all of the
attributes that are necessary, and gives users a straight-forward way to
handle and pass the exception accordingly.
implemented exception handling for missing attributes
plus fixed some minor spelling errors.
Comment threadlibnessus/parser.py Outdated
@bmx0r

Copy link
Copy Markdown
Owner

Thanks a lot for you contribution to this project.
do you mind doing two things:

  • check my comment as i believe you made a typo :)
  • change travis.yml
-- pip install pep8 --use-mirrors
-- pip install pyflakes --use-mirrors
+- pip install pep8
+- pip install pyflakes

we should then have a partial green build
And then i will probably merge :)

@Script-Nomad

Script-Nomad commented Nov 20, 2018

Copy link
Copy Markdown
CollaboratorAuthor

Alright. Changes are pushed. Thanks for catching that typo. Travis doesn't seem to like the pep8 . check though. 🤷‍♂️

There is another issue I encountered with a separate Nessus file that resulted in similar issues that cause the parser to fail. This is because it can't find all the necessary attributes, even though it would normally be handled by for report_item in root.findall("ReportItem") here.
https://github.com/bmx0r/python-libnessus/blob/master/libnessus/parser.py#L75

The problem is that this area of the code is meant to be transparent to the user, or at least I assume so based on the number of private method calls. Raising the error here won't really have any useful effect for the user if they create a NessusReport object, which calls the parser...which parses the hosts...which parses the items, and so on.

The issue is that there is at least one "Info" plugin that is missing these required attributes by design, and there may be more in the future. Ultimately, I think it would be more ideal to accept all ReportItems regardless of what attributes are missing and let the user decide whether or not to accept them or throw them away accordingly by raising the MissingAttribute exception to the top when the user tries to access an item property that is empty.

This is my temporary workaround just so that I can parse the file and throw away the invalid ReportItems:

 for report_item in root.findall("ReportItem"):
try:
_new_item = cls.parse_reportitem(report_item)
except MissingAttribute:
continue
_vuln_list.append(_new_item)

Obviously not a permanent solution. I'm willing to help change the implementation, but I'll wait to know how you would want to handle the issue.

  1. Would you rather accept bad/empty attributes, or throw away bad ReportItems and risk missing details?
  2. Would you want to warn the user in either case and how would you want to handle that?

I've noticed that in every case, the problematic plugin is the "You didn't set Credentials" Info finding which is a bit annoying. It's actually completely empty, except for the name and description. So far, it looks like this is the only problematic plugin.

annoying_nessus_plugin

Curse you Tenable -.-

@bmx0r

Copy link
Copy Markdown
Owner

Hey,
Sorry for this late answer...
You can solve this has it better suit you, but it would be nice to put at least a warning message.
Would you like to do this in the same PR? or will you open another one?
Regards

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Hey, no worries. Sorry that I'm a bit late too. I can work on this and should be able to solve the issues of encountering empty plugins. I think the best solution is to catch any exceptions due to incomplete plugins and warn the user. I have a feeling that if I push this PR now that it will likely break things down the line, so I will fix things here when I have the time.

Implemented pythonic log parser
Added debug logging + PEP 257 Docstring changes
Implementing libnessus logging
Edited to PEP 257 compliant docstrings
Implemented "Strict" parsing option and log warnings to alert users when invalid report items are encountered
@Script-Nomad

Script-Nomad commented Jun 4, 2019

Copy link
Copy Markdown
CollaboratorAuthor

Hi there @bmx0r Sorry this sat for so long, but I finally managed to gather some time to implement some changes that I think are a reasonable approach to dealing with the Bad/MissingAttributes situation. In order to prevent existing codebases that depend on libnessus from breaking, I implemented an optional "strict" boolean argument to all of the parser functions which get passed down to the private functions. By default, it is set to false. As long as strict parsing is false, it will simply warn users when it encounters an invalid ReportItem object due to missing attributes. Users may explicitly set strict parsing to True, which will throw an exception and force the user to deal with it as they please.

So far my testing of this has not introduced any other problems and works as intended. Unfortunately, I am not at all familiar with Travis-CI and cannot get it to behave. As far as I can tell, the code meets all the reasonable PEP8 standards, and works on Python 3.3+. I have not tested any versions prior to that. If it is explicitly required that all of the Travis-CI build checks pass, let me know, but otherwise, I believe this is good to go. If any specific changes are needed, mark them and I'll be happy to address.

Cheers 🍻

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Fixed the Travis-CI build config.
As it turns out, pyflakes no longer works with python<=2.6 or python<=3.3, so I had to remove them from the config.
Pep8 checks still cause the build to fail because of issues in some of the imported modules, rather than the actual libnessus code, so they have been omitted.
The libnessus code itself is still pep8 compliant. 👍

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

@bmx0r Just wanted to drop a reminder in here in case this hasn't been seen yet. If changes are needed, let me know. Would be nice if this could get merged into pypi so that a local fork isn't needed anymore. 😆

@bmx0r
bmx0r merged commit 88bdf57 into bmx0r:masterAug 15, 2019
@bmx0r

Copy link
Copy Markdown
Owner

and finnaly: https://pypi.org/project/python-libnessus/
Sorry for the delay ;)
A thanks you a lot for the contribution :)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Script-Nomad@bmx0r
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

New/exception classes - #3

Merged
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes
Aug 15, 2019
Merged

New/exception classes#3
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes

Conversation

@Script-Nomad

Copy link
Copy Markdown
Collaborator

I had a need for this in order to implement my own parser for nessus's xml output and ran into some trouble that prompted me to implement a few changes to the library. The most significant change was the creation of a new exceptions class file in the base of the library (because it should be accessible from a base import).

The purpose of the exception class file is to introduce custom exceptions for the library which should be created for edge-case scenarios such as the one I encountered while parsing a nessus file. Specifically the error was caused by a configuration change in the scan for nessus to not scan the local host. This apparently creates a unique pluginID that is missing most of the ordinarily common and essential attributes that libnessus requires.

Aside from this tweak, it's mostly just adding some new functionality (like getting the CVE as a string) and some minor spelling mistakes I encountered while casually making changes.

Thanks in advance for reviewing this and your work on this library. Extremely useful stuff. 👍

It's always preferable to create custom exceptions for weird edge-cases
like missing attributes from the Report object. This is esspecially
helpful in cases where certain nessus plugins don't return all of the
attributes that are necessary, and gives users a straight-forward way to
handle and pass the exception accordingly.
implemented exception handling for missing attributes
plus fixed some minor spelling errors.
Comment threadlibnessus/parser.py Outdated
@bmx0r

Copy link
Copy Markdown
Owner

Thanks a lot for you contribution to this project.
do you mind doing two things:

  • check my comment as i believe you made a typo :)
  • change travis.yml
-- pip install pep8 --use-mirrors
-- pip install pyflakes --use-mirrors
+- pip install pep8
+- pip install pyflakes

we should then have a partial green build
And then i will probably merge :)

@Script-Nomad

Script-Nomad commented Nov 20, 2018

Copy link
Copy Markdown
CollaboratorAuthor

Alright. Changes are pushed. Thanks for catching that typo. Travis doesn't seem to like the pep8 . check though. 🤷‍♂️

There is another issue I encountered with a separate Nessus file that resulted in similar issues that cause the parser to fail. This is because it can't find all the necessary attributes, even though it would normally be handled by for report_item in root.findall("ReportItem") here.
https://github.com/bmx0r/python-libnessus/blob/master/libnessus/parser.py#L75

The problem is that this area of the code is meant to be transparent to the user, or at least I assume so based on the number of private method calls. Raising the error here won't really have any useful effect for the user if they create a NessusReport object, which calls the parser...which parses the hosts...which parses the items, and so on.

The issue is that there is at least one "Info" plugin that is missing these required attributes by design, and there may be more in the future. Ultimately, I think it would be more ideal to accept all ReportItems regardless of what attributes are missing and let the user decide whether or not to accept them or throw them away accordingly by raising the MissingAttribute exception to the top when the user tries to access an item property that is empty.

This is my temporary workaround just so that I can parse the file and throw away the invalid ReportItems:

 for report_item in root.findall("ReportItem"):
try:
_new_item = cls.parse_reportitem(report_item)
except MissingAttribute:
continue
_vuln_list.append(_new_item)

Obviously not a permanent solution. I'm willing to help change the implementation, but I'll wait to know how you would want to handle the issue.

  1. Would you rather accept bad/empty attributes, or throw away bad ReportItems and risk missing details?
  2. Would you want to warn the user in either case and how would you want to handle that?

I've noticed that in every case, the problematic plugin is the "You didn't set Credentials" Info finding which is a bit annoying. It's actually completely empty, except for the name and description. So far, it looks like this is the only problematic plugin.

annoying_nessus_plugin

Curse you Tenable -.-

@bmx0r

Copy link
Copy Markdown
Owner

Hey,
Sorry for this late answer...
You can solve this has it better suit you, but it would be nice to put at least a warning message.
Would you like to do this in the same PR? or will you open another one?
Regards

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Hey, no worries. Sorry that I'm a bit late too. I can work on this and should be able to solve the issues of encountering empty plugins. I think the best solution is to catch any exceptions due to incomplete plugins and warn the user. I have a feeling that if I push this PR now that it will likely break things down the line, so I will fix things here when I have the time.

Implemented pythonic log parser
Added debug logging + PEP 257 Docstring changes
Implementing libnessus logging
Edited to PEP 257 compliant docstrings
Implemented "Strict" parsing option and log warnings to alert users when invalid report items are encountered
@Script-Nomad

Script-Nomad commented Jun 4, 2019

Copy link
Copy Markdown
CollaboratorAuthor

Hi there @bmx0r Sorry this sat for so long, but I finally managed to gather some time to implement some changes that I think are a reasonable approach to dealing with the Bad/MissingAttributes situation. In order to prevent existing codebases that depend on libnessus from breaking, I implemented an optional "strict" boolean argument to all of the parser functions which get passed down to the private functions. By default, it is set to false. As long as strict parsing is false, it will simply warn users when it encounters an invalid ReportItem object due to missing attributes. Users may explicitly set strict parsing to True, which will throw an exception and force the user to deal with it as they please.

So far my testing of this has not introduced any other problems and works as intended. Unfortunately, I am not at all familiar with Travis-CI and cannot get it to behave. As far as I can tell, the code meets all the reasonable PEP8 standards, and works on Python 3.3+. I have not tested any versions prior to that. If it is explicitly required that all of the Travis-CI build checks pass, let me know, but otherwise, I believe this is good to go. If any specific changes are needed, mark them and I'll be happy to address.

Cheers 🍻

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Fixed the Travis-CI build config.
As it turns out, pyflakes no longer works with python<=2.6 or python<=3.3, so I had to remove them from the config.
Pep8 checks still cause the build to fail because of issues in some of the imported modules, rather than the actual libnessus code, so they have been omitted.
The libnessus code itself is still pep8 compliant. 👍

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

@bmx0r Just wanted to drop a reminder in here in case this hasn't been seen yet. If changes are needed, let me know. Would be nice if this could get merged into pypi so that a local fork isn't needed anymore. 😆

@bmx0r
bmx0r merged commit 88bdf57 into bmx0r:masterAug 15, 2019
@bmx0r

Copy link
Copy Markdown
Owner

and finnaly: https://pypi.org/project/python-libnessus/
Sorry for the delay ;)
A thanks you a lot for the contribution :)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Script-Nomad@bmx0r
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

New/exception classes - #3

Merged
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes
Aug 15, 2019
Merged

New/exception classes#3
bmx0r merged 10 commits into
bmx0r:masterfrom
Script-Nomad:new/exception_classes

Conversation

@Script-Nomad

Copy link
Copy Markdown
Collaborator

I had a need for this in order to implement my own parser for nessus's xml output and ran into some trouble that prompted me to implement a few changes to the library. The most significant change was the creation of a new exceptions class file in the base of the library (because it should be accessible from a base import).

The purpose of the exception class file is to introduce custom exceptions for the library which should be created for edge-case scenarios such as the one I encountered while parsing a nessus file. Specifically the error was caused by a configuration change in the scan for nessus to not scan the local host. This apparently creates a unique pluginID that is missing most of the ordinarily common and essential attributes that libnessus requires.

Aside from this tweak, it's mostly just adding some new functionality (like getting the CVE as a string) and some minor spelling mistakes I encountered while casually making changes.

Thanks in advance for reviewing this and your work on this library. Extremely useful stuff. 👍

It's always preferable to create custom exceptions for weird edge-cases
like missing attributes from the Report object. This is esspecially
helpful in cases where certain nessus plugins don't return all of the
attributes that are necessary, and gives users a straight-forward way to
handle and pass the exception accordingly.
implemented exception handling for missing attributes
plus fixed some minor spelling errors.
Comment threadlibnessus/parser.py Outdated
@bmx0r

Copy link
Copy Markdown
Owner

Thanks a lot for you contribution to this project.
do you mind doing two things:

  • check my comment as i believe you made a typo :)
  • change travis.yml
-- pip install pep8 --use-mirrors
-- pip install pyflakes --use-mirrors
+- pip install pep8
+- pip install pyflakes

we should then have a partial green build
And then i will probably merge :)

@Script-Nomad

Script-Nomad commented Nov 20, 2018

Copy link
Copy Markdown
CollaboratorAuthor

Alright. Changes are pushed. Thanks for catching that typo. Travis doesn't seem to like the pep8 . check though. 🤷‍♂️

There is another issue I encountered with a separate Nessus file that resulted in similar issues that cause the parser to fail. This is because it can't find all the necessary attributes, even though it would normally be handled by for report_item in root.findall("ReportItem") here.
https://github.com/bmx0r/python-libnessus/blob/master/libnessus/parser.py#L75

The problem is that this area of the code is meant to be transparent to the user, or at least I assume so based on the number of private method calls. Raising the error here won't really have any useful effect for the user if they create a NessusReport object, which calls the parser...which parses the hosts...which parses the items, and so on.

The issue is that there is at least one "Info" plugin that is missing these required attributes by design, and there may be more in the future. Ultimately, I think it would be more ideal to accept all ReportItems regardless of what attributes are missing and let the user decide whether or not to accept them or throw them away accordingly by raising the MissingAttribute exception to the top when the user tries to access an item property that is empty.

This is my temporary workaround just so that I can parse the file and throw away the invalid ReportItems:

 for report_item in root.findall("ReportItem"):
try:
_new_item = cls.parse_reportitem(report_item)
except MissingAttribute:
continue
_vuln_list.append(_new_item)

Obviously not a permanent solution. I'm willing to help change the implementation, but I'll wait to know how you would want to handle the issue.

  1. Would you rather accept bad/empty attributes, or throw away bad ReportItems and risk missing details?
  2. Would you want to warn the user in either case and how would you want to handle that?

I've noticed that in every case, the problematic plugin is the "You didn't set Credentials" Info finding which is a bit annoying. It's actually completely empty, except for the name and description. So far, it looks like this is the only problematic plugin.

annoying_nessus_plugin

Curse you Tenable -.-

@bmx0r

Copy link
Copy Markdown
Owner

Hey,
Sorry for this late answer...
You can solve this has it better suit you, but it would be nice to put at least a warning message.
Would you like to do this in the same PR? or will you open another one?
Regards

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Hey, no worries. Sorry that I'm a bit late too. I can work on this and should be able to solve the issues of encountering empty plugins. I think the best solution is to catch any exceptions due to incomplete plugins and warn the user. I have a feeling that if I push this PR now that it will likely break things down the line, so I will fix things here when I have the time.

Implemented pythonic log parser
Added debug logging + PEP 257 Docstring changes
Implementing libnessus logging
Edited to PEP 257 compliant docstrings
Implemented "Strict" parsing option and log warnings to alert users when invalid report items are encountered
@Script-Nomad

Script-Nomad commented Jun 4, 2019

Copy link
Copy Markdown
CollaboratorAuthor

Hi there @bmx0r Sorry this sat for so long, but I finally managed to gather some time to implement some changes that I think are a reasonable approach to dealing with the Bad/MissingAttributes situation. In order to prevent existing codebases that depend on libnessus from breaking, I implemented an optional "strict" boolean argument to all of the parser functions which get passed down to the private functions. By default, it is set to false. As long as strict parsing is false, it will simply warn users when it encounters an invalid ReportItem object due to missing attributes. Users may explicitly set strict parsing to True, which will throw an exception and force the user to deal with it as they please.

So far my testing of this has not introduced any other problems and works as intended. Unfortunately, I am not at all familiar with Travis-CI and cannot get it to behave. As far as I can tell, the code meets all the reasonable PEP8 standards, and works on Python 3.3+. I have not tested any versions prior to that. If it is explicitly required that all of the Travis-CI build checks pass, let me know, but otherwise, I believe this is good to go. If any specific changes are needed, mark them and I'll be happy to address.

Cheers 🍻

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

Fixed the Travis-CI build config.
As it turns out, pyflakes no longer works with python<=2.6 or python<=3.3, so I had to remove them from the config.
Pep8 checks still cause the build to fail because of issues in some of the imported modules, rather than the actual libnessus code, so they have been omitted.
The libnessus code itself is still pep8 compliant. 👍

@Script-Nomad

Copy link
Copy Markdown
CollaboratorAuthor

@bmx0r Just wanted to drop a reminder in here in case this hasn't been seen yet. If changes are needed, let me know. Would be nice if this could get merged into pypi so that a local fork isn't needed anymore. 😆

@bmx0r
bmx0r merged commit 88bdf57 into bmx0r:masterAug 15, 2019
@bmx0r

Copy link
Copy Markdown
Owner

and finnaly: https://pypi.org/project/python-libnessus/
Sorry for the delay ;)
A thanks you a lot for the contribution :)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Script-Nomad@bmx0r