Add ability to select path segments using Python re regexprs - #146

Closed
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp
Closed

Add ability to select path segments using Python re regexprs#146
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp

Conversation

@AlainLich

Copy link
Copy Markdown

Hi,
In case there is an interest, this allows to use python re regular expressions, like in following examples:

import dpath
dpath.options.DPATH_ACCEPT_RE_REGEXP = True
js = loadJson()
selPath = 'Config/{(Env|Cmd)}'
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = [ re.compile('(Config|Graph)') , re.compile('(Env|Cmd|Data)') ]
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = '{(Config|Graph)}/{(Env|Cmd|Data)}'
x = dpath.util.search(js.lod, selPath)
print(x)

I found this useful navigating JSON.

Remarks:

. This runs under Python3 as shown on https://github.com/AlainLich/dpath-python/actions/runs/896380770 ;
. the trickwith dpath.options.DPATH_ACCEPT_RE_REGEXP, means the option has to be enabled make this compatible
with earlier versions and will not break if users have path segments in curly brackets.

WARNING: this will not work under Python2.7, since it requires Python3 features like 'f' strings; however I might
do a compatible version if there is interest.

Changes (most in random tests):

  1. quite straightforward changes in dpath/segments.py
  2. added DPATH_ACCEPT_RE_REGEXP in options as explained above
  3. in dpath/util.pyre.compile extended regexprs once when they are found
  4. added `test/test_path_ext.py ̀ to test this extension
  5. made all tests using hypothesis run under unittest, in particular this permits to select and reproduce a single test function
  6. debugged a rare situation in the random test generation, reproduced in file issues/err_walk.py where a path segment
    gets generated which is interpreted by the bash globbing function resulting in a test failure (e.g. ̀seg[0]) which is globbed into something accepting seg0` but not its literal version with the brackets

…tests in conformance with options.ALLOW_EMPTY_STRING_KEYS
Still required to add tests for regext path segments.
Ran tests with Python-3 and Python-2.7 (nose)
Still needed: test and documentation of added functionality
Random test has shown 1 issue not related to new feature, non repeatable suspect binary string
with single null byte;documented in issues/nonrepeatableErr.txt.
…ments
Uncontrolled test complexity apparently caused by translation of "***.."" into
costly regexp in test framework; now simplified.
…y, test generator
coming up with keys interpreted literally when walking tree, but interpreted as a bash glob
when matching. See in file issues/err_walk.py. (e.g '[0]' is also a glob meaning '0' ...)
@moomoohk

Copy link
Copy Markdown
Collaborator

@AlainLich Hey, are you still interested in working on this PR?

@AlainLich

Copy link
Copy Markdown
Author

@moomoohk Hello, I have not touched this for a year and a half, could be interested to work on this PR, likely at a slow pace, since this would facilitate my own use (loading via pip ).

However, only on Python3... not interested in Python2 compatibility and such.

@moomoohk

Copy link
Copy Markdown
Collaborator

That's great to hear :)
Slow pace is alright by me, as well as Python 3+. On that note, I've decided to drop Python 2 support entirely a while ago!
Minimum supported version is 3.7 as of now.
I've refactored the codebase quite a bit in the last year and a half. Mainly modernizing, upgrading to Python 3 and blowing some dust off...
If you have any questions feel free to reach out here or on Gitter.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@moomoohk

Copy link
Copy Markdown
Collaborator

Yes indeed. Please start by merging master into your branch and take it from there.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@AlainLich

Copy link
Copy Markdown
Author

Hi, I have entered a new PR #177 with revised code integrated in the current commit for branch master.

This PR should be closed.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@AlainLich@moomoohk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Add ability to select path segments using Python re regexprs - #146

Closed
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp
Closed

Add ability to select path segments using Python re regexprs#146
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp

Conversation

@AlainLich

Copy link
Copy Markdown

Hi,
In case there is an interest, this allows to use python re regular expressions, like in following examples:

import dpath
dpath.options.DPATH_ACCEPT_RE_REGEXP = True
js = loadJson()
selPath = 'Config/{(Env|Cmd)}'
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = [ re.compile('(Config|Graph)') , re.compile('(Env|Cmd|Data)') ]
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = '{(Config|Graph)}/{(Env|Cmd|Data)}'
x = dpath.util.search(js.lod, selPath)
print(x)

I found this useful navigating JSON.

Remarks:

. This runs under Python3 as shown on https://github.com/AlainLich/dpath-python/actions/runs/896380770 ;
. the trickwith dpath.options.DPATH_ACCEPT_RE_REGEXP, means the option has to be enabled make this compatible
with earlier versions and will not break if users have path segments in curly brackets.

WARNING: this will not work under Python2.7, since it requires Python3 features like 'f' strings; however I might
do a compatible version if there is interest.

Changes (most in random tests):

  1. quite straightforward changes in dpath/segments.py
  2. added DPATH_ACCEPT_RE_REGEXP in options as explained above
  3. in dpath/util.pyre.compile extended regexprs once when they are found
  4. added `test/test_path_ext.py ̀ to test this extension
  5. made all tests using hypothesis run under unittest, in particular this permits to select and reproduce a single test function
  6. debugged a rare situation in the random test generation, reproduced in file issues/err_walk.py where a path segment
    gets generated which is interpreted by the bash globbing function resulting in a test failure (e.g. ̀seg[0]) which is globbed into something accepting seg0` but not its literal version with the brackets

…tests in conformance with options.ALLOW_EMPTY_STRING_KEYS
Still required to add tests for regext path segments.
Ran tests with Python-3 and Python-2.7 (nose)
Still needed: test and documentation of added functionality
Random test has shown 1 issue not related to new feature, non repeatable suspect binary string
with single null byte;documented in issues/nonrepeatableErr.txt.
…ments
Uncontrolled test complexity apparently caused by translation of "***.."" into
costly regexp in test framework; now simplified.
…y, test generator
coming up with keys interpreted literally when walking tree, but interpreted as a bash glob
when matching. See in file issues/err_walk.py. (e.g '[0]' is also a glob meaning '0' ...)
@moomoohk

Copy link
Copy Markdown
Collaborator

@AlainLich Hey, are you still interested in working on this PR?

@AlainLich

Copy link
Copy Markdown
Author

@moomoohk Hello, I have not touched this for a year and a half, could be interested to work on this PR, likely at a slow pace, since this would facilitate my own use (loading via pip ).

However, only on Python3... not interested in Python2 compatibility and such.

@moomoohk

Copy link
Copy Markdown
Collaborator

That's great to hear :)
Slow pace is alright by me, as well as Python 3+. On that note, I've decided to drop Python 2 support entirely a while ago!
Minimum supported version is 3.7 as of now.
I've refactored the codebase quite a bit in the last year and a half. Mainly modernizing, upgrading to Python 3 and blowing some dust off...
If you have any questions feel free to reach out here or on Gitter.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@moomoohk

Copy link
Copy Markdown
Collaborator

Yes indeed. Please start by merging master into your branch and take it from there.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@AlainLich

Copy link
Copy Markdown
Author

Hi, I have entered a new PR #177 with revised code integrated in the current commit for branch master.

This PR should be closed.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@AlainLich@moomoohk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Add ability to select path segments using Python re regexprs - #146

Closed
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp
Closed

Add ability to select path segments using Python re regexprs#146
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp

Conversation

@AlainLich

Copy link
Copy Markdown

Hi,
In case there is an interest, this allows to use python re regular expressions, like in following examples:

import dpath
dpath.options.DPATH_ACCEPT_RE_REGEXP = True
js = loadJson()
selPath = 'Config/{(Env|Cmd)}'
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = [ re.compile('(Config|Graph)') , re.compile('(Env|Cmd|Data)') ]
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = '{(Config|Graph)}/{(Env|Cmd|Data)}'
x = dpath.util.search(js.lod, selPath)
print(x)

I found this useful navigating JSON.

Remarks:

. This runs under Python3 as shown on https://github.com/AlainLich/dpath-python/actions/runs/896380770 ;
. the trickwith dpath.options.DPATH_ACCEPT_RE_REGEXP, means the option has to be enabled make this compatible
with earlier versions and will not break if users have path segments in curly brackets.

WARNING: this will not work under Python2.7, since it requires Python3 features like 'f' strings; however I might
do a compatible version if there is interest.

Changes (most in random tests):

  1. quite straightforward changes in dpath/segments.py
  2. added DPATH_ACCEPT_RE_REGEXP in options as explained above
  3. in dpath/util.pyre.compile extended regexprs once when they are found
  4. added `test/test_path_ext.py ̀ to test this extension
  5. made all tests using hypothesis run under unittest, in particular this permits to select and reproduce a single test function
  6. debugged a rare situation in the random test generation, reproduced in file issues/err_walk.py where a path segment
    gets generated which is interpreted by the bash globbing function resulting in a test failure (e.g. ̀seg[0]) which is globbed into something accepting seg0` but not its literal version with the brackets

…tests in conformance with options.ALLOW_EMPTY_STRING_KEYS
Still required to add tests for regext path segments.
Ran tests with Python-3 and Python-2.7 (nose)
Still needed: test and documentation of added functionality
Random test has shown 1 issue not related to new feature, non repeatable suspect binary string
with single null byte;documented in issues/nonrepeatableErr.txt.
…ments
Uncontrolled test complexity apparently caused by translation of "***.."" into
costly regexp in test framework; now simplified.
…y, test generator
coming up with keys interpreted literally when walking tree, but interpreted as a bash glob
when matching. See in file issues/err_walk.py. (e.g '[0]' is also a glob meaning '0' ...)
@moomoohk

Copy link
Copy Markdown
Collaborator

@AlainLich Hey, are you still interested in working on this PR?

@AlainLich

Copy link
Copy Markdown
Author

@moomoohk Hello, I have not touched this for a year and a half, could be interested to work on this PR, likely at a slow pace, since this would facilitate my own use (loading via pip ).

However, only on Python3... not interested in Python2 compatibility and such.

@moomoohk

Copy link
Copy Markdown
Collaborator

That's great to hear :)
Slow pace is alright by me, as well as Python 3+. On that note, I've decided to drop Python 2 support entirely a while ago!
Minimum supported version is 3.7 as of now.
I've refactored the codebase quite a bit in the last year and a half. Mainly modernizing, upgrading to Python 3 and blowing some dust off...
If you have any questions feel free to reach out here or on Gitter.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@moomoohk

Copy link
Copy Markdown
Collaborator

Yes indeed. Please start by merging master into your branch and take it from there.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@AlainLich

Copy link
Copy Markdown
Author

Hi, I have entered a new PR #177 with revised code integrated in the current commit for branch master.

This PR should be closed.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@AlainLich@moomoohk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Add ability to select path segments using Python re regexprs - #146

Closed
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp
Closed

Add ability to select path segments using Python re regexprs#146
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp

Conversation

@AlainLich

Copy link
Copy Markdown

Hi,
In case there is an interest, this allows to use python re regular expressions, like in following examples:

import dpath
dpath.options.DPATH_ACCEPT_RE_REGEXP = True
js = loadJson()
selPath = 'Config/{(Env|Cmd)}'
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = [ re.compile('(Config|Graph)') , re.compile('(Env|Cmd|Data)') ]
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = '{(Config|Graph)}/{(Env|Cmd|Data)}'
x = dpath.util.search(js.lod, selPath)
print(x)

I found this useful navigating JSON.

Remarks:

. This runs under Python3 as shown on https://github.com/AlainLich/dpath-python/actions/runs/896380770 ;
. the trickwith dpath.options.DPATH_ACCEPT_RE_REGEXP, means the option has to be enabled make this compatible
with earlier versions and will not break if users have path segments in curly brackets.

WARNING: this will not work under Python2.7, since it requires Python3 features like 'f' strings; however I might
do a compatible version if there is interest.

Changes (most in random tests):

  1. quite straightforward changes in dpath/segments.py
  2. added DPATH_ACCEPT_RE_REGEXP in options as explained above
  3. in dpath/util.pyre.compile extended regexprs once when they are found
  4. added `test/test_path_ext.py ̀ to test this extension
  5. made all tests using hypothesis run under unittest, in particular this permits to select and reproduce a single test function
  6. debugged a rare situation in the random test generation, reproduced in file issues/err_walk.py where a path segment
    gets generated which is interpreted by the bash globbing function resulting in a test failure (e.g. ̀seg[0]) which is globbed into something accepting seg0` but not its literal version with the brackets

…tests in conformance with options.ALLOW_EMPTY_STRING_KEYS
Still required to add tests for regext path segments.
Ran tests with Python-3 and Python-2.7 (nose)
Still needed: test and documentation of added functionality
Random test has shown 1 issue not related to new feature, non repeatable suspect binary string
with single null byte;documented in issues/nonrepeatableErr.txt.
…ments
Uncontrolled test complexity apparently caused by translation of "***.."" into
costly regexp in test framework; now simplified.
…y, test generator
coming up with keys interpreted literally when walking tree, but interpreted as a bash glob
when matching. See in file issues/err_walk.py. (e.g '[0]' is also a glob meaning '0' ...)
@moomoohk

Copy link
Copy Markdown
Collaborator

@AlainLich Hey, are you still interested in working on this PR?

@AlainLich

Copy link
Copy Markdown
Author

@moomoohk Hello, I have not touched this for a year and a half, could be interested to work on this PR, likely at a slow pace, since this would facilitate my own use (loading via pip ).

However, only on Python3... not interested in Python2 compatibility and such.

@moomoohk

Copy link
Copy Markdown
Collaborator

That's great to hear :)
Slow pace is alright by me, as well as Python 3+. On that note, I've decided to drop Python 2 support entirely a while ago!
Minimum supported version is 3.7 as of now.
I've refactored the codebase quite a bit in the last year and a half. Mainly modernizing, upgrading to Python 3 and blowing some dust off...
If you have any questions feel free to reach out here or on Gitter.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@moomoohk

Copy link
Copy Markdown
Collaborator

Yes indeed. Please start by merging master into your branch and take it from there.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@AlainLich

Copy link
Copy Markdown
Author

Hi, I have entered a new PR #177 with revised code integrated in the current commit for branch master.

This PR should be closed.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@AlainLich@moomoohk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Add ability to select path segments using Python re regexprs - #146

Closed
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp
Closed

Add ability to select path segments using Python re regexprs#146
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp

Conversation

@AlainLich

Copy link
Copy Markdown

Hi,
In case there is an interest, this allows to use python re regular expressions, like in following examples:

import dpath
dpath.options.DPATH_ACCEPT_RE_REGEXP = True
js = loadJson()
selPath = 'Config/{(Env|Cmd)}'
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = [ re.compile('(Config|Graph)') , re.compile('(Env|Cmd|Data)') ]
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = '{(Config|Graph)}/{(Env|Cmd|Data)}'
x = dpath.util.search(js.lod, selPath)
print(x)

I found this useful navigating JSON.

Remarks:

. This runs under Python3 as shown on https://github.com/AlainLich/dpath-python/actions/runs/896380770 ;
. the trickwith dpath.options.DPATH_ACCEPT_RE_REGEXP, means the option has to be enabled make this compatible
with earlier versions and will not break if users have path segments in curly brackets.

WARNING: this will not work under Python2.7, since it requires Python3 features like 'f' strings; however I might
do a compatible version if there is interest.

Changes (most in random tests):

  1. quite straightforward changes in dpath/segments.py
  2. added DPATH_ACCEPT_RE_REGEXP in options as explained above
  3. in dpath/util.pyre.compile extended regexprs once when they are found
  4. added `test/test_path_ext.py ̀ to test this extension
  5. made all tests using hypothesis run under unittest, in particular this permits to select and reproduce a single test function
  6. debugged a rare situation in the random test generation, reproduced in file issues/err_walk.py where a path segment
    gets generated which is interpreted by the bash globbing function resulting in a test failure (e.g. ̀seg[0]) which is globbed into something accepting seg0` but not its literal version with the brackets

…tests in conformance with options.ALLOW_EMPTY_STRING_KEYS
Still required to add tests for regext path segments.
Ran tests with Python-3 and Python-2.7 (nose)
Still needed: test and documentation of added functionality
Random test has shown 1 issue not related to new feature, non repeatable suspect binary string
with single null byte;documented in issues/nonrepeatableErr.txt.
…ments
Uncontrolled test complexity apparently caused by translation of "***.."" into
costly regexp in test framework; now simplified.
…y, test generator
coming up with keys interpreted literally when walking tree, but interpreted as a bash glob
when matching. See in file issues/err_walk.py. (e.g '[0]' is also a glob meaning '0' ...)
@moomoohk

Copy link
Copy Markdown
Collaborator

@AlainLich Hey, are you still interested in working on this PR?

@AlainLich

Copy link
Copy Markdown
Author

@moomoohk Hello, I have not touched this for a year and a half, could be interested to work on this PR, likely at a slow pace, since this would facilitate my own use (loading via pip ).

However, only on Python3... not interested in Python2 compatibility and such.

@moomoohk

Copy link
Copy Markdown
Collaborator

That's great to hear :)
Slow pace is alright by me, as well as Python 3+. On that note, I've decided to drop Python 2 support entirely a while ago!
Minimum supported version is 3.7 as of now.
I've refactored the codebase quite a bit in the last year and a half. Mainly modernizing, upgrading to Python 3 and blowing some dust off...
If you have any questions feel free to reach out here or on Gitter.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@moomoohk

Copy link
Copy Markdown
Collaborator

Yes indeed. Please start by merging master into your branch and take it from there.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@AlainLich

Copy link
Copy Markdown
Author

Hi, I have entered a new PR #177 with revised code integrated in the current commit for branch master.

This PR should be closed.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@AlainLich@moomoohk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Add ability to select path segments using Python re regexprs - #146

Closed
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp
Closed

Add ability to select path segments using Python re regexprs#146
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp

Conversation

@AlainLich

Copy link
Copy Markdown

Hi,
In case there is an interest, this allows to use python re regular expressions, like in following examples:

import dpath
dpath.options.DPATH_ACCEPT_RE_REGEXP = True
js = loadJson()
selPath = 'Config/{(Env|Cmd)}'
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = [ re.compile('(Config|Graph)') , re.compile('(Env|Cmd|Data)') ]
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = '{(Config|Graph)}/{(Env|Cmd|Data)}'
x = dpath.util.search(js.lod, selPath)
print(x)

I found this useful navigating JSON.

Remarks:

. This runs under Python3 as shown on https://github.com/AlainLich/dpath-python/actions/runs/896380770 ;
. the trickwith dpath.options.DPATH_ACCEPT_RE_REGEXP, means the option has to be enabled make this compatible
with earlier versions and will not break if users have path segments in curly brackets.

WARNING: this will not work under Python2.7, since it requires Python3 features like 'f' strings; however I might
do a compatible version if there is interest.

Changes (most in random tests):

  1. quite straightforward changes in dpath/segments.py
  2. added DPATH_ACCEPT_RE_REGEXP in options as explained above
  3. in dpath/util.pyre.compile extended regexprs once when they are found
  4. added `test/test_path_ext.py ̀ to test this extension
  5. made all tests using hypothesis run under unittest, in particular this permits to select and reproduce a single test function
  6. debugged a rare situation in the random test generation, reproduced in file issues/err_walk.py where a path segment
    gets generated which is interpreted by the bash globbing function resulting in a test failure (e.g. ̀seg[0]) which is globbed into something accepting seg0` but not its literal version with the brackets

…tests in conformance with options.ALLOW_EMPTY_STRING_KEYS
Still required to add tests for regext path segments.
Ran tests with Python-3 and Python-2.7 (nose)
Still needed: test and documentation of added functionality
Random test has shown 1 issue not related to new feature, non repeatable suspect binary string
with single null byte;documented in issues/nonrepeatableErr.txt.
…ments
Uncontrolled test complexity apparently caused by translation of "***.."" into
costly regexp in test framework; now simplified.
…y, test generator
coming up with keys interpreted literally when walking tree, but interpreted as a bash glob
when matching. See in file issues/err_walk.py. (e.g '[0]' is also a glob meaning '0' ...)
@moomoohk

Copy link
Copy Markdown
Collaborator

@AlainLich Hey, are you still interested in working on this PR?

@AlainLich

Copy link
Copy Markdown
Author

@moomoohk Hello, I have not touched this for a year and a half, could be interested to work on this PR, likely at a slow pace, since this would facilitate my own use (loading via pip ).

However, only on Python3... not interested in Python2 compatibility and such.

@moomoohk

Copy link
Copy Markdown
Collaborator

That's great to hear :)
Slow pace is alright by me, as well as Python 3+. On that note, I've decided to drop Python 2 support entirely a while ago!
Minimum supported version is 3.7 as of now.
I've refactored the codebase quite a bit in the last year and a half. Mainly modernizing, upgrading to Python 3 and blowing some dust off...
If you have any questions feel free to reach out here or on Gitter.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@moomoohk

Copy link
Copy Markdown
Collaborator

Yes indeed. Please start by merging master into your branch and take it from there.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@AlainLich

Copy link
Copy Markdown
Author

Hi, I have entered a new PR #177 with revised code integrated in the current commit for branch master.

This PR should be closed.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@AlainLich@moomoohk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Add ability to select path segments using Python re regexprs - #146

Closed
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp
Closed

Add ability to select path segments using Python re regexprs#146
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp

Conversation

@AlainLich

Copy link
Copy Markdown

Hi,
In case there is an interest, this allows to use python re regular expressions, like in following examples:

import dpath
dpath.options.DPATH_ACCEPT_RE_REGEXP = True
js = loadJson()
selPath = 'Config/{(Env|Cmd)}'
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = [ re.compile('(Config|Graph)') , re.compile('(Env|Cmd|Data)') ]
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = '{(Config|Graph)}/{(Env|Cmd|Data)}'
x = dpath.util.search(js.lod, selPath)
print(x)

I found this useful navigating JSON.

Remarks:

. This runs under Python3 as shown on https://github.com/AlainLich/dpath-python/actions/runs/896380770 ;
. the trickwith dpath.options.DPATH_ACCEPT_RE_REGEXP, means the option has to be enabled make this compatible
with earlier versions and will not break if users have path segments in curly brackets.

WARNING: this will not work under Python2.7, since it requires Python3 features like 'f' strings; however I might
do a compatible version if there is interest.

Changes (most in random tests):

  1. quite straightforward changes in dpath/segments.py
  2. added DPATH_ACCEPT_RE_REGEXP in options as explained above
  3. in dpath/util.pyre.compile extended regexprs once when they are found
  4. added `test/test_path_ext.py ̀ to test this extension
  5. made all tests using hypothesis run under unittest, in particular this permits to select and reproduce a single test function
  6. debugged a rare situation in the random test generation, reproduced in file issues/err_walk.py where a path segment
    gets generated which is interpreted by the bash globbing function resulting in a test failure (e.g. ̀seg[0]) which is globbed into something accepting seg0` but not its literal version with the brackets

…tests in conformance with options.ALLOW_EMPTY_STRING_KEYS
Still required to add tests for regext path segments.
Ran tests with Python-3 and Python-2.7 (nose)
Still needed: test and documentation of added functionality
Random test has shown 1 issue not related to new feature, non repeatable suspect binary string
with single null byte;documented in issues/nonrepeatableErr.txt.
…ments
Uncontrolled test complexity apparently caused by translation of "***.."" into
costly regexp in test framework; now simplified.
…y, test generator
coming up with keys interpreted literally when walking tree, but interpreted as a bash glob
when matching. See in file issues/err_walk.py. (e.g '[0]' is also a glob meaning '0' ...)
@moomoohk

Copy link
Copy Markdown
Collaborator

@AlainLich Hey, are you still interested in working on this PR?

@AlainLich

Copy link
Copy Markdown
Author

@moomoohk Hello, I have not touched this for a year and a half, could be interested to work on this PR, likely at a slow pace, since this would facilitate my own use (loading via pip ).

However, only on Python3... not interested in Python2 compatibility and such.

@moomoohk

Copy link
Copy Markdown
Collaborator

That's great to hear :)
Slow pace is alright by me, as well as Python 3+. On that note, I've decided to drop Python 2 support entirely a while ago!
Minimum supported version is 3.7 as of now.
I've refactored the codebase quite a bit in the last year and a half. Mainly modernizing, upgrading to Python 3 and blowing some dust off...
If you have any questions feel free to reach out here or on Gitter.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@moomoohk

Copy link
Copy Markdown
Collaborator

Yes indeed. Please start by merging master into your branch and take it from there.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@AlainLich

Copy link
Copy Markdown
Author

Hi, I have entered a new PR #177 with revised code integrated in the current commit for branch master.

This PR should be closed.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@AlainLich@moomoohk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Add ability to select path segments using Python re regexprs - #146

Closed
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp
Closed

Add ability to select path segments using Python re regexprs#146
AlainLich wants to merge 14 commits into
dpath-maintainers:masterfrom
AlainLich:AL-addRegexp

Conversation

@AlainLich

Copy link
Copy Markdown

Hi,
In case there is an interest, this allows to use python re regular expressions, like in following examples:

import dpath
dpath.options.DPATH_ACCEPT_RE_REGEXP = True
js = loadJson()
selPath = 'Config/{(Env|Cmd)}'
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = [ re.compile('(Config|Graph)') , re.compile('(Env|Cmd|Data)') ]
x = dpath.util.search(js.lod, selPath)
print(x)
selPath = '{(Config|Graph)}/{(Env|Cmd|Data)}'
x = dpath.util.search(js.lod, selPath)
print(x)

I found this useful navigating JSON.

Remarks:

. This runs under Python3 as shown on https://github.com/AlainLich/dpath-python/actions/runs/896380770 ;
. the trickwith dpath.options.DPATH_ACCEPT_RE_REGEXP, means the option has to be enabled make this compatible
with earlier versions and will not break if users have path segments in curly brackets.

WARNING: this will not work under Python2.7, since it requires Python3 features like 'f' strings; however I might
do a compatible version if there is interest.

Changes (most in random tests):

  1. quite straightforward changes in dpath/segments.py
  2. added DPATH_ACCEPT_RE_REGEXP in options as explained above
  3. in dpath/util.pyre.compile extended regexprs once when they are found
  4. added `test/test_path_ext.py ̀ to test this extension
  5. made all tests using hypothesis run under unittest, in particular this permits to select and reproduce a single test function
  6. debugged a rare situation in the random test generation, reproduced in file issues/err_walk.py where a path segment
    gets generated which is interpreted by the bash globbing function resulting in a test failure (e.g. ̀seg[0]) which is globbed into something accepting seg0` but not its literal version with the brackets

…tests in conformance with options.ALLOW_EMPTY_STRING_KEYS
Still required to add tests for regext path segments.
Ran tests with Python-3 and Python-2.7 (nose)
Still needed: test and documentation of added functionality
Random test has shown 1 issue not related to new feature, non repeatable suspect binary string
with single null byte;documented in issues/nonrepeatableErr.txt.
…ments
Uncontrolled test complexity apparently caused by translation of "***.."" into
costly regexp in test framework; now simplified.
…y, test generator
coming up with keys interpreted literally when walking tree, but interpreted as a bash glob
when matching. See in file issues/err_walk.py. (e.g '[0]' is also a glob meaning '0' ...)
@moomoohk

Copy link
Copy Markdown
Collaborator

@AlainLich Hey, are you still interested in working on this PR?

@AlainLich

Copy link
Copy Markdown
Author

@moomoohk Hello, I have not touched this for a year and a half, could be interested to work on this PR, likely at a slow pace, since this would facilitate my own use (loading via pip ).

However, only on Python3... not interested in Python2 compatibility and such.

@moomoohk

Copy link
Copy Markdown
Collaborator

That's great to hear :)
Slow pace is alright by me, as well as Python 3+. On that note, I've decided to drop Python 2 support entirely a while ago!
Minimum supported version is 3.7 as of now.
I've refactored the codebase quite a bit in the last year and a half. Mainly modernizing, upgrading to Python 3 and blowing some dust off...
If you have any questions feel free to reach out here or on Gitter.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@moomoohk

Copy link
Copy Markdown
Collaborator

Yes indeed. Please start by merging master into your branch and take it from there.

@AlainLich

Copy link
Copy Markdown
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch masteror any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@AlainLich

Copy link
Copy Markdown
Author

Hi, I have entered a new PR #177 with revised code integrated in the current commit for branch master.

This PR should be closed.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@AlainLich@moomoohk