expose libuv uv_fs_event functionality - #474

Merged
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master
Sep 9, 2022
Merged

expose libuv uv_fs_event functionality#474
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master

Conversation

@jensbjorgensen

Copy link
Copy Markdown
Contributor

It's quite natural for filesystem monitoring to be part of the event loop, and libuv has done a cross-platform implementation. Actually asyncio should probably expose this as well--maybe they will draw from this example implementation to inform their own api?

I wasn't sure what the best way to expose the FS_EVENT_* constants was--I preferred to have them refer directly to the constants defined in libuv but I'm a cython newb and I couldn't get the right incantation. In the merge below they are hardcoded in uvloop.const. Maybe you can suggest a better way.

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok, looks like some items for me to look at. the specific tests I added passed but I see there are formatting issues, sorry I hadn't run those, will update

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the tests I've added are passing, seems like the test suite just takes too long to run for some reason?

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the failures in 3.9 are apparently due to asyncio.events._TransProtPair not being defined in python 3.9

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok! I got test_sourcecode.py fixed up (which I didn't break but ... bonus!), that'd be awesome if one of you maintainers could take a look

@fantixfantix left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a new feature!

Comment thread.gitignore Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/handles/fsevent.pxd Outdated
Comment threaduvloop/const.py Outdated
Comment threaduvloop/loop.pyx Outdated
@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

please have a look at my latest push and thanks for your time reviewing the PR :-)

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

I think everything is lined up now with your comments? Let me know if there's anything else.

@jensbjorgensen

jensbjorgensen commented Jul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

ping-ping?

@fantix

fantix commented Aug 13, 2022

Copy link
Copy Markdown
Member

Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the get_uv_loop_ptr() API accepted (#310) for easier integration with the libuv loop.

@jensbjorgensen

jensbjorgensen commented Aug 15, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

@fantix

Copy link
Copy Markdown
Member

@jensbjorgensen We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implementedrecursive flag, and merge the rest as it is.

Please feel free to let me know if you have any comments, or create a new PR if this is already merged.

@jensbjorgensen

jensbjorgensen commented Sep 1, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

* Move const into fsevent.pyx and import with uvloop.loop
* Use libuv const to replace the redeclaration
* Hide the _monitor_fs API
* Duck-typing the asyncio.Handle API
* Dropped recursive flag which is not functional on Linux
* Proper UVHandle start and stop for fs_event
* Callback with context
* Errors in callback won't stop the watch now
* Dropped the libuv loop pointer (already implemented in other PR)
This happens when executing chmod on a subdir on Linux
Comment threadtests/test_fs_event.py Outdated
Comment threaduvloop/loop.pyx
1st1
1st1 approved these changes Sep 6, 2022
@fantix
fantix merged commit 74d381e into MagicStack:masterSep 9, 2022
fantix added a commit that referenced this pull request Sep 13, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
@fantixfantix mentioned this pull request Sep 13, 2022
fantix added a commit that referenced this pull request Sep 14, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jensbjorgensen@fantix@1st1
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all \u003cpre\u003e\u003ccode\u003e blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks"); } } catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); } })(); (function(){ try { var __m = "github.com"; var __re = new RegExp('^' + "github\\.com" + '
Skip to content

expose libuv uv_fs_event functionality - #474

Merged
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master
Sep 9, 2022
Merged

expose libuv uv_fs_event functionality#474
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master

Conversation

@jensbjorgensen

Copy link
Copy Markdown
Contributor

It's quite natural for filesystem monitoring to be part of the event loop, and libuv has done a cross-platform implementation. Actually asyncio should probably expose this as well--maybe they will draw from this example implementation to inform their own api?

I wasn't sure what the best way to expose the FS_EVENT_* constants was--I preferred to have them refer directly to the constants defined in libuv but I'm a cython newb and I couldn't get the right incantation. In the merge below they are hardcoded in uvloop.const. Maybe you can suggest a better way.

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok, looks like some items for me to look at. the specific tests I added passed but I see there are formatting issues, sorry I hadn't run those, will update

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the tests I've added are passing, seems like the test suite just takes too long to run for some reason?

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the failures in 3.9 are apparently due to asyncio.events._TransProtPair not being defined in python 3.9

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok! I got test_sourcecode.py fixed up (which I didn't break but ... bonus!), that'd be awesome if one of you maintainers could take a look

@fantixfantix left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a new feature!

Comment thread.gitignore Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/handles/fsevent.pxd Outdated
Comment threaduvloop/const.py Outdated
Comment threaduvloop/loop.pyx Outdated
@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

please have a look at my latest push and thanks for your time reviewing the PR :-)

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

I think everything is lined up now with your comments? Let me know if there's anything else.

@jensbjorgensen

jensbjorgensen commented Jul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

ping-ping?

@fantix

fantix commented Aug 13, 2022

Copy link
Copy Markdown
Member

Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the get_uv_loop_ptr() API accepted (#310) for easier integration with the libuv loop.

@jensbjorgensen

jensbjorgensen commented Aug 15, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

@fantix

Copy link
Copy Markdown
Member

@jensbjorgensen We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implementedrecursive flag, and merge the rest as it is.

Please feel free to let me know if you have any comments, or create a new PR if this is already merged.

@jensbjorgensen

jensbjorgensen commented Sep 1, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

* Move const into fsevent.pyx and import with uvloop.loop
* Use libuv const to replace the redeclaration
* Hide the _monitor_fs API
* Duck-typing the asyncio.Handle API
* Dropped recursive flag which is not functional on Linux
* Proper UVHandle start and stop for fs_event
* Callback with context
* Errors in callback won't stop the watch now
* Dropped the libuv loop pointer (already implemented in other PR)
This happens when executing chmod on a subdir on Linux
Comment threadtests/test_fs_event.py Outdated
Comment threaduvloop/loop.pyx
1st1
1st1 approved these changes Sep 6, 2022
@fantix
fantix merged commit 74d381e into MagicStack:masterSep 9, 2022
fantix added a commit that referenced this pull request Sep 13, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
@fantixfantix mentioned this pull request Sep 13, 2022
fantix added a commit that referenced this pull request Sep 14, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jensbjorgensen@fantix@1st1
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

expose libuv uv_fs_event functionality - #474

Merged
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master
Sep 9, 2022
Merged

expose libuv uv_fs_event functionality#474
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master

Conversation

@jensbjorgensen

Copy link
Copy Markdown
Contributor

It's quite natural for filesystem monitoring to be part of the event loop, and libuv has done a cross-platform implementation. Actually asyncio should probably expose this as well--maybe they will draw from this example implementation to inform their own api?

I wasn't sure what the best way to expose the FS_EVENT_* constants was--I preferred to have them refer directly to the constants defined in libuv but I'm a cython newb and I couldn't get the right incantation. In the merge below they are hardcoded in uvloop.const. Maybe you can suggest a better way.

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok, looks like some items for me to look at. the specific tests I added passed but I see there are formatting issues, sorry I hadn't run those, will update

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the tests I've added are passing, seems like the test suite just takes too long to run for some reason?

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the failures in 3.9 are apparently due to asyncio.events._TransProtPair not being defined in python 3.9

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok! I got test_sourcecode.py fixed up (which I didn't break but ... bonus!), that'd be awesome if one of you maintainers could take a look

@fantixfantix left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a new feature!

Comment thread.gitignore Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/handles/fsevent.pxd Outdated
Comment threaduvloop/const.py Outdated
Comment threaduvloop/loop.pyx Outdated
@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

please have a look at my latest push and thanks for your time reviewing the PR :-)

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

I think everything is lined up now with your comments? Let me know if there's anything else.

@jensbjorgensen

jensbjorgensen commented Jul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

ping-ping?

@fantix

fantix commented Aug 13, 2022

Copy link
Copy Markdown
Member

Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the get_uv_loop_ptr() API accepted (#310) for easier integration with the libuv loop.

@jensbjorgensen

jensbjorgensen commented Aug 15, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

@fantix

Copy link
Copy Markdown
Member

@jensbjorgensen We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implementedrecursive flag, and merge the rest as it is.

Please feel free to let me know if you have any comments, or create a new PR if this is already merged.

@jensbjorgensen

jensbjorgensen commented Sep 1, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

* Move const into fsevent.pyx and import with uvloop.loop
* Use libuv const to replace the redeclaration
* Hide the _monitor_fs API
* Duck-typing the asyncio.Handle API
* Dropped recursive flag which is not functional on Linux
* Proper UVHandle start and stop for fs_event
* Callback with context
* Errors in callback won't stop the watch now
* Dropped the libuv loop pointer (already implemented in other PR)
This happens when executing chmod on a subdir on Linux
Comment threadtests/test_fs_event.py Outdated
Comment threaduvloop/loop.pyx
1st1
1st1 approved these changes Sep 6, 2022
@fantix
fantix merged commit 74d381e into MagicStack:masterSep 9, 2022
fantix added a commit that referenced this pull request Sep 13, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
@fantixfantix mentioned this pull request Sep 13, 2022
fantix added a commit that referenced this pull request Sep 14, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jensbjorgensen@fantix@1st1
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length \u003e 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

expose libuv uv_fs_event functionality - #474

Merged
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master
Sep 9, 2022
Merged

expose libuv uv_fs_event functionality#474
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master

Conversation

@jensbjorgensen

Copy link
Copy Markdown
Contributor

It's quite natural for filesystem monitoring to be part of the event loop, and libuv has done a cross-platform implementation. Actually asyncio should probably expose this as well--maybe they will draw from this example implementation to inform their own api?

I wasn't sure what the best way to expose the FS_EVENT_* constants was--I preferred to have them refer directly to the constants defined in libuv but I'm a cython newb and I couldn't get the right incantation. In the merge below they are hardcoded in uvloop.const. Maybe you can suggest a better way.

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok, looks like some items for me to look at. the specific tests I added passed but I see there are formatting issues, sorry I hadn't run those, will update

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the tests I've added are passing, seems like the test suite just takes too long to run for some reason?

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the failures in 3.9 are apparently due to asyncio.events._TransProtPair not being defined in python 3.9

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok! I got test_sourcecode.py fixed up (which I didn't break but ... bonus!), that'd be awesome if one of you maintainers could take a look

@fantixfantix left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a new feature!

Comment thread.gitignore Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/handles/fsevent.pxd Outdated
Comment threaduvloop/const.py Outdated
Comment threaduvloop/loop.pyx Outdated
@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

please have a look at my latest push and thanks for your time reviewing the PR :-)

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

I think everything is lined up now with your comments? Let me know if there's anything else.

@jensbjorgensen

jensbjorgensen commented Jul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

ping-ping?

@fantix

fantix commented Aug 13, 2022

Copy link
Copy Markdown
Member

Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the get_uv_loop_ptr() API accepted (#310) for easier integration with the libuv loop.

@jensbjorgensen

jensbjorgensen commented Aug 15, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

@fantix

Copy link
Copy Markdown
Member

@jensbjorgensen We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implementedrecursive flag, and merge the rest as it is.

Please feel free to let me know if you have any comments, or create a new PR if this is already merged.

@jensbjorgensen

jensbjorgensen commented Sep 1, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

* Move const into fsevent.pyx and import with uvloop.loop
* Use libuv const to replace the redeclaration
* Hide the _monitor_fs API
* Duck-typing the asyncio.Handle API
* Dropped recursive flag which is not functional on Linux
* Proper UVHandle start and stop for fs_event
* Callback with context
* Errors in callback won't stop the watch now
* Dropped the libuv loop pointer (already implemented in other PR)
This happens when executing chmod on a subdir on Linux
Comment threadtests/test_fs_event.py Outdated
Comment threaduvloop/loop.pyx
1st1
1st1 approved these changes Sep 6, 2022
@fantix
fantix merged commit 74d381e into MagicStack:masterSep 9, 2022
fantix added a commit that referenced this pull request Sep 13, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
@fantixfantix mentioned this pull request Sep 13, 2022
fantix added a commit that referenced this pull request Sep 14, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jensbjorgensen@fantix@1st1
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

expose libuv uv_fs_event functionality - #474

Merged
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master
Sep 9, 2022
Merged

expose libuv uv_fs_event functionality#474
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master

Conversation

@jensbjorgensen

Copy link
Copy Markdown
Contributor

It's quite natural for filesystem monitoring to be part of the event loop, and libuv has done a cross-platform implementation. Actually asyncio should probably expose this as well--maybe they will draw from this example implementation to inform their own api?

I wasn't sure what the best way to expose the FS_EVENT_* constants was--I preferred to have them refer directly to the constants defined in libuv but I'm a cython newb and I couldn't get the right incantation. In the merge below they are hardcoded in uvloop.const. Maybe you can suggest a better way.

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok, looks like some items for me to look at. the specific tests I added passed but I see there are formatting issues, sorry I hadn't run those, will update

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the tests I've added are passing, seems like the test suite just takes too long to run for some reason?

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the failures in 3.9 are apparently due to asyncio.events._TransProtPair not being defined in python 3.9

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok! I got test_sourcecode.py fixed up (which I didn't break but ... bonus!), that'd be awesome if one of you maintainers could take a look

@fantixfantix left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a new feature!

Comment thread.gitignore Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/handles/fsevent.pxd Outdated
Comment threaduvloop/const.py Outdated
Comment threaduvloop/loop.pyx Outdated
@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

please have a look at my latest push and thanks for your time reviewing the PR :-)

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

I think everything is lined up now with your comments? Let me know if there's anything else.

@jensbjorgensen

jensbjorgensen commented Jul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

ping-ping?

@fantix

fantix commented Aug 13, 2022

Copy link
Copy Markdown
Member

Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the get_uv_loop_ptr() API accepted (#310) for easier integration with the libuv loop.

@jensbjorgensen

jensbjorgensen commented Aug 15, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

@fantix

Copy link
Copy Markdown
Member

@jensbjorgensen We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implementedrecursive flag, and merge the rest as it is.

Please feel free to let me know if you have any comments, or create a new PR if this is already merged.

@jensbjorgensen

jensbjorgensen commented Sep 1, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

* Move const into fsevent.pyx and import with uvloop.loop
* Use libuv const to replace the redeclaration
* Hide the _monitor_fs API
* Duck-typing the asyncio.Handle API
* Dropped recursive flag which is not functional on Linux
* Proper UVHandle start and stop for fs_event
* Callback with context
* Errors in callback won't stop the watch now
* Dropped the libuv loop pointer (already implemented in other PR)
This happens when executing chmod on a subdir on Linux
Comment threadtests/test_fs_event.py Outdated
Comment threaduvloop/loop.pyx
1st1
1st1 approved these changes Sep 6, 2022
@fantix
fantix merged commit 74d381e into MagicStack:masterSep 9, 2022
fantix added a commit that referenced this pull request Sep 13, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
@fantixfantix mentioned this pull request Sep 13, 2022
fantix added a commit that referenced this pull request Sep 14, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jensbjorgensen@fantix@1st1
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

expose libuv uv_fs_event functionality - #474

Merged
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master
Sep 9, 2022
Merged

expose libuv uv_fs_event functionality#474
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master

Conversation

@jensbjorgensen

Copy link
Copy Markdown
Contributor

It's quite natural for filesystem monitoring to be part of the event loop, and libuv has done a cross-platform implementation. Actually asyncio should probably expose this as well--maybe they will draw from this example implementation to inform their own api?

I wasn't sure what the best way to expose the FS_EVENT_* constants was--I preferred to have them refer directly to the constants defined in libuv but I'm a cython newb and I couldn't get the right incantation. In the merge below they are hardcoded in uvloop.const. Maybe you can suggest a better way.

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok, looks like some items for me to look at. the specific tests I added passed but I see there are formatting issues, sorry I hadn't run those, will update

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the tests I've added are passing, seems like the test suite just takes too long to run for some reason?

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the failures in 3.9 are apparently due to asyncio.events._TransProtPair not being defined in python 3.9

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok! I got test_sourcecode.py fixed up (which I didn't break but ... bonus!), that'd be awesome if one of you maintainers could take a look

@fantixfantix left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a new feature!

Comment thread.gitignore Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/handles/fsevent.pxd Outdated
Comment threaduvloop/const.py Outdated
Comment threaduvloop/loop.pyx Outdated
@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

please have a look at my latest push and thanks for your time reviewing the PR :-)

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

I think everything is lined up now with your comments? Let me know if there's anything else.

@jensbjorgensen

jensbjorgensen commented Jul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

ping-ping?

@fantix

fantix commented Aug 13, 2022

Copy link
Copy Markdown
Member

Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the get_uv_loop_ptr() API accepted (#310) for easier integration with the libuv loop.

@jensbjorgensen

jensbjorgensen commented Aug 15, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

@fantix

Copy link
Copy Markdown
Member

@jensbjorgensen We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implementedrecursive flag, and merge the rest as it is.

Please feel free to let me know if you have any comments, or create a new PR if this is already merged.

@jensbjorgensen

jensbjorgensen commented Sep 1, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

* Move const into fsevent.pyx and import with uvloop.loop
* Use libuv const to replace the redeclaration
* Hide the _monitor_fs API
* Duck-typing the asyncio.Handle API
* Dropped recursive flag which is not functional on Linux
* Proper UVHandle start and stop for fs_event
* Callback with context
* Errors in callback won't stop the watch now
* Dropped the libuv loop pointer (already implemented in other PR)
This happens when executing chmod on a subdir on Linux
Comment threadtests/test_fs_event.py Outdated
Comment threaduvloop/loop.pyx
1st1
1st1 approved these changes Sep 6, 2022
@fantix
fantix merged commit 74d381e into MagicStack:masterSep 9, 2022
fantix added a commit that referenced this pull request Sep 13, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
@fantixfantix mentioned this pull request Sep 13, 2022
fantix added a commit that referenced this pull request Sep 14, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jensbjorgensen@fantix@1st1
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

expose libuv uv_fs_event functionality - #474

Merged
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master
Sep 9, 2022
Merged

expose libuv uv_fs_event functionality#474
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master

Conversation

@jensbjorgensen

Copy link
Copy Markdown
Contributor

It's quite natural for filesystem monitoring to be part of the event loop, and libuv has done a cross-platform implementation. Actually asyncio should probably expose this as well--maybe they will draw from this example implementation to inform their own api?

I wasn't sure what the best way to expose the FS_EVENT_* constants was--I preferred to have them refer directly to the constants defined in libuv but I'm a cython newb and I couldn't get the right incantation. In the merge below they are hardcoded in uvloop.const. Maybe you can suggest a better way.

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok, looks like some items for me to look at. the specific tests I added passed but I see there are formatting issues, sorry I hadn't run those, will update

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the tests I've added are passing, seems like the test suite just takes too long to run for some reason?

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the failures in 3.9 are apparently due to asyncio.events._TransProtPair not being defined in python 3.9

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok! I got test_sourcecode.py fixed up (which I didn't break but ... bonus!), that'd be awesome if one of you maintainers could take a look

@fantixfantix left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a new feature!

Comment thread.gitignore Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/handles/fsevent.pxd Outdated
Comment threaduvloop/const.py Outdated
Comment threaduvloop/loop.pyx Outdated
@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

please have a look at my latest push and thanks for your time reviewing the PR :-)

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

I think everything is lined up now with your comments? Let me know if there's anything else.

@jensbjorgensen

jensbjorgensen commented Jul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

ping-ping?

@fantix

fantix commented Aug 13, 2022

Copy link
Copy Markdown
Member

Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the get_uv_loop_ptr() API accepted (#310) for easier integration with the libuv loop.

@jensbjorgensen

jensbjorgensen commented Aug 15, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

@fantix

Copy link
Copy Markdown
Member

@jensbjorgensen We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implementedrecursive flag, and merge the rest as it is.

Please feel free to let me know if you have any comments, or create a new PR if this is already merged.

@jensbjorgensen

jensbjorgensen commented Sep 1, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

* Move const into fsevent.pyx and import with uvloop.loop
* Use libuv const to replace the redeclaration
* Hide the _monitor_fs API
* Duck-typing the asyncio.Handle API
* Dropped recursive flag which is not functional on Linux
* Proper UVHandle start and stop for fs_event
* Callback with context
* Errors in callback won't stop the watch now
* Dropped the libuv loop pointer (already implemented in other PR)
This happens when executing chmod on a subdir on Linux
Comment threadtests/test_fs_event.py Outdated
Comment threaduvloop/loop.pyx
1st1
1st1 approved these changes Sep 6, 2022
@fantix
fantix merged commit 74d381e into MagicStack:masterSep 9, 2022
fantix added a commit that referenced this pull request Sep 13, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
@fantixfantix mentioned this pull request Sep 13, 2022
fantix added a commit that referenced this pull request Sep 14, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jensbjorgensen@fantix@1st1
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

expose libuv uv_fs_event functionality - #474

Merged
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master
Sep 9, 2022
Merged

expose libuv uv_fs_event functionality#474
fantix merged 18 commits into
MagicStack:masterfrom
jensbjorgensen:master

Conversation

@jensbjorgensen

Copy link
Copy Markdown
Contributor

It's quite natural for filesystem monitoring to be part of the event loop, and libuv has done a cross-platform implementation. Actually asyncio should probably expose this as well--maybe they will draw from this example implementation to inform their own api?

I wasn't sure what the best way to expose the FS_EVENT_* constants was--I preferred to have them refer directly to the constants defined in libuv but I'm a cython newb and I couldn't get the right incantation. In the merge below they are hardcoded in uvloop.const. Maybe you can suggest a better way.

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok, looks like some items for me to look at. the specific tests I added passed but I see there are formatting issues, sorry I hadn't run those, will update

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the tests I've added are passing, seems like the test suite just takes too long to run for some reason?

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

the failures in 3.9 are apparently due to asyncio.events._TransProtPair not being defined in python 3.9

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

ok! I got test_sourcecode.py fixed up (which I didn't break but ... bonus!), that'd be awesome if one of you maintainers could take a look

@fantixfantix left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a new feature!

Comment thread.gitignore Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/loop.pyx Outdated
Comment threaduvloop/handles/fsevent.pxd Outdated
Comment threaduvloop/const.py Outdated
Comment threaduvloop/loop.pyx Outdated
@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

please have a look at my latest push and thanks for your time reviewing the PR :-)

@jensbjorgensen

Copy link
Copy Markdown
ContributorAuthor

I think everything is lined up now with your comments? Let me know if there's anything else.

@jensbjorgensen

jensbjorgensen commented Jul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

ping-ping?

@fantix

fantix commented Aug 13, 2022

Copy link
Copy Markdown
Member

Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the get_uv_loop_ptr() API accepted (#310) for easier integration with the libuv loop.

@jensbjorgensen

jensbjorgensen commented Aug 15, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

@fantix

Copy link
Copy Markdown
Member

@jensbjorgensen We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implementedrecursive flag, and merge the rest as it is.

Please feel free to let me know if you have any comments, or create a new PR if this is already merged.

@jensbjorgensen

jensbjorgensen commented Sep 1, 2022 via email

Copy link
Copy Markdown
ContributorAuthor

* Move const into fsevent.pyx and import with uvloop.loop
* Use libuv const to replace the redeclaration
* Hide the _monitor_fs API
* Duck-typing the asyncio.Handle API
* Dropped recursive flag which is not functional on Linux
* Proper UVHandle start and stop for fs_event
* Callback with context
* Errors in callback won't stop the watch now
* Dropped the libuv loop pointer (already implemented in other PR)
This happens when executing chmod on a subdir on Linux
Comment threadtests/test_fs_event.py Outdated
Comment threaduvloop/loop.pyx
1st1
1st1 approved these changes Sep 6, 2022
@fantix
fantix merged commit 74d381e into MagicStack:masterSep 9, 2022
fantix added a commit that referenced this pull request Sep 13, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
@fantixfantix mentioned this pull request Sep 13, 2022
fantix added a commit that referenced this pull request Sep 14, 2022
This release adds Python 3.11 support, updates bundled libuv to 1.43.0
and fixes a handful of issues.
Changes
=======
* Expose uv_loop_t pointer for integration with other C-extensions (#310)
(by @pranavtbhat in b332eb8 for #310)
* Support python 3.11+ (#473)
(by @zeroday0619 in 8e42921 for #473)
* Expose libuv uv_fs_event functionality (#474)
(by @jensbjorgensen@fantix in 74d381e for #474)
* Activate debug mode when `-X dev` is used
(by @jack1142 in 637a77a)
* Expose uv_version() for libuv API compatibility (#491)
(by @fantix in 089f6cb for #491)
* Fix loop.getaddrinfo() and tests (#495)
(by @fantix in 598b16f for #495)
* Bump to libuv 1.43.0
(by @fantix in 94e5e53)
Fixes
=====
* _TransProtPair is no longer defined in asyncio.events
(by @jensbjorgensen in fae5f7f)
* use a TypeVar for asyncio.BaseProtocol (#478)
(by @graingert in 3aacb35 for #478)
* Fix segfault in TimerHandle.when() after cleared
(by @jensbjorgensen in c39afff for #469)
* Avoid self._errpipe_write double close (#466)
(by @graingert in 72140d7 for #466)
* Fix typo in test (#456)
(by @kianmeng in 033d52d for #456)
* Fix potential infinite loop (#446)
(by @kfur in ada43c0 for #446)
* use a stack of self._fds_to_close to prevent double closes (#481)
(by @graingert in 3214cf6 for #481)
* Fix incorrect main thread id value forking from a thread (#453)
(by @horpto@fantix in e7934c8 for #453)
* create_subprocess_exec should treat env={} as empty environment (#439) (#454)
(by @byllyfish in e04637e for #439)
* Queue write only after processing all buffers (#445)
(by @jakirkham@fantix in 9c6ecb6 for #445)
* Drop Python 3.6 support for thread ident
(by @fantix in 9c37930)
* bugfix: write to another transport in resume_writing() fails (#498)
(by @fantix in d2deffe for #498)
Build
=====
* Upgrade GitHub Actions (#477) (#480)
(by @cclauss in fcbf422 for #477, 1008694 for #480)
* typo `same as same`
(by @YoSTEALTH in fedba80)
* setup.py: allow to override extra_compile_args (#443)
(by @giuliobenetti in a130375 for #443)
* Drop hack in setup.py in finalize_options (492)
(by @fantix in 2f1bc83 for #492)
* Fix tests invocation on release CI worklow (#489)
(by @ben9923 in d6a2b59 for #489)
Documentation
=============
* use asyncio.Runner loop_factory on 3.11+ (#472)
(by @graingert in 31ba48c for #472)
* Fix CI badge in docs, remove remaining Travis CI references from docs
(by @Nothing4You in c6901a7)
* Fix typo in README
(by @monosans in 73d7253)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jensbjorgensen@fantix@1st1