Skip to content

Updated submodules to the release 4.5.4 and added python loader support - #563

Merged
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support
Oct 20, 2021
Merged

Updated submodules to the release 4.5.4 and added python loader support#563
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support

Conversation

@asenyaev

Copy link
Copy Markdown
Contributor

No description provided.

@alalekalalek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Comment threadscripts/__init__.py Outdated
@@ -0,0 +1,23 @@
from .cv2 import *
from .data import *

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we don't need these 2 lines

"data" submodule is imported automatically.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we replace import cv2 to import cv2.cv2 for native_module in __init__.py, then we need first line, second can be removed.

Besides, I removed lines with a replacing, because rewrote config file, so, removed it.

Comment threadscripts/__init__.py
try:
from .version import ci_build, headless

ci_and_not_headless = ci_build and not headless

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can verify that case that locally?

Update: Tried CI_BUILD=1, but it fails on missing files. So it is not easy.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to build with CI_BUILD=1 and also define ENABLE_HEADLESS=0, ENABLE_CONTRIB=0 and SDIST=0.

After the build you can reproduce these lines in the python interpreter. What I've done:

from cv2.version import ci_build, headless
ci_and_not_headless = ci_build and not headless
print (ci_and_not_headless)

In addition, to understand ci_and_not_headless working or not you can print QT_QPA_FONTDIR environment variable. Example:

import cv2, os
print(os.environ["QT_QPA_FONTDIR"])

Comment threadsetup.py Outdated
% cmake_install_dir, 'r') as opencv_init:
opencv_init_data = ""
for line in opencv_init:
opencv_init_replacement = line.replace('importlib.import_module("cv2")', 'importlib.import_module("cv2.cv2")')

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config-X.Y.py defines where we find native binary extension:

PYTHON_EXTENSIONS_PATHS = [
os.path.join(LOADER_DIR, 'python-3.9')
] + PYTHON_EXTENSIONS_PATHS

Currently it is placed to

site-packages/cv2/cv2.cpython-39-x86_64-linux-gnu.so

which is not consistent (lines have no effect)

It would work automatically with import_module("cv2")

  1. if we move binary file to python-X.Y subdirectory (as it is located originally by CMake install)

    site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so
    
  2. Or we replace these config lines to

    PYTHON_EXTENSIONS_PATHS = [
    LOADER_DIR
    ] + PYTHON_EXTENSIONS_PATHS
    

Note: We need to check/update RPATH to point properly to "Lib" directory with dependencies.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the logic when cv2.cpython-39-x86_64-linux-gnu.so locates under site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so, and then libs cannot work properly, because there are the wrong path to libs in config.py.

Added replacing lines into __init__.py file, to define the proper path.

Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we copy all artifacts of "cmake install" automatically "as is"?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is packages = ["cv2", "cv2.data"] line in this file. Perhaps it should be updated somehow (can we remove cv2.data?)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we copy all artifacts of "cmake install" as is, then there will be several folders and files which we do not want to have in a package. We control what should be in a package using rearrange_cmake_output_data in setup.py.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about cv2.data we have to keep it if we want to use haarcascades, because there are no __init__.py file in "cmake install" and we defining it here. However, we can write into a file after building. What do you think about anything of it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@egeakmanegeakman mentioned this pull request Oct 16, 2021
Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@sergregory

Copy link
Copy Markdown
Collaborator

@asenyaev please also merge #565 before the release

@asenyaev

Copy link
Copy Markdown
ContributorAuthor

@asenyaev please also merge #565 before the release

Sure, I'll do it.

@alalek

Copy link
Copy Markdown
Member

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

@asenyaev

asenyaev commented Oct 21, 2021

Copy link
Copy Markdown
ContributorAuthor

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

As far as I remember, we always attached the latest commit for submodules. But in this case, when we have a wrong opencv version, we have to define a tag.

cclauss added a commit to cclauss/opencv-python that referenced this pull request Nov 25, 2021
Related to changes made in opencv#563
$ `flake8 . --count --builtins=ml_ops --select=E9,F63,F7,F82,Y --show-source --statistics`
```
./opencv-python/scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./opencv-python/scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^
./opencv-python/scripts/__init__.py:15:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:16:5: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:16:49: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:17:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:17:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:21:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:22:5: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:22:36: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:23:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
./opencv-python/scripts/__init__.py:23:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
12 F821 undefined name 'LOADER_DIR'
12
```
Comment threadscripts/__init__.py
Comment on lines +1 to +3
PYTHON_EXTENSIONS_PATHS = [
LOADER_DIR
] + PYTHON_EXTENSIONS_PATHS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are undefined names in Python code.
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@asenyaev@sergregory@alalek@asmorkalov@cclauss
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Updated submodules to the release 4.5.4 and added python loader support by asenyaev · Pull Request #563 · opencv/opencv-python · GitHub
Skip to content

Updated submodules to the release 4.5.4 and added python loader support - #563

Merged
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support
Oct 20, 2021
Merged

Updated submodules to the release 4.5.4 and added python loader support#563
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support

Conversation

@asenyaev

Copy link
Copy Markdown
Contributor

No description provided.

@alalekalalek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Comment threadscripts/__init__.py Outdated
@@ -0,0 +1,23 @@
from .cv2 import *
from .data import *

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we don't need these 2 lines

"data" submodule is imported automatically.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we replace import cv2 to import cv2.cv2 for native_module in __init__.py, then we need first line, second can be removed.

Besides, I removed lines with a replacing, because rewrote config file, so, removed it.

Comment threadscripts/__init__.py
try:
from .version import ci_build, headless

ci_and_not_headless = ci_build and not headless

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can verify that case that locally?

Update: Tried CI_BUILD=1, but it fails on missing files. So it is not easy.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to build with CI_BUILD=1 and also define ENABLE_HEADLESS=0, ENABLE_CONTRIB=0 and SDIST=0.

After the build you can reproduce these lines in the python interpreter. What I've done:

from cv2.version import ci_build, headless
ci_and_not_headless = ci_build and not headless
print (ci_and_not_headless)

In addition, to understand ci_and_not_headless working or not you can print QT_QPA_FONTDIR environment variable. Example:

import cv2, os
print(os.environ["QT_QPA_FONTDIR"])

Comment threadsetup.py Outdated
% cmake_install_dir, 'r') as opencv_init:
opencv_init_data = ""
for line in opencv_init:
opencv_init_replacement = line.replace('importlib.import_module("cv2")', 'importlib.import_module("cv2.cv2")')

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config-X.Y.py defines where we find native binary extension:

PYTHON_EXTENSIONS_PATHS = [
os.path.join(LOADER_DIR, 'python-3.9')
] + PYTHON_EXTENSIONS_PATHS

Currently it is placed to

site-packages/cv2/cv2.cpython-39-x86_64-linux-gnu.so

which is not consistent (lines have no effect)

It would work automatically with import_module("cv2")

  1. if we move binary file to python-X.Y subdirectory (as it is located originally by CMake install)

    site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so
    
  2. Or we replace these config lines to

    PYTHON_EXTENSIONS_PATHS = [
    LOADER_DIR
    ] + PYTHON_EXTENSIONS_PATHS
    

Note: We need to check/update RPATH to point properly to "Lib" directory with dependencies.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the logic when cv2.cpython-39-x86_64-linux-gnu.so locates under site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so, and then libs cannot work properly, because there are the wrong path to libs in config.py.

Added replacing lines into __init__.py file, to define the proper path.

Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we copy all artifacts of "cmake install" automatically "as is"?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is packages = ["cv2", "cv2.data"] line in this file. Perhaps it should be updated somehow (can we remove cv2.data?)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we copy all artifacts of "cmake install" as is, then there will be several folders and files which we do not want to have in a package. We control what should be in a package using rearrange_cmake_output_data in setup.py.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about cv2.data we have to keep it if we want to use haarcascades, because there are no __init__.py file in "cmake install" and we defining it here. However, we can write into a file after building. What do you think about anything of it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@egeakmanegeakman mentioned this pull request Oct 16, 2021
Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@sergregory

Copy link
Copy Markdown
Collaborator

@asenyaev please also merge #565 before the release

@asenyaev

Copy link
Copy Markdown
ContributorAuthor

@asenyaev please also merge #565 before the release

Sure, I'll do it.

@alalek

Copy link
Copy Markdown
Member

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

@asenyaev

asenyaev commented Oct 21, 2021

Copy link
Copy Markdown
ContributorAuthor

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

As far as I remember, we always attached the latest commit for submodules. But in this case, when we have a wrong opencv version, we have to define a tag.

cclauss added a commit to cclauss/opencv-python that referenced this pull request Nov 25, 2021
Related to changes made in opencv#563
$ `flake8 . --count --builtins=ml_ops --select=E9,F63,F7,F82,Y --show-source --statistics`
```
./opencv-python/scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./opencv-python/scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^
./opencv-python/scripts/__init__.py:15:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:16:5: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:16:49: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:17:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:17:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:21:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:22:5: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:22:36: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:23:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
./opencv-python/scripts/__init__.py:23:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
12 F821 undefined name 'LOADER_DIR'
12
```
Comment threadscripts/__init__.py
Comment on lines +1 to +3
PYTHON_EXTENSIONS_PATHS = [
LOADER_DIR
] + PYTHON_EXTENSIONS_PATHS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are undefined names in Python code.
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@asenyaev@sergregory@alalek@asmorkalov@cclauss
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Updated submodules to the release 4.5.4 and added python loader support by asenyaev · Pull Request #563 · opencv/opencv-python · GitHub
Skip to content

Updated submodules to the release 4.5.4 and added python loader support - #563

Merged
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support
Oct 20, 2021
Merged

Updated submodules to the release 4.5.4 and added python loader support#563
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support

Conversation

@asenyaev

Copy link
Copy Markdown
Contributor

No description provided.

@alalekalalek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Comment threadscripts/__init__.py Outdated
@@ -0,0 +1,23 @@
from .cv2 import *
from .data import *

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we don't need these 2 lines

"data" submodule is imported automatically.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we replace import cv2 to import cv2.cv2 for native_module in __init__.py, then we need first line, second can be removed.

Besides, I removed lines with a replacing, because rewrote config file, so, removed it.

Comment threadscripts/__init__.py
try:
from .version import ci_build, headless

ci_and_not_headless = ci_build and not headless

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can verify that case that locally?

Update: Tried CI_BUILD=1, but it fails on missing files. So it is not easy.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to build with CI_BUILD=1 and also define ENABLE_HEADLESS=0, ENABLE_CONTRIB=0 and SDIST=0.

After the build you can reproduce these lines in the python interpreter. What I've done:

from cv2.version import ci_build, headless
ci_and_not_headless = ci_build and not headless
print (ci_and_not_headless)

In addition, to understand ci_and_not_headless working or not you can print QT_QPA_FONTDIR environment variable. Example:

import cv2, os
print(os.environ["QT_QPA_FONTDIR"])

Comment threadsetup.py Outdated
% cmake_install_dir, 'r') as opencv_init:
opencv_init_data = ""
for line in opencv_init:
opencv_init_replacement = line.replace('importlib.import_module("cv2")', 'importlib.import_module("cv2.cv2")')

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config-X.Y.py defines where we find native binary extension:

PYTHON_EXTENSIONS_PATHS = [
os.path.join(LOADER_DIR, 'python-3.9')
] + PYTHON_EXTENSIONS_PATHS

Currently it is placed to

site-packages/cv2/cv2.cpython-39-x86_64-linux-gnu.so

which is not consistent (lines have no effect)

It would work automatically with import_module("cv2")

  1. if we move binary file to python-X.Y subdirectory (as it is located originally by CMake install)

    site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so
    
  2. Or we replace these config lines to

    PYTHON_EXTENSIONS_PATHS = [
    LOADER_DIR
    ] + PYTHON_EXTENSIONS_PATHS
    

Note: We need to check/update RPATH to point properly to "Lib" directory with dependencies.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the logic when cv2.cpython-39-x86_64-linux-gnu.so locates under site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so, and then libs cannot work properly, because there are the wrong path to libs in config.py.

Added replacing lines into __init__.py file, to define the proper path.

Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we copy all artifacts of "cmake install" automatically "as is"?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is packages = ["cv2", "cv2.data"] line in this file. Perhaps it should be updated somehow (can we remove cv2.data?)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we copy all artifacts of "cmake install" as is, then there will be several folders and files which we do not want to have in a package. We control what should be in a package using rearrange_cmake_output_data in setup.py.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about cv2.data we have to keep it if we want to use haarcascades, because there are no __init__.py file in "cmake install" and we defining it here. However, we can write into a file after building. What do you think about anything of it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@egeakmanegeakman mentioned this pull request Oct 16, 2021
Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@sergregory

Copy link
Copy Markdown
Collaborator

@asenyaev please also merge #565 before the release

@asenyaev

Copy link
Copy Markdown
ContributorAuthor

@asenyaev please also merge #565 before the release

Sure, I'll do it.

@alalek

Copy link
Copy Markdown
Member

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

@asenyaev

asenyaev commented Oct 21, 2021

Copy link
Copy Markdown
ContributorAuthor

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

As far as I remember, we always attached the latest commit for submodules. But in this case, when we have a wrong opencv version, we have to define a tag.

cclauss added a commit to cclauss/opencv-python that referenced this pull request Nov 25, 2021
Related to changes made in opencv#563
$ `flake8 . --count --builtins=ml_ops --select=E9,F63,F7,F82,Y --show-source --statistics`
```
./opencv-python/scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./opencv-python/scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^
./opencv-python/scripts/__init__.py:15:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:16:5: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:16:49: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:17:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:17:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:21:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:22:5: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:22:36: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:23:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
./opencv-python/scripts/__init__.py:23:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
12 F821 undefined name 'LOADER_DIR'
12
```
Comment threadscripts/__init__.py
Comment on lines +1 to +3
PYTHON_EXTENSIONS_PATHS = [
LOADER_DIR
] + PYTHON_EXTENSIONS_PATHS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are undefined names in Python code.
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@asenyaev@sergregory@alalek@asmorkalov@cclauss
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Updated submodules to the release 4.5.4 and added python loader support by asenyaev · Pull Request #563 · opencv/opencv-python · GitHub
Skip to content

Updated submodules to the release 4.5.4 and added python loader support - #563

Merged
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support
Oct 20, 2021
Merged

Updated submodules to the release 4.5.4 and added python loader support#563
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support

Conversation

@asenyaev

Copy link
Copy Markdown
Contributor

No description provided.

@alalekalalek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Comment threadscripts/__init__.py Outdated
@@ -0,0 +1,23 @@
from .cv2 import *
from .data import *

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we don't need these 2 lines

"data" submodule is imported automatically.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we replace import cv2 to import cv2.cv2 for native_module in __init__.py, then we need first line, second can be removed.

Besides, I removed lines with a replacing, because rewrote config file, so, removed it.

Comment threadscripts/__init__.py
try:
from .version import ci_build, headless

ci_and_not_headless = ci_build and not headless

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can verify that case that locally?

Update: Tried CI_BUILD=1, but it fails on missing files. So it is not easy.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to build with CI_BUILD=1 and also define ENABLE_HEADLESS=0, ENABLE_CONTRIB=0 and SDIST=0.

After the build you can reproduce these lines in the python interpreter. What I've done:

from cv2.version import ci_build, headless
ci_and_not_headless = ci_build and not headless
print (ci_and_not_headless)

In addition, to understand ci_and_not_headless working or not you can print QT_QPA_FONTDIR environment variable. Example:

import cv2, os
print(os.environ["QT_QPA_FONTDIR"])

Comment threadsetup.py Outdated
% cmake_install_dir, 'r') as opencv_init:
opencv_init_data = ""
for line in opencv_init:
opencv_init_replacement = line.replace('importlib.import_module("cv2")', 'importlib.import_module("cv2.cv2")')

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config-X.Y.py defines where we find native binary extension:

PYTHON_EXTENSIONS_PATHS = [
os.path.join(LOADER_DIR, 'python-3.9')
] + PYTHON_EXTENSIONS_PATHS

Currently it is placed to

site-packages/cv2/cv2.cpython-39-x86_64-linux-gnu.so

which is not consistent (lines have no effect)

It would work automatically with import_module("cv2")

  1. if we move binary file to python-X.Y subdirectory (as it is located originally by CMake install)

    site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so
    
  2. Or we replace these config lines to

    PYTHON_EXTENSIONS_PATHS = [
    LOADER_DIR
    ] + PYTHON_EXTENSIONS_PATHS
    

Note: We need to check/update RPATH to point properly to "Lib" directory with dependencies.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the logic when cv2.cpython-39-x86_64-linux-gnu.so locates under site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so, and then libs cannot work properly, because there are the wrong path to libs in config.py.

Added replacing lines into __init__.py file, to define the proper path.

Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we copy all artifacts of "cmake install" automatically "as is"?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is packages = ["cv2", "cv2.data"] line in this file. Perhaps it should be updated somehow (can we remove cv2.data?)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we copy all artifacts of "cmake install" as is, then there will be several folders and files which we do not want to have in a package. We control what should be in a package using rearrange_cmake_output_data in setup.py.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about cv2.data we have to keep it if we want to use haarcascades, because there are no __init__.py file in "cmake install" and we defining it here. However, we can write into a file after building. What do you think about anything of it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@egeakmanegeakman mentioned this pull request Oct 16, 2021
Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@sergregory

Copy link
Copy Markdown
Collaborator

@asenyaev please also merge #565 before the release

@asenyaev

Copy link
Copy Markdown
ContributorAuthor

@asenyaev please also merge #565 before the release

Sure, I'll do it.

@alalek

Copy link
Copy Markdown
Member

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

@asenyaev

asenyaev commented Oct 21, 2021

Copy link
Copy Markdown
ContributorAuthor

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

As far as I remember, we always attached the latest commit for submodules. But in this case, when we have a wrong opencv version, we have to define a tag.

cclauss added a commit to cclauss/opencv-python that referenced this pull request Nov 25, 2021
Related to changes made in opencv#563
$ `flake8 . --count --builtins=ml_ops --select=E9,F63,F7,F82,Y --show-source --statistics`
```
./opencv-python/scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./opencv-python/scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^
./opencv-python/scripts/__init__.py:15:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:16:5: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:16:49: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:17:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:17:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:21:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:22:5: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:22:36: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:23:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
./opencv-python/scripts/__init__.py:23:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
12 F821 undefined name 'LOADER_DIR'
12
```
Comment threadscripts/__init__.py
Comment on lines +1 to +3
PYTHON_EXTENSIONS_PATHS = [
LOADER_DIR
] + PYTHON_EXTENSIONS_PATHS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are undefined names in Python code.
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@asenyaev@sergregory@alalek@asmorkalov@cclauss
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Updated submodules to the release 4.5.4 and added python loader support by asenyaev · Pull Request #563 · opencv/opencv-python · GitHub
Skip to content

Updated submodules to the release 4.5.4 and added python loader support - #563

Merged
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support
Oct 20, 2021
Merged

Updated submodules to the release 4.5.4 and added python loader support#563
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support

Conversation

@asenyaev

Copy link
Copy Markdown
Contributor

No description provided.

@alalekalalek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Comment threadscripts/__init__.py Outdated
@@ -0,0 +1,23 @@
from .cv2 import *
from .data import *

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we don't need these 2 lines

"data" submodule is imported automatically.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we replace import cv2 to import cv2.cv2 for native_module in __init__.py, then we need first line, second can be removed.

Besides, I removed lines with a replacing, because rewrote config file, so, removed it.

Comment threadscripts/__init__.py
try:
from .version import ci_build, headless

ci_and_not_headless = ci_build and not headless

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can verify that case that locally?

Update: Tried CI_BUILD=1, but it fails on missing files. So it is not easy.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to build with CI_BUILD=1 and also define ENABLE_HEADLESS=0, ENABLE_CONTRIB=0 and SDIST=0.

After the build you can reproduce these lines in the python interpreter. What I've done:

from cv2.version import ci_build, headless
ci_and_not_headless = ci_build and not headless
print (ci_and_not_headless)

In addition, to understand ci_and_not_headless working or not you can print QT_QPA_FONTDIR environment variable. Example:

import cv2, os
print(os.environ["QT_QPA_FONTDIR"])

Comment threadsetup.py Outdated
% cmake_install_dir, 'r') as opencv_init:
opencv_init_data = ""
for line in opencv_init:
opencv_init_replacement = line.replace('importlib.import_module("cv2")', 'importlib.import_module("cv2.cv2")')

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config-X.Y.py defines where we find native binary extension:

PYTHON_EXTENSIONS_PATHS = [
os.path.join(LOADER_DIR, 'python-3.9')
] + PYTHON_EXTENSIONS_PATHS

Currently it is placed to

site-packages/cv2/cv2.cpython-39-x86_64-linux-gnu.so

which is not consistent (lines have no effect)

It would work automatically with import_module("cv2")

  1. if we move binary file to python-X.Y subdirectory (as it is located originally by CMake install)

    site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so
    
  2. Or we replace these config lines to

    PYTHON_EXTENSIONS_PATHS = [
    LOADER_DIR
    ] + PYTHON_EXTENSIONS_PATHS
    

Note: We need to check/update RPATH to point properly to "Lib" directory with dependencies.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the logic when cv2.cpython-39-x86_64-linux-gnu.so locates under site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so, and then libs cannot work properly, because there are the wrong path to libs in config.py.

Added replacing lines into __init__.py file, to define the proper path.

Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we copy all artifacts of "cmake install" automatically "as is"?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is packages = ["cv2", "cv2.data"] line in this file. Perhaps it should be updated somehow (can we remove cv2.data?)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we copy all artifacts of "cmake install" as is, then there will be several folders and files which we do not want to have in a package. We control what should be in a package using rearrange_cmake_output_data in setup.py.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about cv2.data we have to keep it if we want to use haarcascades, because there are no __init__.py file in "cmake install" and we defining it here. However, we can write into a file after building. What do you think about anything of it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@egeakmanegeakman mentioned this pull request Oct 16, 2021
Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@sergregory

Copy link
Copy Markdown
Collaborator

@asenyaev please also merge #565 before the release

@asenyaev

Copy link
Copy Markdown
ContributorAuthor

@asenyaev please also merge #565 before the release

Sure, I'll do it.

@alalek

Copy link
Copy Markdown
Member

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

@asenyaev

asenyaev commented Oct 21, 2021

Copy link
Copy Markdown
ContributorAuthor

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

As far as I remember, we always attached the latest commit for submodules. But in this case, when we have a wrong opencv version, we have to define a tag.

cclauss added a commit to cclauss/opencv-python that referenced this pull request Nov 25, 2021
Related to changes made in opencv#563
$ `flake8 . --count --builtins=ml_ops --select=E9,F63,F7,F82,Y --show-source --statistics`
```
./opencv-python/scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./opencv-python/scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^
./opencv-python/scripts/__init__.py:15:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:16:5: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:16:49: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:17:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:17:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:21:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:22:5: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:22:36: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:23:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
./opencv-python/scripts/__init__.py:23:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
12 F821 undefined name 'LOADER_DIR'
12
```
Comment threadscripts/__init__.py
Comment on lines +1 to +3
PYTHON_EXTENSIONS_PATHS = [
LOADER_DIR
] + PYTHON_EXTENSIONS_PATHS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are undefined names in Python code.
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@asenyaev@sergregory@alalek@asmorkalov@cclauss
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Updated submodules to the release 4.5.4 and added python loader support by asenyaev · Pull Request #563 · opencv/opencv-python · GitHub
Skip to content

Updated submodules to the release 4.5.4 and added python loader support - #563

Merged
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support
Oct 20, 2021
Merged

Updated submodules to the release 4.5.4 and added python loader support#563
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support

Conversation

@asenyaev

Copy link
Copy Markdown
Contributor

No description provided.

@alalekalalek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Comment threadscripts/__init__.py Outdated
@@ -0,0 +1,23 @@
from .cv2 import *
from .data import *

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we don't need these 2 lines

"data" submodule is imported automatically.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we replace import cv2 to import cv2.cv2 for native_module in __init__.py, then we need first line, second can be removed.

Besides, I removed lines with a replacing, because rewrote config file, so, removed it.

Comment threadscripts/__init__.py
try:
from .version import ci_build, headless

ci_and_not_headless = ci_build and not headless

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can verify that case that locally?

Update: Tried CI_BUILD=1, but it fails on missing files. So it is not easy.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to build with CI_BUILD=1 and also define ENABLE_HEADLESS=0, ENABLE_CONTRIB=0 and SDIST=0.

After the build you can reproduce these lines in the python interpreter. What I've done:

from cv2.version import ci_build, headless
ci_and_not_headless = ci_build and not headless
print (ci_and_not_headless)

In addition, to understand ci_and_not_headless working or not you can print QT_QPA_FONTDIR environment variable. Example:

import cv2, os
print(os.environ["QT_QPA_FONTDIR"])

Comment threadsetup.py Outdated
% cmake_install_dir, 'r') as opencv_init:
opencv_init_data = ""
for line in opencv_init:
opencv_init_replacement = line.replace('importlib.import_module("cv2")', 'importlib.import_module("cv2.cv2")')

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config-X.Y.py defines where we find native binary extension:

PYTHON_EXTENSIONS_PATHS = [
os.path.join(LOADER_DIR, 'python-3.9')
] + PYTHON_EXTENSIONS_PATHS

Currently it is placed to

site-packages/cv2/cv2.cpython-39-x86_64-linux-gnu.so

which is not consistent (lines have no effect)

It would work automatically with import_module("cv2")

  1. if we move binary file to python-X.Y subdirectory (as it is located originally by CMake install)

    site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so
    
  2. Or we replace these config lines to

    PYTHON_EXTENSIONS_PATHS = [
    LOADER_DIR
    ] + PYTHON_EXTENSIONS_PATHS
    

Note: We need to check/update RPATH to point properly to "Lib" directory with dependencies.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the logic when cv2.cpython-39-x86_64-linux-gnu.so locates under site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so, and then libs cannot work properly, because there are the wrong path to libs in config.py.

Added replacing lines into __init__.py file, to define the proper path.

Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we copy all artifacts of "cmake install" automatically "as is"?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is packages = ["cv2", "cv2.data"] line in this file. Perhaps it should be updated somehow (can we remove cv2.data?)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we copy all artifacts of "cmake install" as is, then there will be several folders and files which we do not want to have in a package. We control what should be in a package using rearrange_cmake_output_data in setup.py.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about cv2.data we have to keep it if we want to use haarcascades, because there are no __init__.py file in "cmake install" and we defining it here. However, we can write into a file after building. What do you think about anything of it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@egeakmanegeakman mentioned this pull request Oct 16, 2021
Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@sergregory

Copy link
Copy Markdown
Collaborator

@asenyaev please also merge #565 before the release

@asenyaev

Copy link
Copy Markdown
ContributorAuthor

@asenyaev please also merge #565 before the release

Sure, I'll do it.

@alalek

Copy link
Copy Markdown
Member

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

@asenyaev

asenyaev commented Oct 21, 2021

Copy link
Copy Markdown
ContributorAuthor

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

As far as I remember, we always attached the latest commit for submodules. But in this case, when we have a wrong opencv version, we have to define a tag.

cclauss added a commit to cclauss/opencv-python that referenced this pull request Nov 25, 2021
Related to changes made in opencv#563
$ `flake8 . --count --builtins=ml_ops --select=E9,F63,F7,F82,Y --show-source --statistics`
```
./opencv-python/scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./opencv-python/scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^
./opencv-python/scripts/__init__.py:15:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:16:5: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:16:49: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:17:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:17:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:21:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:22:5: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:22:36: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:23:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
./opencv-python/scripts/__init__.py:23:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
12 F821 undefined name 'LOADER_DIR'
12
```
Comment threadscripts/__init__.py
Comment on lines +1 to +3
PYTHON_EXTENSIONS_PATHS = [
LOADER_DIR
] + PYTHON_EXTENSIONS_PATHS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are undefined names in Python code.
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@asenyaev@sergregory@alalek@asmorkalov@cclauss
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Updated submodules to the release 4.5.4 and added python loader support by asenyaev · Pull Request #563 · opencv/opencv-python · GitHub
Skip to content

Updated submodules to the release 4.5.4 and added python loader support - #563

Merged
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support
Oct 20, 2021
Merged

Updated submodules to the release 4.5.4 and added python loader support#563
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support

Conversation

@asenyaev

Copy link
Copy Markdown
Contributor

No description provided.

@alalekalalek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Comment threadscripts/__init__.py Outdated
@@ -0,0 +1,23 @@
from .cv2 import *
from .data import *

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we don't need these 2 lines

"data" submodule is imported automatically.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we replace import cv2 to import cv2.cv2 for native_module in __init__.py, then we need first line, second can be removed.

Besides, I removed lines with a replacing, because rewrote config file, so, removed it.

Comment threadscripts/__init__.py
try:
from .version import ci_build, headless

ci_and_not_headless = ci_build and not headless

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can verify that case that locally?

Update: Tried CI_BUILD=1, but it fails on missing files. So it is not easy.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to build with CI_BUILD=1 and also define ENABLE_HEADLESS=0, ENABLE_CONTRIB=0 and SDIST=0.

After the build you can reproduce these lines in the python interpreter. What I've done:

from cv2.version import ci_build, headless
ci_and_not_headless = ci_build and not headless
print (ci_and_not_headless)

In addition, to understand ci_and_not_headless working or not you can print QT_QPA_FONTDIR environment variable. Example:

import cv2, os
print(os.environ["QT_QPA_FONTDIR"])

Comment threadsetup.py Outdated
% cmake_install_dir, 'r') as opencv_init:
opencv_init_data = ""
for line in opencv_init:
opencv_init_replacement = line.replace('importlib.import_module("cv2")', 'importlib.import_module("cv2.cv2")')

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config-X.Y.py defines where we find native binary extension:

PYTHON_EXTENSIONS_PATHS = [
os.path.join(LOADER_DIR, 'python-3.9')
] + PYTHON_EXTENSIONS_PATHS

Currently it is placed to

site-packages/cv2/cv2.cpython-39-x86_64-linux-gnu.so

which is not consistent (lines have no effect)

It would work automatically with import_module("cv2")

  1. if we move binary file to python-X.Y subdirectory (as it is located originally by CMake install)

    site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so
    
  2. Or we replace these config lines to

    PYTHON_EXTENSIONS_PATHS = [
    LOADER_DIR
    ] + PYTHON_EXTENSIONS_PATHS
    

Note: We need to check/update RPATH to point properly to "Lib" directory with dependencies.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the logic when cv2.cpython-39-x86_64-linux-gnu.so locates under site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so, and then libs cannot work properly, because there are the wrong path to libs in config.py.

Added replacing lines into __init__.py file, to define the proper path.

Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we copy all artifacts of "cmake install" automatically "as is"?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is packages = ["cv2", "cv2.data"] line in this file. Perhaps it should be updated somehow (can we remove cv2.data?)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we copy all artifacts of "cmake install" as is, then there will be several folders and files which we do not want to have in a package. We control what should be in a package using rearrange_cmake_output_data in setup.py.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about cv2.data we have to keep it if we want to use haarcascades, because there are no __init__.py file in "cmake install" and we defining it here. However, we can write into a file after building. What do you think about anything of it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@egeakmanegeakman mentioned this pull request Oct 16, 2021
Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@sergregory

Copy link
Copy Markdown
Collaborator

@asenyaev please also merge #565 before the release

@asenyaev

Copy link
Copy Markdown
ContributorAuthor

@asenyaev please also merge #565 before the release

Sure, I'll do it.

@alalek

Copy link
Copy Markdown
Member

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

@asenyaev

asenyaev commented Oct 21, 2021

Copy link
Copy Markdown
ContributorAuthor

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

As far as I remember, we always attached the latest commit for submodules. But in this case, when we have a wrong opencv version, we have to define a tag.

cclauss added a commit to cclauss/opencv-python that referenced this pull request Nov 25, 2021
Related to changes made in opencv#563
$ `flake8 . --count --builtins=ml_ops --select=E9,F63,F7,F82,Y --show-source --statistics`
```
./opencv-python/scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./opencv-python/scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^
./opencv-python/scripts/__init__.py:15:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:16:5: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:16:49: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:17:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:17:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:21:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:22:5: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:22:36: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:23:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
./opencv-python/scripts/__init__.py:23:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
12 F821 undefined name 'LOADER_DIR'
12
```
Comment threadscripts/__init__.py
Comment on lines +1 to +3
PYTHON_EXTENSIONS_PATHS = [
LOADER_DIR
] + PYTHON_EXTENSIONS_PATHS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are undefined names in Python code.
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@asenyaev@sergregory@alalek@asmorkalov@cclauss
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Updated submodules to the release 4.5.4 and added python loader support by asenyaev · Pull Request #563 · opencv/opencv-python · GitHub
Skip to content

Updated submodules to the release 4.5.4 and added python loader support - #563

Merged
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support
Oct 20, 2021
Merged

Updated submodules to the release 4.5.4 and added python loader support#563
asenyaev merged 3 commits into
opencv:masterfrom
asenyaev:asen/python_loader_support

Conversation

@asenyaev

Copy link
Copy Markdown
Contributor

No description provided.

@alalekalalek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Comment threadscripts/__init__.py Outdated
@@ -0,0 +1,23 @@
from .cv2 import *
from .data import *

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we don't need these 2 lines

"data" submodule is imported automatically.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we replace import cv2 to import cv2.cv2 for native_module in __init__.py, then we need first line, second can be removed.

Besides, I removed lines with a replacing, because rewrote config file, so, removed it.

Comment threadscripts/__init__.py
try:
from .version import ci_build, headless

ci_and_not_headless = ci_build and not headless

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can verify that case that locally?

Update: Tried CI_BUILD=1, but it fails on missing files. So it is not easy.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to build with CI_BUILD=1 and also define ENABLE_HEADLESS=0, ENABLE_CONTRIB=0 and SDIST=0.

After the build you can reproduce these lines in the python interpreter. What I've done:

from cv2.version import ci_build, headless
ci_and_not_headless = ci_build and not headless
print (ci_and_not_headless)

In addition, to understand ci_and_not_headless working or not you can print QT_QPA_FONTDIR environment variable. Example:

import cv2, os
print(os.environ["QT_QPA_FONTDIR"])

Comment threadsetup.py Outdated
% cmake_install_dir, 'r') as opencv_init:
opencv_init_data = ""
for line in opencv_init:
opencv_init_replacement = line.replace('importlib.import_module("cv2")', 'importlib.import_module("cv2.cv2")')

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config-X.Y.py defines where we find native binary extension:

PYTHON_EXTENSIONS_PATHS = [
os.path.join(LOADER_DIR, 'python-3.9')
] + PYTHON_EXTENSIONS_PATHS

Currently it is placed to

site-packages/cv2/cv2.cpython-39-x86_64-linux-gnu.so

which is not consistent (lines have no effect)

It would work automatically with import_module("cv2")

  1. if we move binary file to python-X.Y subdirectory (as it is located originally by CMake install)

    site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so
    
  2. Or we replace these config lines to

    PYTHON_EXTENSIONS_PATHS = [
    LOADER_DIR
    ] + PYTHON_EXTENSIONS_PATHS
    

Note: We need to check/update RPATH to point properly to "Lib" directory with dependencies.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the logic when cv2.cpython-39-x86_64-linux-gnu.so locates under site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so, and then libs cannot work properly, because there are the wrong path to libs in config.py.

Added replacing lines into __init__.py file, to define the proper path.

Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

@alalekalalekOct 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we copy all artifacts of "cmake install" automatically "as is"?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is packages = ["cv2", "cv2.data"] line in this file. Perhaps it should be updated somehow (can we remove cv2.data?)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we copy all artifacts of "cmake install" as is, then there will be several folders and files which we do not want to have in a package. We control what should be in a package using rearrange_cmake_output_data in setup.py.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about cv2.data we have to keep it if we want to use haarcascades, because there are no __init__.py file in "cmake install" and we defining it here. However, we can write into a file after building. What do you think about anything of it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@egeakmanegeakman mentioned this pull request Oct 16, 2021
Comment threadsetup.py
Comment on lines +125 to +136
"cv2.gapi": [
"python/cv2" + r"/gapi/.*\.py"
],
"cv2.mat_wrapper": [
"python/cv2" + r"/mat_wrapper/.*\.py"
],
"cv2.misc": [
"python/cv2" + r"/misc/.*\.py"
],
"cv2.utils": [
"python/cv2" + r"/utils/.*\.py"
],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

@sergregory

Copy link
Copy Markdown
Collaborator

@asenyaev please also merge #565 before the release

@asenyaev

Copy link
Copy Markdown
ContributorAuthor

@asenyaev please also merge #565 before the release

Sure, I'll do it.

@alalek

Copy link
Copy Markdown
Member

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

@asenyaev

asenyaev commented Oct 21, 2021

Copy link
Copy Markdown
ContributorAuthor

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

As far as I remember, we always attached the latest commit for submodules. But in this case, when we have a wrong opencv version, we have to define a tag.

cclauss added a commit to cclauss/opencv-python that referenced this pull request Nov 25, 2021
Related to changes made in opencv#563
$ `flake8 . --count --builtins=ml_ops --select=E9,F63,F7,F82,Y --show-source --statistics`
```
./opencv-python/scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./opencv-python/scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^
./opencv-python/scripts/__init__.py:15:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:16:5: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:16:49: F821 undefined name 'os'
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join(
^
./opencv-python/scripts/__init__.py:17:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:17:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "plugins"
^
./opencv-python/scripts/__init__.py:21:4: F821 undefined name 'sys'
if sys.platform.startswith("linux") and ci_and_not_headless:
^
./opencv-python/scripts/__init__.py:22:5: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:22:36: F821 undefined name 'os'
os.environ["QT_QPA_FONTDIR"] = os.path.join(
^
./opencv-python/scripts/__init__.py:23:9: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
./opencv-python/scripts/__init__.py:23:25: F821 undefined name 'os'
os.path.dirname(os.path.abspath(__file__)), "qt", "fonts"
^
12 F821 undefined name 'LOADER_DIR'
12
```
Comment threadscripts/__init__.py
Comment on lines +1 to +3
PYTHON_EXTENSIONS_PATHS = [
LOADER_DIR
] + PYTHON_EXTENSIONS_PATHS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are undefined names in Python code.
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR'
LOADER_DIR
^
./scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS'
] + PYTHON_EXTENSIONS_PATHS
^

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@asenyaev@sergregory@alalek@asmorkalov@cclauss