GH-882: Add support for loading native library from a user specified location - #883

Merged
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882
Oct 27, 2025
Merged

GH-882: Add support for loading native library from a user specified location#883
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882

Conversation

@pepijnve

Copy link
Copy Markdown
Contributor

What's Changed

Add an opt-in code path that attempts to load the native library relative to the path specified by the arrow.cdata.library.path system property.

Closes#882.

@pepijnvepepijnve changed the title Add support for loading native library from a user specified locationGH-882: Add support for loading native library from a user specified locationOct 10, 2025
@github-actions

This comment has been minimized.

@lidavidmlidavidm added the enhancement PRs that add or improve features. label Oct 10, 2025
@github-actionsgithub-actionsBot added this to the 18.4.0 milestone Oct 10, 2025
@lidavidm

Copy link
Copy Markdown
Member

Is there precedent for this kind of flag in other JNI libraries?

@pepijnve

pepijnve commented Oct 10, 2025

Copy link
Copy Markdown
ContributorAuthor

Is there precedent for this kind of flag in other JNI libraries?

See for instance https://github.com/xerial/sqlite-jdbc/blob/755cf1d0223a83b97909fca43c8027e3e02bc021/src/main/java/org/sqlite/SQLiteJDBCLoader.java#L300 and https://github.com/jnr/jffi/blob/master/src/main/java/com/kenai/jffi/internal/StubLoader.java#L302

A possible alternative could be to check java.library.path indirectly by calling System.load first in the try/catch block. I did not go for this option since it might make existing systems pick up a library from somewhere unintentionally. The custom system property makes this explicitly opt-in.

As described in the issue, loading executable code from /tmp is problematic since it's sometimes forbidden by security settings of the host OS. This is an actual deployment problem for the system I work on since we have no control over those settings.

@lidavidm

Copy link
Copy Markdown
Member

Thanks. I'll give this a fuller review but if there's precedent then the idea is good by me.

return;
}
} catch (UnsatisfiedLinkError e) {
// Ignore this error and fall back to extracting from the JAR file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, would it be better to error if the property is provided but the file doesn't exist? That way you don't unexpectedly/silently fall back when you intended to load from somewhere else.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my specific application the native library will be have been stripped from the jar and loading will fail so the end result will be the same either way.

I modeled this on examples like the ones I referenced where possible alternatives are tried in a specified order. Happy to change this to fail early though. Let me know what you would prefer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we can leave it as is then.

@lidavidm

Copy link
Copy Markdown
Member

Ah, the JNI CI is still broken for now...we can queue this for after #865

@lidavidm

Copy link
Copy Markdown
Member

Ah but in the meantime do you mind helping the linter out?

Error: Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-c-data: The following files had format violations:
Error: src/main/java/org/apache/arrow/c/jni/JniLoader.java
Error: @@ -91,8 +91,7 @@
Error: ······}
Error: ····}
Error: Error: -····final·String·libraryToLoad·=
Error: -········name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: +····final·String·libraryToLoad·=·name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: ····try·{
Error: ······File·temp·=
Error: ··········File.createTempFile("jnilib-",·".tmp",·new·File(System.getProperty("java.io.tmpdir")));
Error: Run 'mvn spotless:apply' to fix these violations.
Error: -> [Help 1]

@lidavidm

Copy link
Copy Markdown
Member

Ok, please rebase and CI should be fixed!

@lidavidmlidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failure is #893

@lidavidm
lidavidm merged commit ed81e59 into apache:mainOct 27, 2025
23 of 25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementPRs that add or improve features.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow cdata native library to be loaded from user specified location

2 participants

@pepijnve@lidavidm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

GH-882: Add support for loading native library from a user specified location - #883

Merged
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882
Oct 27, 2025
Merged

GH-882: Add support for loading native library from a user specified location#883
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882

Conversation

@pepijnve

Copy link
Copy Markdown
Contributor

What's Changed

Add an opt-in code path that attempts to load the native library relative to the path specified by the arrow.cdata.library.path system property.

Closes#882.

@pepijnvepepijnve changed the title Add support for loading native library from a user specified locationGH-882: Add support for loading native library from a user specified locationOct 10, 2025
@github-actions

This comment has been minimized.

@lidavidmlidavidm added the enhancement PRs that add or improve features. label Oct 10, 2025
@github-actionsgithub-actionsBot added this to the 18.4.0 milestone Oct 10, 2025
@lidavidm

Copy link
Copy Markdown
Member

Is there precedent for this kind of flag in other JNI libraries?

@pepijnve

pepijnve commented Oct 10, 2025

Copy link
Copy Markdown
ContributorAuthor

Is there precedent for this kind of flag in other JNI libraries?

See for instance https://github.com/xerial/sqlite-jdbc/blob/755cf1d0223a83b97909fca43c8027e3e02bc021/src/main/java/org/sqlite/SQLiteJDBCLoader.java#L300 and https://github.com/jnr/jffi/blob/master/src/main/java/com/kenai/jffi/internal/StubLoader.java#L302

A possible alternative could be to check java.library.path indirectly by calling System.load first in the try/catch block. I did not go for this option since it might make existing systems pick up a library from somewhere unintentionally. The custom system property makes this explicitly opt-in.

As described in the issue, loading executable code from /tmp is problematic since it's sometimes forbidden by security settings of the host OS. This is an actual deployment problem for the system I work on since we have no control over those settings.

@lidavidm

Copy link
Copy Markdown
Member

Thanks. I'll give this a fuller review but if there's precedent then the idea is good by me.

return;
}
} catch (UnsatisfiedLinkError e) {
// Ignore this error and fall back to extracting from the JAR file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, would it be better to error if the property is provided but the file doesn't exist? That way you don't unexpectedly/silently fall back when you intended to load from somewhere else.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my specific application the native library will be have been stripped from the jar and loading will fail so the end result will be the same either way.

I modeled this on examples like the ones I referenced where possible alternatives are tried in a specified order. Happy to change this to fail early though. Let me know what you would prefer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we can leave it as is then.

@lidavidm

Copy link
Copy Markdown
Member

Ah, the JNI CI is still broken for now...we can queue this for after #865

@lidavidm

Copy link
Copy Markdown
Member

Ah but in the meantime do you mind helping the linter out?

Error: Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-c-data: The following files had format violations:
Error: src/main/java/org/apache/arrow/c/jni/JniLoader.java
Error: @@ -91,8 +91,7 @@
Error: ······}
Error: ····}
Error: Error: -····final·String·libraryToLoad·=
Error: -········name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: +····final·String·libraryToLoad·=·name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: ····try·{
Error: ······File·temp·=
Error: ··········File.createTempFile("jnilib-",·".tmp",·new·File(System.getProperty("java.io.tmpdir")));
Error: Run 'mvn spotless:apply' to fix these violations.
Error: -> [Help 1]

@lidavidm

Copy link
Copy Markdown
Member

Ok, please rebase and CI should be fixed!

@lidavidmlidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failure is #893

@lidavidm
lidavidm merged commit ed81e59 into apache:mainOct 27, 2025
23 of 25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementPRs that add or improve features.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow cdata native library to be loaded from user specified location

2 participants

@pepijnve@lidavidm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

GH-882: Add support for loading native library from a user specified location - #883

Merged
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882
Oct 27, 2025
Merged

GH-882: Add support for loading native library from a user specified location#883
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882

Conversation

@pepijnve

Copy link
Copy Markdown
Contributor

What's Changed

Add an opt-in code path that attempts to load the native library relative to the path specified by the arrow.cdata.library.path system property.

Closes#882.

@pepijnvepepijnve changed the title Add support for loading native library from a user specified locationGH-882: Add support for loading native library from a user specified locationOct 10, 2025
@github-actions

This comment has been minimized.

@lidavidmlidavidm added the enhancement PRs that add or improve features. label Oct 10, 2025
@github-actionsgithub-actionsBot added this to the 18.4.0 milestone Oct 10, 2025
@lidavidm

Copy link
Copy Markdown
Member

Is there precedent for this kind of flag in other JNI libraries?

@pepijnve

pepijnve commented Oct 10, 2025

Copy link
Copy Markdown
ContributorAuthor

Is there precedent for this kind of flag in other JNI libraries?

See for instance https://github.com/xerial/sqlite-jdbc/blob/755cf1d0223a83b97909fca43c8027e3e02bc021/src/main/java/org/sqlite/SQLiteJDBCLoader.java#L300 and https://github.com/jnr/jffi/blob/master/src/main/java/com/kenai/jffi/internal/StubLoader.java#L302

A possible alternative could be to check java.library.path indirectly by calling System.load first in the try/catch block. I did not go for this option since it might make existing systems pick up a library from somewhere unintentionally. The custom system property makes this explicitly opt-in.

As described in the issue, loading executable code from /tmp is problematic since it's sometimes forbidden by security settings of the host OS. This is an actual deployment problem for the system I work on since we have no control over those settings.

@lidavidm

Copy link
Copy Markdown
Member

Thanks. I'll give this a fuller review but if there's precedent then the idea is good by me.

return;
}
} catch (UnsatisfiedLinkError e) {
// Ignore this error and fall back to extracting from the JAR file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, would it be better to error if the property is provided but the file doesn't exist? That way you don't unexpectedly/silently fall back when you intended to load from somewhere else.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my specific application the native library will be have been stripped from the jar and loading will fail so the end result will be the same either way.

I modeled this on examples like the ones I referenced where possible alternatives are tried in a specified order. Happy to change this to fail early though. Let me know what you would prefer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we can leave it as is then.

@lidavidm

Copy link
Copy Markdown
Member

Ah, the JNI CI is still broken for now...we can queue this for after #865

@lidavidm

Copy link
Copy Markdown
Member

Ah but in the meantime do you mind helping the linter out?

Error: Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-c-data: The following files had format violations:
Error: src/main/java/org/apache/arrow/c/jni/JniLoader.java
Error: @@ -91,8 +91,7 @@
Error: ······}
Error: ····}
Error: Error: -····final·String·libraryToLoad·=
Error: -········name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: +····final·String·libraryToLoad·=·name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: ····try·{
Error: ······File·temp·=
Error: ··········File.createTempFile("jnilib-",·".tmp",·new·File(System.getProperty("java.io.tmpdir")));
Error: Run 'mvn spotless:apply' to fix these violations.
Error: -> [Help 1]

@lidavidm

Copy link
Copy Markdown
Member

Ok, please rebase and CI should be fixed!

@lidavidmlidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failure is #893

@lidavidm
lidavidm merged commit ed81e59 into apache:mainOct 27, 2025
23 of 25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementPRs that add or improve features.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow cdata native library to be loaded from user specified location

2 participants

@pepijnve@lidavidm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

GH-882: Add support for loading native library from a user specified location - #883

Merged
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882
Oct 27, 2025
Merged

GH-882: Add support for loading native library from a user specified location#883
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882

Conversation

@pepijnve

Copy link
Copy Markdown
Contributor

What's Changed

Add an opt-in code path that attempts to load the native library relative to the path specified by the arrow.cdata.library.path system property.

Closes#882.

@pepijnvepepijnve changed the title Add support for loading native library from a user specified locationGH-882: Add support for loading native library from a user specified locationOct 10, 2025
@github-actions

This comment has been minimized.

@lidavidmlidavidm added the enhancement PRs that add or improve features. label Oct 10, 2025
@github-actionsgithub-actionsBot added this to the 18.4.0 milestone Oct 10, 2025
@lidavidm

Copy link
Copy Markdown
Member

Is there precedent for this kind of flag in other JNI libraries?

@pepijnve

pepijnve commented Oct 10, 2025

Copy link
Copy Markdown
ContributorAuthor

Is there precedent for this kind of flag in other JNI libraries?

See for instance https://github.com/xerial/sqlite-jdbc/blob/755cf1d0223a83b97909fca43c8027e3e02bc021/src/main/java/org/sqlite/SQLiteJDBCLoader.java#L300 and https://github.com/jnr/jffi/blob/master/src/main/java/com/kenai/jffi/internal/StubLoader.java#L302

A possible alternative could be to check java.library.path indirectly by calling System.load first in the try/catch block. I did not go for this option since it might make existing systems pick up a library from somewhere unintentionally. The custom system property makes this explicitly opt-in.

As described in the issue, loading executable code from /tmp is problematic since it's sometimes forbidden by security settings of the host OS. This is an actual deployment problem for the system I work on since we have no control over those settings.

@lidavidm

Copy link
Copy Markdown
Member

Thanks. I'll give this a fuller review but if there's precedent then the idea is good by me.

return;
}
} catch (UnsatisfiedLinkError e) {
// Ignore this error and fall back to extracting from the JAR file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, would it be better to error if the property is provided but the file doesn't exist? That way you don't unexpectedly/silently fall back when you intended to load from somewhere else.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my specific application the native library will be have been stripped from the jar and loading will fail so the end result will be the same either way.

I modeled this on examples like the ones I referenced where possible alternatives are tried in a specified order. Happy to change this to fail early though. Let me know what you would prefer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we can leave it as is then.

@lidavidm

Copy link
Copy Markdown
Member

Ah, the JNI CI is still broken for now...we can queue this for after #865

@lidavidm

Copy link
Copy Markdown
Member

Ah but in the meantime do you mind helping the linter out?

Error: Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-c-data: The following files had format violations:
Error: src/main/java/org/apache/arrow/c/jni/JniLoader.java
Error: @@ -91,8 +91,7 @@
Error: ······}
Error: ····}
Error: Error: -····final·String·libraryToLoad·=
Error: -········name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: +····final·String·libraryToLoad·=·name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: ····try·{
Error: ······File·temp·=
Error: ··········File.createTempFile("jnilib-",·".tmp",·new·File(System.getProperty("java.io.tmpdir")));
Error: Run 'mvn spotless:apply' to fix these violations.
Error: -> [Help 1]

@lidavidm

Copy link
Copy Markdown
Member

Ok, please rebase and CI should be fixed!

@lidavidmlidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failure is #893

@lidavidm
lidavidm merged commit ed81e59 into apache:mainOct 27, 2025
23 of 25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementPRs that add or improve features.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow cdata native library to be loaded from user specified location

2 participants

@pepijnve@lidavidm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

GH-882: Add support for loading native library from a user specified location - #883

Merged
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882
Oct 27, 2025
Merged

GH-882: Add support for loading native library from a user specified location#883
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882

Conversation

@pepijnve

Copy link
Copy Markdown
Contributor

What's Changed

Add an opt-in code path that attempts to load the native library relative to the path specified by the arrow.cdata.library.path system property.

Closes#882.

@pepijnvepepijnve changed the title Add support for loading native library from a user specified locationGH-882: Add support for loading native library from a user specified locationOct 10, 2025
@github-actions

This comment has been minimized.

@lidavidmlidavidm added the enhancement PRs that add or improve features. label Oct 10, 2025
@github-actionsgithub-actionsBot added this to the 18.4.0 milestone Oct 10, 2025
@lidavidm

Copy link
Copy Markdown
Member

Is there precedent for this kind of flag in other JNI libraries?

@pepijnve

pepijnve commented Oct 10, 2025

Copy link
Copy Markdown
ContributorAuthor

Is there precedent for this kind of flag in other JNI libraries?

See for instance https://github.com/xerial/sqlite-jdbc/blob/755cf1d0223a83b97909fca43c8027e3e02bc021/src/main/java/org/sqlite/SQLiteJDBCLoader.java#L300 and https://github.com/jnr/jffi/blob/master/src/main/java/com/kenai/jffi/internal/StubLoader.java#L302

A possible alternative could be to check java.library.path indirectly by calling System.load first in the try/catch block. I did not go for this option since it might make existing systems pick up a library from somewhere unintentionally. The custom system property makes this explicitly opt-in.

As described in the issue, loading executable code from /tmp is problematic since it's sometimes forbidden by security settings of the host OS. This is an actual deployment problem for the system I work on since we have no control over those settings.

@lidavidm

Copy link
Copy Markdown
Member

Thanks. I'll give this a fuller review but if there's precedent then the idea is good by me.

return;
}
} catch (UnsatisfiedLinkError e) {
// Ignore this error and fall back to extracting from the JAR file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, would it be better to error if the property is provided but the file doesn't exist? That way you don't unexpectedly/silently fall back when you intended to load from somewhere else.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my specific application the native library will be have been stripped from the jar and loading will fail so the end result will be the same either way.

I modeled this on examples like the ones I referenced where possible alternatives are tried in a specified order. Happy to change this to fail early though. Let me know what you would prefer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we can leave it as is then.

@lidavidm

Copy link
Copy Markdown
Member

Ah, the JNI CI is still broken for now...we can queue this for after #865

@lidavidm

Copy link
Copy Markdown
Member

Ah but in the meantime do you mind helping the linter out?

Error: Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-c-data: The following files had format violations:
Error: src/main/java/org/apache/arrow/c/jni/JniLoader.java
Error: @@ -91,8 +91,7 @@
Error: ······}
Error: ····}
Error: Error: -····final·String·libraryToLoad·=
Error: -········name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: +····final·String·libraryToLoad·=·name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: ····try·{
Error: ······File·temp·=
Error: ··········File.createTempFile("jnilib-",·".tmp",·new·File(System.getProperty("java.io.tmpdir")));
Error: Run 'mvn spotless:apply' to fix these violations.
Error: -> [Help 1]

@lidavidm

Copy link
Copy Markdown
Member

Ok, please rebase and CI should be fixed!

@lidavidmlidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failure is #893

@lidavidm
lidavidm merged commit ed81e59 into apache:mainOct 27, 2025
23 of 25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementPRs that add or improve features.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow cdata native library to be loaded from user specified location

2 participants

@pepijnve@lidavidm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

GH-882: Add support for loading native library from a user specified location - #883

Merged
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882
Oct 27, 2025
Merged

GH-882: Add support for loading native library from a user specified location#883
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882

Conversation

@pepijnve

Copy link
Copy Markdown
Contributor

What's Changed

Add an opt-in code path that attempts to load the native library relative to the path specified by the arrow.cdata.library.path system property.

Closes#882.

@pepijnvepepijnve changed the title Add support for loading native library from a user specified locationGH-882: Add support for loading native library from a user specified locationOct 10, 2025
@github-actions

This comment has been minimized.

@lidavidmlidavidm added the enhancement PRs that add or improve features. label Oct 10, 2025
@github-actionsgithub-actionsBot added this to the 18.4.0 milestone Oct 10, 2025
@lidavidm

Copy link
Copy Markdown
Member

Is there precedent for this kind of flag in other JNI libraries?

@pepijnve

pepijnve commented Oct 10, 2025

Copy link
Copy Markdown
ContributorAuthor

Is there precedent for this kind of flag in other JNI libraries?

See for instance https://github.com/xerial/sqlite-jdbc/blob/755cf1d0223a83b97909fca43c8027e3e02bc021/src/main/java/org/sqlite/SQLiteJDBCLoader.java#L300 and https://github.com/jnr/jffi/blob/master/src/main/java/com/kenai/jffi/internal/StubLoader.java#L302

A possible alternative could be to check java.library.path indirectly by calling System.load first in the try/catch block. I did not go for this option since it might make existing systems pick up a library from somewhere unintentionally. The custom system property makes this explicitly opt-in.

As described in the issue, loading executable code from /tmp is problematic since it's sometimes forbidden by security settings of the host OS. This is an actual deployment problem for the system I work on since we have no control over those settings.

@lidavidm

Copy link
Copy Markdown
Member

Thanks. I'll give this a fuller review but if there's precedent then the idea is good by me.

return;
}
} catch (UnsatisfiedLinkError e) {
// Ignore this error and fall back to extracting from the JAR file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, would it be better to error if the property is provided but the file doesn't exist? That way you don't unexpectedly/silently fall back when you intended to load from somewhere else.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my specific application the native library will be have been stripped from the jar and loading will fail so the end result will be the same either way.

I modeled this on examples like the ones I referenced where possible alternatives are tried in a specified order. Happy to change this to fail early though. Let me know what you would prefer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we can leave it as is then.

@lidavidm

Copy link
Copy Markdown
Member

Ah, the JNI CI is still broken for now...we can queue this for after #865

@lidavidm

Copy link
Copy Markdown
Member

Ah but in the meantime do you mind helping the linter out?

Error: Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-c-data: The following files had format violations:
Error: src/main/java/org/apache/arrow/c/jni/JniLoader.java
Error: @@ -91,8 +91,7 @@
Error: ······}
Error: ····}
Error: Error: -····final·String·libraryToLoad·=
Error: -········name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: +····final·String·libraryToLoad·=·name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: ····try·{
Error: ······File·temp·=
Error: ··········File.createTempFile("jnilib-",·".tmp",·new·File(System.getProperty("java.io.tmpdir")));
Error: Run 'mvn spotless:apply' to fix these violations.
Error: -> [Help 1]

@lidavidm

Copy link
Copy Markdown
Member

Ok, please rebase and CI should be fixed!

@lidavidmlidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failure is #893

@lidavidm
lidavidm merged commit ed81e59 into apache:mainOct 27, 2025
23 of 25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementPRs that add or improve features.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow cdata native library to be loaded from user specified location

2 participants

@pepijnve@lidavidm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

GH-882: Add support for loading native library from a user specified location - #883

Merged
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882
Oct 27, 2025
Merged

GH-882: Add support for loading native library from a user specified location#883
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882

Conversation

@pepijnve

Copy link
Copy Markdown
Contributor

What's Changed

Add an opt-in code path that attempts to load the native library relative to the path specified by the arrow.cdata.library.path system property.

Closes#882.

@pepijnvepepijnve changed the title Add support for loading native library from a user specified locationGH-882: Add support for loading native library from a user specified locationOct 10, 2025
@github-actions

This comment has been minimized.

@lidavidmlidavidm added the enhancement PRs that add or improve features. label Oct 10, 2025
@github-actionsgithub-actionsBot added this to the 18.4.0 milestone Oct 10, 2025
@lidavidm

Copy link
Copy Markdown
Member

Is there precedent for this kind of flag in other JNI libraries?

@pepijnve

pepijnve commented Oct 10, 2025

Copy link
Copy Markdown
ContributorAuthor

Is there precedent for this kind of flag in other JNI libraries?

See for instance https://github.com/xerial/sqlite-jdbc/blob/755cf1d0223a83b97909fca43c8027e3e02bc021/src/main/java/org/sqlite/SQLiteJDBCLoader.java#L300 and https://github.com/jnr/jffi/blob/master/src/main/java/com/kenai/jffi/internal/StubLoader.java#L302

A possible alternative could be to check java.library.path indirectly by calling System.load first in the try/catch block. I did not go for this option since it might make existing systems pick up a library from somewhere unintentionally. The custom system property makes this explicitly opt-in.

As described in the issue, loading executable code from /tmp is problematic since it's sometimes forbidden by security settings of the host OS. This is an actual deployment problem for the system I work on since we have no control over those settings.

@lidavidm

Copy link
Copy Markdown
Member

Thanks. I'll give this a fuller review but if there's precedent then the idea is good by me.

return;
}
} catch (UnsatisfiedLinkError e) {
// Ignore this error and fall back to extracting from the JAR file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, would it be better to error if the property is provided but the file doesn't exist? That way you don't unexpectedly/silently fall back when you intended to load from somewhere else.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my specific application the native library will be have been stripped from the jar and loading will fail so the end result will be the same either way.

I modeled this on examples like the ones I referenced where possible alternatives are tried in a specified order. Happy to change this to fail early though. Let me know what you would prefer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we can leave it as is then.

@lidavidm

Copy link
Copy Markdown
Member

Ah, the JNI CI is still broken for now...we can queue this for after #865

@lidavidm

Copy link
Copy Markdown
Member

Ah but in the meantime do you mind helping the linter out?

Error: Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-c-data: The following files had format violations:
Error: src/main/java/org/apache/arrow/c/jni/JniLoader.java
Error: @@ -91,8 +91,7 @@
Error: ······}
Error: ····}
Error: Error: -····final·String·libraryToLoad·=
Error: -········name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: +····final·String·libraryToLoad·=·name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: ····try·{
Error: ······File·temp·=
Error: ··········File.createTempFile("jnilib-",·".tmp",·new·File(System.getProperty("java.io.tmpdir")));
Error: Run 'mvn spotless:apply' to fix these violations.
Error: -> [Help 1]

@lidavidm

Copy link
Copy Markdown
Member

Ok, please rebase and CI should be fixed!

@lidavidmlidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failure is #893

@lidavidm
lidavidm merged commit ed81e59 into apache:mainOct 27, 2025
23 of 25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementPRs that add or improve features.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow cdata native library to be loaded from user specified location

2 participants

@pepijnve@lidavidm
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

GH-882: Add support for loading native library from a user specified location - #883

Merged
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882
Oct 27, 2025
Merged

GH-882: Add support for loading native library from a user specified location#883
lidavidm merged 1 commit into
apache:mainfrom
pepijnve:issue_882

Conversation

@pepijnve

Copy link
Copy Markdown
Contributor

What's Changed

Add an opt-in code path that attempts to load the native library relative to the path specified by the arrow.cdata.library.path system property.

Closes#882.

@pepijnvepepijnve changed the title Add support for loading native library from a user specified locationGH-882: Add support for loading native library from a user specified locationOct 10, 2025
@github-actions

This comment has been minimized.

@lidavidmlidavidm added the enhancement PRs that add or improve features. label Oct 10, 2025
@github-actionsgithub-actionsBot added this to the 18.4.0 milestone Oct 10, 2025
@lidavidm

Copy link
Copy Markdown
Member

Is there precedent for this kind of flag in other JNI libraries?

@pepijnve

pepijnve commented Oct 10, 2025

Copy link
Copy Markdown
ContributorAuthor

Is there precedent for this kind of flag in other JNI libraries?

See for instance https://github.com/xerial/sqlite-jdbc/blob/755cf1d0223a83b97909fca43c8027e3e02bc021/src/main/java/org/sqlite/SQLiteJDBCLoader.java#L300 and https://github.com/jnr/jffi/blob/master/src/main/java/com/kenai/jffi/internal/StubLoader.java#L302

A possible alternative could be to check java.library.path indirectly by calling System.load first in the try/catch block. I did not go for this option since it might make existing systems pick up a library from somewhere unintentionally. The custom system property makes this explicitly opt-in.

As described in the issue, loading executable code from /tmp is problematic since it's sometimes forbidden by security settings of the host OS. This is an actual deployment problem for the system I work on since we have no control over those settings.

@lidavidm

Copy link
Copy Markdown
Member

Thanks. I'll give this a fuller review but if there's precedent then the idea is good by me.

return;
}
} catch (UnsatisfiedLinkError e) {
// Ignore this error and fall back to extracting from the JAR file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, would it be better to error if the property is provided but the file doesn't exist? That way you don't unexpectedly/silently fall back when you intended to load from somewhere else.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my specific application the native library will be have been stripped from the jar and loading will fail so the end result will be the same either way.

I modeled this on examples like the ones I referenced where possible alternatives are tried in a specified order. Happy to change this to fail early though. Let me know what you would prefer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we can leave it as is then.

@lidavidm

Copy link
Copy Markdown
Member

Ah, the JNI CI is still broken for now...we can queue this for after #865

@lidavidm

Copy link
Copy Markdown
Member

Ah but in the meantime do you mind helping the linter out?

Error: Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-c-data: The following files had format violations:
Error: src/main/java/org/apache/arrow/c/jni/JniLoader.java
Error: @@ -91,8 +91,7 @@
Error: ······}
Error: ····}
Error: Error: -····final·String·libraryToLoad·=
Error: -········name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: +····final·String·libraryToLoad·=·name·+·"/"·+·getNormalizedArch()·+·"/"·+·libraryName;
Error: ····try·{
Error: ······File·temp·=
Error: ··········File.createTempFile("jnilib-",·".tmp",·new·File(System.getProperty("java.io.tmpdir")));
Error: Run 'mvn spotless:apply' to fix these violations.
Error: -> [Help 1]

@lidavidm

Copy link
Copy Markdown
Member

Ok, please rebase and CI should be fixed!

@lidavidmlidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI failure is #893

@lidavidm
lidavidm merged commit ed81e59 into apache:mainOct 27, 2025
23 of 25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementPRs that add or improve features.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow cdata native library to be loaded from user specified location

2 participants

@pepijnve@lidavidm