Skip to content

GH-32123: [R] Expose azure blob filesystem - #49553

Merged
jonkeane merged 89 commits into
apache:mainfrom
marberts:32123-expose-azure-blob-filesystem
Jul 13, 2026
Merged

GH-32123: [R] Expose azure blob filesystem#49553
jonkeane merged 89 commits into
apache:mainfrom
marberts:32123-expose-azure-blob-filesystem

Conversation

@marberts

@marbertsmarberts commented Mar 19, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Rationale for this change

This PR adds support for Azure. The Arrow R package already has support for AWS and GCS, and the Arrow C++ library has had support for Azure for a couple years now. Support for Azure is already available in pyarrow.

This would close#32123.

What changes are included in this PR?

  • A new class AzureFileSystem that's analogous to S3FileSystem/GcsFileSystem, along with a helper function az_container() that's analogous to s3_bucket()/gcs_bucket().

  • Updates to src/filesystem.cpp to interact with the machinery in arrow/filesystem/azurefs.h.

  • Updates to the configuration and build scripts to support building with support for Azure.

  • Updates to the vignettes on cloud storage, installation, and developer setup.

Are these changes tested?

Yes. See tests/testthat/test-azure.R.

Are there any user-facing changes?

Yes. There is a new function az_container(), serving the analogous role to s3_bucket()/gcs_bucket(), along with an R6 class AzureFileSystem, again serving the same role as S3FileSystem and GcsFileSystem. There is also a function arrow_with_azure() to indicate if Arrow was built with support for Azure.

@github-actions

github-actionsBot commented Mar 19, 2026

Copy link
Copy Markdown

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

@marbertsmarberts changed the title 32123 expose azure blob filesystemGH-32123 [R]: Expose azure blob filesystemMar 19, 2026
@marbertsmarberts changed the title GH-32123 [R]: Expose azure blob filesystemGH-32123: [R] Expose azure blob filesystemMar 19, 2026
@github-actions

github-actionsBot commented Mar 19, 2026

Copy link
Copy Markdown

⚠️ GitHub issue #32123has been automatically assigned in GitHub to PR creator.

@marberts

marberts commented Mar 20, 2026•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

The CI is failing because the Azure C++ SDK depends on libxml2.

  • Update tools/nixlibs.R to check for libxml2 in with_cloud_support(), turning off the Azure feature flag if libxml2 isn't found.
    • What about configure and configure.win?
    • Is with_cloud_support() the only function that needs changing in tools/nixlibs.R?
  • Update the install.Rmd and developers/setup.Rmd vignettes to note that libxml2 is a dependency of Azure (same as with curl and openssl).
  • Update DESCRIPTION to include libxml2 in SystemRequirements.

@marberts

marberts commented Mar 24, 2026

Copy link
Copy Markdown
ContributorAuthor

It looks like the failing CI job is from a warning in R CMD check about a non-API call to a C function, due to all the changes in R-devel. I don't think this is due to our PR.

At this point, I'm fairly confident that we've implemented the Azure features correctly. The only thing I'm not sure about is the setup in configure, configure.win, and tools/nixlibs.R. The Azure C++ SDK used by Arrow requires libxml2, which is a new system dependency for this package. I don't fully understand how these configuration scripts work, so I wouldn't be surprised if they're not updated correctly to reflect the new system dependency.

@jonkeane, @thisisnic, @assignUser Please let me or @Collinbrown95 know if there are any changes needed to this PR.

@thisisnic

thisisnic commented Mar 24, 2026

Copy link
Copy Markdown
Member

Thanks for the PR @marberts - if you rebase from the main branch now, the non-API call stuff should be resolved.

Your best bet for the configure etc stuff is looking at the history of that file or older PRs to see how this kind of thing was handled there.

This PR where the GCS bindings were added might help: #13404, but let us know if you have any questions!

@thisisnic

thisisnic commented Mar 24, 2026

Copy link
Copy Markdown
Member

We'll also need additional CI jobs to test this - the PR I linked to above contains examples of what we added there.

@marberts

marberts commented Mar 24, 2026

Copy link
Copy Markdown
ContributorAuthor

Great, thanks @thisisnic!

@marberts

marberts commented Mar 24, 2026•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

To do list.

@marberts
marberts marked this pull request as draft March 25, 2026 23:32
@Collinbrown95

Collinbrown95 commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

@github-actions crossbow submit -g r

@github-actions

github-actionsBot commented Mar 26, 2026

Copy link
Copy Markdown 
Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23576717645

@thisisnic

thisisnic commented Mar 26, 2026

Copy link
Copy Markdown
Member

@github-actions crossbow submit -g r

@github-actions

github-actionsBot commented Mar 26, 2026

Copy link
Copy Markdown

Revision: 4da7280

Submitted crossbow builds: ursacomputing/crossbow @ actions-92325d6543

TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions

@marberts

marberts commented Mar 29, 2026

Copy link
Copy Markdown
ContributorAuthor

@github-actions crossbow submit -g r

@github-actions

github-actionsBot commented Mar 29, 2026

Copy link
Copy Markdown 
Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23698292050

@thisisnic

thisisnic commented Mar 30, 2026

Copy link
Copy Markdown
Member

Some of those CI failures are from things which have been fixed on main - you'll need to rebase your branch.

Collinbrown95and others added 17 commits July 10, 2026 07:12
@Collinbrown95@jonkeane
cleanup azurefs test function code
2bc10ba
@marberts@jonkeane
Trying PKGBUILD and windows build script again
169ca19
@marberts@jonkeane
Updated PKGBUILD
d39eef4
@Collinbrown95@jonkeane
debug ci: add azure dependencies that CI tries to build from source.
fb850f1
The hypothesis is that the errors in the build logs are caused by incompatibilities between WIL (Windows Implementation Library) and MinGW GCC, so install them directly instead of trying to build from source.
@marberts@jonkeane
Removed tmp.md
7619f01
@marberts@jonkeane
Removed windows build and added macos
ff68542
@marberts@jonkeane
azure off for windows ci
9ea60df
@marberts@jonkeane
Disable azure in configure.win
5a3eed4
@Collinbrown95@jonkeane
Add placeholder comment to flag present issues with MinGW and the Azu…
8f5d9ba
…re C++ SDK
@Collinbrown95@jonkeane
remove debugging script to install azure sdk from source
addcfa5
@marberts@jonkeane
Updated vignettes to reflect Azure off on Windows
b58fcc5
@marberts@jonkeane
Update github.packages.yml
d5cd943
@marberts@jonkeane
Fixing lint in filesystem.cpp
211c1fc
@marberts@jonkeane
Fixed warning in cpp code
c7dd3e5
@marberts@jonkeane
Added build info for azure to tasks for mac + a couple code comments
24c918b
@marberts@jonkeane
Fixed line endings on github.packages yaml
a8ad6a9
@marberts@jonkeane
libxml2 warning in with_cloud_support() is now conditional on azure b…
84f3b74
…eing requested
@jonkeane
jonkeaneforce-pushed the 32123-expose-azure-blob-filesystem branch from 1ea9150 to 84f3b74CompareJuly 10, 2026 12:13
@jonkeane

jonkeane commented Jul 10, 2026

Copy link
Copy Markdown
Member

@github-actions crossbow submit -g r

@jonkeane

jonkeane commented Jul 10, 2026

Copy link
Copy Markdown
Member

Thank you for your continued patience on this! I've rebased this against main and am running our extended CI to confirm if those failures we saw before were related to things on main from the last branching point.

@github-actions

github-actionsBot commented Jul 10, 2026

Copy link
Copy Markdown

Revision: 84f3b74

Submitted crossbow builds: ursacomputing/crossbow @ actions-3cda80c98b

TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-r-wasmGitHub Actions

@jonkeane

jonkeane commented Jul 10, 2026

Copy link
Copy Markdown
Member

@github-actions crossbow submit r-binary-packages

@github-actions

github-actionsBot commented Jul 10, 2026

Copy link
Copy Markdown

Revision: 9dd6793

Submitted crossbow builds: ursacomputing/crossbow @ actions-a47c3da823

TaskStatus
r-binary-packagesGitHub Actions

@jonkeane

jonkeane commented Jul 10, 2026

Copy link
Copy Markdown
Member

I pushed a change to make sure that the binary runners have libxml, etc. The other failures are unrelated to this branch I believe (though some aren't on https://crossbow.arrow-dev.org/ yet, so they might even be r-devel transitory actually!)

jonkeane
jonkeane approved these changes Jul 10, 2026
View reviewed changes

@jonkeanejonkeane left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing to push on this, I've rebased and made sure that CI is passing. I'm inclined to merge, but will give others (and especially @thisisnic) a day or two to speak up if they don't want me to.

@github-actionsgithub-actionsBot removed the awaiting change review Awaiting change review label Jul 10, 2026
@marberts

marberts commented Jul 11, 2026

Copy link
Copy Markdown
ContributorAuthor

Awesome! Thank you.

@jonkeanejonkeane mentioned this pull request Jul 13, 2026
Closed
@conbench-apache-arrow

conbench-apache-arrowBot commented Jul 15, 2026

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b03b4c5.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@thisisnicthisisnicthisisnic left review comments
@jonkeanejonkeanejonkeane approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@koukouAwaiting requested review from koukou is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees

No one assigned

Labels

Component: R

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[R] Expose Azure Blob Storage filesystem

5 participants

@marberts@thisisnic@Collinbrown95@jonkeane@kou
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
GH-32123: [R] Expose azure blob filesystem by marberts · Pull Request #49553 · apache/arrow · GitHub
Skip to content
GH-32123: [R] Expose azure blob filesystem - #49553
Merged
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Jul 13, 2026
Merged
GH-32123: [R] Expose azure blob filesystem#49553
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Conversation
@marberts
@marbertsmarberts
commented
Mar 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
This PR adds support for Azure. The Arrow R package already has support for AWS and GCS, and the Arrow C++ library has had support for Azure for a couple years now. Support for Azure is already available in pyarrow.
This would close#32123.
What changes are included in this PR?
  • A new class AzureFileSystem that's analogous to S3FileSystem/GcsFileSystem, along with a helper function az_container() that's analogous to s3_bucket()/gcs_bucket().
  • Updates to src/filesystem.cpp to interact with the machinery in arrow/filesystem/azurefs.h.
  • Updates to the configuration and build scripts to support building with support for Azure.
  • Updates to the vignettes on cloud storage, installation, and developer setup.
Are these changes tested?
Yes. See tests/testthat/test-azure.R.
Are there any user-facing changes?
Yes. There is a new function az_container(), serving the analogous role to s3_bucket()/gcs_bucket(), along with an R6 class AzureFileSystem, again serving the same role as S3FileSystem and GcsFileSystem. There is also a function arrow_with_azure() to indicate if Arrow was built with support for Azure.
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

Thanks for opening a pull request!
If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or
MINOR: [${COMPONENT}] ${SUMMARY}

See also:
@marbertsmarberts
changed the title
32123 expose azure blob filesystemGH-32123 [R]: Expose azure blob filesystemMar 19, 2026
@marbertsmarberts
changed the title
GH-32123 [R]: Expose azure blob filesystemGH-32123: [R] Expose azure blob filesystemMar 19, 2026
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

⚠️ GitHub issue #32123has been automatically assigned in GitHub to PR creator.
@marberts
marberts
commented
Mar 20, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
The CI is failing because the Azure C++ SDK depends on libxml2.
  •  Update tools/nixlibs.R to check for libxml2 in with_cloud_support(), turning off the Azure feature flag if libxml2 isn't found.

    •  What about configure and configure.win?
    •  Is with_cloud_support() the only function that needs changing in tools/nixlibs.R?
  •  Update the install.Rmd and developers/setup.Rmd vignettes to note that libxml2 is a dependency of Azure (same as with curl and openssl).
  •  Update DESCRIPTION to include libxml2 in SystemRequirements.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
It looks like the failing CI job is from a warning in R CMD check about a non-API call to a C function, due to all the changes in R-devel. I don't think this is due to our PR.
At this point, I'm fairly confident that we've implemented the Azure features correctly. The only thing I'm not sure about is the setup in configure, configure.win, and tools/nixlibs.R. The Azure C++ SDK used by Arrow requires libxml2, which is a new system dependency for this package. I don't fully understand how these configuration scripts work, so I wouldn't be surprised if they're not updated correctly to reflect the new system dependency.
@jonkeane, @thisisnic, @assignUser Please let me or @Collinbrown95 know if there are any changes needed to this PR.
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
Thanks for the PR @marberts - if you rebase from the main branch now, the non-API call stuff should be resolved.
Your best bet for the configure etc stuff is looking at the history of that file or older PRs to see how this kind of thing was handled there.
This PR where the GCS bindings were added might help: #13404, but let us know if you have any questions!
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
We'll also need additional CI jobs to test this - the PR I linked to above contains examples of what we added there.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
Great, thanks @thisisnic!
@marberts
marberts
commented
Mar 24, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
To do list.
@marberts
marberts
marked this pull request as draft
March 25, 2026 23:32
@Collinbrown95
Collinbrown95
commented
Mar 26, 2026

Copy link


Copy Markdown

Contributor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23576717645

@thisisnic
thisisnic
commented
Mar 26, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Revision: 4da7280
Submitted crossbow builds: ursacomputing/crossbow @ actions-92325d6543
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
@marberts
marberts
commented
Mar 29, 2026

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 29, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23698292050

@thisisnic
thisisnic
commented
Mar 30, 2026

Copy link


Copy Markdown

Member
Some of those CI failures are from things which have been fixed on main - you'll need to rebase your branch.
Collinbrown95and others
added 17 commits
July 10, 2026 07:12
@Collinbrown95@jonkeane
cleanup azurefs test function code
2bc10ba
@marberts@jonkeane
Trying PKGBUILD and windows build script again
169ca19
@marberts@jonkeane
Updated PKGBUILD
d39eef4
@Collinbrown95@jonkeane
debug ci: add azure dependencies that CI tries to build from source.
fb850f1
The hypothesis is that the errors in the build logs are caused by incompatibilities between WIL (Windows Implementation Library) and MinGW GCC, so install them directly instead of trying to build from source.
@marberts@jonkeane
Removed tmp.md
7619f01
@marberts@jonkeane
Removed windows build and added macos
ff68542
@marberts@jonkeane
azure off for windows ci
9ea60df
@marberts@jonkeane
Disable azure in configure.win
5a3eed4
@Collinbrown95@jonkeane
Add placeholder comment to flag present issues with MinGW and the Azu…
8f5d9ba
…re C++ SDK
@Collinbrown95@jonkeane
remove debugging script to install azure sdk from source
addcfa5
@marberts@jonkeane
Updated vignettes to reflect Azure off on Windows
b58fcc5
@marberts@jonkeane
Update github.packages.yml
d5cd943
@marberts@jonkeane
Fixing lint in filesystem.cpp
211c1fc
@marberts@jonkeane
Fixed warning in cpp code
c7dd3e5
@marberts@jonkeane
Added build info for azure to tasks for mac + a couple code comments
24c918b
@marberts@jonkeane
Fixed line endings on github.packages yaml
a8ad6a9
@marberts@jonkeane
libxml2 warning in with_cloud_support() is now conditional on azure b…
84f3b74
…eing requested
@jonkeane
jonkeaneforce-pushed
the
32123-expose-azure-blob-filesystem
branch
from
1ea9150 to
84f3b74CompareJuly 10, 2026 12:13
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
Thank you for your continued patience on this! I've rebased this against main and am running our extended CI to confirm if those failures we saw before were related to things on main from the last branching point.
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 84f3b74
Submitted crossbow builds: ursacomputing/crossbow @ actions-3cda80c98b
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-r-wasmGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit r-binary-packages
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 9dd6793
Submitted crossbow builds: ursacomputing/crossbow @ actions-a47c3da823
TaskStatus
r-binary-packagesGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
I pushed a change to make sure that the binary runners have libxml, etc. The other failures are unrelated to this branch I believe (though some aren't on https://crossbow.arrow-dev.org/ yet, so they might even be r-devel transitory actually!)
jonkeane
jonkeane
approved these changes
Jul 10, 2026
View reviewed changes
@jonkeanejonkeane
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing to push on this, I've rebased and made sure that CI is passing. I'm inclined to merge, but will give others (and especially @thisisnic) a day or two to speak up if they don't want me to.
@github-actionsgithub-actionsBot
removed
the 
awaiting change review
Awaiting change review
label
Jul 10, 2026
@marberts
marberts
commented
Jul 11, 2026

Copy link


Copy Markdown

ContributorAuthor
Awesome! Thank you.
@jonkeanejonkeane
mentioned this pull request
Jul 13, 2026

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 15, 2026

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b03b4c5.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@thisisnicthisisnicthisisnic left review comments
@jonkeanejonkeanejonkeane approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@koukouAwaiting requested review from koukou is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[R] Expose Azure Blob Storage filesystem


5 participants

@marberts@thisisnic@Collinbrown95@jonkeane@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GH-32123: [R] Expose azure blob filesystem by marberts · Pull Request #49553 · apache/arrow · GitHub
Skip to content
GH-32123: [R] Expose azure blob filesystem - #49553
Merged
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Jul 13, 2026
Merged
GH-32123: [R] Expose azure blob filesystem#49553
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Conversation
@marberts
@marbertsmarberts
commented
Mar 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
This PR adds support for Azure. The Arrow R package already has support for AWS and GCS, and the Arrow C++ library has had support for Azure for a couple years now. Support for Azure is already available in pyarrow.
This would close#32123.
What changes are included in this PR?
  • A new class AzureFileSystem that's analogous to S3FileSystem/GcsFileSystem, along with a helper function az_container() that's analogous to s3_bucket()/gcs_bucket().
  • Updates to src/filesystem.cpp to interact with the machinery in arrow/filesystem/azurefs.h.
  • Updates to the configuration and build scripts to support building with support for Azure.
  • Updates to the vignettes on cloud storage, installation, and developer setup.
Are these changes tested?
Yes. See tests/testthat/test-azure.R.
Are there any user-facing changes?
Yes. There is a new function az_container(), serving the analogous role to s3_bucket()/gcs_bucket(), along with an R6 class AzureFileSystem, again serving the same role as S3FileSystem and GcsFileSystem. There is also a function arrow_with_azure() to indicate if Arrow was built with support for Azure.
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

Thanks for opening a pull request!
If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or
MINOR: [${COMPONENT}] ${SUMMARY}

See also:
@marbertsmarberts
changed the title
32123 expose azure blob filesystemGH-32123 [R]: Expose azure blob filesystemMar 19, 2026
@marbertsmarberts
changed the title
GH-32123 [R]: Expose azure blob filesystemGH-32123: [R] Expose azure blob filesystemMar 19, 2026
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

⚠️ GitHub issue #32123has been automatically assigned in GitHub to PR creator.
@marberts
marberts
commented
Mar 20, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
The CI is failing because the Azure C++ SDK depends on libxml2.
  •  Update tools/nixlibs.R to check for libxml2 in with_cloud_support(), turning off the Azure feature flag if libxml2 isn't found.

    •  What about configure and configure.win?
    •  Is with_cloud_support() the only function that needs changing in tools/nixlibs.R?
  •  Update the install.Rmd and developers/setup.Rmd vignettes to note that libxml2 is a dependency of Azure (same as with curl and openssl).
  •  Update DESCRIPTION to include libxml2 in SystemRequirements.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
It looks like the failing CI job is from a warning in R CMD check about a non-API call to a C function, due to all the changes in R-devel. I don't think this is due to our PR.
At this point, I'm fairly confident that we've implemented the Azure features correctly. The only thing I'm not sure about is the setup in configure, configure.win, and tools/nixlibs.R. The Azure C++ SDK used by Arrow requires libxml2, which is a new system dependency for this package. I don't fully understand how these configuration scripts work, so I wouldn't be surprised if they're not updated correctly to reflect the new system dependency.
@jonkeane, @thisisnic, @assignUser Please let me or @Collinbrown95 know if there are any changes needed to this PR.
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
Thanks for the PR @marberts - if you rebase from the main branch now, the non-API call stuff should be resolved.
Your best bet for the configure etc stuff is looking at the history of that file or older PRs to see how this kind of thing was handled there.
This PR where the GCS bindings were added might help: #13404, but let us know if you have any questions!
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
We'll also need additional CI jobs to test this - the PR I linked to above contains examples of what we added there.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
Great, thanks @thisisnic!
@marberts
marberts
commented
Mar 24, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
To do list.
@marberts
marberts
marked this pull request as draft
March 25, 2026 23:32
@Collinbrown95
Collinbrown95
commented
Mar 26, 2026

Copy link


Copy Markdown

Contributor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23576717645

@thisisnic
thisisnic
commented
Mar 26, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Revision: 4da7280
Submitted crossbow builds: ursacomputing/crossbow @ actions-92325d6543
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
@marberts
marberts
commented
Mar 29, 2026

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 29, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23698292050

@thisisnic
thisisnic
commented
Mar 30, 2026

Copy link


Copy Markdown

Member
Some of those CI failures are from things which have been fixed on main - you'll need to rebase your branch.
Collinbrown95and others
added 17 commits
July 10, 2026 07:12
@Collinbrown95@jonkeane
cleanup azurefs test function code
2bc10ba
@marberts@jonkeane
Trying PKGBUILD and windows build script again
169ca19
@marberts@jonkeane
Updated PKGBUILD
d39eef4
@Collinbrown95@jonkeane
debug ci: add azure dependencies that CI tries to build from source.
fb850f1
The hypothesis is that the errors in the build logs are caused by incompatibilities between WIL (Windows Implementation Library) and MinGW GCC, so install them directly instead of trying to build from source.
@marberts@jonkeane
Removed tmp.md
7619f01
@marberts@jonkeane
Removed windows build and added macos
ff68542
@marberts@jonkeane
azure off for windows ci
9ea60df
@marberts@jonkeane
Disable azure in configure.win
5a3eed4
@Collinbrown95@jonkeane
Add placeholder comment to flag present issues with MinGW and the Azu…
8f5d9ba
…re C++ SDK
@Collinbrown95@jonkeane
remove debugging script to install azure sdk from source
addcfa5
@marberts@jonkeane
Updated vignettes to reflect Azure off on Windows
b58fcc5
@marberts@jonkeane
Update github.packages.yml
d5cd943
@marberts@jonkeane
Fixing lint in filesystem.cpp
211c1fc
@marberts@jonkeane
Fixed warning in cpp code
c7dd3e5
@marberts@jonkeane
Added build info for azure to tasks for mac + a couple code comments
24c918b
@marberts@jonkeane
Fixed line endings on github.packages yaml
a8ad6a9
@marberts@jonkeane
libxml2 warning in with_cloud_support() is now conditional on azure b…
84f3b74
…eing requested
@jonkeane
jonkeaneforce-pushed
the
32123-expose-azure-blob-filesystem
branch
from
1ea9150 to
84f3b74CompareJuly 10, 2026 12:13
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
Thank you for your continued patience on this! I've rebased this against main and am running our extended CI to confirm if those failures we saw before were related to things on main from the last branching point.
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 84f3b74
Submitted crossbow builds: ursacomputing/crossbow @ actions-3cda80c98b
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-r-wasmGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit r-binary-packages
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 9dd6793
Submitted crossbow builds: ursacomputing/crossbow @ actions-a47c3da823
TaskStatus
r-binary-packagesGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
I pushed a change to make sure that the binary runners have libxml, etc. The other failures are unrelated to this branch I believe (though some aren't on https://crossbow.arrow-dev.org/ yet, so they might even be r-devel transitory actually!)
jonkeane
jonkeane
approved these changes
Jul 10, 2026
View reviewed changes
@jonkeanejonkeane
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing to push on this, I've rebased and made sure that CI is passing. I'm inclined to merge, but will give others (and especially @thisisnic) a day or two to speak up if they don't want me to.
@github-actionsgithub-actionsBot
removed
the 
awaiting change review
Awaiting change review
label
Jul 10, 2026
@marberts
marberts
commented
Jul 11, 2026

Copy link


Copy Markdown

ContributorAuthor
Awesome! Thank you.
@jonkeanejonkeane
mentioned this pull request
Jul 13, 2026

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 15, 2026

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b03b4c5.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@thisisnicthisisnicthisisnic left review comments
@jonkeanejonkeanejonkeane approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@koukouAwaiting requested review from koukou is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[R] Expose Azure Blob Storage filesystem


5 participants

@marberts@thisisnic@Collinbrown95@jonkeane@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GH-32123: [R] Expose azure blob filesystem by marberts · Pull Request #49553 · apache/arrow · GitHub
Skip to content
GH-32123: [R] Expose azure blob filesystem - #49553
Merged
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Jul 13, 2026
Merged
GH-32123: [R] Expose azure blob filesystem#49553
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Conversation
@marberts
@marbertsmarberts
commented
Mar 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
This PR adds support for Azure. The Arrow R package already has support for AWS and GCS, and the Arrow C++ library has had support for Azure for a couple years now. Support for Azure is already available in pyarrow.
This would close#32123.
What changes are included in this PR?
  • A new class AzureFileSystem that's analogous to S3FileSystem/GcsFileSystem, along with a helper function az_container() that's analogous to s3_bucket()/gcs_bucket().
  • Updates to src/filesystem.cpp to interact with the machinery in arrow/filesystem/azurefs.h.
  • Updates to the configuration and build scripts to support building with support for Azure.
  • Updates to the vignettes on cloud storage, installation, and developer setup.
Are these changes tested?
Yes. See tests/testthat/test-azure.R.
Are there any user-facing changes?
Yes. There is a new function az_container(), serving the analogous role to s3_bucket()/gcs_bucket(), along with an R6 class AzureFileSystem, again serving the same role as S3FileSystem and GcsFileSystem. There is also a function arrow_with_azure() to indicate if Arrow was built with support for Azure.
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

Thanks for opening a pull request!
If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or
MINOR: [${COMPONENT}] ${SUMMARY}

See also:
@marbertsmarberts
changed the title
32123 expose azure blob filesystemGH-32123 [R]: Expose azure blob filesystemMar 19, 2026
@marbertsmarberts
changed the title
GH-32123 [R]: Expose azure blob filesystemGH-32123: [R] Expose azure blob filesystemMar 19, 2026
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

⚠️ GitHub issue #32123has been automatically assigned in GitHub to PR creator.
@marberts
marberts
commented
Mar 20, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
The CI is failing because the Azure C++ SDK depends on libxml2.
  •  Update tools/nixlibs.R to check for libxml2 in with_cloud_support(), turning off the Azure feature flag if libxml2 isn't found.

    •  What about configure and configure.win?
    •  Is with_cloud_support() the only function that needs changing in tools/nixlibs.R?
  •  Update the install.Rmd and developers/setup.Rmd vignettes to note that libxml2 is a dependency of Azure (same as with curl and openssl).
  •  Update DESCRIPTION to include libxml2 in SystemRequirements.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
It looks like the failing CI job is from a warning in R CMD check about a non-API call to a C function, due to all the changes in R-devel. I don't think this is due to our PR.
At this point, I'm fairly confident that we've implemented the Azure features correctly. The only thing I'm not sure about is the setup in configure, configure.win, and tools/nixlibs.R. The Azure C++ SDK used by Arrow requires libxml2, which is a new system dependency for this package. I don't fully understand how these configuration scripts work, so I wouldn't be surprised if they're not updated correctly to reflect the new system dependency.
@jonkeane, @thisisnic, @assignUser Please let me or @Collinbrown95 know if there are any changes needed to this PR.
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
Thanks for the PR @marberts - if you rebase from the main branch now, the non-API call stuff should be resolved.
Your best bet for the configure etc stuff is looking at the history of that file or older PRs to see how this kind of thing was handled there.
This PR where the GCS bindings were added might help: #13404, but let us know if you have any questions!
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
We'll also need additional CI jobs to test this - the PR I linked to above contains examples of what we added there.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
Great, thanks @thisisnic!
@marberts
marberts
commented
Mar 24, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
To do list.
@marberts
marberts
marked this pull request as draft
March 25, 2026 23:32
@Collinbrown95
Collinbrown95
commented
Mar 26, 2026

Copy link


Copy Markdown

Contributor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23576717645

@thisisnic
thisisnic
commented
Mar 26, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Revision: 4da7280
Submitted crossbow builds: ursacomputing/crossbow @ actions-92325d6543
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
@marberts
marberts
commented
Mar 29, 2026

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 29, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23698292050

@thisisnic
thisisnic
commented
Mar 30, 2026

Copy link


Copy Markdown

Member
Some of those CI failures are from things which have been fixed on main - you'll need to rebase your branch.
Collinbrown95and others
added 17 commits
July 10, 2026 07:12
@Collinbrown95@jonkeane
cleanup azurefs test function code
2bc10ba
@marberts@jonkeane
Trying PKGBUILD and windows build script again
169ca19
@marberts@jonkeane
Updated PKGBUILD
d39eef4
@Collinbrown95@jonkeane
debug ci: add azure dependencies that CI tries to build from source.
fb850f1
The hypothesis is that the errors in the build logs are caused by incompatibilities between WIL (Windows Implementation Library) and MinGW GCC, so install them directly instead of trying to build from source.
@marberts@jonkeane
Removed tmp.md
7619f01
@marberts@jonkeane
Removed windows build and added macos
ff68542
@marberts@jonkeane
azure off for windows ci
9ea60df
@marberts@jonkeane
Disable azure in configure.win
5a3eed4
@Collinbrown95@jonkeane
Add placeholder comment to flag present issues with MinGW and the Azu…
8f5d9ba
…re C++ SDK
@Collinbrown95@jonkeane
remove debugging script to install azure sdk from source
addcfa5
@marberts@jonkeane
Updated vignettes to reflect Azure off on Windows
b58fcc5
@marberts@jonkeane
Update github.packages.yml
d5cd943
@marberts@jonkeane
Fixing lint in filesystem.cpp
211c1fc
@marberts@jonkeane
Fixed warning in cpp code
c7dd3e5
@marberts@jonkeane
Added build info for azure to tasks for mac + a couple code comments
24c918b
@marberts@jonkeane
Fixed line endings on github.packages yaml
a8ad6a9
@marberts@jonkeane
libxml2 warning in with_cloud_support() is now conditional on azure b…
84f3b74
…eing requested
@jonkeane
jonkeaneforce-pushed
the
32123-expose-azure-blob-filesystem
branch
from
1ea9150 to
84f3b74CompareJuly 10, 2026 12:13
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
Thank you for your continued patience on this! I've rebased this against main and am running our extended CI to confirm if those failures we saw before were related to things on main from the last branching point.
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 84f3b74
Submitted crossbow builds: ursacomputing/crossbow @ actions-3cda80c98b
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-r-wasmGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit r-binary-packages
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 9dd6793
Submitted crossbow builds: ursacomputing/crossbow @ actions-a47c3da823
TaskStatus
r-binary-packagesGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
I pushed a change to make sure that the binary runners have libxml, etc. The other failures are unrelated to this branch I believe (though some aren't on https://crossbow.arrow-dev.org/ yet, so they might even be r-devel transitory actually!)
jonkeane
jonkeane
approved these changes
Jul 10, 2026
View reviewed changes
@jonkeanejonkeane
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing to push on this, I've rebased and made sure that CI is passing. I'm inclined to merge, but will give others (and especially @thisisnic) a day or two to speak up if they don't want me to.
@github-actionsgithub-actionsBot
removed
the 
awaiting change review
Awaiting change review
label
Jul 10, 2026
@marberts
marberts
commented
Jul 11, 2026

Copy link


Copy Markdown

ContributorAuthor
Awesome! Thank you.
@jonkeanejonkeane
mentioned this pull request
Jul 13, 2026

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 15, 2026

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b03b4c5.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@thisisnicthisisnicthisisnic left review comments
@jonkeanejonkeanejonkeane approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@koukouAwaiting requested review from koukou is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[R] Expose Azure Blob Storage filesystem


5 participants

@marberts@thisisnic@Collinbrown95@jonkeane@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
GH-32123: [R] Expose azure blob filesystem by marberts · Pull Request #49553 · apache/arrow · GitHub
Skip to content
GH-32123: [R] Expose azure blob filesystem - #49553
Merged
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Jul 13, 2026
Merged
GH-32123: [R] Expose azure blob filesystem#49553
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Conversation
@marberts
@marbertsmarberts
commented
Mar 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
This PR adds support for Azure. The Arrow R package already has support for AWS and GCS, and the Arrow C++ library has had support for Azure for a couple years now. Support for Azure is already available in pyarrow.
This would close#32123.
What changes are included in this PR?
  • A new class AzureFileSystem that's analogous to S3FileSystem/GcsFileSystem, along with a helper function az_container() that's analogous to s3_bucket()/gcs_bucket().
  • Updates to src/filesystem.cpp to interact with the machinery in arrow/filesystem/azurefs.h.
  • Updates to the configuration and build scripts to support building with support for Azure.
  • Updates to the vignettes on cloud storage, installation, and developer setup.
Are these changes tested?
Yes. See tests/testthat/test-azure.R.
Are there any user-facing changes?
Yes. There is a new function az_container(), serving the analogous role to s3_bucket()/gcs_bucket(), along with an R6 class AzureFileSystem, again serving the same role as S3FileSystem and GcsFileSystem. There is also a function arrow_with_azure() to indicate if Arrow was built with support for Azure.
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

Thanks for opening a pull request!
If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or
MINOR: [${COMPONENT}] ${SUMMARY}

See also:
@marbertsmarberts
changed the title
32123 expose azure blob filesystemGH-32123 [R]: Expose azure blob filesystemMar 19, 2026
@marbertsmarberts
changed the title
GH-32123 [R]: Expose azure blob filesystemGH-32123: [R] Expose azure blob filesystemMar 19, 2026
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

⚠️ GitHub issue #32123has been automatically assigned in GitHub to PR creator.
@marberts
marberts
commented
Mar 20, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
The CI is failing because the Azure C++ SDK depends on libxml2.
  •  Update tools/nixlibs.R to check for libxml2 in with_cloud_support(), turning off the Azure feature flag if libxml2 isn't found.

    •  What about configure and configure.win?
    •  Is with_cloud_support() the only function that needs changing in tools/nixlibs.R?
  •  Update the install.Rmd and developers/setup.Rmd vignettes to note that libxml2 is a dependency of Azure (same as with curl and openssl).
  •  Update DESCRIPTION to include libxml2 in SystemRequirements.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
It looks like the failing CI job is from a warning in R CMD check about a non-API call to a C function, due to all the changes in R-devel. I don't think this is due to our PR.
At this point, I'm fairly confident that we've implemented the Azure features correctly. The only thing I'm not sure about is the setup in configure, configure.win, and tools/nixlibs.R. The Azure C++ SDK used by Arrow requires libxml2, which is a new system dependency for this package. I don't fully understand how these configuration scripts work, so I wouldn't be surprised if they're not updated correctly to reflect the new system dependency.
@jonkeane, @thisisnic, @assignUser Please let me or @Collinbrown95 know if there are any changes needed to this PR.
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
Thanks for the PR @marberts - if you rebase from the main branch now, the non-API call stuff should be resolved.
Your best bet for the configure etc stuff is looking at the history of that file or older PRs to see how this kind of thing was handled there.
This PR where the GCS bindings were added might help: #13404, but let us know if you have any questions!
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
We'll also need additional CI jobs to test this - the PR I linked to above contains examples of what we added there.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
Great, thanks @thisisnic!
@marberts
marberts
commented
Mar 24, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
To do list.
@marberts
marberts
marked this pull request as draft
March 25, 2026 23:32
@Collinbrown95
Collinbrown95
commented
Mar 26, 2026

Copy link


Copy Markdown

Contributor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23576717645

@thisisnic
thisisnic
commented
Mar 26, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Revision: 4da7280
Submitted crossbow builds: ursacomputing/crossbow @ actions-92325d6543
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
@marberts
marberts
commented
Mar 29, 2026

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 29, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23698292050

@thisisnic
thisisnic
commented
Mar 30, 2026

Copy link


Copy Markdown

Member
Some of those CI failures are from things which have been fixed on main - you'll need to rebase your branch.
Collinbrown95and others
added 17 commits
July 10, 2026 07:12
@Collinbrown95@jonkeane
cleanup azurefs test function code
2bc10ba
@marberts@jonkeane
Trying PKGBUILD and windows build script again
169ca19
@marberts@jonkeane
Updated PKGBUILD
d39eef4
@Collinbrown95@jonkeane
debug ci: add azure dependencies that CI tries to build from source.
fb850f1
The hypothesis is that the errors in the build logs are caused by incompatibilities between WIL (Windows Implementation Library) and MinGW GCC, so install them directly instead of trying to build from source.
@marberts@jonkeane
Removed tmp.md
7619f01
@marberts@jonkeane
Removed windows build and added macos
ff68542
@marberts@jonkeane
azure off for windows ci
9ea60df
@marberts@jonkeane
Disable azure in configure.win
5a3eed4
@Collinbrown95@jonkeane
Add placeholder comment to flag present issues with MinGW and the Azu…
8f5d9ba
…re C++ SDK
@Collinbrown95@jonkeane
remove debugging script to install azure sdk from source
addcfa5
@marberts@jonkeane
Updated vignettes to reflect Azure off on Windows
b58fcc5
@marberts@jonkeane
Update github.packages.yml
d5cd943
@marberts@jonkeane
Fixing lint in filesystem.cpp
211c1fc
@marberts@jonkeane
Fixed warning in cpp code
c7dd3e5
@marberts@jonkeane
Added build info for azure to tasks for mac + a couple code comments
24c918b
@marberts@jonkeane
Fixed line endings on github.packages yaml
a8ad6a9
@marberts@jonkeane
libxml2 warning in with_cloud_support() is now conditional on azure b…
84f3b74
…eing requested
@jonkeane
jonkeaneforce-pushed
the
32123-expose-azure-blob-filesystem
branch
from
1ea9150 to
84f3b74CompareJuly 10, 2026 12:13
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
Thank you for your continued patience on this! I've rebased this against main and am running our extended CI to confirm if those failures we saw before were related to things on main from the last branching point.
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 84f3b74
Submitted crossbow builds: ursacomputing/crossbow @ actions-3cda80c98b
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-r-wasmGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit r-binary-packages
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 9dd6793
Submitted crossbow builds: ursacomputing/crossbow @ actions-a47c3da823
TaskStatus
r-binary-packagesGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
I pushed a change to make sure that the binary runners have libxml, etc. The other failures are unrelated to this branch I believe (though some aren't on https://crossbow.arrow-dev.org/ yet, so they might even be r-devel transitory actually!)
jonkeane
jonkeane
approved these changes
Jul 10, 2026
View reviewed changes
@jonkeanejonkeane
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing to push on this, I've rebased and made sure that CI is passing. I'm inclined to merge, but will give others (and especially @thisisnic) a day or two to speak up if they don't want me to.
@github-actionsgithub-actionsBot
removed
the 
awaiting change review
Awaiting change review
label
Jul 10, 2026
@marberts
marberts
commented
Jul 11, 2026

Copy link


Copy Markdown

ContributorAuthor
Awesome! Thank you.
@jonkeanejonkeane
mentioned this pull request
Jul 13, 2026

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 15, 2026

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b03b4c5.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@thisisnicthisisnicthisisnic left review comments
@jonkeanejonkeanejonkeane approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@koukouAwaiting requested review from koukou is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[R] Expose Azure Blob Storage filesystem


5 participants

@marberts@thisisnic@Collinbrown95@jonkeane@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GH-32123: [R] Expose azure blob filesystem by marberts · Pull Request #49553 · apache/arrow · GitHub
Skip to content
GH-32123: [R] Expose azure blob filesystem - #49553
Merged
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Jul 13, 2026
Merged
GH-32123: [R] Expose azure blob filesystem#49553
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Conversation
@marberts
@marbertsmarberts
commented
Mar 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
This PR adds support for Azure. The Arrow R package already has support for AWS and GCS, and the Arrow C++ library has had support for Azure for a couple years now. Support for Azure is already available in pyarrow.
This would close#32123.
What changes are included in this PR?
  • A new class AzureFileSystem that's analogous to S3FileSystem/GcsFileSystem, along with a helper function az_container() that's analogous to s3_bucket()/gcs_bucket().
  • Updates to src/filesystem.cpp to interact with the machinery in arrow/filesystem/azurefs.h.
  • Updates to the configuration and build scripts to support building with support for Azure.
  • Updates to the vignettes on cloud storage, installation, and developer setup.
Are these changes tested?
Yes. See tests/testthat/test-azure.R.
Are there any user-facing changes?
Yes. There is a new function az_container(), serving the analogous role to s3_bucket()/gcs_bucket(), along with an R6 class AzureFileSystem, again serving the same role as S3FileSystem and GcsFileSystem. There is also a function arrow_with_azure() to indicate if Arrow was built with support for Azure.
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

Thanks for opening a pull request!
If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or
MINOR: [${COMPONENT}] ${SUMMARY}

See also:
@marbertsmarberts
changed the title
32123 expose azure blob filesystemGH-32123 [R]: Expose azure blob filesystemMar 19, 2026
@marbertsmarberts
changed the title
GH-32123 [R]: Expose azure blob filesystemGH-32123: [R] Expose azure blob filesystemMar 19, 2026
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

⚠️ GitHub issue #32123has been automatically assigned in GitHub to PR creator.
@marberts
marberts
commented
Mar 20, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
The CI is failing because the Azure C++ SDK depends on libxml2.
  •  Update tools/nixlibs.R to check for libxml2 in with_cloud_support(), turning off the Azure feature flag if libxml2 isn't found.

    •  What about configure and configure.win?
    •  Is with_cloud_support() the only function that needs changing in tools/nixlibs.R?
  •  Update the install.Rmd and developers/setup.Rmd vignettes to note that libxml2 is a dependency of Azure (same as with curl and openssl).
  •  Update DESCRIPTION to include libxml2 in SystemRequirements.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
It looks like the failing CI job is from a warning in R CMD check about a non-API call to a C function, due to all the changes in R-devel. I don't think this is due to our PR.
At this point, I'm fairly confident that we've implemented the Azure features correctly. The only thing I'm not sure about is the setup in configure, configure.win, and tools/nixlibs.R. The Azure C++ SDK used by Arrow requires libxml2, which is a new system dependency for this package. I don't fully understand how these configuration scripts work, so I wouldn't be surprised if they're not updated correctly to reflect the new system dependency.
@jonkeane, @thisisnic, @assignUser Please let me or @Collinbrown95 know if there are any changes needed to this PR.
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
Thanks for the PR @marberts - if you rebase from the main branch now, the non-API call stuff should be resolved.
Your best bet for the configure etc stuff is looking at the history of that file or older PRs to see how this kind of thing was handled there.
This PR where the GCS bindings were added might help: #13404, but let us know if you have any questions!
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
We'll also need additional CI jobs to test this - the PR I linked to above contains examples of what we added there.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
Great, thanks @thisisnic!
@marberts
marberts
commented
Mar 24, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
To do list.
@marberts
marberts
marked this pull request as draft
March 25, 2026 23:32
@Collinbrown95
Collinbrown95
commented
Mar 26, 2026

Copy link


Copy Markdown

Contributor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23576717645

@thisisnic
thisisnic
commented
Mar 26, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Revision: 4da7280
Submitted crossbow builds: ursacomputing/crossbow @ actions-92325d6543
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
@marberts
marberts
commented
Mar 29, 2026

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 29, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23698292050

@thisisnic
thisisnic
commented
Mar 30, 2026

Copy link


Copy Markdown

Member
Some of those CI failures are from things which have been fixed on main - you'll need to rebase your branch.
Collinbrown95and others
added 17 commits
July 10, 2026 07:12
@Collinbrown95@jonkeane
cleanup azurefs test function code
2bc10ba
@marberts@jonkeane
Trying PKGBUILD and windows build script again
169ca19
@marberts@jonkeane
Updated PKGBUILD
d39eef4
@Collinbrown95@jonkeane
debug ci: add azure dependencies that CI tries to build from source.
fb850f1
The hypothesis is that the errors in the build logs are caused by incompatibilities between WIL (Windows Implementation Library) and MinGW GCC, so install them directly instead of trying to build from source.
@marberts@jonkeane
Removed tmp.md
7619f01
@marberts@jonkeane
Removed windows build and added macos
ff68542
@marberts@jonkeane
azure off for windows ci
9ea60df
@marberts@jonkeane
Disable azure in configure.win
5a3eed4
@Collinbrown95@jonkeane
Add placeholder comment to flag present issues with MinGW and the Azu…
8f5d9ba
…re C++ SDK
@Collinbrown95@jonkeane
remove debugging script to install azure sdk from source
addcfa5
@marberts@jonkeane
Updated vignettes to reflect Azure off on Windows
b58fcc5
@marberts@jonkeane
Update github.packages.yml
d5cd943
@marberts@jonkeane
Fixing lint in filesystem.cpp
211c1fc
@marberts@jonkeane
Fixed warning in cpp code
c7dd3e5
@marberts@jonkeane
Added build info for azure to tasks for mac + a couple code comments
24c918b
@marberts@jonkeane
Fixed line endings on github.packages yaml
a8ad6a9
@marberts@jonkeane
libxml2 warning in with_cloud_support() is now conditional on azure b…
84f3b74
…eing requested
@jonkeane
jonkeaneforce-pushed
the
32123-expose-azure-blob-filesystem
branch
from
1ea9150 to
84f3b74CompareJuly 10, 2026 12:13
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
Thank you for your continued patience on this! I've rebased this against main and am running our extended CI to confirm if those failures we saw before were related to things on main from the last branching point.
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 84f3b74
Submitted crossbow builds: ursacomputing/crossbow @ actions-3cda80c98b
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-r-wasmGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit r-binary-packages
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 9dd6793
Submitted crossbow builds: ursacomputing/crossbow @ actions-a47c3da823
TaskStatus
r-binary-packagesGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
I pushed a change to make sure that the binary runners have libxml, etc. The other failures are unrelated to this branch I believe (though some aren't on https://crossbow.arrow-dev.org/ yet, so they might even be r-devel transitory actually!)
jonkeane
jonkeane
approved these changes
Jul 10, 2026
View reviewed changes
@jonkeanejonkeane
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing to push on this, I've rebased and made sure that CI is passing. I'm inclined to merge, but will give others (and especially @thisisnic) a day or two to speak up if they don't want me to.
@github-actionsgithub-actionsBot
removed
the 
awaiting change review
Awaiting change review
label
Jul 10, 2026
@marberts
marberts
commented
Jul 11, 2026

Copy link


Copy Markdown

ContributorAuthor
Awesome! Thank you.
@jonkeanejonkeane
mentioned this pull request
Jul 13, 2026

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 15, 2026

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b03b4c5.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@thisisnicthisisnicthisisnic left review comments
@jonkeanejonkeanejonkeane approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@koukouAwaiting requested review from koukou is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[R] Expose Azure Blob Storage filesystem


5 participants

@marberts@thisisnic@Collinbrown95@jonkeane@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GH-32123: [R] Expose azure blob filesystem by marberts · Pull Request #49553 · apache/arrow · GitHub
Skip to content
GH-32123: [R] Expose azure blob filesystem - #49553
Merged
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Jul 13, 2026
Merged
GH-32123: [R] Expose azure blob filesystem#49553
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Conversation
@marberts
@marbertsmarberts
commented
Mar 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
This PR adds support for Azure. The Arrow R package already has support for AWS and GCS, and the Arrow C++ library has had support for Azure for a couple years now. Support for Azure is already available in pyarrow.
This would close#32123.
What changes are included in this PR?
  • A new class AzureFileSystem that's analogous to S3FileSystem/GcsFileSystem, along with a helper function az_container() that's analogous to s3_bucket()/gcs_bucket().
  • Updates to src/filesystem.cpp to interact with the machinery in arrow/filesystem/azurefs.h.
  • Updates to the configuration and build scripts to support building with support for Azure.
  • Updates to the vignettes on cloud storage, installation, and developer setup.
Are these changes tested?
Yes. See tests/testthat/test-azure.R.
Are there any user-facing changes?
Yes. There is a new function az_container(), serving the analogous role to s3_bucket()/gcs_bucket(), along with an R6 class AzureFileSystem, again serving the same role as S3FileSystem and GcsFileSystem. There is also a function arrow_with_azure() to indicate if Arrow was built with support for Azure.
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

Thanks for opening a pull request!
If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or
MINOR: [${COMPONENT}] ${SUMMARY}

See also:
@marbertsmarberts
changed the title
32123 expose azure blob filesystemGH-32123 [R]: Expose azure blob filesystemMar 19, 2026
@marbertsmarberts
changed the title
GH-32123 [R]: Expose azure blob filesystemGH-32123: [R] Expose azure blob filesystemMar 19, 2026
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

⚠️ GitHub issue #32123has been automatically assigned in GitHub to PR creator.
@marberts
marberts
commented
Mar 20, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
The CI is failing because the Azure C++ SDK depends on libxml2.
  •  Update tools/nixlibs.R to check for libxml2 in with_cloud_support(), turning off the Azure feature flag if libxml2 isn't found.

    •  What about configure and configure.win?
    •  Is with_cloud_support() the only function that needs changing in tools/nixlibs.R?
  •  Update the install.Rmd and developers/setup.Rmd vignettes to note that libxml2 is a dependency of Azure (same as with curl and openssl).
  •  Update DESCRIPTION to include libxml2 in SystemRequirements.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
It looks like the failing CI job is from a warning in R CMD check about a non-API call to a C function, due to all the changes in R-devel. I don't think this is due to our PR.
At this point, I'm fairly confident that we've implemented the Azure features correctly. The only thing I'm not sure about is the setup in configure, configure.win, and tools/nixlibs.R. The Azure C++ SDK used by Arrow requires libxml2, which is a new system dependency for this package. I don't fully understand how these configuration scripts work, so I wouldn't be surprised if they're not updated correctly to reflect the new system dependency.
@jonkeane, @thisisnic, @assignUser Please let me or @Collinbrown95 know if there are any changes needed to this PR.
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
Thanks for the PR @marberts - if you rebase from the main branch now, the non-API call stuff should be resolved.
Your best bet for the configure etc stuff is looking at the history of that file or older PRs to see how this kind of thing was handled there.
This PR where the GCS bindings were added might help: #13404, but let us know if you have any questions!
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
We'll also need additional CI jobs to test this - the PR I linked to above contains examples of what we added there.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
Great, thanks @thisisnic!
@marberts
marberts
commented
Mar 24, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
To do list.
@marberts
marberts
marked this pull request as draft
March 25, 2026 23:32
@Collinbrown95
Collinbrown95
commented
Mar 26, 2026

Copy link


Copy Markdown

Contributor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23576717645

@thisisnic
thisisnic
commented
Mar 26, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Revision: 4da7280
Submitted crossbow builds: ursacomputing/crossbow @ actions-92325d6543
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
@marberts
marberts
commented
Mar 29, 2026

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 29, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23698292050

@thisisnic
thisisnic
commented
Mar 30, 2026

Copy link


Copy Markdown

Member
Some of those CI failures are from things which have been fixed on main - you'll need to rebase your branch.
Collinbrown95and others
added 17 commits
July 10, 2026 07:12
@Collinbrown95@jonkeane
cleanup azurefs test function code
2bc10ba
@marberts@jonkeane
Trying PKGBUILD and windows build script again
169ca19
@marberts@jonkeane
Updated PKGBUILD
d39eef4
@Collinbrown95@jonkeane
debug ci: add azure dependencies that CI tries to build from source.
fb850f1
The hypothesis is that the errors in the build logs are caused by incompatibilities between WIL (Windows Implementation Library) and MinGW GCC, so install them directly instead of trying to build from source.
@marberts@jonkeane
Removed tmp.md
7619f01
@marberts@jonkeane
Removed windows build and added macos
ff68542
@marberts@jonkeane
azure off for windows ci
9ea60df
@marberts@jonkeane
Disable azure in configure.win
5a3eed4
@Collinbrown95@jonkeane
Add placeholder comment to flag present issues with MinGW and the Azu…
8f5d9ba
…re C++ SDK
@Collinbrown95@jonkeane
remove debugging script to install azure sdk from source
addcfa5
@marberts@jonkeane
Updated vignettes to reflect Azure off on Windows
b58fcc5
@marberts@jonkeane
Update github.packages.yml
d5cd943
@marberts@jonkeane
Fixing lint in filesystem.cpp
211c1fc
@marberts@jonkeane
Fixed warning in cpp code
c7dd3e5
@marberts@jonkeane
Added build info for azure to tasks for mac + a couple code comments
24c918b
@marberts@jonkeane
Fixed line endings on github.packages yaml
a8ad6a9
@marberts@jonkeane
libxml2 warning in with_cloud_support() is now conditional on azure b…
84f3b74
…eing requested
@jonkeane
jonkeaneforce-pushed
the
32123-expose-azure-blob-filesystem
branch
from
1ea9150 to
84f3b74CompareJuly 10, 2026 12:13
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
Thank you for your continued patience on this! I've rebased this against main and am running our extended CI to confirm if those failures we saw before were related to things on main from the last branching point.
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 84f3b74
Submitted crossbow builds: ursacomputing/crossbow @ actions-3cda80c98b
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-r-wasmGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit r-binary-packages
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 9dd6793
Submitted crossbow builds: ursacomputing/crossbow @ actions-a47c3da823
TaskStatus
r-binary-packagesGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
I pushed a change to make sure that the binary runners have libxml, etc. The other failures are unrelated to this branch I believe (though some aren't on https://crossbow.arrow-dev.org/ yet, so they might even be r-devel transitory actually!)
jonkeane
jonkeane
approved these changes
Jul 10, 2026
View reviewed changes
@jonkeanejonkeane
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing to push on this, I've rebased and made sure that CI is passing. I'm inclined to merge, but will give others (and especially @thisisnic) a day or two to speak up if they don't want me to.
@github-actionsgithub-actionsBot
removed
the 
awaiting change review
Awaiting change review
label
Jul 10, 2026
@marberts
marberts
commented
Jul 11, 2026

Copy link


Copy Markdown

ContributorAuthor
Awesome! Thank you.
@jonkeanejonkeane
mentioned this pull request
Jul 13, 2026

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 15, 2026

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b03b4c5.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@thisisnicthisisnicthisisnic left review comments
@jonkeanejonkeanejonkeane approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@koukouAwaiting requested review from koukou is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[R] Expose Azure Blob Storage filesystem


5 participants

@marberts@thisisnic@Collinbrown95@jonkeane@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
GH-32123: [R] Expose azure blob filesystem by marberts · Pull Request #49553 · apache/arrow · GitHub
Skip to content
GH-32123: [R] Expose azure blob filesystem - #49553
Merged
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Jul 13, 2026
Merged
GH-32123: [R] Expose azure blob filesystem#49553
jonkeane merged 89 commits into
apache:mainfrom 
marberts:32123-expose-azure-blob-filesystem
Conversation
@marberts
@marbertsmarberts
commented
Mar 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
This PR adds support for Azure. The Arrow R package already has support for AWS and GCS, and the Arrow C++ library has had support for Azure for a couple years now. Support for Azure is already available in pyarrow.
This would close#32123.
What changes are included in this PR?
  • A new class AzureFileSystem that's analogous to S3FileSystem/GcsFileSystem, along with a helper function az_container() that's analogous to s3_bucket()/gcs_bucket().
  • Updates to src/filesystem.cpp to interact with the machinery in arrow/filesystem/azurefs.h.
  • Updates to the configuration and build scripts to support building with support for Azure.
  • Updates to the vignettes on cloud storage, installation, and developer setup.
Are these changes tested?
Yes. See tests/testthat/test-azure.R.
Are there any user-facing changes?
Yes. There is a new function az_container(), serving the analogous role to s3_bucket()/gcs_bucket(), along with an R6 class AzureFileSystem, again serving the same role as S3FileSystem and GcsFileSystem. There is also a function arrow_with_azure() to indicate if Arrow was built with support for Azure.
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

Thanks for opening a pull request!
If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or
MINOR: [${COMPONENT}] ${SUMMARY}

See also:
@marbertsmarberts
changed the title
32123 expose azure blob filesystemGH-32123 [R]: Expose azure blob filesystemMar 19, 2026
@marbertsmarberts
changed the title
GH-32123 [R]: Expose azure blob filesystemGH-32123: [R] Expose azure blob filesystemMar 19, 2026
@github-actions
github-actionsBot
commented
Mar 19, 2026

Copy link


Copy Markdown

⚠️ GitHub issue #32123has been automatically assigned in GitHub to PR creator.
@marberts
marberts
commented
Mar 20, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
The CI is failing because the Azure C++ SDK depends on libxml2.
  •  Update tools/nixlibs.R to check for libxml2 in with_cloud_support(), turning off the Azure feature flag if libxml2 isn't found.

    •  What about configure and configure.win?
    •  Is with_cloud_support() the only function that needs changing in tools/nixlibs.R?
  •  Update the install.Rmd and developers/setup.Rmd vignettes to note that libxml2 is a dependency of Azure (same as with curl and openssl).
  •  Update DESCRIPTION to include libxml2 in SystemRequirements.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
It looks like the failing CI job is from a warning in R CMD check about a non-API call to a C function, due to all the changes in R-devel. I don't think this is due to our PR.
At this point, I'm fairly confident that we've implemented the Azure features correctly. The only thing I'm not sure about is the setup in configure, configure.win, and tools/nixlibs.R. The Azure C++ SDK used by Arrow requires libxml2, which is a new system dependency for this package. I don't fully understand how these configuration scripts work, so I wouldn't be surprised if they're not updated correctly to reflect the new system dependency.
@jonkeane, @thisisnic, @assignUser Please let me or @Collinbrown95 know if there are any changes needed to this PR.
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
Thanks for the PR @marberts - if you rebase from the main branch now, the non-API call stuff should be resolved.
Your best bet for the configure etc stuff is looking at the history of that file or older PRs to see how this kind of thing was handled there.
This PR where the GCS bindings were added might help: #13404, but let us know if you have any questions!
@thisisnic
thisisnic
commented
Mar 24, 2026

Copy link


Copy Markdown

Member
We'll also need additional CI jobs to test this - the PR I linked to above contains examples of what we added there.
@marberts
marberts
commented
Mar 24, 2026

Copy link


Copy Markdown

ContributorAuthor
Great, thanks @thisisnic!
@marberts
marberts
commented
Mar 24, 2026•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
To do list.
@marberts
marberts
marked this pull request as draft
March 25, 2026 23:32
@Collinbrown95
Collinbrown95
commented
Mar 26, 2026

Copy link


Copy Markdown

Contributor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23576717645

@thisisnic
thisisnic
commented
Mar 26, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 26, 2026

Copy link


Copy Markdown

Revision: 4da7280
Submitted crossbow builds: ursacomputing/crossbow @ actions-92325d6543
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
@marberts
marberts
commented
Mar 29, 2026

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g r
@github-actions
github-actionsBot
commented
Mar 29, 2026

Copy link


Copy Markdown

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/23698292050

@thisisnic
thisisnic
commented
Mar 30, 2026

Copy link


Copy Markdown

Member
Some of those CI failures are from things which have been fixed on main - you'll need to rebase your branch.
Collinbrown95and others
added 17 commits
July 10, 2026 07:12
@Collinbrown95@jonkeane
cleanup azurefs test function code
2bc10ba
@marberts@jonkeane
Trying PKGBUILD and windows build script again
169ca19
@marberts@jonkeane
Updated PKGBUILD
d39eef4
@Collinbrown95@jonkeane
debug ci: add azure dependencies that CI tries to build from source.
fb850f1
The hypothesis is that the errors in the build logs are caused by incompatibilities between WIL (Windows Implementation Library) and MinGW GCC, so install them directly instead of trying to build from source.
@marberts@jonkeane
Removed tmp.md
7619f01
@marberts@jonkeane
Removed windows build and added macos
ff68542
@marberts@jonkeane
azure off for windows ci
9ea60df
@marberts@jonkeane
Disable azure in configure.win
5a3eed4
@Collinbrown95@jonkeane
Add placeholder comment to flag present issues with MinGW and the Azu…
8f5d9ba
…re C++ SDK
@Collinbrown95@jonkeane
remove debugging script to install azure sdk from source
addcfa5
@marberts@jonkeane
Updated vignettes to reflect Azure off on Windows
b58fcc5
@marberts@jonkeane
Update github.packages.yml
d5cd943
@marberts@jonkeane
Fixing lint in filesystem.cpp
211c1fc
@marberts@jonkeane
Fixed warning in cpp code
c7dd3e5
@marberts@jonkeane
Added build info for azure to tasks for mac + a couple code comments
24c918b
@marberts@jonkeane
Fixed line endings on github.packages yaml
a8ad6a9
@marberts@jonkeane
libxml2 warning in with_cloud_support() is now conditional on azure b…
84f3b74
…eing requested
@jonkeane
jonkeaneforce-pushed
the
32123-expose-azure-blob-filesystem
branch
from
1ea9150 to
84f3b74CompareJuly 10, 2026 12:13
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g r
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
Thank you for your continued patience on this! I've rebased this against main and am running our extended CI to confirm if those failures we saw before were related to things on main from the last branching point.
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 84f3b74
Submitted crossbow builds: ursacomputing/crossbow @ actions-3cda80c98b
TaskStatus
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-r-alpine-linux-cranGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-fedora-clangGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-offline-maximalGitHub Actions
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-r-wasmGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
@github-actions crossbow submit r-binary-packages
@github-actions
github-actionsBot
commented
Jul 10, 2026

Copy link


Copy Markdown

Revision: 9dd6793
Submitted crossbow builds: ursacomputing/crossbow @ actions-a47c3da823
TaskStatus
r-binary-packagesGitHub Actions
@jonkeane
jonkeane
commented
Jul 10, 2026

Copy link


Copy Markdown

Member
I pushed a change to make sure that the binary runners have libxml, etc. The other failures are unrelated to this branch I believe (though some aren't on https://crossbow.arrow-dev.org/ yet, so they might even be r-devel transitory actually!)
jonkeane
jonkeane
approved these changes
Jul 10, 2026
View reviewed changes
@jonkeanejonkeane
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing to push on this, I've rebased and made sure that CI is passing. I'm inclined to merge, but will give others (and especially @thisisnic) a day or two to speak up if they don't want me to.
@github-actionsgithub-actionsBot
removed
the 
awaiting change review
Awaiting change review
label
Jul 10, 2026
@marberts
marberts
commented
Jul 11, 2026

Copy link


Copy Markdown

ContributorAuthor
Awesome! Thank you.
@jonkeanejonkeane
mentioned this pull request
Jul 13, 2026

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 15, 2026

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b03b4c5.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@thisisnicthisisnicthisisnic left review comments
@jonkeanejonkeanejonkeane approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@koukouAwaiting requested review from koukou is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[R] Expose Azure Blob Storage filesystem


5 participants

@marberts@thisisnic@Collinbrown95@jonkeane@kou