[C++] Handle URI paths for filesystems? #34386

Closed
#34420
Closed
[C++] Handle URI paths for filesystems?#34386
#34420
Assignees
westonpace
Labels
Component: C++Type: enhancement
Milestone
13.0.0

Description

@lidavidm
lidavidm
opened on Feb 28, 2023

Describe the enhancement requested

Given a URI like s3://bucket/foo/bar.parquet, right now, you can turn that URI into a FileSystem instance + a non-URI path. That is useful, but then the FileSystem APIs don't work with URIs. For instance, it's explicitly rejected here:

arrow/cpp/src/arrow/filesystem/s3fs.cc

Lines 416 to 419 in 0844924

if (internal::IsLikelyUri(s)) {
returnStatus::Invalid(
"Expected an S3 object path of the form 'bucket/key...', got a URI: '", s, "'");
}

So if you have a set of URIs and want to work with them with filesystems, you have to parse the URIs yourself. Given we already do the URI munging internally, it might be nice for FileSystem instances to expose a method to parse URIs into a path as well (raising an error if the URI is incompatible).

Component(s)

C++

Metadata

Metadata

Assignees

Labels

Component: C++Type: enhancement

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
     blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

    Skip to content
    [C++] Handle URI paths for filesystems? #34386
    Closed
    #34420
    Closed
    [C++] Handle URI paths for filesystems?#34386
    #34420
    Assignees
    westonpace
    Labels
    Component: C++Type: enhancement
    Milestone
    13.0.0
    Description
    @lidavidm
    lidavidm
    opened on Feb 28, 2023
    Describe the enhancement requested
    Given a URI like s3://bucket/foo/bar.parquet, right now, you can turn that URI into a FileSystem instance + a non-URI path. That is useful, but then the FileSystem APIs don't work with URIs. For instance, it's explicitly rejected here:
    arrow/cpp/src/arrow/filesystem/s3fs.cc
    
Lines 416 to 419
in
0844924
    if (internal::IsLikelyUri(s)) { 
    returnStatus::Invalid( 
    "Expected an S3 object path of the form 'bucket/key...', got a URI: '", s, "'"); 
     } 
    So if you have a set of URIs and want to work with them with filesystems, you have to parse the URIs yourself. Given we already do the URI munging internally, it might be nice for FileSystem instances to expose a method to parse URIs into a path as well (raising an error if the URI is incompatible).
    Component(s)
    C++
    Metadata
    Metadata
    Assignees
    Labels
    Component: C++Type: enhancement
    Type
    No type
    Projects
    No projects
      Milestone
      Relationships
      None yet
      Development
      No branches or pull requests
      Issue actions
      
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

      Skip to content
      [C++] Handle URI paths for filesystems? #34386
      Closed
      #34420
      Closed
      [C++] Handle URI paths for filesystems?#34386
      #34420
      Assignees
      westonpace
      Labels
      Component: C++Type: enhancement
      Milestone
      13.0.0
      Description
      @lidavidm
      lidavidm
      opened on Feb 28, 2023
      Describe the enhancement requested
      Given a URI like s3://bucket/foo/bar.parquet, right now, you can turn that URI into a FileSystem instance + a non-URI path. That is useful, but then the FileSystem APIs don't work with URIs. For instance, it's explicitly rejected here:
      arrow/cpp/src/arrow/filesystem/s3fs.cc
      
Lines 416 to 419
in
0844924
      if (internal::IsLikelyUri(s)) { 
      returnStatus::Invalid( 
      "Expected an S3 object path of the form 'bucket/key...', got a URI: '", s, "'"); 
       } 
      So if you have a set of URIs and want to work with them with filesystems, you have to parse the URIs yourself. Given we already do the URI munging internally, it might be nice for FileSystem instances to expose a method to parse URIs into a path as well (raising an error if the URI is incompatible).
      Component(s)
      C++
      Metadata
      Metadata
      Assignees
      Labels
      Component: C++Type: enhancement
      Type
      No type
      Projects
      No projects
        Milestone
        Relationships
        None yet
        Development
        No branches or pull requests
        Issue actions
        
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

        Skip to content
        [C++] Handle URI paths for filesystems? #34386
        Closed
        #34420
        Closed
        [C++] Handle URI paths for filesystems?#34386
        #34420
        Assignees
        westonpace
        Labels
        Component: C++Type: enhancement
        Milestone
        13.0.0
        Description
        @lidavidm
        lidavidm
        opened on Feb 28, 2023
        Describe the enhancement requested
        Given a URI like s3://bucket/foo/bar.parquet, right now, you can turn that URI into a FileSystem instance + a non-URI path. That is useful, but then the FileSystem APIs don't work with URIs. For instance, it's explicitly rejected here:
        arrow/cpp/src/arrow/filesystem/s3fs.cc
        
Lines 416 to 419
in
0844924
        if (internal::IsLikelyUri(s)) { 
        returnStatus::Invalid( 
        "Expected an S3 object path of the form 'bucket/key...', got a URI: '", s, "'"); 
         } 
        So if you have a set of URIs and want to work with them with filesystems, you have to parse the URIs yourself. Given we already do the URI munging internally, it might be nice for FileSystem instances to expose a method to parse URIs into a path as well (raising an error if the URI is incompatible).
        Component(s)
        C++
        Metadata
        Metadata
        Assignees
        Labels
        Component: C++Type: enhancement
        Type
        No type
        Projects
        No projects
          Milestone
          Relationships
          None yet
          Development
          No branches or pull requests
          Issue actions
          
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

          Skip to content
          [C++] Handle URI paths for filesystems? #34386
          Closed
          #34420
          Closed
          [C++] Handle URI paths for filesystems?#34386
          #34420
          Assignees
          westonpace
          Labels
          Component: C++Type: enhancement
          Milestone
          13.0.0
          Description
          @lidavidm
          lidavidm
          opened on Feb 28, 2023
          Describe the enhancement requested
          Given a URI like s3://bucket/foo/bar.parquet, right now, you can turn that URI into a FileSystem instance + a non-URI path. That is useful, but then the FileSystem APIs don't work with URIs. For instance, it's explicitly rejected here:
          arrow/cpp/src/arrow/filesystem/s3fs.cc
          
Lines 416 to 419
in
0844924
          if (internal::IsLikelyUri(s)) { 
          returnStatus::Invalid( 
          "Expected an S3 object path of the form 'bucket/key...', got a URI: '", s, "'"); 
           } 
          So if you have a set of URIs and want to work with them with filesystems, you have to parse the URIs yourself. Given we already do the URI munging internally, it might be nice for FileSystem instances to expose a method to parse URIs into a path as well (raising an error if the URI is incompatible).
          Component(s)
          C++
          Metadata
          Metadata
          Assignees
          Labels
          Component: C++Type: enhancement
          Type
          No type
          Projects
          No projects
            Milestone
            Relationships
            None yet
            Development
            No branches or pull requests
            Issue actions
            
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

            Skip to content
            [C++] Handle URI paths for filesystems? #34386
            Closed
            #34420
            Closed
            [C++] Handle URI paths for filesystems?#34386
            #34420
            Assignees
            westonpace
            Labels
            Component: C++Type: enhancement
            Milestone
            13.0.0
            Description
            @lidavidm
            lidavidm
            opened on Feb 28, 2023
            Describe the enhancement requested
            Given a URI like s3://bucket/foo/bar.parquet, right now, you can turn that URI into a FileSystem instance + a non-URI path. That is useful, but then the FileSystem APIs don't work with URIs. For instance, it's explicitly rejected here:
            arrow/cpp/src/arrow/filesystem/s3fs.cc
            
Lines 416 to 419
in
0844924
            if (internal::IsLikelyUri(s)) { 
            returnStatus::Invalid( 
            "Expected an S3 object path of the form 'bucket/key...', got a URI: '", s, "'"); 
             } 
            So if you have a set of URIs and want to work with them with filesystems, you have to parse the URIs yourself. Given we already do the URI munging internally, it might be nice for FileSystem instances to expose a method to parse URIs into a path as well (raising an error if the URI is incompatible).
            Component(s)
            C++
            Metadata
            Metadata
            Assignees
            Labels
            Component: C++Type: enhancement
            Type
            No type
            Projects
            No projects
              Milestone
              Relationships
              None yet
              Development
              No branches or pull requests
              Issue actions
              
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

              Skip to content
              [C++] Handle URI paths for filesystems? #34386
              Closed
              #34420
              Closed
              [C++] Handle URI paths for filesystems?#34386
              #34420
              Assignees
              westonpace
              Labels
              Component: C++Type: enhancement
              Milestone
              13.0.0
              Description
              @lidavidm
              lidavidm
              opened on Feb 28, 2023
              Describe the enhancement requested
              Given a URI like s3://bucket/foo/bar.parquet, right now, you can turn that URI into a FileSystem instance + a non-URI path. That is useful, but then the FileSystem APIs don't work with URIs. For instance, it's explicitly rejected here:
              arrow/cpp/src/arrow/filesystem/s3fs.cc
              
Lines 416 to 419
in
0844924
              if (internal::IsLikelyUri(s)) { 
              returnStatus::Invalid( 
              "Expected an S3 object path of the form 'bucket/key...', got a URI: '", s, "'"); 
               } 
              So if you have a set of URIs and want to work with them with filesystems, you have to parse the URIs yourself. Given we already do the URI munging internally, it might be nice for FileSystem instances to expose a method to parse URIs into a path as well (raising an error if the URI is incompatible).
              Component(s)
              C++
              Metadata
              Metadata
              Assignees
              Labels
              Component: C++Type: enhancement
              Type
              No type
              Projects
              No projects
                Milestone
                Relationships
                None yet
                Development
                No branches or pull requests
                Issue actions
                
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

                Skip to content
                [C++] Handle URI paths for filesystems? #34386
                Closed
                #34420
                Closed
                [C++] Handle URI paths for filesystems?#34386
                #34420
                Assignees
                westonpace
                Labels
                Component: C++Type: enhancement
                Milestone
                13.0.0
                Description
                @lidavidm
                lidavidm
                opened on Feb 28, 2023
                Describe the enhancement requested
                Given a URI like s3://bucket/foo/bar.parquet, right now, you can turn that URI into a FileSystem instance + a non-URI path. That is useful, but then the FileSystem APIs don't work with URIs. For instance, it's explicitly rejected here:
                arrow/cpp/src/arrow/filesystem/s3fs.cc
                
Lines 416 to 419
in
0844924
                if (internal::IsLikelyUri(s)) { 
                returnStatus::Invalid( 
                "Expected an S3 object path of the form 'bucket/key...', got a URI: '", s, "'"); 
                 } 
                So if you have a set of URIs and want to work with them with filesystems, you have to parse the URIs yourself. Given we already do the URI munging internally, it might be nice for FileSystem instances to expose a method to parse URIs into a path as well (raising an error if the URI is incompatible).
                Component(s)
                C++
                Metadata
                Metadata
                Assignees
                Labels
                Component: C++Type: enhancement
                Type
                No type
                Projects
                No projects
                  Milestone
                  Relationships
                  None yet
                  Development
                  No branches or pull requests
                  Issue actions