[C++][Java] Handle Arrow stream with zero record batch #18090

Closed
Closed
[C++][Java] Handle Arrow stream with zero record batch#18090
Assignees
wesm
Labels
Component: C++Component: JavaType: bug
Milestone
0.14.0

Description

@asfimport
asfimport
opened on Feb 8, 2018

It looks like currently many places of the code assume that there needs to be at least one record batch for streaming format. Is zero-recordbatch not supported by design?

e.g. https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/StreamToFile.java#L45


public static void convert(InputStream in, OutputStream out) throws IOException {
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
try (ArrowStreamReader reader = new ArrowStreamReader(in, allocator)) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
// load the first batch before instantiating the writer so that we have any dictionaries
if (!reader.loadNextBatch()) {
throw new IOException("Unable to read first record batch");
}
...

Pyarrow-0.8.0 does not load 0-recordbatch stream either. It would throw an exception originated from https://github.com/apache/arrow/blob/a95465b8ce7a32feeaae3e13d0a64102ffa590d9/cpp/src/arrow/table.cc#L309:


Status Table::FromRecordBatches(const std::vector<std::shared_ptr<RecordBatch>>& batches,
std::shared_ptr<Table>* table) {
if (batches.size() == 0) {
return Status::Invalid("Must pass at least one record batch");
}
...

Reporter: Jingyuan Wang / @alphalfalfa
Assignee: Wes McKinney / @wesm

PRs and other links:

Note: This issue was originally created as ARROW-2119. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

Labels

Component: C++Component: JavaType: bug

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
     blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

    Skip to content
    [C++][Java] Handle Arrow stream with zero record batch #18090
    Closed
    Closed
    [C++][Java] Handle Arrow stream with zero record batch#18090
    Assignees
    wesm
    Labels
    Component: C++Component: JavaType: bug
    Milestone
    0.14.0
    Description
    @asfimport
    asfimport
    opened on Feb 8, 2018
    It looks like currently many places of the code assume that there needs to be at least one record batch for streaming format. Is zero-recordbatch not supported by design?
    e.g. https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/StreamToFile.java#L45
    
public static void convert(InputStream in, OutputStream out) throws IOException {
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
try (ArrowStreamReader reader = new ArrowStreamReader(in, allocator)) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
// load the first batch before instantiating the writer so that we have any dictionaries
if (!reader.loadNextBatch()) {
throw new IOException("Unable to read first record batch");
}
...

    Pyarrow-0.8.0 does not load 0-recordbatch stream either. It would throw an exception originated from https://github.com/apache/arrow/blob/a95465b8ce7a32feeaae3e13d0a64102ffa590d9/cpp/src/arrow/table.cc#L309:
    
Status Table::FromRecordBatches(const std::vector<std::shared_ptr<RecordBatch>>& batches,
std::shared_ptr<Table>* table) {
if (batches.size() == 0) {
return Status::Invalid("Must pass at least one record batch");
}
...

    Reporter: Jingyuan Wang / @alphalfalfa
    Assignee: Wes McKinney / @wesm
    PRs and other links:
    Note: This issue was originally created as ARROW-2119. Please see the migration documentation for further details.
    Metadata
    Metadata
    Assignees
    Labels
    Component: C++Component: JavaType: bug
    Type
    No type
    Projects
    No projects
      Milestone
      Relationships
      None yet
      Development
      No branches or pull requests
      Issue actions
      
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

      Skip to content
      [C++][Java] Handle Arrow stream with zero record batch #18090
      Closed
      Closed
      [C++][Java] Handle Arrow stream with zero record batch#18090
      Assignees
      wesm
      Labels
      Component: C++Component: JavaType: bug
      Milestone
      0.14.0
      Description
      @asfimport
      asfimport
      opened on Feb 8, 2018
      It looks like currently many places of the code assume that there needs to be at least one record batch for streaming format. Is zero-recordbatch not supported by design?
      e.g. https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/StreamToFile.java#L45
      
public static void convert(InputStream in, OutputStream out) throws IOException {
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
try (ArrowStreamReader reader = new ArrowStreamReader(in, allocator)) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
// load the first batch before instantiating the writer so that we have any dictionaries
if (!reader.loadNextBatch()) {
throw new IOException("Unable to read first record batch");
}
...

      Pyarrow-0.8.0 does not load 0-recordbatch stream either. It would throw an exception originated from https://github.com/apache/arrow/blob/a95465b8ce7a32feeaae3e13d0a64102ffa590d9/cpp/src/arrow/table.cc#L309:
      
Status Table::FromRecordBatches(const std::vector<std::shared_ptr<RecordBatch>>& batches,
std::shared_ptr<Table>* table) {
if (batches.size() == 0) {
return Status::Invalid("Must pass at least one record batch");
}
...

      Reporter: Jingyuan Wang / @alphalfalfa
      Assignee: Wes McKinney / @wesm
      PRs and other links:
      Note: This issue was originally created as ARROW-2119. Please see the migration documentation for further details.
      Metadata
      Metadata
      Assignees
      Labels
      Component: C++Component: JavaType: bug
      Type
      No type
      Projects
      No projects
        Milestone
        Relationships
        None yet
        Development
        No branches or pull requests
        Issue actions
        
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

        Skip to content
        [C++][Java] Handle Arrow stream with zero record batch #18090
        Closed
        Closed
        [C++][Java] Handle Arrow stream with zero record batch#18090
        Assignees
        wesm
        Labels
        Component: C++Component: JavaType: bug
        Milestone
        0.14.0
        Description
        @asfimport
        asfimport
        opened on Feb 8, 2018
        It looks like currently many places of the code assume that there needs to be at least one record batch for streaming format. Is zero-recordbatch not supported by design?
        e.g. https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/StreamToFile.java#L45
        
public static void convert(InputStream in, OutputStream out) throws IOException {
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
try (ArrowStreamReader reader = new ArrowStreamReader(in, allocator)) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
// load the first batch before instantiating the writer so that we have any dictionaries
if (!reader.loadNextBatch()) {
throw new IOException("Unable to read first record batch");
}
...

        Pyarrow-0.8.0 does not load 0-recordbatch stream either. It would throw an exception originated from https://github.com/apache/arrow/blob/a95465b8ce7a32feeaae3e13d0a64102ffa590d9/cpp/src/arrow/table.cc#L309:
        
Status Table::FromRecordBatches(const std::vector<std::shared_ptr<RecordBatch>>& batches,
std::shared_ptr<Table>* table) {
if (batches.size() == 0) {
return Status::Invalid("Must pass at least one record batch");
}
...

        Reporter: Jingyuan Wang / @alphalfalfa
        Assignee: Wes McKinney / @wesm
        PRs and other links:
        Note: This issue was originally created as ARROW-2119. Please see the migration documentation for further details.
        Metadata
        Metadata
        Assignees
        Labels
        Component: C++Component: JavaType: bug
        Type
        No type
        Projects
        No projects
          Milestone
          Relationships
          None yet
          Development
          No branches or pull requests
          Issue actions
          
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

          Skip to content
          [C++][Java] Handle Arrow stream with zero record batch #18090
          Closed
          Closed
          [C++][Java] Handle Arrow stream with zero record batch#18090
          Assignees
          wesm
          Labels
          Component: C++Component: JavaType: bug
          Milestone
          0.14.0
          Description
          @asfimport
          asfimport
          opened on Feb 8, 2018
          It looks like currently many places of the code assume that there needs to be at least one record batch for streaming format. Is zero-recordbatch not supported by design?
          e.g. https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/StreamToFile.java#L45
          
public static void convert(InputStream in, OutputStream out) throws IOException {
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
try (ArrowStreamReader reader = new ArrowStreamReader(in, allocator)) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
// load the first batch before instantiating the writer so that we have any dictionaries
if (!reader.loadNextBatch()) {
throw new IOException("Unable to read first record batch");
}
...

          Pyarrow-0.8.0 does not load 0-recordbatch stream either. It would throw an exception originated from https://github.com/apache/arrow/blob/a95465b8ce7a32feeaae3e13d0a64102ffa590d9/cpp/src/arrow/table.cc#L309:
          
Status Table::FromRecordBatches(const std::vector<std::shared_ptr<RecordBatch>>& batches,
std::shared_ptr<Table>* table) {
if (batches.size() == 0) {
return Status::Invalid("Must pass at least one record batch");
}
...

          Reporter: Jingyuan Wang / @alphalfalfa
          Assignee: Wes McKinney / @wesm
          PRs and other links:
          Note: This issue was originally created as ARROW-2119. Please see the migration documentation for further details.
          Metadata
          Metadata
          Assignees
          Labels
          Component: C++Component: JavaType: bug
          Type
          No type
          Projects
          No projects
            Milestone
            Relationships
            None yet
            Development
            No branches or pull requests
            Issue actions
            
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

            Skip to content
            [C++][Java] Handle Arrow stream with zero record batch #18090
            Closed
            Closed
            [C++][Java] Handle Arrow stream with zero record batch#18090
            Assignees
            wesm
            Labels
            Component: C++Component: JavaType: bug
            Milestone
            0.14.0
            Description
            @asfimport
            asfimport
            opened on Feb 8, 2018
            It looks like currently many places of the code assume that there needs to be at least one record batch for streaming format. Is zero-recordbatch not supported by design?
            e.g. https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/StreamToFile.java#L45
            
public static void convert(InputStream in, OutputStream out) throws IOException {
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
try (ArrowStreamReader reader = new ArrowStreamReader(in, allocator)) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
// load the first batch before instantiating the writer so that we have any dictionaries
if (!reader.loadNextBatch()) {
throw new IOException("Unable to read first record batch");
}
...

            Pyarrow-0.8.0 does not load 0-recordbatch stream either. It would throw an exception originated from https://github.com/apache/arrow/blob/a95465b8ce7a32feeaae3e13d0a64102ffa590d9/cpp/src/arrow/table.cc#L309:
            
Status Table::FromRecordBatches(const std::vector<std::shared_ptr<RecordBatch>>& batches,
std::shared_ptr<Table>* table) {
if (batches.size() == 0) {
return Status::Invalid("Must pass at least one record batch");
}
...

            Reporter: Jingyuan Wang / @alphalfalfa
            Assignee: Wes McKinney / @wesm
            PRs and other links:
            Note: This issue was originally created as ARROW-2119. Please see the migration documentation for further details.
            Metadata
            Metadata
            Assignees
            Labels
            Component: C++Component: JavaType: bug
            Type
            No type
            Projects
            No projects
              Milestone
              Relationships
              None yet
              Development
              No branches or pull requests
              Issue actions
              
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

              Skip to content
              [C++][Java] Handle Arrow stream with zero record batch #18090
              Closed
              Closed
              [C++][Java] Handle Arrow stream with zero record batch#18090
              Assignees
              wesm
              Labels
              Component: C++Component: JavaType: bug
              Milestone
              0.14.0
              Description
              @asfimport
              asfimport
              opened on Feb 8, 2018
              It looks like currently many places of the code assume that there needs to be at least one record batch for streaming format. Is zero-recordbatch not supported by design?
              e.g. https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/StreamToFile.java#L45
              
public static void convert(InputStream in, OutputStream out) throws IOException {
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
try (ArrowStreamReader reader = new ArrowStreamReader(in, allocator)) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
// load the first batch before instantiating the writer so that we have any dictionaries
if (!reader.loadNextBatch()) {
throw new IOException("Unable to read first record batch");
}
...

              Pyarrow-0.8.0 does not load 0-recordbatch stream either. It would throw an exception originated from https://github.com/apache/arrow/blob/a95465b8ce7a32feeaae3e13d0a64102ffa590d9/cpp/src/arrow/table.cc#L309:
              
Status Table::FromRecordBatches(const std::vector<std::shared_ptr<RecordBatch>>& batches,
std::shared_ptr<Table>* table) {
if (batches.size() == 0) {
return Status::Invalid("Must pass at least one record batch");
}
...

              Reporter: Jingyuan Wang / @alphalfalfa
              Assignee: Wes McKinney / @wesm
              PRs and other links:
              Note: This issue was originally created as ARROW-2119. Please see the migration documentation for further details.
              Metadata
              Metadata
              Assignees
              Labels
              Component: C++Component: JavaType: bug
              Type
              No type
              Projects
              No projects
                Milestone
                Relationships
                None yet
                Development
                No branches or pull requests
                Issue actions
                
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

                Skip to content
                [C++][Java] Handle Arrow stream with zero record batch #18090
                Closed
                Closed
                [C++][Java] Handle Arrow stream with zero record batch#18090
                Assignees
                wesm
                Labels
                Component: C++Component: JavaType: bug
                Milestone
                0.14.0
                Description
                @asfimport
                asfimport
                opened on Feb 8, 2018
                It looks like currently many places of the code assume that there needs to be at least one record batch for streaming format. Is zero-recordbatch not supported by design?
                e.g. https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/StreamToFile.java#L45
                
public static void convert(InputStream in, OutputStream out) throws IOException {
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
try (ArrowStreamReader reader = new ArrowStreamReader(in, allocator)) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
// load the first batch before instantiating the writer so that we have any dictionaries
if (!reader.loadNextBatch()) {
throw new IOException("Unable to read first record batch");
}
...

                Pyarrow-0.8.0 does not load 0-recordbatch stream either. It would throw an exception originated from https://github.com/apache/arrow/blob/a95465b8ce7a32feeaae3e13d0a64102ffa590d9/cpp/src/arrow/table.cc#L309:
                
Status Table::FromRecordBatches(const std::vector<std::shared_ptr<RecordBatch>>& batches,
std::shared_ptr<Table>* table) {
if (batches.size() == 0) {
return Status::Invalid("Must pass at least one record batch");
}
...

                Reporter: Jingyuan Wang / @alphalfalfa
                Assignee: Wes McKinney / @wesm
                PRs and other links:
                Note: This issue was originally created as ARROW-2119. Please see the migration documentation for further details.
                Metadata
                Metadata
                Assignees
                Labels
                Component: C++Component: JavaType: bug
                Type
                No type
                Projects
                No projects
                  Milestone
                  Relationships
                  None yet
                  Development
                  No branches or pull requests
                  Issue actions