ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type - #365

Closed
wesm wants to merge 3 commits into
apache:masterfrom
wesm:array-loader
Closed

ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type#365
wesm wants to merge 3 commits into
apache:masterfrom
wesm:array-loader

Conversation

@wesm

@wesmwesm commented Mar 8, 2017

Copy link
Copy Markdown
Member

These are various changes introduced to support the Feather merge in ARROW-452#361

@wesm
Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type
997b7a2
Change-Id: I75eb30bdc12723e29f0c8e2c7c92c296dfbe95a7
@wesmwesm mentioned this pull request Mar 9, 2017
Closed
tebeka
tebeka reviewed Mar 9, 2017
View reviewed changes
Comment threadcpp/src/arrow/type.h
explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
: FixedWidthType(Type::TIMESTAMP), unit(unit) {}

explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)

@tebekatebekaMar 9, 2017

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't timezone default to "UTC"? Also maybe it should be the 2nd parameter?

@wesmwesmMar 9, 2017

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't resolved any questions about storing time zones in the IPC metadata (https://github.com/apache/arrow/blob/master/format/Message.fbs), so I did the bare minimum here to support the pandas and R use cases, where they have the notion of tz-naive and tz-aware timestamps. So I don't want to invest any more energy in it until we have a broader discussion about time zones

@xhochyxhochyMar 9, 2017•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?

@wesmwesmMar 9, 2017

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, wasn't sure what else to use for a "null" value at the moment

@wesm

wesm commented Mar 9, 2017

Copy link
Copy Markdown
MemberAuthor

fixing compiler warnings

wesm added 2 commits March 9, 2017 08:05
@wesm
fix compiler warning
344e6b1
Change-Id: I6dc42fd7b21d910dd9a8444a02048e4f08ec86cf
@wesm
Revert Array::type_id to type_enum since Parquet uses this API
bc22872
Change-Id: I77db11a9c324d4b9055db7490fd3947e651a7d85
@wesm

wesm commented Mar 9, 2017

Copy link
Copy Markdown
MemberAuthor

green build: https://travis-ci.org/wesm/arrow/builds/209347029

xhochy
xhochy approved these changes Mar 9, 2017
View reviewed changes

@xhochyxhochy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except the two questions, this looks good. If they both can be answered with yes, feel free to merge.

Comment threadcpp/src/arrow/type.h
DATE,

// int32_t days since the UNIX epoch
DATE32,

@xhochyxhochyMar 9, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering of the types important yet, i.e. do we match the enums anywhere?

@wesmwesmMar 9, 2017

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, and I don't think we should guarantee ABI stability of the enum values yet anyhow (at some point we will want to)

Comment threadcpp/src/arrow/type.h
explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
: FixedWidthType(Type::TIMESTAMP), unit(unit) {}

explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)

@xhochyxhochyMar 9, 2017•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?

@wesm

wesm commented Mar 9, 2017

Copy link
Copy Markdown
MemberAuthor

thanks

@asfgitasfgit closed this in 6b3ae2aMar 9, 2017
@wesm
wesm deleted the array-loader branch March 9, 2017 21:59
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 8, 2018
@xhochy@wesm
PARQUET-1040: Add missing writer methods
5374737
Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Closesapache#365 from xhochy/PARQUET-1040 and squashes the following commits:
ef359ef [Korn, Uwe] PARQUET-1040: Add missing writer methods
Change-Id: I0d7a5b227b64e85c42106e37bb902cc3bbb15e85
@toddfarmertoddfarmer mentioned this pull request Apr 16, 2017
Closed
@asfimportasfimport mentioned this pull request Apr 16, 2017
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xhochyxhochyxhochy approved these changes
+1 more reviewer
@tebekatebekatebeka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@wesm@xhochy@tebeka
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type - #365
Closed
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Closed
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type#365
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Conversation
@wesm
@wesmwesm
commented
Mar 8, 2017

Copy link


Copy Markdown

Member
These are various changes introduced to support the Feather merge in ARROW-452#361
@wesm
Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type
997b7a2
Change-Id: I75eb30bdc12723e29f0c8e2c7c92c296dfbe95a7
@wesmwesm
mentioned this pull request
Mar 9, 2017
 Closed

tebeka
tebeka
reviewed
Mar 9, 2017
View reviewed changes
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@tebekatebekaMar 9, 2017

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't timezone default to "UTC"? Also maybe it should be the 2nd parameter?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't resolved any questions about storing time zones in the IPC metadata (https://github.com/apache/arrow/blob/master/format/Message.fbs), so I did the bare minimum here to support the pandas and R use cases, where they have the notion of tz-naive and tz-aware timestamps. So I don't want to invest any more energy in it until we have a broader discussion about time zones
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, wasn't sure what else to use for a "null" value at the moment
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
fixing compiler warnings
wesm
added 2 commits
March 9, 2017 08:05
@wesm
fix compiler warning
344e6b1
Change-Id: I6dc42fd7b21d910dd9a8444a02048e4f08ec86cf
@wesm
Revert Array::type_id to type_enum since Parquet uses this API
bc22872
Change-Id: I77db11a9c324d4b9055db7490fd3947e651a7d85
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
green build: https://travis-ci.org/wesm/arrow/builds/209347029
xhochy
xhochy
approved these changes
Mar 9, 2017
View reviewed changes
@xhochyxhochy
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except the two questions, this looks good. If they both can be answered with yes, feel free to merge.
Comment threadcpp/src/arrow/type.h
 DATE,

 // int32_t days since the UNIX epoch
 DATE32,
@xhochyxhochyMar 9, 2017

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering of the types important yet, i.e. do we match the enums anywhere?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, and I don't think we should guarantee ABI stability of the enum values yet anyhow (at some point we will want to)
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
thanks
@asfgitasfgit
closed this
in
6b3ae2aMar 9, 2017
@wesm
wesm
deleted the

array-loader

branch
March 9, 2017 21:59
wesm pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
@xhochy@wesm
PARQUET-1040: Add missing writer methods
5374737
Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Closesapache#365 from xhochy/PARQUET-1040 and squashes the following commits:
ef359ef [Korn, Uwe] PARQUET-1040: Add missing writer methods
Change-Id: I0d7a5b227b64e85c42106e37bb902cc3bbb15e85
@toddfarmertoddfarmer
mentioned this pull request
Apr 16, 2017

Closed

@asfimportasfimport
mentioned this pull request
Apr 16, 2017

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@xhochyxhochyxhochy approved these changes

+1 more reviewer

@tebekatebekatebeka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@wesm@xhochy@tebeka

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type - #365
Closed
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Closed
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type#365
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Conversation
@wesm
@wesmwesm
commented
Mar 8, 2017

Copy link


Copy Markdown

Member
These are various changes introduced to support the Feather merge in ARROW-452#361
@wesm
Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type
997b7a2
Change-Id: I75eb30bdc12723e29f0c8e2c7c92c296dfbe95a7
@wesmwesm
mentioned this pull request
Mar 9, 2017
 Closed

tebeka
tebeka
reviewed
Mar 9, 2017
View reviewed changes
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@tebekatebekaMar 9, 2017

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't timezone default to "UTC"? Also maybe it should be the 2nd parameter?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't resolved any questions about storing time zones in the IPC metadata (https://github.com/apache/arrow/blob/master/format/Message.fbs), so I did the bare minimum here to support the pandas and R use cases, where they have the notion of tz-naive and tz-aware timestamps. So I don't want to invest any more energy in it until we have a broader discussion about time zones
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, wasn't sure what else to use for a "null" value at the moment
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
fixing compiler warnings
wesm
added 2 commits
March 9, 2017 08:05
@wesm
fix compiler warning
344e6b1
Change-Id: I6dc42fd7b21d910dd9a8444a02048e4f08ec86cf
@wesm
Revert Array::type_id to type_enum since Parquet uses this API
bc22872
Change-Id: I77db11a9c324d4b9055db7490fd3947e651a7d85
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
green build: https://travis-ci.org/wesm/arrow/builds/209347029
xhochy
xhochy
approved these changes
Mar 9, 2017
View reviewed changes
@xhochyxhochy
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except the two questions, this looks good. If they both can be answered with yes, feel free to merge.
Comment threadcpp/src/arrow/type.h
 DATE,

 // int32_t days since the UNIX epoch
 DATE32,
@xhochyxhochyMar 9, 2017

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering of the types important yet, i.e. do we match the enums anywhere?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, and I don't think we should guarantee ABI stability of the enum values yet anyhow (at some point we will want to)
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
thanks
@asfgitasfgit
closed this
in
6b3ae2aMar 9, 2017
@wesm
wesm
deleted the

array-loader

branch
March 9, 2017 21:59
wesm pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
@xhochy@wesm
PARQUET-1040: Add missing writer methods
5374737
Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Closesapache#365 from xhochy/PARQUET-1040 and squashes the following commits:
ef359ef [Korn, Uwe] PARQUET-1040: Add missing writer methods
Change-Id: I0d7a5b227b64e85c42106e37bb902cc3bbb15e85
@toddfarmertoddfarmer
mentioned this pull request
Apr 16, 2017

Closed

@asfimportasfimport
mentioned this pull request
Apr 16, 2017

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@xhochyxhochyxhochy approved these changes

+1 more reviewer

@tebekatebekatebeka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@wesm@xhochy@tebeka

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type - #365
Closed
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Closed
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type#365
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Conversation
@wesm
@wesmwesm
commented
Mar 8, 2017

Copy link


Copy Markdown

Member
These are various changes introduced to support the Feather merge in ARROW-452#361
@wesm
Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type
997b7a2
Change-Id: I75eb30bdc12723e29f0c8e2c7c92c296dfbe95a7
@wesmwesm
mentioned this pull request
Mar 9, 2017
 Closed

tebeka
tebeka
reviewed
Mar 9, 2017
View reviewed changes
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@tebekatebekaMar 9, 2017

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't timezone default to "UTC"? Also maybe it should be the 2nd parameter?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't resolved any questions about storing time zones in the IPC metadata (https://github.com/apache/arrow/blob/master/format/Message.fbs), so I did the bare minimum here to support the pandas and R use cases, where they have the notion of tz-naive and tz-aware timestamps. So I don't want to invest any more energy in it until we have a broader discussion about time zones
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, wasn't sure what else to use for a "null" value at the moment
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
fixing compiler warnings
wesm
added 2 commits
March 9, 2017 08:05
@wesm
fix compiler warning
344e6b1
Change-Id: I6dc42fd7b21d910dd9a8444a02048e4f08ec86cf
@wesm
Revert Array::type_id to type_enum since Parquet uses this API
bc22872
Change-Id: I77db11a9c324d4b9055db7490fd3947e651a7d85
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
green build: https://travis-ci.org/wesm/arrow/builds/209347029
xhochy
xhochy
approved these changes
Mar 9, 2017
View reviewed changes
@xhochyxhochy
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except the two questions, this looks good. If they both can be answered with yes, feel free to merge.
Comment threadcpp/src/arrow/type.h
 DATE,

 // int32_t days since the UNIX epoch
 DATE32,
@xhochyxhochyMar 9, 2017

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering of the types important yet, i.e. do we match the enums anywhere?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, and I don't think we should guarantee ABI stability of the enum values yet anyhow (at some point we will want to)
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
thanks
@asfgitasfgit
closed this
in
6b3ae2aMar 9, 2017
@wesm
wesm
deleted the

array-loader

branch
March 9, 2017 21:59
wesm pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
@xhochy@wesm
PARQUET-1040: Add missing writer methods
5374737
Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Closesapache#365 from xhochy/PARQUET-1040 and squashes the following commits:
ef359ef [Korn, Uwe] PARQUET-1040: Add missing writer methods
Change-Id: I0d7a5b227b64e85c42106e37bb902cc3bbb15e85
@toddfarmertoddfarmer
mentioned this pull request
Apr 16, 2017

Closed

@asfimportasfimport
mentioned this pull request
Apr 16, 2017

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@xhochyxhochyxhochy approved these changes

+1 more reviewer

@tebekatebekatebeka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@wesm@xhochy@tebeka

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type - #365
Closed
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Closed
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type#365
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Conversation
@wesm
@wesmwesm
commented
Mar 8, 2017

Copy link


Copy Markdown

Member
These are various changes introduced to support the Feather merge in ARROW-452#361
@wesm
Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type
997b7a2
Change-Id: I75eb30bdc12723e29f0c8e2c7c92c296dfbe95a7
@wesmwesm
mentioned this pull request
Mar 9, 2017
 Closed

tebeka
tebeka
reviewed
Mar 9, 2017
View reviewed changes
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@tebekatebekaMar 9, 2017

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't timezone default to "UTC"? Also maybe it should be the 2nd parameter?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't resolved any questions about storing time zones in the IPC metadata (https://github.com/apache/arrow/blob/master/format/Message.fbs), so I did the bare minimum here to support the pandas and R use cases, where they have the notion of tz-naive and tz-aware timestamps. So I don't want to invest any more energy in it until we have a broader discussion about time zones
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, wasn't sure what else to use for a "null" value at the moment
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
fixing compiler warnings
wesm
added 2 commits
March 9, 2017 08:05
@wesm
fix compiler warning
344e6b1
Change-Id: I6dc42fd7b21d910dd9a8444a02048e4f08ec86cf
@wesm
Revert Array::type_id to type_enum since Parquet uses this API
bc22872
Change-Id: I77db11a9c324d4b9055db7490fd3947e651a7d85
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
green build: https://travis-ci.org/wesm/arrow/builds/209347029
xhochy
xhochy
approved these changes
Mar 9, 2017
View reviewed changes
@xhochyxhochy
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except the two questions, this looks good. If they both can be answered with yes, feel free to merge.
Comment threadcpp/src/arrow/type.h
 DATE,

 // int32_t days since the UNIX epoch
 DATE32,
@xhochyxhochyMar 9, 2017

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering of the types important yet, i.e. do we match the enums anywhere?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, and I don't think we should guarantee ABI stability of the enum values yet anyhow (at some point we will want to)
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
thanks
@asfgitasfgit
closed this
in
6b3ae2aMar 9, 2017
@wesm
wesm
deleted the

array-loader

branch
March 9, 2017 21:59
wesm pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
@xhochy@wesm
PARQUET-1040: Add missing writer methods
5374737
Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Closesapache#365 from xhochy/PARQUET-1040 and squashes the following commits:
ef359ef [Korn, Uwe] PARQUET-1040: Add missing writer methods
Change-Id: I0d7a5b227b64e85c42106e37bb902cc3bbb15e85
@toddfarmertoddfarmer
mentioned this pull request
Apr 16, 2017

Closed

@asfimportasfimport
mentioned this pull request
Apr 16, 2017

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@xhochyxhochyxhochy approved these changes

+1 more reviewer

@tebekatebekatebeka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@wesm@xhochy@tebeka

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type - #365
Closed
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Closed
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type#365
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Conversation
@wesm
@wesmwesm
commented
Mar 8, 2017

Copy link


Copy Markdown

Member
These are various changes introduced to support the Feather merge in ARROW-452#361
@wesm
Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type
997b7a2
Change-Id: I75eb30bdc12723e29f0c8e2c7c92c296dfbe95a7
@wesmwesm
mentioned this pull request
Mar 9, 2017
 Closed

tebeka
tebeka
reviewed
Mar 9, 2017
View reviewed changes
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@tebekatebekaMar 9, 2017

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't timezone default to "UTC"? Also maybe it should be the 2nd parameter?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't resolved any questions about storing time zones in the IPC metadata (https://github.com/apache/arrow/blob/master/format/Message.fbs), so I did the bare minimum here to support the pandas and R use cases, where they have the notion of tz-naive and tz-aware timestamps. So I don't want to invest any more energy in it until we have a broader discussion about time zones
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, wasn't sure what else to use for a "null" value at the moment
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
fixing compiler warnings
wesm
added 2 commits
March 9, 2017 08:05
@wesm
fix compiler warning
344e6b1
Change-Id: I6dc42fd7b21d910dd9a8444a02048e4f08ec86cf
@wesm
Revert Array::type_id to type_enum since Parquet uses this API
bc22872
Change-Id: I77db11a9c324d4b9055db7490fd3947e651a7d85
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
green build: https://travis-ci.org/wesm/arrow/builds/209347029
xhochy
xhochy
approved these changes
Mar 9, 2017
View reviewed changes
@xhochyxhochy
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except the two questions, this looks good. If they both can be answered with yes, feel free to merge.
Comment threadcpp/src/arrow/type.h
 DATE,

 // int32_t days since the UNIX epoch
 DATE32,
@xhochyxhochyMar 9, 2017

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering of the types important yet, i.e. do we match the enums anywhere?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, and I don't think we should guarantee ABI stability of the enum values yet anyhow (at some point we will want to)
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
thanks
@asfgitasfgit
closed this
in
6b3ae2aMar 9, 2017
@wesm
wesm
deleted the

array-loader

branch
March 9, 2017 21:59
wesm pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
@xhochy@wesm
PARQUET-1040: Add missing writer methods
5374737
Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Closesapache#365 from xhochy/PARQUET-1040 and squashes the following commits:
ef359ef [Korn, Uwe] PARQUET-1040: Add missing writer methods
Change-Id: I0d7a5b227b64e85c42106e37bb902cc3bbb15e85
@toddfarmertoddfarmer
mentioned this pull request
Apr 16, 2017

Closed

@asfimportasfimport
mentioned this pull request
Apr 16, 2017

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@xhochyxhochyxhochy approved these changes

+1 more reviewer

@tebekatebekatebeka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@wesm@xhochy@tebeka

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type - #365
Closed
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Closed
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type#365
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Conversation
@wesm
@wesmwesm
commented
Mar 8, 2017

Copy link


Copy Markdown

Member
These are various changes introduced to support the Feather merge in ARROW-452#361
@wesm
Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type
997b7a2
Change-Id: I75eb30bdc12723e29f0c8e2c7c92c296dfbe95a7
@wesmwesm
mentioned this pull request
Mar 9, 2017
 Closed

tebeka
tebeka
reviewed
Mar 9, 2017
View reviewed changes
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@tebekatebekaMar 9, 2017

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't timezone default to "UTC"? Also maybe it should be the 2nd parameter?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't resolved any questions about storing time zones in the IPC metadata (https://github.com/apache/arrow/blob/master/format/Message.fbs), so I did the bare minimum here to support the pandas and R use cases, where they have the notion of tz-naive and tz-aware timestamps. So I don't want to invest any more energy in it until we have a broader discussion about time zones
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, wasn't sure what else to use for a "null" value at the moment
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
fixing compiler warnings
wesm
added 2 commits
March 9, 2017 08:05
@wesm
fix compiler warning
344e6b1
Change-Id: I6dc42fd7b21d910dd9a8444a02048e4f08ec86cf
@wesm
Revert Array::type_id to type_enum since Parquet uses this API
bc22872
Change-Id: I77db11a9c324d4b9055db7490fd3947e651a7d85
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
green build: https://travis-ci.org/wesm/arrow/builds/209347029
xhochy
xhochy
approved these changes
Mar 9, 2017
View reviewed changes
@xhochyxhochy
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except the two questions, this looks good. If they both can be answered with yes, feel free to merge.
Comment threadcpp/src/arrow/type.h
 DATE,

 // int32_t days since the UNIX epoch
 DATE32,
@xhochyxhochyMar 9, 2017

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering of the types important yet, i.e. do we match the enums anywhere?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, and I don't think we should guarantee ABI stability of the enum values yet anyhow (at some point we will want to)
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
thanks
@asfgitasfgit
closed this
in
6b3ae2aMar 9, 2017
@wesm
wesm
deleted the

array-loader

branch
March 9, 2017 21:59
wesm pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
@xhochy@wesm
PARQUET-1040: Add missing writer methods
5374737
Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Closesapache#365 from xhochy/PARQUET-1040 and squashes the following commits:
ef359ef [Korn, Uwe] PARQUET-1040: Add missing writer methods
Change-Id: I0d7a5b227b64e85c42106e37bb902cc3bbb15e85
@toddfarmertoddfarmer
mentioned this pull request
Apr 16, 2017

Closed

@asfimportasfimport
mentioned this pull request
Apr 16, 2017

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@xhochyxhochyxhochy approved these changes

+1 more reviewer

@tebekatebekatebeka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@wesm@xhochy@tebeka

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type - #365
Closed
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Closed
ARROW-605: [C++] Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type#365
wesm wants to merge 3 commits into
apache:masterfrom 
wesm:array-loader
Conversation
@wesm
@wesmwesm
commented
Mar 8, 2017

Copy link


Copy Markdown

Member
These are various changes introduced to support the Feather merge in ARROW-452#361
@wesm
Refactor IPC adapter code into generic ArrayLoader class. Add Date32Type
997b7a2
Change-Id: I75eb30bdc12723e29f0c8e2c7c92c296dfbe95a7
@wesmwesm
mentioned this pull request
Mar 9, 2017
 Closed

tebeka
tebeka
reviewed
Mar 9, 2017
View reviewed changes
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@tebekatebekaMar 9, 2017

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't timezone default to "UTC"? Also maybe it should be the 2nd parameter?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't resolved any questions about storing time zones in the IPC metadata (https://github.com/apache/arrow/blob/master/format/Message.fbs), so I did the bare minimum here to support the pandas and R use cases, where they have the notion of tz-naive and tz-aware timestamps. So I don't want to invest any more energy in it until we have a broader discussion about time zones
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, wasn't sure what else to use for a "null" value at the moment
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
fixing compiler warnings
wesm
added 2 commits
March 9, 2017 08:05
@wesm
fix compiler warning
344e6b1
Change-Id: I6dc42fd7b21d910dd9a8444a02048e4f08ec86cf
@wesm
Revert Array::type_id to type_enum since Parquet uses this API
bc22872
Change-Id: I77db11a9c324d4b9055db7490fd3947e651a7d85
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
green build: https://travis-ci.org/wesm/arrow/builds/209347029
xhochy
xhochy
approved these changes
Mar 9, 2017
View reviewed changes
@xhochyxhochy
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except the two questions, this looks good. If they both can be answered with yes, feel free to merge.
Comment threadcpp/src/arrow/type.h
 DATE,

 // int32_t days since the UNIX epoch
 DATE32,
@xhochyxhochyMar 9, 2017

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering of the types important yet, i.e. do we match the enums anywhere?
@wesmwesmMar 9, 2017

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, and I don't think we should guarantee ABI stability of the enum values yet anyhow (at some point we will want to)
Comment threadcpp/src/arrow/type.h
 explicit TimestampType(TimeUnit unit = TimeUnit::MILLI)
 : FixedWidthType(Type::TIMESTAMP), unit(unit) {}

 explicit TimestampType(const std::string& timezone, TimeUnit unit = TimeUnit::MILLI)
@xhochyxhochyMar 9, 2017•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty string thus indicates a tz-naive timestamp?
@wesm
wesm
commented
Mar 9, 2017

Copy link


Copy Markdown

MemberAuthor
thanks
@asfgitasfgit
closed this
in
6b3ae2aMar 9, 2017
@wesm
wesm
deleted the

array-loader

branch
March 9, 2017 21:59
wesm pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
@xhochy@wesm
PARQUET-1040: Add missing writer methods
5374737
Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Closesapache#365 from xhochy/PARQUET-1040 and squashes the following commits:
ef359ef [Korn, Uwe] PARQUET-1040: Add missing writer methods
Change-Id: I0d7a5b227b64e85c42106e37bb902cc3bbb15e85
@toddfarmertoddfarmer
mentioned this pull request
Apr 16, 2017

Closed

@asfimportasfimport
mentioned this pull request
Apr 16, 2017

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@xhochyxhochyxhochy approved these changes

+1 more reviewer

@tebekatebekatebeka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@wesm@xhochy@tebeka