Skip to content

GH-39257: [JS] LargeBinary - #39258

Merged
domoritz merged 4 commits into
apache:mainfrom
domoritz:dom/largeBinary-2
Dec 18, 2023
Merged

GH-39257: [JS] LargeBinary#39258
domoritz merged 4 commits into
apache:mainfrom
domoritz:dom/largeBinary-2

Conversation

@domoritz

@domoritzdomoritz commented Dec 17, 2023

Copy link
Copy Markdown
Member

Comment on lines +39 to +54
protected _flushPending(pending: Map<number, Uint8Array | undefined>, pendingLength: number) {
const offsets = this._offsets;
const data = this._values.reserve(pendingLength).buffer;
let offset = 0;
for (const [index, value] of pending) {
if (value === undefined) {
offsets.set(index, BigInt(0));
} else {
const length = value.length;
data.set(value, offset);
offsets.set(index, BigInt(length));
offset += length;
}
}
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to activate the placeholder in builder/largeutf8.ts in this PR?

Seems like you could probably do the same for getByteLength, too.

(LargeUtf8Builder.prototype as any)._flushPending = (LargeBinaryBuilder.prototype as any)._flushPending;

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, just updated the pull request.

@bmschmidtbmschmidt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, fwiw, except for the duplicated code between the utf8 and binary builders. The new unification stuff is nice.

@domoritz

Copy link
Copy Markdown
MemberAuthor

Oh good catch. I forgot to remove that duplicate code.

@domoritz

Copy link
Copy Markdown
MemberAuthor

CC @kylebarron

@kylebarronkylebarron left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@github-actionsgithub-actionsBot added awaiting changes Awaiting changes awaiting change review Awaiting change review and removed awaiting committer review Awaiting committer review awaiting changes Awaiting changes labels Dec 18, 2023
@domoritz
domoritz merged commit 4ec6544 into apache:mainDec 18, 2023
@domoritz
domoritz deleted the dom/largeBinary-2 branch December 18, 2023 03:27
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 4ec6544.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 7 possible false positives for unstable benchmarks that are known to sometimes produce them.

dgreiss pushed a commit to dgreiss/arrow that referenced this pull request Feb 19, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[JS] Support LargeBinary type

3 participants

@domoritz@bmschmidt@kylebarron