Skip to content

JSON encoding engine instrumentation - #3012

Closed
jonmmease wants to merge 18 commits into
orjson_encodingfrom
orjson_encoding_instrumentation
Closed

JSON encoding engine instrumentation#3012
jonmmease wants to merge 18 commits into
orjson_encodingfrom
orjson_encoding_instrumentation

Conversation

@jonmmease

@jonmmeasejonmmease commented Jan 8, 2021

Copy link
Copy Markdown
Contributor

This PR should not be merged!

This PR is instrumented for testing the new JSON encoding engine being introduced in #2955 (comment). Idea from @nicolaskruchten in #2955 (comment).

With this branch, every call to to_plotly_json performs JSON encoding with all three engines, then decodes the result and checks that identical dictionaries are produced. When they are not identical, and exception is raised.

Locally, nearly all of the documentation examples work. The two exceptions are imshow and ml-tsne-umap-projections. Both of these fail due to difference in how the orjson encoding handles floating-point numbers with precision less than 64 bits.

When encoding a float32 numpy array, the legacy and json encoders will convert this to a list of regular Python 64-bit floats before encoding them. This results extra digits in the JSON representation. In contrast, the orjson encoder will only write out the number of digits corresponding to the original 32-bit values.

@jonmmease
jonmmease marked this pull request as draft January 8, 2021 21:02
@jonmmeasejonmmease mentioned this pull request Jan 8, 2021
Merged
1 task
@nicolaskruchten

nicolaskruchten commented Jan 10, 2021

Copy link
Copy Markdown
Contributor

awesome! To get a bit more mileage out of our extant tests which don't call fig.show() maybe add a JSON call to the end of px._core.make_figure() or something? Maybe consider adding it to all the update_* methods so that we check a bunch of intermediate representations as well? Super-slow, I know, but doing it just a couple of times would do ;)

@jonmmease

jonmmease commented Jan 12, 2021

Copy link
Copy Markdown
ContributorAuthor

Yeah, those are good ideas. I'll add that sometime over the next couple of days

@jonmmease

jonmmease commented Jan 13, 2021

Copy link
Copy Markdown
ContributorAuthor

@nicolaskruchten, I added the extra instrumentation points in be81b16 and everything looks the same as far as I can tell. See what you think!

@nicolaskruchten

nicolaskruchten commented Jan 13, 2021

Copy link
Copy Markdown
Contributor

OK, are you just running these locally or...? CI is failing because orjson isn't installed so I assume your goal wasn't to get CI to go green here, right?

@jonmmease

jonmmease commented Jan 13, 2021

Copy link
Copy Markdown
ContributorAuthor

Yeah, I was running locally. And on CI I was mostly looking at the docs. I had the orjson stuff working on CI, but maybe that's in the other PR. I'll take a look.

@jonmmease

jonmmease commented Jan 13, 2021

Copy link
Copy Markdown
ContributorAuthor

Merged in #3022, and a couple CI fixes and now all existing tests are passing with the instrumented version.

@archmoj
archmoj deleted the orjson_encoding_instrumentation branch November 23, 2021 23:34
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jonmmease@nicolaskruchten
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
JSON encoding engine instrumentation by jonmmease · Pull Request #3012 · plotly/plotly.py · GitHub
Skip to content
JSON encoding engine instrumentation - #3012
Closed
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Closed
JSON encoding engine instrumentation#3012
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Conversation
@jonmmease
@jonmmeasejonmmease
commented
Jan 8, 2021

Copy link


Copy Markdown

Contributor
This PR should not be merged!
This PR is instrumented for testing the new JSON encoding engine being introduced in #2955 (comment). Idea from @nicolaskruchten in #2955 (comment).
With this branch, every call to to_plotly_json performs JSON encoding with all three engines, then decodes the result and checks that identical dictionaries are produced. When they are not identical, and exception is raised.
Locally, nearly all of the documentation examples work. The two exceptions are imshow and ml-tsne-umap-projections. Both of these fail due to difference in how the orjson encoding handles floating-point numbers with precision less than 64 bits.
When encoding a float32 numpy array, the legacy and json encoders will convert this to a list of regular Python 64-bit floats before encoding them. This results extra digits in the JSON representation. In contrast, the orjson encoder will only write out the number of digits corresponding to the original 32-bit values.
@jonmmease
jonmmease
marked this pull request as draft
January 8, 2021 21:02
@jonmmeasejonmmease
mentioned this pull request
Jan 8, 2021
 Merged

1 task
@nicolaskruchten
nicolaskruchten
commented
Jan 10, 2021

Copy link


Copy Markdown

Contributor
awesome! To get a bit more mileage out of our extant tests which don't call fig.show() maybe add a JSON call to the end of px._core.make_figure() or something? Maybe consider adding it to all the update_* methods so that we check a bunch of intermediate representations as well? Super-slow, I know, but doing it just a couple of times would do ;)
@jonmmease
jonmmease
commented
Jan 12, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, those are good ideas. I'll add that sometime over the next couple of days
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
@nicolaskruchten, I added the extra instrumentation points in be81b16 and everything looks the same as far as I can tell. See what you think!
@nicolaskruchten
nicolaskruchten
commented
Jan 13, 2021

Copy link


Copy Markdown

Contributor
OK, are you just running these locally or...? CI is failing because orjson isn't installed so I assume your goal wasn't to get CI to go green here, right?
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, I was running locally. And on CI I was mostly looking at the docs. I had the orjson stuff working on CI, but maybe that's in the other PR. I'll take a look.
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Merged in #3022, and a couple CI fixes and now all existing tests are passing with the instrumented version.
@archmoj
archmoj
deleted the

orjson_encoding_instrumentation

branch
November 23, 2021 23:34
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jonmmease@nicolaskruchten

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
JSON encoding engine instrumentation by jonmmease · Pull Request #3012 · plotly/plotly.py · GitHub
Skip to content
JSON encoding engine instrumentation - #3012
Closed
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Closed
JSON encoding engine instrumentation#3012
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Conversation
@jonmmease
@jonmmeasejonmmease
commented
Jan 8, 2021

Copy link


Copy Markdown

Contributor
This PR should not be merged!
This PR is instrumented for testing the new JSON encoding engine being introduced in #2955 (comment). Idea from @nicolaskruchten in #2955 (comment).
With this branch, every call to to_plotly_json performs JSON encoding with all three engines, then decodes the result and checks that identical dictionaries are produced. When they are not identical, and exception is raised.
Locally, nearly all of the documentation examples work. The two exceptions are imshow and ml-tsne-umap-projections. Both of these fail due to difference in how the orjson encoding handles floating-point numbers with precision less than 64 bits.
When encoding a float32 numpy array, the legacy and json encoders will convert this to a list of regular Python 64-bit floats before encoding them. This results extra digits in the JSON representation. In contrast, the orjson encoder will only write out the number of digits corresponding to the original 32-bit values.
@jonmmease
jonmmease
marked this pull request as draft
January 8, 2021 21:02
@jonmmeasejonmmease
mentioned this pull request
Jan 8, 2021
 Merged

1 task
@nicolaskruchten
nicolaskruchten
commented
Jan 10, 2021

Copy link


Copy Markdown

Contributor
awesome! To get a bit more mileage out of our extant tests which don't call fig.show() maybe add a JSON call to the end of px._core.make_figure() or something? Maybe consider adding it to all the update_* methods so that we check a bunch of intermediate representations as well? Super-slow, I know, but doing it just a couple of times would do ;)
@jonmmease
jonmmease
commented
Jan 12, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, those are good ideas. I'll add that sometime over the next couple of days
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
@nicolaskruchten, I added the extra instrumentation points in be81b16 and everything looks the same as far as I can tell. See what you think!
@nicolaskruchten
nicolaskruchten
commented
Jan 13, 2021

Copy link


Copy Markdown

Contributor
OK, are you just running these locally or...? CI is failing because orjson isn't installed so I assume your goal wasn't to get CI to go green here, right?
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, I was running locally. And on CI I was mostly looking at the docs. I had the orjson stuff working on CI, but maybe that's in the other PR. I'll take a look.
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Merged in #3022, and a couple CI fixes and now all existing tests are passing with the instrumented version.
@archmoj
archmoj
deleted the

orjson_encoding_instrumentation

branch
November 23, 2021 23:34
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jonmmease@nicolaskruchten

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
JSON encoding engine instrumentation by jonmmease · Pull Request #3012 · plotly/plotly.py · GitHub
Skip to content
JSON encoding engine instrumentation - #3012
Closed
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Closed
JSON encoding engine instrumentation#3012
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Conversation
@jonmmease
@jonmmeasejonmmease
commented
Jan 8, 2021

Copy link


Copy Markdown

Contributor
This PR should not be merged!
This PR is instrumented for testing the new JSON encoding engine being introduced in #2955 (comment). Idea from @nicolaskruchten in #2955 (comment).
With this branch, every call to to_plotly_json performs JSON encoding with all three engines, then decodes the result and checks that identical dictionaries are produced. When they are not identical, and exception is raised.
Locally, nearly all of the documentation examples work. The two exceptions are imshow and ml-tsne-umap-projections. Both of these fail due to difference in how the orjson encoding handles floating-point numbers with precision less than 64 bits.
When encoding a float32 numpy array, the legacy and json encoders will convert this to a list of regular Python 64-bit floats before encoding them. This results extra digits in the JSON representation. In contrast, the orjson encoder will only write out the number of digits corresponding to the original 32-bit values.
@jonmmease
jonmmease
marked this pull request as draft
January 8, 2021 21:02
@jonmmeasejonmmease
mentioned this pull request
Jan 8, 2021
 Merged

1 task
@nicolaskruchten
nicolaskruchten
commented
Jan 10, 2021

Copy link


Copy Markdown

Contributor
awesome! To get a bit more mileage out of our extant tests which don't call fig.show() maybe add a JSON call to the end of px._core.make_figure() or something? Maybe consider adding it to all the update_* methods so that we check a bunch of intermediate representations as well? Super-slow, I know, but doing it just a couple of times would do ;)
@jonmmease
jonmmease
commented
Jan 12, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, those are good ideas. I'll add that sometime over the next couple of days
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
@nicolaskruchten, I added the extra instrumentation points in be81b16 and everything looks the same as far as I can tell. See what you think!
@nicolaskruchten
nicolaskruchten
commented
Jan 13, 2021

Copy link


Copy Markdown

Contributor
OK, are you just running these locally or...? CI is failing because orjson isn't installed so I assume your goal wasn't to get CI to go green here, right?
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, I was running locally. And on CI I was mostly looking at the docs. I had the orjson stuff working on CI, but maybe that's in the other PR. I'll take a look.
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Merged in #3022, and a couple CI fixes and now all existing tests are passing with the instrumented version.
@archmoj
archmoj
deleted the

orjson_encoding_instrumentation

branch
November 23, 2021 23:34
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jonmmease@nicolaskruchten

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
JSON encoding engine instrumentation by jonmmease · Pull Request #3012 · plotly/plotly.py · GitHub
Skip to content
JSON encoding engine instrumentation - #3012
Closed
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Closed
JSON encoding engine instrumentation#3012
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Conversation
@jonmmease
@jonmmeasejonmmease
commented
Jan 8, 2021

Copy link


Copy Markdown

Contributor
This PR should not be merged!
This PR is instrumented for testing the new JSON encoding engine being introduced in #2955 (comment). Idea from @nicolaskruchten in #2955 (comment).
With this branch, every call to to_plotly_json performs JSON encoding with all three engines, then decodes the result and checks that identical dictionaries are produced. When they are not identical, and exception is raised.
Locally, nearly all of the documentation examples work. The two exceptions are imshow and ml-tsne-umap-projections. Both of these fail due to difference in how the orjson encoding handles floating-point numbers with precision less than 64 bits.
When encoding a float32 numpy array, the legacy and json encoders will convert this to a list of regular Python 64-bit floats before encoding them. This results extra digits in the JSON representation. In contrast, the orjson encoder will only write out the number of digits corresponding to the original 32-bit values.
@jonmmease
jonmmease
marked this pull request as draft
January 8, 2021 21:02
@jonmmeasejonmmease
mentioned this pull request
Jan 8, 2021
 Merged

1 task
@nicolaskruchten
nicolaskruchten
commented
Jan 10, 2021

Copy link


Copy Markdown

Contributor
awesome! To get a bit more mileage out of our extant tests which don't call fig.show() maybe add a JSON call to the end of px._core.make_figure() or something? Maybe consider adding it to all the update_* methods so that we check a bunch of intermediate representations as well? Super-slow, I know, but doing it just a couple of times would do ;)
@jonmmease
jonmmease
commented
Jan 12, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, those are good ideas. I'll add that sometime over the next couple of days
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
@nicolaskruchten, I added the extra instrumentation points in be81b16 and everything looks the same as far as I can tell. See what you think!
@nicolaskruchten
nicolaskruchten
commented
Jan 13, 2021

Copy link


Copy Markdown

Contributor
OK, are you just running these locally or...? CI is failing because orjson isn't installed so I assume your goal wasn't to get CI to go green here, right?
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, I was running locally. And on CI I was mostly looking at the docs. I had the orjson stuff working on CI, but maybe that's in the other PR. I'll take a look.
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Merged in #3022, and a couple CI fixes and now all existing tests are passing with the instrumented version.
@archmoj
archmoj
deleted the

orjson_encoding_instrumentation

branch
November 23, 2021 23:34
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jonmmease@nicolaskruchten

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
JSON encoding engine instrumentation by jonmmease · Pull Request #3012 · plotly/plotly.py · GitHub
Skip to content
JSON encoding engine instrumentation - #3012
Closed
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Closed
JSON encoding engine instrumentation#3012
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Conversation
@jonmmease
@jonmmeasejonmmease
commented
Jan 8, 2021

Copy link


Copy Markdown

Contributor
This PR should not be merged!
This PR is instrumented for testing the new JSON encoding engine being introduced in #2955 (comment). Idea from @nicolaskruchten in #2955 (comment).
With this branch, every call to to_plotly_json performs JSON encoding with all three engines, then decodes the result and checks that identical dictionaries are produced. When they are not identical, and exception is raised.
Locally, nearly all of the documentation examples work. The two exceptions are imshow and ml-tsne-umap-projections. Both of these fail due to difference in how the orjson encoding handles floating-point numbers with precision less than 64 bits.
When encoding a float32 numpy array, the legacy and json encoders will convert this to a list of regular Python 64-bit floats before encoding them. This results extra digits in the JSON representation. In contrast, the orjson encoder will only write out the number of digits corresponding to the original 32-bit values.
@jonmmease
jonmmease
marked this pull request as draft
January 8, 2021 21:02
@jonmmeasejonmmease
mentioned this pull request
Jan 8, 2021
 Merged

1 task
@nicolaskruchten
nicolaskruchten
commented
Jan 10, 2021

Copy link


Copy Markdown

Contributor
awesome! To get a bit more mileage out of our extant tests which don't call fig.show() maybe add a JSON call to the end of px._core.make_figure() or something? Maybe consider adding it to all the update_* methods so that we check a bunch of intermediate representations as well? Super-slow, I know, but doing it just a couple of times would do ;)
@jonmmease
jonmmease
commented
Jan 12, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, those are good ideas. I'll add that sometime over the next couple of days
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
@nicolaskruchten, I added the extra instrumentation points in be81b16 and everything looks the same as far as I can tell. See what you think!
@nicolaskruchten
nicolaskruchten
commented
Jan 13, 2021

Copy link


Copy Markdown

Contributor
OK, are you just running these locally or...? CI is failing because orjson isn't installed so I assume your goal wasn't to get CI to go green here, right?
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, I was running locally. And on CI I was mostly looking at the docs. I had the orjson stuff working on CI, but maybe that's in the other PR. I'll take a look.
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Merged in #3022, and a couple CI fixes and now all existing tests are passing with the instrumented version.
@archmoj
archmoj
deleted the

orjson_encoding_instrumentation

branch
November 23, 2021 23:34
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jonmmease@nicolaskruchten

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
JSON encoding engine instrumentation by jonmmease · Pull Request #3012 · plotly/plotly.py · GitHub
Skip to content
JSON encoding engine instrumentation - #3012
Closed
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Closed
JSON encoding engine instrumentation#3012
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Conversation
@jonmmease
@jonmmeasejonmmease
commented
Jan 8, 2021

Copy link


Copy Markdown

Contributor
This PR should not be merged!
This PR is instrumented for testing the new JSON encoding engine being introduced in #2955 (comment). Idea from @nicolaskruchten in #2955 (comment).
With this branch, every call to to_plotly_json performs JSON encoding with all three engines, then decodes the result and checks that identical dictionaries are produced. When they are not identical, and exception is raised.
Locally, nearly all of the documentation examples work. The two exceptions are imshow and ml-tsne-umap-projections. Both of these fail due to difference in how the orjson encoding handles floating-point numbers with precision less than 64 bits.
When encoding a float32 numpy array, the legacy and json encoders will convert this to a list of regular Python 64-bit floats before encoding them. This results extra digits in the JSON representation. In contrast, the orjson encoder will only write out the number of digits corresponding to the original 32-bit values.
@jonmmease
jonmmease
marked this pull request as draft
January 8, 2021 21:02
@jonmmeasejonmmease
mentioned this pull request
Jan 8, 2021
 Merged

1 task
@nicolaskruchten
nicolaskruchten
commented
Jan 10, 2021

Copy link


Copy Markdown

Contributor
awesome! To get a bit more mileage out of our extant tests which don't call fig.show() maybe add a JSON call to the end of px._core.make_figure() or something? Maybe consider adding it to all the update_* methods so that we check a bunch of intermediate representations as well? Super-slow, I know, but doing it just a couple of times would do ;)
@jonmmease
jonmmease
commented
Jan 12, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, those are good ideas. I'll add that sometime over the next couple of days
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
@nicolaskruchten, I added the extra instrumentation points in be81b16 and everything looks the same as far as I can tell. See what you think!
@nicolaskruchten
nicolaskruchten
commented
Jan 13, 2021

Copy link


Copy Markdown

Contributor
OK, are you just running these locally or...? CI is failing because orjson isn't installed so I assume your goal wasn't to get CI to go green here, right?
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, I was running locally. And on CI I was mostly looking at the docs. I had the orjson stuff working on CI, but maybe that's in the other PR. I'll take a look.
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Merged in #3022, and a couple CI fixes and now all existing tests are passing with the instrumented version.
@archmoj
archmoj
deleted the

orjson_encoding_instrumentation

branch
November 23, 2021 23:34
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jonmmease@nicolaskruchten

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
JSON encoding engine instrumentation by jonmmease · Pull Request #3012 · plotly/plotly.py · GitHub
Skip to content
JSON encoding engine instrumentation - #3012
Closed
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Closed
JSON encoding engine instrumentation#3012
jonmmease wants to merge 18 commits into
orjson_encodingfrom 
orjson_encoding_instrumentation
Conversation
@jonmmease
@jonmmeasejonmmease
commented
Jan 8, 2021

Copy link


Copy Markdown

Contributor
This PR should not be merged!
This PR is instrumented for testing the new JSON encoding engine being introduced in #2955 (comment). Idea from @nicolaskruchten in #2955 (comment).
With this branch, every call to to_plotly_json performs JSON encoding with all three engines, then decodes the result and checks that identical dictionaries are produced. When they are not identical, and exception is raised.
Locally, nearly all of the documentation examples work. The two exceptions are imshow and ml-tsne-umap-projections. Both of these fail due to difference in how the orjson encoding handles floating-point numbers with precision less than 64 bits.
When encoding a float32 numpy array, the legacy and json encoders will convert this to a list of regular Python 64-bit floats before encoding them. This results extra digits in the JSON representation. In contrast, the orjson encoder will only write out the number of digits corresponding to the original 32-bit values.
@jonmmease
jonmmease
marked this pull request as draft
January 8, 2021 21:02
@jonmmeasejonmmease
mentioned this pull request
Jan 8, 2021
 Merged

1 task
@nicolaskruchten
nicolaskruchten
commented
Jan 10, 2021

Copy link


Copy Markdown

Contributor
awesome! To get a bit more mileage out of our extant tests which don't call fig.show() maybe add a JSON call to the end of px._core.make_figure() or something? Maybe consider adding it to all the update_* methods so that we check a bunch of intermediate representations as well? Super-slow, I know, but doing it just a couple of times would do ;)
@jonmmease
jonmmease
commented
Jan 12, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, those are good ideas. I'll add that sometime over the next couple of days
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
@nicolaskruchten, I added the extra instrumentation points in be81b16 and everything looks the same as far as I can tell. See what you think!
@nicolaskruchten
nicolaskruchten
commented
Jan 13, 2021

Copy link


Copy Markdown

Contributor
OK, are you just running these locally or...? CI is failing because orjson isn't installed so I assume your goal wasn't to get CI to go green here, right?
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Yeah, I was running locally. And on CI I was mostly looking at the docs. I had the orjson stuff working on CI, but maybe that's in the other PR. I'll take a look.
@jonmmease
jonmmease
commented
Jan 13, 2021

Copy link


Copy Markdown

ContributorAuthor
Merged in #3022, and a couple CI fixes and now all existing tests are passing with the instrumented version.
@archmoj
archmoj
deleted the

orjson_encoding_instrumentation

branch
November 23, 2021 23:34
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@jonmmease@nicolaskruchten