Skip to content

polyval: Add runtime PCLMULQDQ detection - #11

Closed
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection
Closed

polyval: Add runtime PCLMULQDQ detection#11
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection

Conversation

@tarcieri

Copy link
Copy Markdown
Member

When the std feature is enabled (which it is now by default), this adds runtime detection for PCLMULQDQ support on x86/x86_64 architectures.

The detection happens once at the time Polyval is instantiated. The polyval::field::Element type has been changed into an enum which remembers the detection result, and its API changed to operate on bytestring representations of POLYVAL field elements.

This appears to have a negligible performance impact.

Comment threadpolyval/src/field.rs
target_feature = "sse2",
target_feature = "sse4.1",
any(target_arch = "x86", target_arch = "x86_64")
))]

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional gating this implementation is using is gross. Open to alternatives... perhaps the cfg-if crate?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried cfg-if. Unfortunately it doesn't seem to support nesting, and the resulting code ended up looking just as bad

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made things a little better by using cfg!. Still a lot of annoying, redundant gating though

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My kingdom for cfg_alias: rust-lang/cargo#7260 (comment)

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 44ce8e5 to 231a23bCompareSeptember 18, 2019 20:13
@WildCryptoFox

Copy link
Copy Markdown

Everything looks good to me.

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch 4 times, most recently from 65f6879 to 86e59ebCompareSeptember 19, 2019 02:48
@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov WDYT? I'm calling this "done" but I'd be interested to know any ideas for potential simplifications/cleanups

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 86e59eb to 87cc950CompareSeptember 19, 2019 03:40
When the `std` feature is enabled (which it is now by default), this
adds runtime detection for PCLMULQDQ support on x86/x86_64
architectures.
The detection happens once at the time `Polyval` is instantated. The
`polyval::field::Element` type has been changed into an enum which
remembers the detection result, and its API changed to operate on
bytestring representations of POLYVAL field elements.
This appears to have a negligable performance impact.
@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 87cc950 to f0db4e4CompareSeptember 19, 2019 03:43
@WildCryptoFoxWildCryptoFox mentioned this pull request Sep 19, 2019
4 tasks
Comment threadpolyval/src/field.rs
pub fn from_bytes(bytes: Block) -> Self {
Element(bytes.into())
if cfg!(feature = "std") {
if is_x86_feature_detected!("pclmulqdq") {

@newpavlovnewpavlovSep 19, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite like that it does not work on no_std targets. I think we better to use RDRAND directly and cache results in atomic.

@newpavlov

Copy link
Copy Markdown
Member

IIUC this implementation branches between Soft and Clmul more than once per single call to Ghash, while ideally it should branch only ones.

Also I think you are using cfg a bit incorrectly. If CLMUL and other necessary target features are enabled, we should remove the software fallback completely.

One approach which we can take looks roughly like this:

traitElement{ .. }implElementforClmul{ .. }implElementforSoft{ .. }structGhashImpl<E:Element>{ .. }enumGhashPrivate{#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]Clmul(GhashImpl<Clmul>),// disable soft fallback if CLMUL is enabled by user#[cfg(not(all( any(target_arch = "x86", target_arch = "x86_64"), all(target_feature = "pclmulqdq", ..))))]Soft(GhashImpl<Soft>),}structGhash(GhashPrivate);

But I am not sure how good inlining will work with #[target_feature(enable = "sse4")] added to trait impl. The main idea is what we want to switch to a full implementation for given conditions as high as possible and do it only once per Ghash call.

@tarcieri

tarcieri commented Sep 19, 2019

Copy link
Copy Markdown
MemberAuthor

@newpavlov upon further reflection, I agree runtime detection isn't helpful. It would be extremely helpful for, say, AES-NI, where it isn't available inside KVM and therefore relocatable binaries produced for architectures that support it fail in these environments, but from what I can tell CLMUL isn't similarly impacted.

That said, your suggested approach is pretty much what the existing implementation already does.

I think I can eliminate the need for an enum and a trait entirely though via a simple newtype which only ensures the relevant APIs are equivalent though, which I'll take a stab at in a separate PR.

It seems we can also fake cfg aliases in build scripts per rust-lang/cargo#7260 (comment), so I might take a look at that as a way to simplify the target gating.

Closing this PR out.

@tarcieri
tarcieri deleted the polyval/runtime-detection branch September 19, 2019 14:37
@newpavlov

Copy link
Copy Markdown
Member

I wouldn't say it isn't helpful. But if we want to squeeze a maximum performance, I think we should push switch between target dependent implementations as high as possible (i.e. we should not do runtime detections for aes, ctr, ghash, but do a single detection and switch in a high-level construct as AES-GCM-SIV), and unfortunately right now Rust is quite poorly equipped for doing that, especially in mult-crate contexts.

@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov I'm down to actually implement AES-GCM-SIV first, then circle back on this

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tarcieri@WildCryptoFox@newpavlov
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
polyval: Add runtime PCLMULQDQ detection by tarcieri · Pull Request #11 · RustCrypto/universal-hashes · GitHub
Skip to content

polyval: Add runtime PCLMULQDQ detection - #11

Closed
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection
Closed

polyval: Add runtime PCLMULQDQ detection#11
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection

Conversation

@tarcieri

Copy link
Copy Markdown
Member

When the std feature is enabled (which it is now by default), this adds runtime detection for PCLMULQDQ support on x86/x86_64 architectures.

The detection happens once at the time Polyval is instantiated. The polyval::field::Element type has been changed into an enum which remembers the detection result, and its API changed to operate on bytestring representations of POLYVAL field elements.

This appears to have a negligible performance impact.

Comment threadpolyval/src/field.rs
target_feature = "sse2",
target_feature = "sse4.1",
any(target_arch = "x86", target_arch = "x86_64")
))]

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional gating this implementation is using is gross. Open to alternatives... perhaps the cfg-if crate?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried cfg-if. Unfortunately it doesn't seem to support nesting, and the resulting code ended up looking just as bad

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made things a little better by using cfg!. Still a lot of annoying, redundant gating though

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My kingdom for cfg_alias: rust-lang/cargo#7260 (comment)

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 44ce8e5 to 231a23bCompareSeptember 18, 2019 20:13
@WildCryptoFox

Copy link
Copy Markdown

Everything looks good to me.

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch 4 times, most recently from 65f6879 to 86e59ebCompareSeptember 19, 2019 02:48
@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov WDYT? I'm calling this "done" but I'd be interested to know any ideas for potential simplifications/cleanups

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 86e59eb to 87cc950CompareSeptember 19, 2019 03:40
When the `std` feature is enabled (which it is now by default), this
adds runtime detection for PCLMULQDQ support on x86/x86_64
architectures.
The detection happens once at the time `Polyval` is instantated. The
`polyval::field::Element` type has been changed into an enum which
remembers the detection result, and its API changed to operate on
bytestring representations of POLYVAL field elements.
This appears to have a negligable performance impact.
@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 87cc950 to f0db4e4CompareSeptember 19, 2019 03:43
@WildCryptoFoxWildCryptoFox mentioned this pull request Sep 19, 2019
4 tasks
Comment threadpolyval/src/field.rs
pub fn from_bytes(bytes: Block) -> Self {
Element(bytes.into())
if cfg!(feature = "std") {
if is_x86_feature_detected!("pclmulqdq") {

@newpavlovnewpavlovSep 19, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite like that it does not work on no_std targets. I think we better to use RDRAND directly and cache results in atomic.

@newpavlov

Copy link
Copy Markdown
Member

IIUC this implementation branches between Soft and Clmul more than once per single call to Ghash, while ideally it should branch only ones.

Also I think you are using cfg a bit incorrectly. If CLMUL and other necessary target features are enabled, we should remove the software fallback completely.

One approach which we can take looks roughly like this:

traitElement{ .. }implElementforClmul{ .. }implElementforSoft{ .. }structGhashImpl<E:Element>{ .. }enumGhashPrivate{#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]Clmul(GhashImpl<Clmul>),// disable soft fallback if CLMUL is enabled by user#[cfg(not(all( any(target_arch = "x86", target_arch = "x86_64"), all(target_feature = "pclmulqdq", ..))))]Soft(GhashImpl<Soft>),}structGhash(GhashPrivate);

But I am not sure how good inlining will work with #[target_feature(enable = "sse4")] added to trait impl. The main idea is what we want to switch to a full implementation for given conditions as high as possible and do it only once per Ghash call.

@tarcieri

tarcieri commented Sep 19, 2019

Copy link
Copy Markdown
MemberAuthor

@newpavlov upon further reflection, I agree runtime detection isn't helpful. It would be extremely helpful for, say, AES-NI, where it isn't available inside KVM and therefore relocatable binaries produced for architectures that support it fail in these environments, but from what I can tell CLMUL isn't similarly impacted.

That said, your suggested approach is pretty much what the existing implementation already does.

I think I can eliminate the need for an enum and a trait entirely though via a simple newtype which only ensures the relevant APIs are equivalent though, which I'll take a stab at in a separate PR.

It seems we can also fake cfg aliases in build scripts per rust-lang/cargo#7260 (comment), so I might take a look at that as a way to simplify the target gating.

Closing this PR out.

@tarcieri
tarcieri deleted the polyval/runtime-detection branch September 19, 2019 14:37
@newpavlov

Copy link
Copy Markdown
Member

I wouldn't say it isn't helpful. But if we want to squeeze a maximum performance, I think we should push switch between target dependent implementations as high as possible (i.e. we should not do runtime detections for aes, ctr, ghash, but do a single detection and switch in a high-level construct as AES-GCM-SIV), and unfortunately right now Rust is quite poorly equipped for doing that, especially in mult-crate contexts.

@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov I'm down to actually implement AES-GCM-SIV first, then circle back on this

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tarcieri@WildCryptoFox@newpavlov
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' polyval: Add runtime PCLMULQDQ detection by tarcieri · Pull Request #11 · RustCrypto/universal-hashes · GitHub
Skip to content

polyval: Add runtime PCLMULQDQ detection - #11

Closed
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection
Closed

polyval: Add runtime PCLMULQDQ detection#11
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection

Conversation

@tarcieri

Copy link
Copy Markdown
Member

When the std feature is enabled (which it is now by default), this adds runtime detection for PCLMULQDQ support on x86/x86_64 architectures.

The detection happens once at the time Polyval is instantiated. The polyval::field::Element type has been changed into an enum which remembers the detection result, and its API changed to operate on bytestring representations of POLYVAL field elements.

This appears to have a negligible performance impact.

Comment threadpolyval/src/field.rs
target_feature = "sse2",
target_feature = "sse4.1",
any(target_arch = "x86", target_arch = "x86_64")
))]

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional gating this implementation is using is gross. Open to alternatives... perhaps the cfg-if crate?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried cfg-if. Unfortunately it doesn't seem to support nesting, and the resulting code ended up looking just as bad

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made things a little better by using cfg!. Still a lot of annoying, redundant gating though

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My kingdom for cfg_alias: rust-lang/cargo#7260 (comment)

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 44ce8e5 to 231a23bCompareSeptember 18, 2019 20:13
@WildCryptoFox

Copy link
Copy Markdown

Everything looks good to me.

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch 4 times, most recently from 65f6879 to 86e59ebCompareSeptember 19, 2019 02:48
@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov WDYT? I'm calling this "done" but I'd be interested to know any ideas for potential simplifications/cleanups

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 86e59eb to 87cc950CompareSeptember 19, 2019 03:40
When the `std` feature is enabled (which it is now by default), this
adds runtime detection for PCLMULQDQ support on x86/x86_64
architectures.
The detection happens once at the time `Polyval` is instantated. The
`polyval::field::Element` type has been changed into an enum which
remembers the detection result, and its API changed to operate on
bytestring representations of POLYVAL field elements.
This appears to have a negligable performance impact.
@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 87cc950 to f0db4e4CompareSeptember 19, 2019 03:43
@WildCryptoFoxWildCryptoFox mentioned this pull request Sep 19, 2019
4 tasks
Comment threadpolyval/src/field.rs
pub fn from_bytes(bytes: Block) -> Self {
Element(bytes.into())
if cfg!(feature = "std") {
if is_x86_feature_detected!("pclmulqdq") {

@newpavlovnewpavlovSep 19, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite like that it does not work on no_std targets. I think we better to use RDRAND directly and cache results in atomic.

@newpavlov

Copy link
Copy Markdown
Member

IIUC this implementation branches between Soft and Clmul more than once per single call to Ghash, while ideally it should branch only ones.

Also I think you are using cfg a bit incorrectly. If CLMUL and other necessary target features are enabled, we should remove the software fallback completely.

One approach which we can take looks roughly like this:

traitElement{ .. }implElementforClmul{ .. }implElementforSoft{ .. }structGhashImpl<E:Element>{ .. }enumGhashPrivate{#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]Clmul(GhashImpl<Clmul>),// disable soft fallback if CLMUL is enabled by user#[cfg(not(all( any(target_arch = "x86", target_arch = "x86_64"), all(target_feature = "pclmulqdq", ..))))]Soft(GhashImpl<Soft>),}structGhash(GhashPrivate);

But I am not sure how good inlining will work with #[target_feature(enable = "sse4")] added to trait impl. The main idea is what we want to switch to a full implementation for given conditions as high as possible and do it only once per Ghash call.

@tarcieri

tarcieri commented Sep 19, 2019

Copy link
Copy Markdown
MemberAuthor

@newpavlov upon further reflection, I agree runtime detection isn't helpful. It would be extremely helpful for, say, AES-NI, where it isn't available inside KVM and therefore relocatable binaries produced for architectures that support it fail in these environments, but from what I can tell CLMUL isn't similarly impacted.

That said, your suggested approach is pretty much what the existing implementation already does.

I think I can eliminate the need for an enum and a trait entirely though via a simple newtype which only ensures the relevant APIs are equivalent though, which I'll take a stab at in a separate PR.

It seems we can also fake cfg aliases in build scripts per rust-lang/cargo#7260 (comment), so I might take a look at that as a way to simplify the target gating.

Closing this PR out.

@tarcieri
tarcieri deleted the polyval/runtime-detection branch September 19, 2019 14:37
@newpavlov

Copy link
Copy Markdown
Member

I wouldn't say it isn't helpful. But if we want to squeeze a maximum performance, I think we should push switch between target dependent implementations as high as possible (i.e. we should not do runtime detections for aes, ctr, ghash, but do a single detection and switch in a high-level construct as AES-GCM-SIV), and unfortunately right now Rust is quite poorly equipped for doing that, especially in mult-crate contexts.

@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov I'm down to actually implement AES-GCM-SIV first, then circle back on this

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tarcieri@WildCryptoFox@newpavlov
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' polyval: Add runtime PCLMULQDQ detection by tarcieri · Pull Request #11 · RustCrypto/universal-hashes · GitHub
Skip to content

polyval: Add runtime PCLMULQDQ detection - #11

Closed
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection
Closed

polyval: Add runtime PCLMULQDQ detection#11
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection

Conversation

@tarcieri

Copy link
Copy Markdown
Member

When the std feature is enabled (which it is now by default), this adds runtime detection for PCLMULQDQ support on x86/x86_64 architectures.

The detection happens once at the time Polyval is instantiated. The polyval::field::Element type has been changed into an enum which remembers the detection result, and its API changed to operate on bytestring representations of POLYVAL field elements.

This appears to have a negligible performance impact.

Comment threadpolyval/src/field.rs
target_feature = "sse2",
target_feature = "sse4.1",
any(target_arch = "x86", target_arch = "x86_64")
))]

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional gating this implementation is using is gross. Open to alternatives... perhaps the cfg-if crate?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried cfg-if. Unfortunately it doesn't seem to support nesting, and the resulting code ended up looking just as bad

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made things a little better by using cfg!. Still a lot of annoying, redundant gating though

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My kingdom for cfg_alias: rust-lang/cargo#7260 (comment)

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 44ce8e5 to 231a23bCompareSeptember 18, 2019 20:13
@WildCryptoFox

Copy link
Copy Markdown

Everything looks good to me.

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch 4 times, most recently from 65f6879 to 86e59ebCompareSeptember 19, 2019 02:48
@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov WDYT? I'm calling this "done" but I'd be interested to know any ideas for potential simplifications/cleanups

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 86e59eb to 87cc950CompareSeptember 19, 2019 03:40
When the `std` feature is enabled (which it is now by default), this
adds runtime detection for PCLMULQDQ support on x86/x86_64
architectures.
The detection happens once at the time `Polyval` is instantated. The
`polyval::field::Element` type has been changed into an enum which
remembers the detection result, and its API changed to operate on
bytestring representations of POLYVAL field elements.
This appears to have a negligable performance impact.
@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 87cc950 to f0db4e4CompareSeptember 19, 2019 03:43
@WildCryptoFoxWildCryptoFox mentioned this pull request Sep 19, 2019
4 tasks
Comment threadpolyval/src/field.rs
pub fn from_bytes(bytes: Block) -> Self {
Element(bytes.into())
if cfg!(feature = "std") {
if is_x86_feature_detected!("pclmulqdq") {

@newpavlovnewpavlovSep 19, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite like that it does not work on no_std targets. I think we better to use RDRAND directly and cache results in atomic.

@newpavlov

Copy link
Copy Markdown
Member

IIUC this implementation branches between Soft and Clmul more than once per single call to Ghash, while ideally it should branch only ones.

Also I think you are using cfg a bit incorrectly. If CLMUL and other necessary target features are enabled, we should remove the software fallback completely.

One approach which we can take looks roughly like this:

traitElement{ .. }implElementforClmul{ .. }implElementforSoft{ .. }structGhashImpl<E:Element>{ .. }enumGhashPrivate{#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]Clmul(GhashImpl<Clmul>),// disable soft fallback if CLMUL is enabled by user#[cfg(not(all( any(target_arch = "x86", target_arch = "x86_64"), all(target_feature = "pclmulqdq", ..))))]Soft(GhashImpl<Soft>),}structGhash(GhashPrivate);

But I am not sure how good inlining will work with #[target_feature(enable = "sse4")] added to trait impl. The main idea is what we want to switch to a full implementation for given conditions as high as possible and do it only once per Ghash call.

@tarcieri

tarcieri commented Sep 19, 2019

Copy link
Copy Markdown
MemberAuthor

@newpavlov upon further reflection, I agree runtime detection isn't helpful. It would be extremely helpful for, say, AES-NI, where it isn't available inside KVM and therefore relocatable binaries produced for architectures that support it fail in these environments, but from what I can tell CLMUL isn't similarly impacted.

That said, your suggested approach is pretty much what the existing implementation already does.

I think I can eliminate the need for an enum and a trait entirely though via a simple newtype which only ensures the relevant APIs are equivalent though, which I'll take a stab at in a separate PR.

It seems we can also fake cfg aliases in build scripts per rust-lang/cargo#7260 (comment), so I might take a look at that as a way to simplify the target gating.

Closing this PR out.

@tarcieri
tarcieri deleted the polyval/runtime-detection branch September 19, 2019 14:37
@newpavlov

Copy link
Copy Markdown
Member

I wouldn't say it isn't helpful. But if we want to squeeze a maximum performance, I think we should push switch between target dependent implementations as high as possible (i.e. we should not do runtime detections for aes, ctr, ghash, but do a single detection and switch in a high-level construct as AES-GCM-SIV), and unfortunately right now Rust is quite poorly equipped for doing that, especially in mult-crate contexts.

@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov I'm down to actually implement AES-GCM-SIV first, then circle back on this

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tarcieri@WildCryptoFox@newpavlov
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' polyval: Add runtime PCLMULQDQ detection by tarcieri · Pull Request #11 · RustCrypto/universal-hashes · GitHub
Skip to content

polyval: Add runtime PCLMULQDQ detection - #11

Closed
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection
Closed

polyval: Add runtime PCLMULQDQ detection#11
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection

Conversation

@tarcieri

Copy link
Copy Markdown
Member

When the std feature is enabled (which it is now by default), this adds runtime detection for PCLMULQDQ support on x86/x86_64 architectures.

The detection happens once at the time Polyval is instantiated. The polyval::field::Element type has been changed into an enum which remembers the detection result, and its API changed to operate on bytestring representations of POLYVAL field elements.

This appears to have a negligible performance impact.

Comment threadpolyval/src/field.rs
target_feature = "sse2",
target_feature = "sse4.1",
any(target_arch = "x86", target_arch = "x86_64")
))]

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional gating this implementation is using is gross. Open to alternatives... perhaps the cfg-if crate?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried cfg-if. Unfortunately it doesn't seem to support nesting, and the resulting code ended up looking just as bad

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made things a little better by using cfg!. Still a lot of annoying, redundant gating though

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My kingdom for cfg_alias: rust-lang/cargo#7260 (comment)

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 44ce8e5 to 231a23bCompareSeptember 18, 2019 20:13
@WildCryptoFox

Copy link
Copy Markdown

Everything looks good to me.

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch 4 times, most recently from 65f6879 to 86e59ebCompareSeptember 19, 2019 02:48
@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov WDYT? I'm calling this "done" but I'd be interested to know any ideas for potential simplifications/cleanups

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 86e59eb to 87cc950CompareSeptember 19, 2019 03:40
When the `std` feature is enabled (which it is now by default), this
adds runtime detection for PCLMULQDQ support on x86/x86_64
architectures.
The detection happens once at the time `Polyval` is instantated. The
`polyval::field::Element` type has been changed into an enum which
remembers the detection result, and its API changed to operate on
bytestring representations of POLYVAL field elements.
This appears to have a negligable performance impact.
@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 87cc950 to f0db4e4CompareSeptember 19, 2019 03:43
@WildCryptoFoxWildCryptoFox mentioned this pull request Sep 19, 2019
4 tasks
Comment threadpolyval/src/field.rs
pub fn from_bytes(bytes: Block) -> Self {
Element(bytes.into())
if cfg!(feature = "std") {
if is_x86_feature_detected!("pclmulqdq") {

@newpavlovnewpavlovSep 19, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite like that it does not work on no_std targets. I think we better to use RDRAND directly and cache results in atomic.

@newpavlov

Copy link
Copy Markdown
Member

IIUC this implementation branches between Soft and Clmul more than once per single call to Ghash, while ideally it should branch only ones.

Also I think you are using cfg a bit incorrectly. If CLMUL and other necessary target features are enabled, we should remove the software fallback completely.

One approach which we can take looks roughly like this:

traitElement{ .. }implElementforClmul{ .. }implElementforSoft{ .. }structGhashImpl<E:Element>{ .. }enumGhashPrivate{#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]Clmul(GhashImpl<Clmul>),// disable soft fallback if CLMUL is enabled by user#[cfg(not(all( any(target_arch = "x86", target_arch = "x86_64"), all(target_feature = "pclmulqdq", ..))))]Soft(GhashImpl<Soft>),}structGhash(GhashPrivate);

But I am not sure how good inlining will work with #[target_feature(enable = "sse4")] added to trait impl. The main idea is what we want to switch to a full implementation for given conditions as high as possible and do it only once per Ghash call.

@tarcieri

tarcieri commented Sep 19, 2019

Copy link
Copy Markdown
MemberAuthor

@newpavlov upon further reflection, I agree runtime detection isn't helpful. It would be extremely helpful for, say, AES-NI, where it isn't available inside KVM and therefore relocatable binaries produced for architectures that support it fail in these environments, but from what I can tell CLMUL isn't similarly impacted.

That said, your suggested approach is pretty much what the existing implementation already does.

I think I can eliminate the need for an enum and a trait entirely though via a simple newtype which only ensures the relevant APIs are equivalent though, which I'll take a stab at in a separate PR.

It seems we can also fake cfg aliases in build scripts per rust-lang/cargo#7260 (comment), so I might take a look at that as a way to simplify the target gating.

Closing this PR out.

@tarcieri
tarcieri deleted the polyval/runtime-detection branch September 19, 2019 14:37
@newpavlov

Copy link
Copy Markdown
Member

I wouldn't say it isn't helpful. But if we want to squeeze a maximum performance, I think we should push switch between target dependent implementations as high as possible (i.e. we should not do runtime detections for aes, ctr, ghash, but do a single detection and switch in a high-level construct as AES-GCM-SIV), and unfortunately right now Rust is quite poorly equipped for doing that, especially in mult-crate contexts.

@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov I'm down to actually implement AES-GCM-SIV first, then circle back on this

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tarcieri@WildCryptoFox@newpavlov
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' polyval: Add runtime PCLMULQDQ detection by tarcieri · Pull Request #11 · RustCrypto/universal-hashes · GitHub
Skip to content

polyval: Add runtime PCLMULQDQ detection - #11

Closed
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection
Closed

polyval: Add runtime PCLMULQDQ detection#11
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection

Conversation

@tarcieri

Copy link
Copy Markdown
Member

When the std feature is enabled (which it is now by default), this adds runtime detection for PCLMULQDQ support on x86/x86_64 architectures.

The detection happens once at the time Polyval is instantiated. The polyval::field::Element type has been changed into an enum which remembers the detection result, and its API changed to operate on bytestring representations of POLYVAL field elements.

This appears to have a negligible performance impact.

Comment threadpolyval/src/field.rs
target_feature = "sse2",
target_feature = "sse4.1",
any(target_arch = "x86", target_arch = "x86_64")
))]

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional gating this implementation is using is gross. Open to alternatives... perhaps the cfg-if crate?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried cfg-if. Unfortunately it doesn't seem to support nesting, and the resulting code ended up looking just as bad

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made things a little better by using cfg!. Still a lot of annoying, redundant gating though

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My kingdom for cfg_alias: rust-lang/cargo#7260 (comment)

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 44ce8e5 to 231a23bCompareSeptember 18, 2019 20:13
@WildCryptoFox

Copy link
Copy Markdown

Everything looks good to me.

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch 4 times, most recently from 65f6879 to 86e59ebCompareSeptember 19, 2019 02:48
@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov WDYT? I'm calling this "done" but I'd be interested to know any ideas for potential simplifications/cleanups

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 86e59eb to 87cc950CompareSeptember 19, 2019 03:40
When the `std` feature is enabled (which it is now by default), this
adds runtime detection for PCLMULQDQ support on x86/x86_64
architectures.
The detection happens once at the time `Polyval` is instantated. The
`polyval::field::Element` type has been changed into an enum which
remembers the detection result, and its API changed to operate on
bytestring representations of POLYVAL field elements.
This appears to have a negligable performance impact.
@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 87cc950 to f0db4e4CompareSeptember 19, 2019 03:43
@WildCryptoFoxWildCryptoFox mentioned this pull request Sep 19, 2019
4 tasks
Comment threadpolyval/src/field.rs
pub fn from_bytes(bytes: Block) -> Self {
Element(bytes.into())
if cfg!(feature = "std") {
if is_x86_feature_detected!("pclmulqdq") {

@newpavlovnewpavlovSep 19, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite like that it does not work on no_std targets. I think we better to use RDRAND directly and cache results in atomic.

@newpavlov

Copy link
Copy Markdown
Member

IIUC this implementation branches between Soft and Clmul more than once per single call to Ghash, while ideally it should branch only ones.

Also I think you are using cfg a bit incorrectly. If CLMUL and other necessary target features are enabled, we should remove the software fallback completely.

One approach which we can take looks roughly like this:

traitElement{ .. }implElementforClmul{ .. }implElementforSoft{ .. }structGhashImpl<E:Element>{ .. }enumGhashPrivate{#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]Clmul(GhashImpl<Clmul>),// disable soft fallback if CLMUL is enabled by user#[cfg(not(all( any(target_arch = "x86", target_arch = "x86_64"), all(target_feature = "pclmulqdq", ..))))]Soft(GhashImpl<Soft>),}structGhash(GhashPrivate);

But I am not sure how good inlining will work with #[target_feature(enable = "sse4")] added to trait impl. The main idea is what we want to switch to a full implementation for given conditions as high as possible and do it only once per Ghash call.

@tarcieri

tarcieri commented Sep 19, 2019

Copy link
Copy Markdown
MemberAuthor

@newpavlov upon further reflection, I agree runtime detection isn't helpful. It would be extremely helpful for, say, AES-NI, where it isn't available inside KVM and therefore relocatable binaries produced for architectures that support it fail in these environments, but from what I can tell CLMUL isn't similarly impacted.

That said, your suggested approach is pretty much what the existing implementation already does.

I think I can eliminate the need for an enum and a trait entirely though via a simple newtype which only ensures the relevant APIs are equivalent though, which I'll take a stab at in a separate PR.

It seems we can also fake cfg aliases in build scripts per rust-lang/cargo#7260 (comment), so I might take a look at that as a way to simplify the target gating.

Closing this PR out.

@tarcieri
tarcieri deleted the polyval/runtime-detection branch September 19, 2019 14:37
@newpavlov

Copy link
Copy Markdown
Member

I wouldn't say it isn't helpful. But if we want to squeeze a maximum performance, I think we should push switch between target dependent implementations as high as possible (i.e. we should not do runtime detections for aes, ctr, ghash, but do a single detection and switch in a high-level construct as AES-GCM-SIV), and unfortunately right now Rust is quite poorly equipped for doing that, especially in mult-crate contexts.

@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov I'm down to actually implement AES-GCM-SIV first, then circle back on this

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tarcieri@WildCryptoFox@newpavlov
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' polyval: Add runtime PCLMULQDQ detection by tarcieri · Pull Request #11 · RustCrypto/universal-hashes · GitHub
Skip to content

polyval: Add runtime PCLMULQDQ detection - #11

Closed
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection
Closed

polyval: Add runtime PCLMULQDQ detection#11
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection

Conversation

@tarcieri

Copy link
Copy Markdown
Member

When the std feature is enabled (which it is now by default), this adds runtime detection for PCLMULQDQ support on x86/x86_64 architectures.

The detection happens once at the time Polyval is instantiated. The polyval::field::Element type has been changed into an enum which remembers the detection result, and its API changed to operate on bytestring representations of POLYVAL field elements.

This appears to have a negligible performance impact.

Comment threadpolyval/src/field.rs
target_feature = "sse2",
target_feature = "sse4.1",
any(target_arch = "x86", target_arch = "x86_64")
))]

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional gating this implementation is using is gross. Open to alternatives... perhaps the cfg-if crate?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried cfg-if. Unfortunately it doesn't seem to support nesting, and the resulting code ended up looking just as bad

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made things a little better by using cfg!. Still a lot of annoying, redundant gating though

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My kingdom for cfg_alias: rust-lang/cargo#7260 (comment)

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 44ce8e5 to 231a23bCompareSeptember 18, 2019 20:13
@WildCryptoFox

Copy link
Copy Markdown

Everything looks good to me.

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch 4 times, most recently from 65f6879 to 86e59ebCompareSeptember 19, 2019 02:48
@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov WDYT? I'm calling this "done" but I'd be interested to know any ideas for potential simplifications/cleanups

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 86e59eb to 87cc950CompareSeptember 19, 2019 03:40
When the `std` feature is enabled (which it is now by default), this
adds runtime detection for PCLMULQDQ support on x86/x86_64
architectures.
The detection happens once at the time `Polyval` is instantated. The
`polyval::field::Element` type has been changed into an enum which
remembers the detection result, and its API changed to operate on
bytestring representations of POLYVAL field elements.
This appears to have a negligable performance impact.
@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 87cc950 to f0db4e4CompareSeptember 19, 2019 03:43
@WildCryptoFoxWildCryptoFox mentioned this pull request Sep 19, 2019
4 tasks
Comment threadpolyval/src/field.rs
pub fn from_bytes(bytes: Block) -> Self {
Element(bytes.into())
if cfg!(feature = "std") {
if is_x86_feature_detected!("pclmulqdq") {

@newpavlovnewpavlovSep 19, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite like that it does not work on no_std targets. I think we better to use RDRAND directly and cache results in atomic.

@newpavlov

Copy link
Copy Markdown
Member

IIUC this implementation branches between Soft and Clmul more than once per single call to Ghash, while ideally it should branch only ones.

Also I think you are using cfg a bit incorrectly. If CLMUL and other necessary target features are enabled, we should remove the software fallback completely.

One approach which we can take looks roughly like this:

traitElement{ .. }implElementforClmul{ .. }implElementforSoft{ .. }structGhashImpl<E:Element>{ .. }enumGhashPrivate{#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]Clmul(GhashImpl<Clmul>),// disable soft fallback if CLMUL is enabled by user#[cfg(not(all( any(target_arch = "x86", target_arch = "x86_64"), all(target_feature = "pclmulqdq", ..))))]Soft(GhashImpl<Soft>),}structGhash(GhashPrivate);

But I am not sure how good inlining will work with #[target_feature(enable = "sse4")] added to trait impl. The main idea is what we want to switch to a full implementation for given conditions as high as possible and do it only once per Ghash call.

@tarcieri

tarcieri commented Sep 19, 2019

Copy link
Copy Markdown
MemberAuthor

@newpavlov upon further reflection, I agree runtime detection isn't helpful. It would be extremely helpful for, say, AES-NI, where it isn't available inside KVM and therefore relocatable binaries produced for architectures that support it fail in these environments, but from what I can tell CLMUL isn't similarly impacted.

That said, your suggested approach is pretty much what the existing implementation already does.

I think I can eliminate the need for an enum and a trait entirely though via a simple newtype which only ensures the relevant APIs are equivalent though, which I'll take a stab at in a separate PR.

It seems we can also fake cfg aliases in build scripts per rust-lang/cargo#7260 (comment), so I might take a look at that as a way to simplify the target gating.

Closing this PR out.

@tarcieri
tarcieri deleted the polyval/runtime-detection branch September 19, 2019 14:37
@newpavlov

Copy link
Copy Markdown
Member

I wouldn't say it isn't helpful. But if we want to squeeze a maximum performance, I think we should push switch between target dependent implementations as high as possible (i.e. we should not do runtime detections for aes, ctr, ghash, but do a single detection and switch in a high-level construct as AES-GCM-SIV), and unfortunately right now Rust is quite poorly equipped for doing that, especially in mult-crate contexts.

@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov I'm down to actually implement AES-GCM-SIV first, then circle back on this

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tarcieri@WildCryptoFox@newpavlov
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); polyval: Add runtime PCLMULQDQ detection by tarcieri · Pull Request #11 · RustCrypto/universal-hashes · GitHub
Skip to content

polyval: Add runtime PCLMULQDQ detection - #11

Closed
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection
Closed

polyval: Add runtime PCLMULQDQ detection#11
tarcieri wants to merge 1 commit into
masterfrom
polyval/runtime-detection

Conversation

@tarcieri

Copy link
Copy Markdown
Member

When the std feature is enabled (which it is now by default), this adds runtime detection for PCLMULQDQ support on x86/x86_64 architectures.

The detection happens once at the time Polyval is instantiated. The polyval::field::Element type has been changed into an enum which remembers the detection result, and its API changed to operate on bytestring representations of POLYVAL field elements.

This appears to have a negligible performance impact.

Comment threadpolyval/src/field.rs
target_feature = "sse2",
target_feature = "sse4.1",
any(target_arch = "x86", target_arch = "x86_64")
))]

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional gating this implementation is using is gross. Open to alternatives... perhaps the cfg-if crate?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried cfg-if. Unfortunately it doesn't seem to support nesting, and the resulting code ended up looking just as bad

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made things a little better by using cfg!. Still a lot of annoying, redundant gating though

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My kingdom for cfg_alias: rust-lang/cargo#7260 (comment)

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 44ce8e5 to 231a23bCompareSeptember 18, 2019 20:13
@WildCryptoFox

Copy link
Copy Markdown

Everything looks good to me.

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch 4 times, most recently from 65f6879 to 86e59ebCompareSeptember 19, 2019 02:48
@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov WDYT? I'm calling this "done" but I'd be interested to know any ideas for potential simplifications/cleanups

@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 86e59eb to 87cc950CompareSeptember 19, 2019 03:40
When the `std` feature is enabled (which it is now by default), this
adds runtime detection for PCLMULQDQ support on x86/x86_64
architectures.
The detection happens once at the time `Polyval` is instantated. The
`polyval::field::Element` type has been changed into an enum which
remembers the detection result, and its API changed to operate on
bytestring representations of POLYVAL field elements.
This appears to have a negligable performance impact.
@tarcieri
tarcieriforce-pushed the polyval/runtime-detection branch from 87cc950 to f0db4e4CompareSeptember 19, 2019 03:43
@WildCryptoFoxWildCryptoFox mentioned this pull request Sep 19, 2019
4 tasks
Comment threadpolyval/src/field.rs
pub fn from_bytes(bytes: Block) -> Self {
Element(bytes.into())
if cfg!(feature = "std") {
if is_x86_feature_detected!("pclmulqdq") {

@newpavlovnewpavlovSep 19, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite like that it does not work on no_std targets. I think we better to use RDRAND directly and cache results in atomic.

@newpavlov

Copy link
Copy Markdown
Member

IIUC this implementation branches between Soft and Clmul more than once per single call to Ghash, while ideally it should branch only ones.

Also I think you are using cfg a bit incorrectly. If CLMUL and other necessary target features are enabled, we should remove the software fallback completely.

One approach which we can take looks roughly like this:

traitElement{ .. }implElementforClmul{ .. }implElementforSoft{ .. }structGhashImpl<E:Element>{ .. }enumGhashPrivate{#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]Clmul(GhashImpl<Clmul>),// disable soft fallback if CLMUL is enabled by user#[cfg(not(all( any(target_arch = "x86", target_arch = "x86_64"), all(target_feature = "pclmulqdq", ..))))]Soft(GhashImpl<Soft>),}structGhash(GhashPrivate);

But I am not sure how good inlining will work with #[target_feature(enable = "sse4")] added to trait impl. The main idea is what we want to switch to a full implementation for given conditions as high as possible and do it only once per Ghash call.

@tarcieri

tarcieri commented Sep 19, 2019

Copy link
Copy Markdown
MemberAuthor

@newpavlov upon further reflection, I agree runtime detection isn't helpful. It would be extremely helpful for, say, AES-NI, where it isn't available inside KVM and therefore relocatable binaries produced for architectures that support it fail in these environments, but from what I can tell CLMUL isn't similarly impacted.

That said, your suggested approach is pretty much what the existing implementation already does.

I think I can eliminate the need for an enum and a trait entirely though via a simple newtype which only ensures the relevant APIs are equivalent though, which I'll take a stab at in a separate PR.

It seems we can also fake cfg aliases in build scripts per rust-lang/cargo#7260 (comment), so I might take a look at that as a way to simplify the target gating.

Closing this PR out.

@tarcieri
tarcieri deleted the polyval/runtime-detection branch September 19, 2019 14:37
@newpavlov

Copy link
Copy Markdown
Member

I wouldn't say it isn't helpful. But if we want to squeeze a maximum performance, I think we should push switch between target dependent implementations as high as possible (i.e. we should not do runtime detections for aes, ctr, ghash, but do a single detection and switch in a high-level construct as AES-GCM-SIV), and unfortunately right now Rust is quite poorly equipped for doing that, especially in mult-crate contexts.

@tarcieri

Copy link
Copy Markdown
MemberAuthor

@newpavlov I'm down to actually implement AES-GCM-SIV first, then circle back on this

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tarcieri@WildCryptoFox@newpavlov