Swapping PyCrypto for pyOpenSSL.

[dhermes]

This was done because PyCrypto does not install easily on Windows. pyOpenSSL is managed by PyCA (the Python crypto authority) and has a mature release process.

This change was influenced by discussions about #1009.

[dhermes]

To "confirm" this implementation matches our current one I did the following:

>>>importjson>>>fromoauth2client.clientimportGoogleCredentials>>>fromOpenSSLimportcrypto>>>fromgcloudimportcredentials>>>creds=GoogleCredentials.get_application_default() # Env. var. -> path to JSON key>>>pkey=crypto.load_privatekey(crypto.FILETYPE_PEM, creds._private_key_pkcs8_text)
>>>data=b'foo'>>>new_signed_bytes=crypto.sign(pkey, data, 'SHA256')
>>>curr_signed_bytes=credentials._get_signature_bytes(creds, data)
>>>new_signed_bytes==curr_signed_bytesTrue

[dhermes]

H/T to @tseaver for pointing out that cryptography was managed by the PyCA which led me to discover that pyOpenSSL was too.

[dhermes]

@jonparrott Is pyOpenSSL supported on App Engine?

[theacodes]

I don't think so; only pycrypto and the core ssl library.

[dhermes]

OK. PyCrypto has a pretty crappy install story (latest version isn't on PyPI, pip install fails on Windows).

It may be best to take the oauth2client approach and just implement using both pyOpenSSL and PyCrypto but require neither in setup.py.

[theacodes]

Sounds reasonable.

[tseaver]

I'd really hate to have our codebase cluttered with both the current PyCrypto-based implementation and the new pyOpenSSL-based one.

[dhermes]

1. The original goal: better Windows experience, is not really possible with PyCrypto since the maintainers don't have great windows support.
2. pyOpenSSL not working on Google App Engine is something we can avoid if we allow PyCrypto as a fallback since it is supported on GAE
3. I wouldn't exactly call it cluttered. We already have both implementations and they are both short (and only used for signing bytes). We've mentioned before we wanted to push the signing upsteam into oauth2client anyhow.

---

Maybe now is the time to bite the bullet and get both implementations into oauth2client?

Then the question becomes

• Are we OK having a component that depends on one of pyOpenSSL and PyCrypto being installed if we don't include either one in our setup.py?

[tseaver]

Are we OK having a component that depends on one of pyOpenSSL and PyCrypto being installed if we don't include either one in our setup.py?

I really don't want to introduce imperative platform-based variations in the requirements (breaking wheel generation, for instance). Is there a PEP 508-supported way to detect GAE?

[theacodes]

Is there a PEP 508-supported way to detect GAE?

No.

[dhermes]

@jonparrott Wrote https://github.com/jonparrott/Darth-Vendor which probably makes him the foremost expert on packaging for GAE 😀

[dhermes]

@jonparrott After some digging I realized this can be done in pure Python using pyasn1, pyasn1-modules and rsa. Is there any vendor-ing issue for these libraries on GAE?

[theacodes]

@dhermes if they have any native components, then yes. Let me verify.

[theacodes]

@dhermes those should be fine. Go for it. 👍

[dhermes]

Good deal. Thanks for doing it for me (I know I could've RTFM instead of wasting your time).

[theacodes]

I know I could've RTFM instead of wasting your time

Considering how much of your time I've monopolized elsewhere, I'd say I owe you.

[dhermes]

As I was starting to do the rsa/pyasn1 implementation I realized GAE users are [covered] by using oauth2client.appengine.AppAssertionCredentials so we don't need to worry if they have support for either pyOpenSSLorPyCrypto.

@jonparrott What does darth vendor do if a package can't be installed / imported in GAE?

[theacodes]

@dhermes nothing. pip handles the installation, not the vendor tool. Pip will happily stage a binary package into the lib directory. :(

[dhermes]

OK. I suppose we could try/except ImportError on the pyOpenSSL import. Though it seems strange to do for something in setup.py.

[tseaver]

Would the try:..except: in setup.py be to make the dependency on PyCrypto be "soft" FBO the GAE environment?

[dhermes]

What did you have in mind?

i.e. Try and except what import? How would it help GAE?

[tseaver]

I'm responding to:

OK. I suppose we could try/except ImportError on the pyOpenSSL import. Though it seems strange to do for something in setup.py.

[dhermes]

I meant try/except in our code, not in setup.py.

[theacodes]

setup.py doesn't run on app engine, it runs on the user's machine when they vendor the package.

[dhermes]

@jonparrott Do you think it's fine to just leave pyopenssl as a dep since on GAE the user has the fancy http://oauth2client.readthedocs.org/en/stable/source/oauth2client.appengine.html module (now moved to contrib)

[theacodes]

I think you're in the clear as long as you don't ever try to use the library within GAE code.

[dhermes]

@tseaver GAE is "no longer" a blocker for using pyopenssl. Remaining issues?

[tseaver]

@dhermes

GAE is "no longer" a blocker for using pyopenssl. Remaining issues?

I thought GAE was the only reason to keep the pyopenssl codepath around at all. Am I mistaken?

[dhermes]

Nope its Windows. Check out the description of this PR.

[dhermes]

I just scared myself into thinking that it'd be a blocker for GAE but forgot that GAE and GCE had custom credentials types

[tseaver]

Oops, I misspoke: I meant I thought the old codepath (PyCrypto) was being kept around FBO GAE. If that isn't so, then lets just punt and make pyOpenSSL the only way we do this.

[dhermes]

Got it. All good to merge?

(PS I am planning on pushing this upstream to oauth2client at some point.)

[tseaver]

LGTM. I had lost track of the fact that you already dropped the old codepath.