Support PEP 420 -- Implicit Namespace Packages

[jiasli]

As Python 2 has been deprecated, we should move away from the pkg_resources approach and use PEP 420 -- Implicit Namespace Packages.

Importing pkg_resources will add a 100ms~200ms delay to CLI's loading time.

$ time python -c "import pkg_resources"

real    0m0.153s
user    0m0.137s
sys     0m0.016s

time python -X importtime -m azure.cli version -h 2>perf.log
tuna perf.log

This is discussed at pypa/setuptools#510 which affects many projects but still doesn't have a proper fix.

Moreover, on Linux, the wrapper ~/env38/bin/az created by Easy Install also requires pkg_resources:

#!/home/admin/env38/bin/python
# EASY-INSTALL-DEV-SCRIPT: 'azure-cli==2.5.1','az'
__requires__ = 'azure-cli==2.5.1'
__import__('pkg_resources').require('azure-cli==2.5.1')
__file__ = '/home/admin/azure-cli/src/azure-cli/az'
with open(__file__) as f:
    exec(compile(f.read(), __file__, 'exec'))

This makes az installed via azdev even slower.

For comparison, directly run azure.cli:

$ time python -m azure.cli version -h

real    0m0.261s
user    0m0.238s
sys     0m0.024s

Run the Easy Install wrapper:

$ time python ~/env38/bin/az version -h

real    0m0.822s
user    0m0.779s
sys     0m0.044s

python -X importtime ~/env38/bin/az version -h 2>perf.log
tuna perf.log

In our Linux release, we package our own wrapper src/azure-cli/az, so it doesn't suffer from the slowness of this Easy Install wrapper.

[yonzhan]

add to S169

[lmazuel]

See also #8164

Since you don't need Python 2, your situation is way simpler than it was when I created that issue, but might give you some insights. Feel free to contact me if you have questions on packaging.

[jiasli]

#2951 aimed to remove pkg_resources/declare_namespace from the final wheel, but in my test with pip and Windows MSI installer, the __init__.py or __init__.pyc file is still there. Why?

This also brings up another question: what is the purpose of azure-cli-nspkg and azure-cli-command_modules-nspkg. Are we free to remove them?

[lmazuel]

Yes, you can kill them now, their only purpose is Python 2, that you don't handle anymore. Remove them all from everywhere.

[haroldrandom]

I think we are going to onboard this, have to change the way to find Python Package in setup.py from find_packages to find_namespace_packages.
And a lot of continuous intgeration job need to change accordingly as we use find_package a lot in CLI Extension Azure/azure-cli-extensions#1608