ABSStore ImportError

[jhamman]

I'm finding that current versions of azure-storage-blob (12.5.0) are not importable when using the ABSStore. This was actually warned in this comment: #528 (comment). Is anyone using this store anymore? Presumably, fsspec's mapper interface should work

Minimal, reproducible code sample, a copy-pastable example if possible

import zarr

store = zarr.storage.ABSStore('carbonplan-scratch',
 prefix=f'cmip-raw-conus/',
 account_name="carbonplan",
 account_key=os.environ["BLOB_ACCOUNT_KEY"])

Problem description

Simply creating the ABSStore raises an ImportError:

---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-7-ad89fee562aa> in <module>
 4 prefix=f'cmip-raw-conus/',
 5 account_name="carbonplan",
----> 6 account_key=os.environ["BLOB_ACCOUNT_KEY"])
 7 store

/srv/conda/envs/notebook/lib/python3.7/site-packages/zarr/storage.py in __init__(self, container, prefix, account_name, account_key, blob_service_kwargs)
 1980 def __init__(self, container, prefix='', account_name=None, account_key=None,
 1981 blob_service_kwargs=None):
-> 1982 from azure.storage.blob import BlockBlobService
 1983 self.container = container
 1984 self.prefix = normalize_storage_path(prefix)

ImportError: cannot import name 'BlockBlobService' from 'azure.storage.blob' (/srv/conda/envs/notebook/lib/python3.7/site-packages/azure/storage/blob/__init__.py)

ABSStore is unusable in its current state.

Version and installation information

• Value of zarr.__version__: '2.4.0'
• Value of numcodecs.__version__: '0.7.2'
• Version of Python interpreter: 3.7.8
• Operating system (Linux/Windows/Mac): Linux
• How Zarr was installed (e.g., "using pip into virtual environment", or "using conda"): conda-forge

conda env export :

Details

name: notebook
channels:
 - conda-forge
 - defaults
dependencies:
 - _libgcc_mutex=0.1=conda_forge
 - _openmp_mutex=4.5=1_gnu
 - abseil-cpp=20200225.2=he1b5a44_2
 - adal=1.2.4=pyh9f0ad1d_0
 - affine=2.3.0=py_0
 - aiofiles=0.5.0=py_0
 - aiohttp=3.6.2=py37h516909a_0
 - alembic=1.4.3=pyh9f0ad1d_0
 - altair=4.1.0=py_1
 - altair_data_server=0.4.1=py_0
 - appdirs=1.4.3=py_1
 - argon2-cffi=20.1.0=py37h8f50634_1
 - arrow-cpp=1.0.1=py37h8c03bf9_3_cpu
 - arviz=0.9.0=py_0
 - asciitree=0.3.3=py_2
 - asn1crypto=1.4.0=pyh9f0ad1d_0
 - async-timeout=3.0.1=py_1000
 - async_generator=1.10=py_0
 - attrs=20.2.0=pyh9f0ad1d_0
 - aws-sdk-cpp=1.7.164=hc831370_1
 - awscli=1.18.141=py37hc8dfbb8_0
 - azure-core=1.8.1=pyh9f0ad1d_0
 - azure-datalake-store=0.0.50=pyh9f0ad1d_0
 - azure-identity=1.4.0=pyh9f0ad1d_0
 - backcall=0.2.0=pyh9f0ad1d_0
 - backports=1.0=py_2
 - backports.functools_lru_cache=1.6.1=py_0
 - beautifulsoup4=4.9.1=py_1
 - binutils_impl_linux-64=2.35=h18a2f87_9
 - binutils_linux-64=2.35=heab0d09_28
 - bleach=3.2.0=pyh9f0ad1d_0
 - blinker=1.4=py_1
 - bokeh=2.2.1=py37hc8dfbb8_0
 - boost-cpp=1.72.0=h8e57a91_0
 - boto3=1.15.0=pyh9f0ad1d_0
 - botocore=1.18.0=pyh9f0ad1d_0
 - bottleneck=1.3.2=py37h03ebfcd_1
 - branca=0.3.1=py_0
 - brotli=1.0.9=he1b5a44_0
 - brotlipy=0.7.0=py37h8f50634_1000
 - bzip2=1.0.8=h516909a_3
 - c-ares=1.16.1=h516909a_3
 - ca-certificates=2020.6.20=hecda079_0
 - cachetools=4.1.1=py_0
 - cachey=0.2.1=pyh9f0ad1d_0
 - cairo=1.16.0=hcf35c78_1003
 - cartopy=0.17.0=py37he1be148_1005
 - certifi=2020.6.20=py37hc8dfbb8_0
 - certipy=0.1.3=py_0
 - cffi=1.14.3=py37h2b28604_0
 - cfgv=3.2.0=py_0
 - cfitsio=3.470=hb60a0a2_2
 - cftime=1.2.1=py37h03ebfcd_0
 - chardet=3.0.4=py37hc8dfbb8_1007
 - click=7.1.2=pyh9f0ad1d_0
 - click-plugins=1.1.1=py_0
 - cligj=0.5.0=py_0
 - cloudpickle=1.6.0=py_0
 - colorama=0.4.3=py_0
 - colorcet=2.0.1=py_0
 - configobj=5.0.6=py_0
 - croniter=0.3.30=py_0
 - cryptography=3.1=py37hb09aad4_0
 - curl=7.68.0=hf8cf82a_0
 - cycler=0.10.0=py_2
 - cython=0.28.5=py37hf484d3e_1000
 - cytoolz=0.10.1=py37h516909a_0
 - dask=2.25.0=py_0
 - dask-core=2.25.0=py_0
 - dask-gateway=0.8.0=py37hc8dfbb8_0
 - dask-kubernetes=0.10.1=py_0
 - dask-labextension=3.0.0=py_0
 - datashader=0.11.1=pyh9f0ad1d_0
 - datashape=0.5.4=py_1
 - dbus=1.13.6=he372182_0
 - decorator=4.4.2=py_0
 - defusedxml=0.6.0=py_0
 - distlib=0.3.1=pyh9f0ad1d_0
 - distributed=2.25.0=py37hc8dfbb8_0
 - docker-py=4.3.1=py37hc8dfbb8_0
 - docker-pycreds=0.4.0=py_0
 - docopt=0.6.2=py_1
 - docutils=0.15.2=py37_0
 - earthengine-api=0.1.235=pyh9f0ad1d_0
 - editdistance=0.5.3=py37h3340039_1
 - entrypoints=0.3=py37hc8dfbb8_1001
 - esmf=8.0.0=hcda5390_0
 - esmpy=8.0.0=py37h5ca1d4c_0
 - expat=2.2.9=he1b5a44_2
 - fastapi=0.61.1=py_0
 - fasteners=0.14.1=py_3
 - fastparquet=0.4.1=py37h03ebfcd_0
 - fastprogress=1.0.0=py_0
 - filelock=3.0.12=pyh9f0ad1d_0
 - fiona=1.8.9.post2=py37hdff7cfa_0
 - fontconfig=2.13.1=h86ecdb6_1001
 - freetype=2.10.2=he06d7ca_0
 - freexl=1.0.5=h516909a_1002
 - fribidi=1.0.10=h516909a_0
 - future=0.18.2=py37hc8dfbb8_1
 - gcc_impl_linux-64=7.5.0=hda68d29_13
 - gcc_linux-64=7.5.0=hf34d7eb_28
 - gcsfs=0.7.1=py_0
 - gdal=2.4.2=py37h5f563d9_4
 - geopandas=0.8.1=py_0
 - geos=3.7.2=he1b5a44_2
 - geotiff=1.5.1=h560c3f3_2
 - geoviews=1.8.1=py_0
 - geoviews-core=1.8.1=py_0
 - gettext=0.19.8.1=hc5be6a0_1002
 - gflags=2.2.2=he1b5a44_1004
 - giflib=5.1.7=h516909a_1
 - glib=2.66.0=h0dae87d_0
 - glog=0.4.0=h49b9bf7_3
 - google-api-core=1.22.2=py37hc8dfbb8_0
 - google-api-python-client=1.12.1=pyh9f0ad1d_0
 - google-auth=1.21.2=py_0
 - google-auth-httplib2=0.0.3=py_3
 - google-auth-oauthlib=0.4.1=py_2
 - google-cloud-core=1.4.1=pyh9f0ad1d_0
 - google-cloud-storage=1.31.0=pyh9f0ad1d_0
 - google-crc32c=1.0.0=py37h193935f_0
 - google-resumable-media=1.0.0=pyh9f0ad1d_0
 - googleapis-common-protos=1.51.0=py37hc8dfbb8_2
 - graphene=1.4=py_0
 - graphite2=1.3.13=he1b5a44_1001
 - graphql-core=3.1.2=py_0
 - graphql-relay=3.0.0=py_0
 - graphviz=2.42.3=h0511662_0
 - grpc-cpp=1.30.2=heedbac9_0
 - grpcio=1.31.0=py37hb0870dc_0
 - gunicorn=20.0.4=py37hc8dfbb8_1
 - gxx_impl_linux-64=7.5.0=h64c220c_13
 - gxx_linux-64=7.5.0=ha781d05_28
 - h11=0.9.0=py_0
 - h5netcdf=0.8.1=py_0
 - h5py=2.10.0=nompi_py37h513d04c_102
 - harfbuzz=2.4.0=h9f30f68_3
 - hdf4=4.2.13=hf30be14_1003
 - hdf5=1.10.5=nompi_h3c11f04_1104
 - heapdict=1.0.1=py_0
 - holoviews=1.13.3=pyh9f0ad1d_0
 - httplib2=0.18.1=pyh9f0ad1d_0
 - httptools=0.1.1=py37h516909a_0
 - hvplot=0.6.0=pyh9f0ad1d_0
 - icu=64.2=he1b5a44_1
 - identify=1.5.3=pyh9f0ad1d_0
 - idna=2.10=pyh9f0ad1d_0
 - imagecodecs-lite=2019.12.3=py37h03ebfcd_1
 - imageio=2.9.0=py_0
 - importlib-metadata=1.7.0=py37hc8dfbb8_0
 - importlib_metadata=1.7.0=0
 - iniconfig=1.0.1=pyh9f0ad1d_0
 - intake=0.6.0=py_0
 - intake-esm=2020.8.15=py_0
 - intake-geopandas=0.2.3=py_0
 - intake-parquet=0.2.3=py_0
 - intake-stac=0.2.3=py_0
 - intake-xarray=0.3.1=py_0
 - ipykernel=5.3.4=py37h43977f1_0
 - ipyleaflet=0.13.3=pyh9f0ad1d_0
 - ipython=7.18.1=py37hc6149b9_0
 - ipython_genutils=0.2.0=py_1
 - ipytree=0.1.8=py_0
 - ipywidgets=7.5.1=pyh9f0ad1d_1
 - isodate=0.6.0=py_1
 - itsdangerous=1.1.0=py_0
 - jedi=0.17.2=py37hc8dfbb8_0
 - jeepney=0.4.3=py_0
 - jinja2=2.11.2=pyh9f0ad1d_0
 - jmespath=0.10.0=pyh9f0ad1d_0
 - joblib=0.16.0=py_0
 - jpeg=9d=h516909a_0
 - json-c=0.13.1=hbfbb72e_1002
 - json5=0.9.4=pyh9f0ad1d_0
 - jsonschema=3.2.0=py37hc8dfbb8_1
 - jupyter-server-proxy=1.5.0=py_0
 - jupyter_client=6.1.7=py_0
 - jupyter_core=4.6.3=py37hc8dfbb8_1
 - jupyter_telemetry=0.0.5=py_0
 - jupyterhub-base=1.1.0=py37hc8dfbb8_4
 - jupyterhub-singleuser=1.1.0=py37hc8dfbb8_4
 - jupyterlab=2.2.8=py_0
 - jupyterlab_pygments=0.1.1=pyh9f0ad1d_0
 - jupyterlab_server=1.2.0=py_0
 - kealib=1.4.13=hec59c27_0
 - kernel-headers_linux-64=2.6.32=h77966d4_13
 - keyring=21.4.0=py37hc8dfbb8_0
 - kiwisolver=1.2.0=py37h99015e2_0
 - krb5=1.16.4=h2fd8d38_0
 - kubernetes_asyncio=11.3.0=pyh9f0ad1d_0
 - lcms2=2.11=hbd6801e_0
 - ld_impl_linux-64=2.35=h769bd43_9
 - libblas=3.8.0=17_openblas
 - libcblas=3.8.0=17_openblas
 - libcrc32c=1.1.1=he1b5a44_2
 - libcurl=7.68.0=hda55be3_0
 - libdap4=3.20.4=hd3bb157_0
 - libedit=3.1.20191231=he28a2e2_2
 - libevent=2.1.10=hcdb4288_2
 - libffi=3.2.1=he1b5a44_1007
 - libgcc-ng=9.3.0=h24d8f2e_16
 - libgdal=2.4.2=hbef8c27_4
 - libgfortran-ng=7.5.0=hdf63c60_16
 - libgomp=9.3.0=h24d8f2e_16
 - libiconv=1.16=h516909a_0
 - libkml=1.3.0=hd79254b_1012
 - liblapack=3.8.0=17_openblas
 - libllvm10=10.0.1=he513fc3_3
 - libnetcdf=4.6.2=h303dfb8_1003
 - libopenblas=0.3.10=pthreads_hb3c22a3_4
 - libpng=1.6.37=hed695b0_2
 - libpq=11.5=hd9ab2ff_2
 - libprotobuf=3.12.4=h8b12597_0
 - libsodium=1.0.18=h516909a_0
 - libspatialindex=1.9.3=he1b5a44_3
 - libspatialite=4.3.0a=he1bb1e1_1029
 - libssh2=1.9.0=hab1572f_5
 - libstdcxx-ng=9.3.0=hdf63c60_16
 - libtiff=4.1.0=hc7e4089_6
 - libtool=2.4.6=h516909a_1005
 - libutf8proc=2.5.0=h516909a_2
 - libuuid=2.32.1=h14c3975_1000
 - libuv=1.34.0=h516909a_0
 - libwebp-base=1.1.0=h516909a_3
 - libxcb=1.13=h14c3975_1002
 - libxml2=2.9.10=hee79883_0
 - libxslt=1.1.33=h31b3aaa_0
 - llvmlite=0.34.0=py37h5202443_1
 - locket=0.2.0=py_2
 - lxml=4.5.2=py37he3881c9_0
 - lz4-c=1.9.2=he1b5a44_3
 - mako=1.1.3=pyh9f0ad1d_0
 - markdown=3.2.2=py_0
 - markupsafe=1.1.1=py37h8f50634_1
 - marshmallow=3.8.0=py_0
 - marshmallow-oneofschema=2.0.1=py_0
 - matplotlib=3.3.2=0
 - matplotlib-base=3.3.2=py37hd478181_0
 - mechanicalsoup=0.12.0=py_0
 - mercantile=1.1.6=pyh9f0ad1d_0
 - mistune=0.8.4=py37h8f50634_1001
 - monotonic=1.5=py_0
 - more-itertools=8.5.0=py_0
 - mpi=1.0=mpich
 - mpich=3.3.2=hc856adb_0
 - mpl-probscale=0.2.4=pyh9f0ad1d_0
 - msal=1.5.0=pyh9f0ad1d_0
 - msal_extensions=0.2.2=py37hc8dfbb8_0
 - msgpack-python=1.0.0=py37h99015e2_1
 - msrest=0.6.19=pyh9f0ad1d_0
 - msrestazure=0.6.2=py_0
 - multidict=4.7.5=py37h8f50634_1
 - multipledispatch=0.6.0=py_0
 - munch=2.5.0=py_0
 - mypy_extensions=0.4.3=py37hc8dfbb8_1
 - nb_conda_kernels=2.2.4=py37hc8dfbb8_0
 - nbclient=0.5.0=py_0
 - nbconvert=6.0.3=py37hc8dfbb8_0
 - nbformat=5.0.7=py_0
 - nbgitpuller=0.9.0=py_0
 - nc-time-axis=1.2.0=py_1
 - ncurses=6.2=he1b5a44_1
 - nest-asyncio=1.4.0=py_0
 - netcdf-fortran=4.4.5=h0789656_1004
 - netcdf4=1.5.1.2=py37h73a1b54_1
 - networkx=2.5=py_0
 - nodeenv=1.5.0=pyh9f0ad1d_0
 - nodejs=13.13.0=hf5d1a2b_0
 - notebook=6.1.4=py37hc8dfbb8_0
 - numba=0.51.2=py37h9fdb41a_0
 - numcodecs=0.7.2=py37h3340039_0
 - numpy=1.19.1=py37h7ea13bd_2
 - oauthlib=3.0.1=py_0
 - olefile=0.46=py_0
 - openjpeg=2.3.1=h981e76c_3
 - openssl=1.1.1g=h516909a_1
 - owslib=0.20.0=py_0
 - packaging=20.4=pyh9f0ad1d_0
 - pamela=1.0.0=py_0
 - pandas=1.1.2=py37h3340039_0
 - pandoc=2.10.1=h516909a_0
 - pandocfilters=1.4.2=py_1
 - panel=0.9.7=py_0
 - pangeo-dask=2020.08.30=1
 - pangeo-notebook=2020.08.31=0
 - pango=1.42.4=h7062337_4
 - param=1.9.3=py_0
 - parquet-cpp=1.5.1=2
 - parso=0.7.1=pyh9f0ad1d_0
 - partd=1.1.0=py_0
 - patsy=0.5.1=py_0
 - pcre=8.44=he1b5a44_0
 - pendulum=2.1.2=py37hc8dfbb8_0
 - pexpect=4.8.0=py37hc8dfbb8_1
 - pickleshare=0.7.5=py37hc8dfbb8_1001
 - pillow=7.2.0=py37h718be6c_1
 - pip=20.2.3=py_0
 - pixman=0.38.0=h516909a_1003
 - pluggy=0.13.1=py37hc8dfbb8_2
 - poppler=0.67.0=h14e79db_8
 - poppler-data=0.4.9=1
 - portalocker=1.7.0=py37hc8dfbb8_0
 - portpicker=1.3.1=py37_0
 - postgresql=11.5=hc63931a_2
 - pre-commit=2.7.1=py37hc8dfbb8_0
 - pre_commit=2.7.1=0
 - prefect=0.13.7=py_0
 - proj4=6.1.0=he751ad9_2
 - prometheus_client=0.8.0=pyh9f0ad1d_0
 - promise=2.3=py37hc8dfbb8_1
 - prompt-toolkit=3.0.7=py_0
 - protobuf=3.12.4=py37h3340039_0
 - psutil=5.7.2=py37h8f50634_0
 - pthread-stubs=0.4=h14c3975_1001
 - ptyprocess=0.6.0=py_1001
 - py=1.9.0=pyh9f0ad1d_0
 - pyarrow=1.0.1=py37hcae2be3_3_cpu
 - pyasn1=0.4.8=py_0
 - pyasn1-modules=0.2.7=py_0
 - pycparser=2.20=pyh9f0ad1d_2
 - pyct=0.4.6=py_0
 - pyct-core=0.4.6=py_0
 - pycurl=7.43.0.5=py37hce7685b_2
 - pydantic=1.6.1=py37h8f50634_0
 - pydap=3.2.2=py37_1000
 - pyepsg=0.4.0=py_0
 - pygments=2.7.1=py_0
 - pyjwt=1.7.1=py_0
 - pykdtree=1.3.1=py37h03ebfcd_1003
 - pyopenssl=19.1.0=py_1
 - pyparsing=2.4.7=pyh9f0ad1d_0
 - pyproj=2.2.1=py37hc44880f_0
 - pyresample=1.16.0=py37h0da4684_0
 - pyrsistent=0.17.3=py37h8f50634_0
 - pyshp=2.1.2=pyh9f0ad1d_0
 - pysocks=1.7.1=py37hc8dfbb8_1
 - pystac=0.5.2=pyh9f0ad1d_0
 - pystan=2.19.1.1=py37h9fdb41a_2
 - pytest=6.0.2=py37hc8dfbb8_0
 - python=3.7.8=h6f2ec95_1_cpython
 - python-box=4.2.3=py_0
 - python-dateutil=2.7.5=py_0
 - python-editor=1.0.4=py_0
 - python-graphviz=0.14.1=pyh9f0ad1d_0
 - python-json-logger=0.1.11=py_0
 - python-kubernetes=11.0.0=py37hc8dfbb8_0
 - python-multipart=0.0.5=py_0
 - python-slugify=4.0.1=pyh9f0ad1d_0
 - python-wget=3.2=py_0
 - python_abi=3.7=1_cp37m
 - pytz=2020.1=pyh9f0ad1d_0
 - pytzdata=2020.1=pyh9f0ad1d_0
 - pyviz_comms=0.7.6=pyh9f0ad1d_0
 - pywavelets=1.1.1=py37h161383b_2
 - pyyaml=5.3.1=py37h8f50634_0
 - pyzmq=19.0.2=py37hac76be4_0
 - rasterio=1.0.25=py37hb3f55d8_1
 - re2=2020.08.01=he1b5a44_1
 - readline=8.0=he28a2e2_2
 - requests=2.24.0=pyh9f0ad1d_0
 - requests-oauthlib=1.3.0=pyh9f0ad1d_0
 - richdem=0.3.4=py37hb3f55d8_0
 - rio-cogeo=1.1.6=py_0
 - rioxarray=0.0.31=py_0
 - rsa=4.4.1=pyh9f0ad1d_0
 - rtree=0.9.4=py37h8526d28_1
 - ruamel.yaml=0.16.12=py37h8f50634_0
 - ruamel.yaml.clib=0.2.2=py37h8f50634_0
 - s3fs=0.4.2=py_0
 - s3transfer=0.3.3=py37hc8dfbb8_1
 - sat-stac=0.4.0=pyh9f0ad1d_0
 - scikit-image=0.17.2=py37h9fdb41a_2
 - scikit-learn=0.23.2=py37h6785257_0
 - scipy=1.5.2=py37hb14ef9d_0
 - seaborn=0.11.0=0
 - seaborn-base=0.11.0=py_0
 - secretstorage=3.1.2=py37hc8dfbb8_1
 - send2trash=1.5.0=py_0
 - setuptools=49.6.0=py37hc8dfbb8_1
 - shapely=1.6.4=py37hec07ddf_1006
 - simpervisor=0.3=py_1
 - simplejson=3.17.2=py37h8f50634_0
 - six=1.15.0=pyh9f0ad1d_0
 - snappy=1.1.8=he1b5a44_3
 - snuggs=1.4.7=py_0
 - sortedcontainers=2.2.2=pyh9f0ad1d_0
 - soupsieve=2.0.1=py_1
 - sqlalchemy=1.3.19=py37h8f50634_0
 - sqlite=3.33.0=h4cf870e_0
 - starlette=0.13.6=py_0
 - starlette-base=0.13.6=py_0
 - statsmodels=0.12.0=py37h8f50634_0
 - supermercado=0.1.1=pyh9f0ad1d_0
 - sysroot_linux-64=2.12=h77966d4_13
 - tabulate=0.8.7=pyh9f0ad1d_0
 - tblib=1.6.0=py_0
 - terminado=0.8.3=py37hc8dfbb8_1
 - testpath=0.4.4=py_0
 - text-unidecode=1.3=py_0
 - threadpoolctl=2.1.0=pyh5ca1d4c_0
 - thrift=0.11.0=py37he1b5a44_1001
 - thrift-cpp=0.13.0=hbe8ec66_4
 - tifffile=2020.6.3=py_0
 - tippecanoe=1.35.0=h792bddf_1
 - tk=8.6.10=hed695b0_0
 - toml=0.10.1=pyh9f0ad1d_0
 - toolz=0.10.0=py_0
 - tornado=6.0.4=py37h8f50634_1
 - tqdm=4.49.0=pyh9f0ad1d_0
 - traitlets=4.3.3=py37hc8dfbb8_1
 - traittypes=0.2.1=py_1
 - typing_extensions=3.7.4.2=py_0
 - tzcode=2020a=h516909a_0
 - ujson=3.2.0=py37h3340039_0
 - unidecode=1.1.1=py_0
 - uritemplate=3.0.1=py_0
 - urllib3=1.25.10=py_0
 - uvicorn=0.11.3=py37hc8dfbb8_1
 - uvloop=0.14.0=py37h8f50634_1
 - virtualenv=20.0.20=py37hc8dfbb8_1
 - wcwidth=0.2.5=pyh9f0ad1d_1
 - webencodings=0.5.1=py_1
 - webob=1.8.6=py_0
 - websocket-client=0.57.0=py37hc8dfbb8_2
 - websockets=8.1=py37h8f50634_1
 - wheel=0.35.1=pyh9f0ad1d_0
 - widgetsnbextension=3.5.1=py37hc8dfbb8_1
 - wsproto=0.13.0=py_0
 - xarray=0.16.0=py_0
 - xerces-c=3.2.2=h8412b87_1004
 - xesmf=0.3.0=py_0
 - xlrd=1.2.0=pyh9f0ad1d_1
 - xorg-kbproto=1.0.7=h14c3975_1002
 - xorg-libice=1.0.10=h516909a_0
 - xorg-libsm=1.2.3=h84519dc_1000
 - xorg-libx11=1.6.12=h516909a_0
 - xorg-libxau=1.0.9=h14c3975_0
 - xorg-libxdmcp=1.1.3=h516909a_0
 - xorg-libxext=1.3.4=h516909a_0
 - xorg-libxpm=3.5.13=h516909a_0
 - xorg-libxrender=0.9.10=h516909a_1002
 - xorg-libxt=1.1.5=h516909a_1003
 - xorg-renderproto=0.11.1=h14c3975_1002
 - xorg-xextproto=7.3.0=h14c3975_1002
 - xorg-xproto=7.0.31=h14c3975_1007
 - xpublish=0.1.0=py_0
 - xz=5.2.5=h516909a_1
 - yaml=0.2.5=h516909a_0
 - yarl=1.3.0=py37h516909a_1000
 - yarn=1.22.5=0
 - zarr=2.4.0=py_0
 - zeromq=4.3.2=he1b5a44_3
 - zict=2.0.0=py_0
 - zipp=3.1.0=py_0
 - zlib=1.2.11=h516909a_1009
 - zstd=1.4.5=h6597ccf_2
 - pip:
 - adlfs==0.5.3+12.g0cbfe53
 - azure-common==1.1.25
 - azure-storage-blob==12.5.0
 - azure-storage-common==2.1.0
 - fsspec==0.8.3+1.g8bec5de
 - numbagg==0.1
 - rechunker==0.2.0
 - rio-tiler-mvt==0.0.1.dev1
 - showit==1.1.4
 - urlpath==1.1.7
 - vtzero==0.0.1b1
prefix: /srv/conda/envs/notebook

cc @tjcrone

[rabernat]

Should we consider deprecating ABSStore in favor of https://github.com/dask/adlfs, which has matured a lot recently?

That would bring our treatment of ABS more in line with the other cloud object stores.

[tjcrone]

@jhamman, the BlockBlobService is no longer available in recent versions of the Azure storage module. BlobServiceClient replaces this in azure.storage.blob, and transitioning to this new approach in ABSStore should be relatively straightforward but it's not clear how easy it would be to maintain backward compatibility. The fsspec mapper interface does work and it may be the best thing to rely on going forward.

[tjcrone]

@rabernat, if adlfs, despite the name, is able to access Azure Blob storage, then this might also be a viable way forward. Data Lake is not Blob, but there is some indication in the adlfs documentation that it can access Blob.

[rabernat]

fsspec-compatible Azure Datake and Azure Blob Storage access

[jhamman]

FWIW, I haven't been able to get adlfs to work yet (I'll open an issue there in a bit).

from adlfs import AzureBlobFileSystem


path = 'carbonplan-data/raw/terraclimate/4000m/raster.zarr'
fs = AzureBlobFileSystem(
 account_name="carbonplan",
 account_key=os.environ["BLOB_ACCOUNT_KEY"]
)
mapper = fs.get_mapper(path)
xr.open_zarr(mapper, consolidated=True) # hangs!!!

While I can get a mapper object, accessing keys from the mapper ends up hanging indefinitely.

Also, since I know you're wondering, read-only access over http is working just fine for these stores.

[tjcrone]

@jhamman as a stop-gap, if you want to use ABSStore you can install a pre-12 (2.1) version of azure-storage-blob:

pip install azure-storage-blob==2.1.0

Updating ABSStore to work with v12 versions of the azure-storage SDK would likely be just a few lines of code. Maintaining backward compatibility would be more difficult but almost certainly doable. I would be happy to continue supporting ABSStore if other Zarr devs support this approach, but I am also happy for us to move in a different direction if that's what we would like to do. This would be a great time and maybe even a great place to have this discussion.