close_credential: TypeError: object NoneType can't be used in 'await' expression

[hginzel]

This simple code

from fsspec import filesystem
from azure.identity import DefaultAzureCredential

protocol = 'az'
account_name = 'blob_account_name'
container_name = 'storage_container'
fs = filesystem(protocol,
  account_name=account_name,
  credential=DefaultAzureCredential())

blobs = fs.ls(container_name)
for blob in blobs[:5]:
  logging.info("\t" + blob)

returns error

Traceback (most recent call last):
  File "/home/h.ginzel/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/weakref.py", line 666, in _exitfunc
    f()
  File "/home/h.ginzel/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/weakref.py", line 590, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/h.ginzel/Copernicus/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/home/h.ginzel/Copernicus/.venv/lib/python3.12/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
                ^^^^^^^^^^
  File "/home/h.ginzel/Copernicus/.venv/lib/python3.12/site-packages/adlfs/utils.py", line 78, in close_credential
    await file_obj.credential.close()
TypeError: object NoneType can't be used in 'await' expression

[martindurant]

Transferred to adlfs repo

[kyleknap]

Hi @hginzel. Could you try using the asynchronous SDK default credentials instead? So, import DefaultAzureCredential from azure.identity.aio instead of azure.identity. It's subtle but the AzureBlobFilesystem expects asynchronous SDK credentials and not synchronous. Mainly because under the hood the implementation uses the asynchronous blob SDK and it is generally advised to match the credential type (e.g. async vs sync) to client type.

Let us know if switching to async credentials works on your end.

I do think this is a sharp edge in adlfs that we should address; it is an easy issue to run into. I'd like to think more about it but some options I'd like us to consider are:

1. Just make synchronous credentials work when supplied to AzureBlobFilesystem (and possibly warn to suggest to use synchronous credentials instead)
2. Improve the error message to be more clear that sync credentials lead to this error
3. Disallow synchronous credentials altogether in AzureBlobFilesystem

Thoughts?

[AlecVivian]

Any updates here? Would like to use this tool for my project. azure.identity.aio.DefaultAzureCredential resolved this for me!