Get rid of strict_optional=false mypy setting

[michelzurkirchen]

dlt version

1.8.0

Describe the problem

I have the following file, with only this source and nothing else.

# demo.py
from dlt.sources.sql_database import sql_database


source = sql_database("sqlite:///demo.db")

Type checking this file with uv run pyright demo.py results in the following error.

demo.py:4:10 - error: Argument of type "None" cannot be assigned to parameter "engine_adapter_callback" of type "(Engine) -> Engine" in function "__call__"
    Type "None" is not assignable to type "(Engine) -> Engine" (reportArgumentType)
1 error, 0 warnings, 0 informations

This seems to be due to engine_adapter_callback not being Optional, even though its default value is set to None and it isn't required for this source to function.

Expected behavior

Type checkers shouldn't throw this error, because engine_adapter_callback isn't necessary for the source to function. This means that engine_adapter_callback should be made optional.

Steps to reproduce

Install the below packages

dlt==1.8.0
pyright==1.1.396

Create a demo source.

# demo.py
from dlt.sources.sql_database import sql_database


source = sql_database("sqlite:///demo.db")

Type check the file with pyright.

uv run pyright demo.py

Pyright now throws an error. My assumption is that other type checkers would throw similar errors.

Operating system

Windows

Runtime environment

Local

Python version

3.11

dlt data source

sql_database

dlt destination

No response

Other deployment details

No response

Additional information

This seems like an easy fix, by changing

engine_adapter_callback: Callable[[Engine], Engine] = None,

to

engine_adapter_callback: Optional[Callable[[Engine], Engine]] = None,

I'd be happy to submit a PR.

[sh-rp]

@michelzurkirchen we have strict_optional=false set in our mypy settings: https://mypy.readthedocs.io/en/stable/config_file.html#confval-strict_optional. So for now you'll have to lint dlt code with this setting (or the equivalent of it in other tools). I also see that it says in the mypy docs that this setting is evil, so we may change it at some point, but there are many places in the code we will probably need to update, so for now, no guarantees ;)

[michelzurkirchen]

I'd be happy to submit a PR specifically for engine_adapter_callback if you think that's helpful. If you want the PR to address strict_optional=false in general I'd need to have a better look at the repo first to determine how much time would go into that.

[sh-rp]

@michelzurkirchen you can submit a pr for this specific change if you like, no problem.