get_origin of typing.Iterable returns collections.abc.Iterable

[saulshanabrook]

In [1]: import typing_inspect; import typing

In [2]: t = typing.Iterable[int]

In [3]: typing_inspect.get_origin(t)
Out[3]: collections.abc.Iterable

I would expect it to return typing.Iterable instead.

Why? I am using the get_origin and get_args to take apart to walk the type tree, transforming certain nodes and collecting some data (to match a template tree against an actual tree). Similar to something you would do in the AST package or with any other traversal of a nested data structure.

This means I want to be be able to recursively fold over the types, which means I have to be able to deconstruct them and put them back together again. Since collections.abc.Iterable[int] is invalid, this breaks this use case.

This is likely the same issue as #27, but I think I have a slightly different use case. If you have general suggestions on better ways to do this kind of fold, instead of using get_origin and get_args, I am welcome to suggestions.

cc @tonyfast who is also doing some type traversal stuff.

[Stewori]

I encountered the same issue on supporting Python 3.7 in pytypes. I tried to resolve it by an internal map, e.g. origin_dict[collections.abc.Iterable] == typing.Iterable but discarded that because it only shifted my core issue: I actually needed a way to tell that e.g. List[T] is a subtype of MutableSequence[T] and so on. This became a hassle in the new typing as well (or even worse). So far I came up with this hackish solution: https://github.com/Stewori/pytypes/blob/master/pytypes/type_util.py#L105 but I will probably have to rewrite it again. This is only the first step to make things somehow work again. If it helps you, you can access it as pytypes.type_bases (avaible only in unreleased master branch as of this writing).

[saulshanabrook]

@Stewori Thanks for the reference. Luckily with metadsl I only support a small subset of Python't built in annotations in type hints, so I can just special case Iterable for now.

I would love to eventually factor out all the type analysis it is doing and contribute it to something like pytypes, if it would make sense there.

[Peilonrayz]

Currently I too use an internal map in typing_inspect_lib. as I couldn't figure out a nicer way to implement this in Python 3.7.

This may be privately available in typing_inspect when I get around to making a PR for #31 after 3.8's released.

from typing_inspect._extra.get_typing import get_typing

def get_origin_old(type_):
    return get_typing(type_)[0]

[saulshanabrook]

For reference python/cpython#13685 was just merged that adds get_origin to typing itself and also normalizes the return type.

[Stewori]

Looking at the changes, that does not seem to address the issue discussed here, nor the somewhat related issue I have with getting the generic base type. Correct me if I overlook something -- what do you mean by "normalizes the return type"?

I would love to eventually factor out all the type analysis it is doing and contribute it to something like pytypes, if it would make sense there.

Sure! Various stuff is already there, feel free to open feature requests or contribute something. It is meant as a toolbox for these things that are beyond scope of typing_inspect.

Currently I too use an internal map in typing_inspect_lib. as I couldn't figure out a nicer way to implement this in Python 3.7.

It occurred to me, in pytypes there is also still this map https://github.com/Stewori/pytypes/blob/master/pytypes/type_util.py#L77, originally used for backwards lookup of the type from its __extra__ content . In light of this issue, I suppose it should be turned into public API.

[saulshanabrook]

Looking at the changes, that does not seem to address the issue discussed here, nor the somewhat related issue I have with getting the generic base type. Correct me if I overlook something -- what do you mean by "normalizes the return type"?

See the change in the docs: assert get_origin(Dict[str, int]) is dict

I believe that is a similar/same issue as this one, because it adds a get_origin command to the standard library. "normalize" comes from the added documentation:

If X is a generic alias for a builtin or :mod:collections class, it gets normalized to the original class.

[Stewori]

Okay, then "normalization" refers exactly to the opposite of what is desired in this issue. One would need a way to opt out of the normalization.

[saulshanabrook]

Okay, then "normalization" refers exactly to the opposite of what is desired in this issue. One would need a way to opt out of the normalization.

Yep. Or have a seperate normalize function that performs this logic if you want it.

[ilevkivskyi]

Since now get_origin() is moved to typing, I think the best way to do this is just to have a global mapping. Potentially, we can even add a flag to get_origin() in typing_inspect that will map the result through this mapping automatically.

[ilevkivskyi]

(I will not have time to work on this any time soon, but PRs are wellcome :-))