[rest_api source] Extend the resolve params to fetch an pass a list of parameters

[francescomucio]

Feature description

Some APIs allow to pass a list of elements to collect further details on some resources, this is a way to limit the number of calls.

The idea would be something like:

        "params": {
            "resource_id": {
                "type": "resolve_list",
                "field": "id",
                "resource": "parent_resource",
                "list_size": 100
            },
        },

A few notes:

• this must work with query params (this is where usually you pass arrays). So this issue needs to be solved too.
• would be nice to sort out this too.

Are you a dlt user?

Yes, I run dlt in production.

Use case

APis accepting arrays of elements

Proposed solution

Add an additional params type

Related issues

No response

[burnash]

Thanks for the suggestion, @francescomucio. Do you have an example API or APIs that do accept a list of values? As far as I know different APIs use different ways of accepting lists (e.g. repeating the query string param ?id=1&id=2&id=3, comma delimited ?id=1,2,3, encoding json-lists and so on.) How would rest_api source know the right way to serialize the list for an API call?

[francescomucio]

Hi @burnash , that's a good point.

Till now I saw only comma delimited, but I guess this is not always the case.
Let me cook up a possible PR and we can discuss it further