Skip to content

dlt tries OAuth in GCP Instead of Service Account Credentials when given Service Account Credentials via credential file #3328

@jstark-gitlab

Description

@jstark-gitlab

dlt version

dlt 1.17.1

Describe the problem

When using GCP service account credentials, referenced either in environment variables or in secrets.toml I get this following error.

`GcpOAuthCredentials` cannot parse the configuration value provided. The value is of type `str` and comes from the sections `('credentials',)` Value may be a secret and is not shown. Details:  The expected representation for `GcpOAuthCredentials` is a string with serialized oauth2 user info and may be wrapped in 'install'/'web' node - depending of oauth2 app type.

Expected behavior

dlt should be using GcpServiceAccountCredentials, which currently can be used when explicitly provided within the pipeline as in

    with open("creds.json", "r") as f:
        creds =  json.load(f)

    service_dict = json.loads(json.dumps(creds))

    gcp_credentials = GcpServiceAccountCredentials()
    gcp_credentials.parse_native_representation(json.dumps(creds))

Steps to reproduce

set up GCP service account credentials and attempt to use them either in secrets.toml as in

[destination.filesystem]
bucket_url = "gs://yourbucket...

[destination.filesystem]
credentials="path/to/your/json"

when running a dlt pipeline for filesystem destination

Operating system

macOS

Runtime environment

Local

Python version

3.10

dlt data source

API

dlt destination

Filesystem & buckets

Other deployment details

No response

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions