dlt version
dlt 1.17.1
Describe the problem
When using GCP service account credentials, referenced either in environment variables or in secrets.toml I get this following error.
`GcpOAuthCredentials` cannot parse the configuration value provided. The value is of type `str` and comes from the sections `('credentials',)` Value may be a secret and is not shown. Details: The expected representation for `GcpOAuthCredentials` is a string with serialized oauth2 user info and may be wrapped in 'install'/'web' node - depending of oauth2 app type.
Expected behavior
dlt should be using GcpServiceAccountCredentials, which currently can be used when explicitly provided within the pipeline as in
with open("creds.json", "r") as f:
creds = json.load(f)
service_dict = json.loads(json.dumps(creds))
gcp_credentials = GcpServiceAccountCredentials()
gcp_credentials.parse_native_representation(json.dumps(creds))
Steps to reproduce
set up GCP service account credentials and attempt to use them either in secrets.toml as in
[destination.filesystem]
bucket_url = "gs://yourbucket...
[destination.filesystem]
credentials="path/to/your/json"
when running a dlt pipeline for filesystem destination
Operating system
macOS
Runtime environment
Local
Python version
3.10
dlt data source
API
dlt destination
Filesystem & buckets
Other deployment details
No response
Additional information
No response
dlt version
dlt 1.17.1
Describe the problem
When using GCP service account credentials, referenced either in environment variables or in
secrets.tomlI get this following error.Expected behavior
dlt should be using
GcpServiceAccountCredentials, which currently can be used when explicitly provided within the pipeline as inSteps to reproduce
set up GCP service account credentials and attempt to use them either in secrets.toml as in
when running a dlt pipeline for filesystem destination
Operating system
macOS
Runtime environment
Local
Python version
3.10
dlt data source
API
dlt destination
Filesystem & buckets
Other deployment details
No response
Additional information
No response