Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package fails to inference type of column for PostgreSQL data_types #5

Open
vbatychko-modeln opened this issue Dec 8, 2023 · 3 comments

Comments

@vbatychko-modeln
Copy link

vbatychko-modeln commented Dec 8, 2023

DBT's data_type can have more broader set of types that are currently I see in package's source. The type can be any of supported in underlying database. For exple, PostgreSQL varchar type:

Error: invalid operation: Error while calling method: Python error: RuntimeError: Unknown column type of volumes.id: varchar(255)

Package version: 0.6.0

@jronsse
Copy link

jronsse commented Dec 8, 2023

Hello !

We kinda have the same issue:

Context

We are currently experimenting with Cube Cloud after a month or so testing things out with Cube Core. We plan on using dbt to generate our models.
We recently updated our process and now include columns' data_type in the generated YAML model files. We use dbt-codegen: generate_model_yaml to generate the model.
We use Databricks SQL Serverless as our data warehouse

Issue encountered

The resulting manifest.json mentions data types that are not supported by cube-dbt:

"store_indice": {
  "name": "store_indice",
  "description": "",
  "meta": {},
  "data_type": "double",
  "constraints": [],
  "quote": null,
  "tags": []
},

Here's a list of datatypes that we might encounter in the manifest

@igorlukanin Would you agree that adding those type aliases to cube-dbt is the way to go? If so I'd be happy to create a PR.

Thanks !

@bzenker-amplify
Copy link

I'm not sure that this qualifies as an issue of the cube_dbt codebase. Cube clarifies what data types are allowed, and the list is abysmally small. So, all that cube_dbt is doing is supporting that list.

You should open an issue on https://github.com/cube-js/cube if you want them to support more data types.

@jmkacz
Copy link
Contributor

jmkacz commented Oct 16, 2024

Looking at this, I wonder about the mapping dict in column,py. It is currently hardcoded. Could this be exposed as a configuration file?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants