You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
djoanes opened this issue
Feb 10, 2023
· 1 comment
Labels
coreA change which affects the synth coreenhancementNew feature or requesthelp wantedExtra attention is neededintegrationA change which affects or introduces an integration
Required Functionality
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
With the current functionality of synth, the output artifacts are all typeless. This causes there to be a necessity for auto type derivation downstream or an explicit redeclaration of the types of all the columns.
Proposed Solution
Add support for new uris, parquet:, avro:. This will allow for richer importing and exporting where the explicitly defined types are preserved.
Use case
I'd like to use synth to generate data and bulk load it into a big data ecosystem (ie. Hadoop, BigQuery)
The text was updated successfully, but these errors were encountered:
Avro integration is possible, avrow should work well. Parquet integration would take a lot more work, as it doesn't seem to have a decent serde implantation.
coreA change which affects the synth coreenhancementNew feature or requesthelp wantedExtra attention is neededintegrationA change which affects or introduces an integration
Required Functionality
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
With the current functionality of synth, the output artifacts are all typeless. This causes there to be a necessity for auto type derivation downstream or an explicit redeclaration of the types of all the columns.
Proposed Solution
Add support for new uris,
parquet:
,avro:
. This will allow for richer importing and exporting where the explicitly defined types are preserved.Use case
I'd like to use synth to generate data and bulk load it into a big data ecosystem (ie. Hadoop, BigQuery)
The text was updated successfully, but these errors were encountered: