Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] change create_table_upload_to_gcs #729

Open
folhesgabriel opened this issue Mar 25, 2024 · 0 comments
Open

[fix] change create_table_upload_to_gcs #729

folhesgabriel opened this issue Mar 25, 2024 · 0 comments

Comments

@folhesgabriel
Copy link
Contributor

problema:

  • A task create_table_upload_to_gcs usada nos flows primeiro cria o dataset e tabela no bigquery e depois sobe os dados para storage;
    - Para criar a tabela, precisa do schema.
    - Para criar o schema a função dump_header_to_csv é utilizada;
    - Se não existe dado no storage, a função não acha o schema e quebra e execução.
  • Caso uma pipeline seja executa e não exista dado no storage, a task create_table_upload_to_gcs vai quebrar.

run com erro: br_ms_sia.producao_ambulatorial

@folhesgabriel folhesgabriel moved this to 📋 A refinar in Roadmap de dados Mar 25, 2024
@folhesgabriel folhesgabriel moved this from 📋 A refinar to 🏁 Priorizado in Roadmap de dados Mar 25, 2024
@folhesgabriel folhesgabriel self-assigned this Mar 25, 2024
@laura-l-amaral laura-l-amaral moved this from 🏁 Priorizado to 🏗 Em andamento in Roadmap de dados Mar 25, 2024
@laura-l-amaral laura-l-amaral moved this from 🏗 Em andamento to 📋 A refinar in Roadmap de dados Mar 25, 2024
@laura-l-amaral laura-l-amaral changed the title [feat] change create_table_upload_to_gcs [fix] change create_table_upload_to_gcs Mar 25, 2024
@laura-l-amaral laura-l-amaral moved this from 📋 A refinar to ⛔ Bloqueada ou Revisão in Roadmap de dados Mar 25, 2024
@laura-l-amaral laura-l-amaral moved this from ⛔ Bloqueada ou Revisão to 📋 Backlog in Roadmap de dados Mar 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 📋 Backlog
Development

No branches or pull requests

1 participant