Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Hubs] Archive data – Change ADF to move parquet to archive/raw container #669

Open
flanakin opened this issue Apr 3, 2024 · 0 comments
Labels
Area: Data factory Data Factory integration Solution: FinOps hubs Data pipeline solution

Comments

@flanakin
Copy link
Contributor

flanakin commented Apr 3, 2024

πŸ“ Scenario

As a FinOps practitioner, I need to move raw data to cold storage in order to minimize storage costs for data that is not accessed frequently

πŸ’Ž Solution

Add an option to move raw data to cold or archive storage after it's ingested into Azure Data Explorer.

ℹ️ Additional context

Add a boolean flag to the config/settings.json file and check that during ingestion to determine whether data should be archived or not.

πŸ™‹β€β™€οΈ Ask for the community

We could use your help:

  1. Please vote this issue up (πŸ‘) to prioritize it.
  2. Leave comments to help us solidify the vision.
@flanakin flanakin added this to the Hubs - ADX support milestone Apr 3, 2024
@microsoft-github-policy-service microsoft-github-policy-service bot added Solution: FinOps hubs Data pipeline solution Needs: Triage πŸ” Untriaged issue needs to be reviewed labels Apr 3, 2024
@flanakin flanakin added Area: Data factory Data Factory integration and removed Needs: Triage πŸ” Untriaged issue needs to be reviewed labels Apr 10, 2024
@flanakin flanakin removed their assignment Apr 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Data factory Data Factory integration Solution: FinOps hubs Data pipeline solution
Projects
Status: πŸ“‹ Backlog
Development

No branches or pull requests

1 participant