v0.11.123

Changelog

• 695c0df add parallel testing
• e52ab45 add state comparison to formatAsset
• 1077a63 added logic in a seperate function
• 0626dff changed flag name
• ec767d3 Add benchmarks for internal command parsing and linting
• 77680e7 Added benchmarks
• 1ddd83d Ensure golang.org/x/net is updated to v0.33.0
• eca7d74 Merge branch 'main' into benchmarks
• e1da266 Merge branch 'main' into mvp-2
• 1a9591f Merge branch 'main' into mvp-2
• 971a1e1 Merge pull request #336 from bruin-data/fail-if-changed
• 2ef3c78 Merge pull request #341 from bruin-data/mvp-2
• 8e3630b Merge pull request #343 from bruin-data/benchmarks
• 9005e3d Merge pull request #344 from bruin-data/update-dependencies
• 40317fb Merge pull request #345 from bruin-data/asana-source/BRU-1282
• 276a2aa Merge pull request #346 from bruin-data/dynamodb-source/BRU-1283
• ce260ca Refactor RestoreState logic and improve error handling in scheduler
• be0119c Refactor Run command and enhance state management
• c5d7ea0 Refactor benchmark timing in lint_test.go to improve accuracy and readability
• 5671ae0 Refactor state management in the Run command and enhance test coverage
• 2c6230b Update benchmark iterations in internal_test.go and lint_test.go to a fixed count of 20 for consistency and improved performance testing
• 1de7637 Update build-test.yml
• 60c293f Update build-test.yml
• 471ae9d Update build-test.yml
• 02c4400 add asana config package
• 9904dc9 add dynamodb client
• adf1c20 add failure
• 4fc02c0 add json success message
• 6af1124 add logic for single asset
• cd6de78 add normalization
• 7c1b3eb add path for windows
• 8070f5a add success
• ae51278 add tests
• 169b8bd added benchmarking in ci
• 35084a0 added integration tests
• f9f0632 added restore logic for flags and input
• 25882d8 added state restore logic
• 53f401e added unit test
• 9a9c3d7 change helper function name
• 73b4678 config manager: add dynamodb connection
• debdf61 config manager: add tests for dynamodb
• e07c993 config: add asana config management
• 516cce0 config: add tests for asana config
• 108b83f connection manager: add connection handler for dynamodb
• c86424d connection: add dynamodb
• e115652 connections: add asana
• b09c605 connections: add dynamodb
• c4c6ab8 delete concurrency in test
• e30ee78 delete unbroken yml test case
• e10e536 examples: simple-pipeline: add asana
• 72954a5 fail-if-changed
• 22ab5e9 fix ci issue
• 782d2e9 fix for windows
• 09a247f fix integration test
• 2323926 fix lint
• 5621ace fix lint
• 47ab1c2 fix lint
• 9e2b00b fix lint
• af43788 fix panic
• 04b6500 fix state dir
• 76c392b fix tests
• defd52f fix unit test
• f66dbcf fix variable names
• 9ab03ab fix: broken testdata
• 1d5cccb handle cases where service account file is not given
• dc2fb81 integration tests: add dynamodb schema
• 3dd936f integration tests: fix schema mismatch
• be4982c lint: disable godox
• 475683a lint: spellfix
• 9170adc logic for multiple assets
• 0557836 misc: bump ingestr version to v0.12.2
• f624fe6 more
• 3b65c71 more
• 7ca6db1 more changes
• 05452a5 more changes
• e0d417a more changes
• dbc4c79 more changes
• 4a17e9d more changes
• 64d36cc more changes
• acec139 more changes
• c939bde more changes
• 543aecb pipeline: add default mapping for asana
• 30fa2d2 pipeline: add dynamodb mapping
• 72d73bd publish schedule type
• a3092fa refactor
• d652f44 refactor
• a712735 refactor json message
• e6be31c revert github action
• 4bbde71 test fail if they take more then 100ms
• be4dd25 tests: fix manager tests on windows
• 7815a15 update flag info

v0.11.122

Changelog

• f2ef55a added integration test for metadata push
• fb38af5 change header
• 71742c3 make pipeline smaller
• f788823 Add VSCode configuration files for Go and Python debugging
• cad85df Add custom check validation for empty queries
• e5e9528 Add duplicate column checks validation rule
• d697e32 Add initial state package with State, Metadata, and SchedulerState types
• 562ce07 Enhance state package by introducing Status constants and updating SchedulerState structure
• d4ebbe7 Fix import statement in main_test.py to use relative import
• cdd04a1 Implement concurrency control in State management
• 3e2e488 Merge branch 'main' into duplicate-name-check
• 87c0488 Merge branch 'main' into duplicate-name-check
• 49276fa Merge branch 'main' into fix/python-test
• 294b250 Merge branch 'main' into fix/python-test-import
• 7fd64a5 Merge branch 'main' into state
• 5b203ef Merge pull request #324 from bruin-data/add-destination-docs
• 8c422ba Merge pull request #326 from bruin-data/shopify-bq-pipeline
• 2a185d8 Merge pull request #327 from bruin-data/duplicate-name-check
• a79d537 Merge pull request #328 from bruin-data/change-integration-test
• b8e1a06 Merge pull request #329 from bruin-data/metadata-integration
• 81b8662 Merge pull request #330 from bruin-data/relative-path
• f63c0d9 Merge pull request #331 from bruin-data/fix/python-test-import
• 55c830e Merge pull request #332 from bruin-data/fix/python-test
• 14ba0e7 Merge pull request #333 from bruin-data/state
• ffdc953 Merge pull request #334 from bruin-data/security-updTE
• 7e0ea38 Merge pull request #335 from bruin-data/fix-dilect
• a1659dd Merge pull request #337 from bruin-data/doc-minor-changes
• fa52631 Merge pull request #340 from bruin-data/patch/add-missing-ssl-params-for-mysql
• 003cc04 Refactor import statement in main_test.py to use relative import
• 240b10e Refactor state package to enhance state management and JSON serialization
• 0958e50 Refactor version management
• 2f5698d Update dialect.go
• fd0743b abs path for gcp
• fe0f971 add bigquery pipeline to integration-tests
• 5e5ef9f add link to docs for installation
• dad9b46 add missing ssl params for mysql connection strings
• 29e783c added docs for destinations
• 04ffb09 added param
• d2d9d02 change demo gif
• 6a3b19e change intro
• cea08f7 change slack button alignment
• bc0b341 change slack location
• 1cb7931 delete expectQuery
• 578f969 differentiate test cases for unix and windows
• dda5d53 fix integration tests locally
• ef1b26d fix json
• ecbdeaf fix lint
• 6598728 fix path
• 90aebb5 fix path
• d1501b2 fix pipeline
• f348549 fix readme links
• 315d2a0 fix test
• aacab2a fix: correct filename in assetTypeDialectMap from "mysql.sql" to "mssql.sql"
• 6dd142d fix: upgrade golang.org/x/crypto to v0.31.0 to resolve security vulnerability
• aed9567 go mod tidy
• 1cf45a7 go mod tidy
• 9fb9295 make lint
• 499ce4e more changes
• d6d815e move parameters to top
• df6d747 remove json
• 2f84880 remove wget
• b291d8f rename-pipeline
• 1403519 revert
• 9fd53ed revert dep
• d2341cb revert dep
• 4483efc select environment when testing connections
• 864abaf small change
• a7cd4b0 undo
• ec160f7 update docs for the new mysql ssl variables
• 61d3680 update expected connection schema
• 09b5fc7 update shopify pipeline

v0.11.121

Changelog

• 8fafa35 doc updates
• 9e6992e Enhance LineageExtractor to track processed assets
• 425a25d Fix Column Lineage Processing in LineageExtractor to Use Asset Name as Key
• ab13338 Merge branch 'main' into patch/speedup-lineage-v2
• 78fecc6 Merge pull request #319 from bruin-data/speedup/lineage
• 2dd2add Merge pull request #320 from Barbar1432/doc-updates
• a09a9de Merge pull request #321 from bruin-data/patch/speedup-lineage-v2
• 474bb04 Merge pull request #322 from bruin-data/hotfix/patch-lineage
• fcb989e Merge pull request #323 from Barbar1432/metadata-push
• 61a2a62 Refactor Column Lineage Processing to Use Processed Assets Map
• d46e7b4 Remove redundant logging in ColumnLineage method of LineageExtractor
• 87ab821 fix bug
• d8368ad fix lint
• 9049bb6 fix styling
• 7f44ae7 send only the necessary metadata
• 9b1b38d send only the necessary metadata to the parser
• 48c36d1 skip non-asset dependencies

v0.11.120

Changelog

• e34c4e0 irrelevant yml files ignored
• 05e3b05 Merge pull request #316 from bruin-data/hotfix/lineage-column-py
• 15929bc Merge pull request #317 from Barbar1432/pipeline-yaml-bug
• 24e88ba Merge pull request #318 from bruin-data/hotfix/revert
• 95bb90f Refactor schema optimization in query parsing
• 2224eff make format
• 4b5dfe8 revert changes
• 6e7aa2f revert column lineage

v0.11.119

Changelog

• f222844 Merge pull request #315 from bruin-data/patch/fix-nested-table-reference-parsing
• 88a3f2a fix nested reference parsing by building schema properly

v0.11.118

Changelog

• 0bb149b added new int test pipeline
• a814c3b fix for mssql
• 47d6c3a Enhance asset expectations by adding 'columns' field to upstreams and updating file paths to absolute locations. This change ensures better clarity in asset dependencies and improves the handling of upstream columns across multiple expectation files.
• 93e2ee4 Enhance lineage processing by removing asset references in expectations and improving column name handling; add asset content wiping in command execution
• 8dd1e74 Ensure 'upstreams' is initialized as an empty array in JSON pipeline files
• d5a7d93 Merge branch 'main' into finetune/column-merge
• 216d01c Merge branch 'main' into finetune/column-merge
• d12141f Merge branch 'main' into fix-empty-lineage
• 90a68b3 Merge branch 'main' into patch/set-upstreams-to-be-empty-array-by-default
• 636cb58 Merge pull request #310 from Barbar1432/materialization-bug
• c5aef48 Merge pull request #311 from bruin-data/finetune/column-merge
• 7c97ea3 Merge pull request #312 from Barbar1432/integration-test-new
• aa02d64 Merge pull request #313 from bruin-data/fix-empty-lineage
• 837263c Merge pull request #314 from bruin-data/patch/set-upstreams-to-be-empty-array-by-default
• 8c9a893 Refactor Upstream structure to use pointer for Columns; update lineage processing to handle upstream columns correctly and initialize empty upstreams in assets
• 9ac8cdf Refactor column upstream handling in tests and lineage extraction
• ab18ad8 Refactor lineage extraction logic and improve column name comparison
• 266cfbe Refactor upstream column handling in lineage tests and extraction logic; remove asset references for cleaner structure
• 74f2d15 Update ColumnLineage struct to require upstream columns in JSON serialization
• 5e454e4 Update Upstream struct to mark 'columns' field as optional in JSON/YAML serialization.
• 434e3c1 Update expectations to include 'upstreams' as an empty array in JSON files
• 677f812 add new cases
• 0327c23 add tests
• dabb3a8 added new integration test
• 3254b0a added test cases
• 5b81bc6 fix for athena
• eee7727 fix for databricks
• bbcced9 fix for duckdb
• 0653958 fix for postgres
• 9c3264d fix tests
• d23febe fix tests
• 0b73ec0 fix windows test
• d4c890b fix-lint
• f8d6d8d more changes
• 695cf92 newline before semicolon
• 101d598 revert
• e136fa4 revert
• 10211d1 revert
• 6a10e1a revert
• 06cef6c revert
• d7f97ef revert
• d1c1341 set upstreams to be an empty array by default

v0.11.117

Changelog

• ed6264e fix table name

v0.11.116

Changelog

• 4da7299 Enhance table and column extraction by including database context in names
• 2c90b9f Merge branch 'main' into sparkhealth/bugs
• 4cd5c15 Merge branch 'main' into sparkhealth/bugs
• ece24e2 Merge pull request #306 from bruin-data/sparkhealth/bugs
• 574bd80 Refactor column and table handling in parser
• be0e3d0 amend tests
• bcd097c lint
• a1d9186 more changes
• 26d7b8f more changes
• af39cbf more changes
• 5c15e97 neat: reafctor
• ae7af08 pipeline: added unit test
• 95d9ee3 pipeline: more changes
• c0b7624 pipeline: more changes
• 4dc0680 update query for new ingestr

v0.11.115

Changelog

• 572b276 bump ingestr
• 3513450 wip: improve demo tape

v0.11.114

Changelog

• 2256f0d mark not running tasks as skipped instead of succeed
• 4761b8f Merge pull request #297 from bruin-data/patch/avoid-compile
• 4ce2b02 Merge pull request #309 from Barbar1432/log-bug
• 3d0a593 Refactor lineage extraction for stateless (#308)
• 64bedbb add dot
• 5352757 avoid compiling every time
• 2a4da04 ci: fix windows e2e test failing
• dfdc43d copy file
• 48e67a3 downstream log bug
• 49c567f fix by OS
• 12e7f1b fix lint
• afb26ff integration tests: fix formatting
• 89718aa rename function