Sync from soda-core #8
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
updates: - [github.com/psf/black: 23.7.0 → 23.9.1](psf/black@23.7.0...23.9.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Vijay Kiran <[email protected]>
* Initial commit with csv tests
* Added Json tests
* Added support for csv, json and parquet file for
duckdb.
Changes to be committed:
modified: soda/duckdb/soda/data_sources/duckdb_data_source.py
modified: soda/duckdb/tests/test_duckdb.py
* Removing unused import
Changes to be committed:
modified: soda/duckdb/tests/test_duckdb.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update soda/duckdb/tests/test_duckdb.py
Removed unused f-string
Co-authored-by: Milan Lukac <[email protected]>
* Update soda/duckdb/tests/test_duckdb.py
Removed unused f-string
Co-authored-by: Milan Lukac <[email protected]>
* Update soda/duckdb/tests/test_duckdb.py
Removed unused f-string
Co-authored-by: Milan Lukac <[email protected]>
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Milan Lukac <[email protected]>
Co-authored-by: data-fool <[email protected]>
* fix: compute value counts in DB rather than in python for categoric distribution checks (#116) * get value counts from db for categoric variables * add/modify unit tests for distro checks * fix unit tests * fix sqlserver limit query * fix db specific unit tests for distro check * fix unit tests and change pr feedback * do not evaluate the check if there are more than 1M groups * fix sqlserver * remove sampling from categoric variables for distro checks * add query logs for distro check * use logs.debug to show the running query in verbose mode rather than using logs.info * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: Add Dask/Pandas data source naming support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add missing method * fix indent * Fix test * remove unused feature test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add SparkDF support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Freshness: support mixed thresholds * fix precommit
Co-authored-by: Milan Lukac <[email protected]>
…1959) * Sqlserver: support quoting tables with whitespace * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support [] as quotes * Fix parser * Use upper case info schema * Remove default casify sql function * Update quoting * Fix quoting * Strip quotes from all payloads using table name * Change how bracket quotes work * Remove schema identifier lowercasing * Fix parser --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Initial contracts transformation proposal * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added schema check capability matching columns * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixing the schema mismatching columns test * Improved test suite and initial docs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added contract docs and missing check configurations * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added reference property to contracts * Added extra comment in programmatic docs and added test to ignore other keys * added setup.py * Preparations for contracts release * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated contracts package version to 3.0.54 * Fix imports * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix CI * Fix CI --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Milan Lukac <[email protected]>
updates: - [github.com/asottile/pyupgrade: v3.16.0 → v3.17.0](asottile/pyupgrade@v3.16.0...v3.17.0) - [github.com/psf/black: 24.4.2 → 24.8.0](psf/black@24.4.2...24.8.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Fixing the correlation issue by syncing on database and schema
This reverts commit ec42e43.
Fixed the Atlan integration glue db-schema switch
* Fixing the lacking data source error message on contract build * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fixing Spark session API * Fixed duckdb version to 1.0.0
* Removing the data source name lower case requirement * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Update README.md to include Job Description * Add source
…upport for SQL Server (#2160) * working fabric data source inheriting from sqlserver * fix failing tests * fix table creation in fabric * restore dev-reqs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add email check for sqlserver and fabric * add test for email format * remove useless line * remove useless line * remove extra deps * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * enable auth with mssparkutils * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add fabric spark auth * Update tbump+version --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Milan Lukac <[email protected]> Co-authored-by: Milan Lukac <[email protected]>
updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/asottile/pyupgrade: v3.17.0 → v3.18.0](asottile/pyupgrade@v3.17.0...v3.18.0) - [github.com/psf/black: 24.8.0 → 24.10.0](psf/black@24.8.0...24.10.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Milan Lukac <[email protected]>
* Comparison row count check secondary datasource filter fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Milan Lukac <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.