Skip to content

Lenient LZ4 check #298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

samikshya-db
Copy link

Currently, LZ4 compression is not used if module is not found (i..e, when lz4 module is not installed). At times customer is running databricks nodejs driver in Azure functions or a non-supported arch compute, in that case it throws

{...}/node_modules/@databricks/sql/dist/utils/lz4.js:10
            throw err;
            ^

Error: {...}/node_modules/lz4/build/Release/xxhash.node: invalid ELF header
    at Module._extensions..node (node:internal/modules/cjs/loader:1460:18)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.<anonymous> (/home/dejhjs3e/projects/databricks-node/node_modules/lz4/lib/utils.js:4:11)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12) {
  code: 'ERR_DLOPEN_FAILED'
}

Making the error handling more lenient inorder to exclude lz4 compression whenever a dependency error is thrown.

This fixes the problems stated in : #289 #275 #266 #270

@samikshya-db samikshya-db deployed to azure-prod May 14, 2025 11:35 — with GitHub Actions Active
@samikshya-db samikshya-db requested a review from shivam2680 May 14, 2025 11:36
Copy link
Contributor

@shivam2680 shivam2680 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you test this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants