Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(encryption): Implement e2e encryption using encryptedStorage composable #363

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

itpropro
Copy link
Member

@itpropro itpropro commented Jan 2, 2024

πŸ”— Linked issue

#24

❓ Type of change

  • πŸ“– Documentation (updates to the documentation, readme, or JSdoc annotations)
  • 🐞 Bug fix (a non-breaking change that fixes an issue)
  • πŸ‘Œ Enhancement (improving an existing functionality like performance)
  • ✨ New feature (a non-breaking change that adds functionality)
  • 🧹 Chore (updates to the build process or auxiliary tools and libraries)
  • ⚠️ Breaking change (fix or feature that would cause existing functionality to change)

πŸ“š Description

Implements a new composable function encryptedStorage similar to prefixStorage that wraps a unstorage instance and offers encryption for values and optionally for keys as well.

  • Adds new composable encryptedStorage that can also be used in combination with others like prefixStorage. Usage is encryptedStorage(createStorage({ driver }), encryptionKey, true) where the last parameter (set to true) defines if keys should also be encrypted
  • Uses @noble/ciphers for encryption, as it is 0 deps and works in Node, Bun, Deno and Workers (tested on CF). It supports AES-GCM-SIV (not implemented in Web/Node Crypto) which is nonce misuse resistant, what we need for deterministic key encryption with the same IV/nonce for keys.
  • For content encryption, XChaCha20-Poly1305 (which is also used in TLS 1.3) is used. It is a fast modern and future proof cipher and safe to use with random nonces. Nonces are generated with getRandomValues from uncrypto. The content is wrapped into a StorageValueEnvelope:
export interface StorageValueEnvelope {
  nonce: string;
  encryptedValue: string;
}
  • The implementation was tested against common drivers like fs, memory, redis, 'lrs' and azure-blob-storage for content as well as key encryption and in combination with prefixStorage:
    const storage = createStorage();
    const pStorage = prefixStorage(storage, "foo");
    const encStorage = encryptedStorage(pStorage, encryptionKey, true);
  • Tests include an encryption example for fs driver, an update to the testDriver utility to optionally test for content and key encryption as well es tests for the storage server and the combination with prefixStorage

πŸ“ Checklist

  • I have linked an issue or discussion.
  • I have updated the documentation accordingly.

@itpropro
Copy link
Member Author

itpropro commented Jan 2, 2024

Would appreciate your feedback @pi0, @danielroe, @Atinux.
If the implementation is fine for you, I would add an option to provide the encryption key via. env variable and write the docs.

Copy link

codecov bot commented Jan 2, 2024

Codecov Report

Attention: 28 lines in your changes are missing coverage. Please review.

Comparison is base (293a2a6) 75.32% compared to head (1c88d64) 76.08%.

Files Patch % Lines
src/utils.ts 83.97% 25 Missing ⚠️
src/drivers/encryption.ts 94.87% 2 Missing ⚠️
src/_utils.ts 98.92% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #363      +/-   ##
==========================================
+ Coverage   75.32%   76.08%   +0.76%     
==========================================
  Files          30       31       +1     
  Lines        3534     3822     +288     
  Branches      494      531      +37     
==========================================
+ Hits         2662     2908     +246     
- Misses        871      913      +42     
  Partials        1        1              

β˜” View full report in Codecov by Sentry.
πŸ“’ Have feedback on the report? Share it here.

@@ -5,7 +5,7 @@ import { BlobsServer } from "@netlify/blobs";
import { resolve } from "path";
import { rm, mkdir } from "node:fs/promises";

describe("drivers: netlify-blobs", async () => {
describe.skip("drivers: netlify-blobs", async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
describe.skip("drivers: netlify-blobs", async () => {
describe("drivers: netlify-blobs", async () => {

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That driver’s tests are extremely flaky, which is why I disabled them and forgot about it, thanks for the find :)
What do you think about the implementation @pi0 ?

@itpropro itpropro requested a review from pi0 April 7, 2024 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants