Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add opfs driver #319

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat: add opfs driver #319

wants to merge 5 commits into from

Conversation

loilo
Copy link

@loilo loilo commented Oct 12, 2023

πŸ”— Linked issue

Resolves #231

❓ Type of change

  • πŸ“– Documentation (updates to the documentation, readme, or JSdoc annotations)
  • 🐞 Bug fix (a non-breaking change that fixes an issue)
  • πŸ‘Œ Enhancement (improving an existing functionality like performance)
  • ✨ New feature (a non-breaking change that adds functionality)
  • 🧹 Chore (updates to the build process or auxiliary tools and libraries)
  • ⚠️ Breaking change (fix or feature that would cause existing functionality to change)

πŸ“š Description

This PR adds a driver for the browser-based origin private file system, as requested in #231.

Strictly speaking, opfs is not an exactly accurate name as the driver actually supports any file system following the File System API. This PR sticks to the opfs naming for two reasons:

  1. At the time of writing, there are two browser APIs following the File System API: the File System Access API, which allows accessing working on a user's actual file system, and the origin private file system (OPFS).
    Of these, only the OPFS is supported across browser engines and is therefore the default file system used by this driver.
  2. To use a more accurate name for the driver, it should somehow refer to the "File System API". This is an incredibly ambiguous term, so something more clear like the OPFS should be preferred.

The driver and its tests are modeled quite closely after the fs-lite driver, especially with regards to the options and the tests.

Some key differences are:

The base Option

The base option is available, yet not required.

Other than with Node's fs module, it's safe for the File System API to operate on its root directory since it only exists in a sandboxed area of the device's file system.

While namespacing with base should be considered a best practice (which is why it's also used in the documentation example), requiring it can be a footgun, especially when working with alternative file systems.

The fs Option

As described above, this driver can work with file systems other than the OPFS.
Therefore, there is the additional fs option which can be passed an alternative FileSystemDirectoryHandle (or a Promise resolving to one).

One way to use this would be to work with the user's real file system in supported browsers.
A naΓ―ve way to use that option would look like this:

opfs({ fs: window.showDirectoryPicker() })

Of course, this code is silly as it has no checks for browser support or other safe guards, but technically, it works.

The readOnly and noClear Options

The readOnly and noClear options have been carried over from the fs/fs-lite drivers, but other than with those, they have also been added to the documentation.

I'm not sure whether they are currently missing from the Node FS driver docs or whether they've been omitted on purpose.

If so, please request a change to drop them from the OPFS documentation as well.

Different hasItem Behavior

The hasItem method returns false when the target path exists but is a directory.

To be fair, I consider this a bug in the fs/fs-lite drivers. Since a directory can not be accessed with getItem, it should not report to exist via hasItem either.

Substitutions for Node Built-ins

Since the driver should run in the browser, it uses naΓ―ve reimplementations of some node:path functions used in fs-lite. They are not as sophisticated as the ones provided by Node, but they should easily be up to the task of handling path-like keys while being a lot leaner in size.

πŸ“ Checklist

  • I have linked an issue or discussion.
  • I have updated the documentation accordingly.

@nuxt-studio
Copy link

nuxt-studio bot commented Oct 12, 2023

βœ… Live Preview ready!

Name Edit Preview Latest Commit
unstorage Edit on Studio β†—οΈŽ View Live Preview babcc16

@loilo
Copy link
Author

loilo commented Oct 13, 2023

An additional thought about naming: I explained my decision to stick with the opfs naming for this driver. However, I only considered the browser as an environment.

To this date, I have not had any involvement with any serverless environments, but I am aware that those are increasingly modeled after standardized Web APIs. So, somebody with more exposure to this topic should probably check if any established cloud provider uses the File System API in their services already. In that case, it may be worth reconsidering the naming of this driver (and possibly also stop defaulting to the OPFS and instead improve the description of the fs option in the docs with examples, so that users searching for "OPFS" also get pointed to the File System API driver).

EDIT: The more I think about this, the more I'm convinced that we probably should name this driver "File System API" or something similar, despite the ambiguities.
It seems way harder for new users to make sense of an OPFS-named driver that can be used for accessing any File System API compatible file system than to just point users wanting OPFS to a generic File System API driver.
Curious for feedback about this.

Copy link
Member

@pi0 pi0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments but really nice work ❀️ πŸ’―

src/drivers/opfs.ts Outdated Show resolved Hide resolved
src/drivers/opfs.ts Outdated Show resolved Hide resolved
src/drivers/utils/opfs-utils.ts Outdated Show resolved Hide resolved
src/drivers/utils/opfs-utils.ts Outdated Show resolved Hide resolved
@pi0
Copy link
Member

pi0 commented Oct 13, 2023

Re naming, i will be make some more research to give feedback on naming once back from holidays. But don't worry we can always iterate to introduce new aliases. Current name is good to me!

@loilo
Copy link
Author

loilo commented Oct 13, 2023

Alright, then let's stick with the name for now. I've dealt with the change requests, so let me know when there's anything else I can help with. Also, enjoy your holidays. πŸ™‚

@loilo
Copy link
Author

loilo commented Dec 1, 2023

Hey all. Is there anything I can help with to finish this one? :)

@ManUtopiK
Copy link

ManUtopiK commented Jan 19, 2024

Hi!
Thanks a lot for your driver!
It works very well, but I have an issue while keeping the handle after a page refresh.

First, I want to save the name of the directory chosen by the user. The only way to do that is to manage the showDirectoryPicker() outside the driver and pass the instance to fs. That works.
Now, I want to mount the opfs driver just after a page refresh. Normally, we can do that as long as the page session is active (like the sessionStorage).
Here is a part of my code :

async function requestNativeBrowserFSPermission(dirHandle: FileSystemDirectoryHandle) {
    const opts: any = {}
    opts.writable = true
    // For Chrome 86 and later...
    opts.mode = 'readwrite'
    const perms = await dirHandle.requestPermission(opts)
  
    return perms === 'granted'
}

let fs
if (source.base === 'directoryPicker') {
    fs = await window.showDirectoryPicker()
    if (!(await requestNativeBrowserFSPermission(fs))) return 
} else {
    const dir = await navigator.storage.getDirectory()
    fs = await dir.getDirectoryHandle(source.base, { create: true })
  
    if (!(await requestNativeBrowserFSPermission(fs))) return
}

storage.mount(`opfs:${fs.name}`, opfsDriver({ fs }))
return fs.name

The first time, source.base = 'directoryPicker', so it mounts the driver and ask to pick a directory.
After chosen a folder, I have to accept one browser popup to read files, and one more popup throw to ask for the permission to write files.
It will be nice to ask one time directly to readwrite.
So, after accepted the two browser popups, I can see the files. But if I refresh the page, the source.base is the name of the previous folder I've chosen (I keep it in the sessionStorage). It doesn't ask for permissions, and I can see that the driver is mounted, but nothing appear 😠
await storage.getKeys() return nothing. There's no error in the console.

I don't know where I'm wrong. Have you got an idea on this ?
Thx!

@pi0 pi0 added the driver label May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Origin Private File System (OPFS) API Support
3 participants