Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Userobj, groupobj and project quotas #727

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Userobj, groupobj and project quotas #727

wants to merge 2 commits into from

Conversation

lundman
Copy link
Contributor

@lundman lundman commented Aug 2, 2019

This includes the commit:
Add types to featureflags in zfs
as well as commits for userobj quotas, and project quotas, all in one PR.
They are rather intertwined.

This is a port from HEAD on ZOL master today - not the original commits (plus updates). Some git history is lost.

Userobj, and Groupobj quotas appear to work:

tank  feature@userobj_accounting     active                         local
tank  feature@project_quota          active                         local

# zfs set userobjquota@lundman=100 tank
# zfs get userobjquota@lundman tank
NAME  PROPERTY              VALUE                 SOURCE
tank  userobjquota@lundman  100                   local

# for i in {1..200}; do touch /Volumes/tank/file$i; done
touch: /Volumes/tank/file101: Disc quota exceeded

#  zfs get userobjused@lundman tank
NAME  PROPERTY             VALUE                SOURCE
tank  userobjused@lundman  50K                  local

(Why is it 50k?! I expect to see 100 - possibly bug in zfs_main.c - research)

# zfs set groupobjquota@staff=100 tank
# zfs get groupobjquota@staff tank
NAME  PROPERTY             VALUE                SOURCE
tank  groupobjquota@staff  100                  local
  • Further testing of userobj, and groupobj quotas required.

  • Further testing with project quotas required.

  • The use of zfs set|get project*quota | project*used should at least work.
  • zfs project command should run, but we expect Inappropriate ioctl for device
    as there are no Projects in XNU.

@lundman
Copy link
Contributor Author

lundman commented Aug 7, 2019

Ah hmm it seems the ret = ioctl(fd, ZFS_IOC_FSGETXATTR, &fsx); is an internal ioctl, so doesn't involve some feature of Linux kernel, and should be implemented.

@JMoVS
Copy link
Contributor

JMoVS commented Feb 13, 2020

Ah hmm it seems the ret = ioctl(fd, ZFS_IOC_FSGETXATTR, &fsx); is an internal ioctl, so doesn't involve some feature of Linux kernel, and should be implemented.

"should be implenented" - is this ready for testing then or should it be implenented first before it gets to testing?

The boolean featureflags in use thus far in ZFS are extremely useful,
but because they take advantage of the zap layer, more interesting data
than just a true/false value can be stored in a featureflag. In redacted
send/receive, this is used to store the list of redaction snapshots for
a redacted dataset.

This change adds the ability for ZFS to store types other than a boolean
in a featureflag. The only other implemented type is a uint64_t array.
It also modifies the interfaces around dataset features to accomodate
the new capabilities, and adds a few new functions to increase
encapsulation.

This functionality will be used by the Redacted Send/Receive feature.

Reviewed-by: Matthew Ahrens <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Paul Dagnelie <[email protected]>
Closes #7981
@lundman lundman force-pushed the userobj_quota branch 2 times, most recently from 6a2a0c4 to c488622 Compare February 14, 2020 01:59
Change vnode ioctl to fcntl

XNU will deny ioctl to any vnode that is not BLK or CHR. It is
expected that fcntl() is used in this case. Change "zfs project" to
use fcntl to query projid.

Potentially there is no way to set project, without an equivalent
tool to Linux "chattr". But pool import should at least work, as
well as quota restrictions.
@yurikoles
Copy link

yurikoles commented Jun 23, 2020

I haven't tested much new functionality of this PR, I just use this branch with my ZFS pool created with current ZoL 0.8.4 without any features disabled by command from OpenZFS wiki on Debian root on ZFS :

zpool create \
    -o ashift=12 \
    -O acltype=posixacl -O canmount=off -O compression=lz4 \
    -O dnodesize=auto -O normalization=formD -O relatime=on \
    -O xattr=sa -O mountpoint=/ -R /mnt \
    rpool ${DISK}-part4

I saw two freezes today when I try to zfs destroy -r <dataset>. This dataset is created from macOS and don't have any features of this PR enabled. Here is its properties:

zfs get -r all rpool/work/gapps
NAME              PROPERTY               VALUE                      SOURCE
rpool/work/gapps  type                   filesystem                 -
rpool/work/gapps  creation               нд чер 21 19:43 2020  -
rpool/work/gapps  used                   29,4G                      -
rpool/work/gapps  available              485G                       -
rpool/work/gapps  referenced             29,4G                      -
rpool/work/gapps  compressratio          1.23x                      -
rpool/work/gapps  mounted                yes                        -
rpool/work/gapps  quota                  none                       default
rpool/work/gapps  reservation            none                       default
rpool/work/gapps  recordsize             128K                       default
rpool/work/gapps  mountpoint             /Volumes/work/gapps        inherited from rpool/work
rpool/work/gapps  sharenfs               off                        default
rpool/work/gapps  checksum               on                         default
rpool/work/gapps  compression            lz4                        inherited from rpool
rpool/work/gapps  atime                  on                         default
rpool/work/gapps  devices                on                         default
rpool/work/gapps  exec                   on                         default
rpool/work/gapps  setuid                 on                         default
rpool/work/gapps  readonly               off                        default
rpool/work/gapps  zoned                  off                        default
rpool/work/gapps  snapdir                hidden                     default
rpool/work/gapps  aclmode                passthrough                default
rpool/work/gapps  aclinherit             restricted                 default
rpool/work/gapps  canmount               on                         default
rpool/work/gapps  xattr                  sa                         inherited from rpool
rpool/work/gapps  copies                 1                          default
rpool/work/gapps  version                5                          -
rpool/work/gapps  utf8only               on                         -
rpool/work/gapps  normalization          formD                      -
rpool/work/gapps  casesensitivity        sensitive                  -
rpool/work/gapps  vscan                  off                        default
rpool/work/gapps  nbmand                 off                        default
rpool/work/gapps  sharesmb               off                        default
rpool/work/gapps  refquota               none                       default
rpool/work/gapps  refreservation         none                       default
rpool/work/gapps  primarycache           all                        default
rpool/work/gapps  secondarycache         all                        default
rpool/work/gapps  usedbysnapshots        0                          -
rpool/work/gapps  usedbydataset          29,4G                      -
rpool/work/gapps  usedbychildren         0                          -
rpool/work/gapps  usedbyrefreservation   0                          -
rpool/work/gapps  logbias                latency                    default
rpool/work/gapps  dedup                  off                        default
rpool/work/gapps  mlslabel               none                       default
rpool/work/gapps  sync                   standard                   default
rpool/work/gapps  dnodesize              auto                       inherited from rpool
rpool/work/gapps  refcompressratio       1.23x                      -
rpool/work/gapps  written                29,4G                      -
rpool/work/gapps  logicalused            36,2G                      -
rpool/work/gapps  logicalreferenced      36,2G                      -
rpool/work/gapps  filesystem_limit       none                       default
rpool/work/gapps  snapshot_limit         none                       default
rpool/work/gapps  filesystem_count       none                       default
rpool/work/gapps  snapshot_count         none                       default
rpool/work/gapps  snapdev                hidden                     default
rpool/work/gapps  com.apple.browse       on                         default
rpool/work/gapps  com.apple.ignoreowner  off                        default
rpool/work/gapps  com.apple.mimic_hfs    off                        default
rpool/work/gapps  com.apple.devdisk      poolonly                   default
rpool/work/gapps  shareafp               off                        default
rpool/work/gapps  redundant_metadata     all                        default
rpool/work/gapps  overlay                off                        default
rpool/work/gapps  encryption             off                        default
rpool/work/gapps  keylocation            none                       default
rpool/work/gapps  keyformat              none                       default
rpool/work/gapps  pbkdf2iters            0                          default
rpool/work/gapps  special_small_blocks   0                          default

It contains only clone of opengapps repo in a subfolder and has mds disabled by file .metadata_never_index and removing corresponding mds directory. That's all I can say about it. I haven't found any mention of zfs in my standard macOS logs GUI.

@yurikoles
Copy link

Oh sorry, not resets, but complete freezes, so GUI is there, but no clock ticks (I have seconds enabled in my clock).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants