Automaticaly determine FS image size based on srcpath (or mountpoint) contents

[Harvie]

Sometimes i need to create image as small as possible.
In cases where i have userspace from buildroot and i don't know the size of that directory beforehand, i have to guess how small the build will be and adjust size accordingly in genimage config.

My proposal for this is following:

In addition to size = 32M, allow as well notation with plus sign, eg: size = +1M or size = +0
This would then look into source directory of filesystem image, eg. using du -s $srcpath | cut -f 1 to get the apparent size of the contents, then add specified margin to it (eg. +1M) and base filesystem size of it. Note that some minimal margin will probably be always necessary, because different filesystems allocate files differently, so when i build VFAT image from EXT4 srcpath, the files will likely take up different amount of space and therefore du will not be 100% accurate.

That way i can have optimaly sized image without having to worry that it will be unnecessarily big or that it will not fill all the files from srcpath.

Similar behaviour should occur when size is not specified at all.

[michaelolbrich]

We need to be careful when implementing this.

Calling du like this depends on the filesystem so you may get different result for the same files. du -s --apparent-size --block-size=1 will make sure that only the file content is counted, not any filesystem overhead. That should keep it deterministic.

The actual space used on the resulting filesystem will be bigger, because:

• directory entries need space as well
• each file in general use full blocks of some size, often 4k or something like that.
• extra space may be needed for a journal
  And probably some other overhead.

So calculating the exact space that will be needed is probably not feasible. So we will always need to add something. So this should always be explicit to say how much.

I've implemented something like that in PTXdist and use it to generate the genimage config. I'm using percentages here. Anything below 100% probably won't work. I mostly use 125% and that seems to work quite well.

I prefer percentages because it scales better but supporting +nM as well should be trivial.

As this is already implemented in PTXdist and that's my main personal use-case, this is not very high on my priority list, so contributions are welcome.

[Harvie]

Another way might be to create the filesystem, populate it and then shrink it :-) However i have a feeling that this might be painful due to differences between filesystems and their semi-broken resizing code.

So we will always need to add something.

That's what i've said :-)

already implemented in PTXdist

Never heard about that. Does it provide some kind of genimage alternative with more features?

I prefer percentages because it scales better

Depends on use case... Lets say that i want some safety margin to accomodate all files. Then percentage is probably way to go. But when i need some additional space because i know i will want to add some static files later or perhaps user will sometimes need to install few packages with given size (eg. the GDB debuger which i don't ship by default). In that case it makes sense to always leave space for debugger without being relative to the size of the image.

[michaelolbrich]

already implemented in PTXdist

Never heard about that. Does it provide some kind of genimage alternative with more features?

It's something like buildroot. It calculates the size and writes it into the config before calling genimage.

I prefer percentages because it scales better

Depends on use case... Lets say that i want some safety margin to accomodate all files. Then percentage is probably way to go. But when i need some additional space because i know i will want to add some static files later or perhaps user will sometimes need to install few packages with given size (eg. the GDB debuger which i don't ship by default). In that case it makes sense to always leave space for debugger without being relative to the size of the image.

Right. So both would be nice. As I said, that's the trivial part.