Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/make-ext4-fs: increase fudge factor from 1.03 to 1.10 #91214

Merged
merged 1 commit into from Jun 21, 2020

Conversation

flokli
Copy link
Contributor

@flokli flokli commented Jun 21, 2020

Turns out, on smaller images (~800MiB uncompressed sdcard image size),
the current fudge factor is way too small to even get the system to the
phase where it can resize itself.

I first tried with 1.05, but it wasn't enough.

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

cc @NixOS/exotic-platform-maintainers

Turns out, on smaller images (~800MiB uncompressed sdcard image size),
the current fudge factor is way too small to even get the system to the
phase where it can resize itself.

I first tried with 1.05, but it wasn't enough.
@matthewbauer
Copy link
Member

I’ve noticed this too. The smaller the system is, the larger fudge factor you need.

I wish we could get rid of the fudge factor altogether. I think it might be better as a constant reserved space instead of a multiplier.

/cc @samueldr

@flokli
Copy link
Contributor Author

flokli commented Jun 21, 2020

We probably need a fixed offset, and a factor - but I assume it also depends on the type/amount of files you add. It's a bummer we can't just ask for the real size it needs to get some space.

I'm not really in favour of making the formula more complicated, by adding more empirically determined numbers, but I'm also not sure if there's any upper limits we're almost hitting, for which we need to be that conservative.

@samueldr
Copy link
Member

I believe the actual solution would be to make use of a tool that is made to create a filesystem with a tool that is better suited for the task of making a filesystem with predefined contents. make_ext4fs is such a tool. It is recommended by the reproducible-builds project to make filesystem images.

This, in turn, does not need fudge factor. It knows about ext4 in intricate levels enough that it builds the filesystem from the inside out, finishing with the structures, leaving enough free space so things just work.

(If you're a specific individual whom I shared a fudge factor thing with yesterday, using make_ext4, I reviewed the comments in the implementation and it's only relevant when computing available space from a partition size.)


So, for this PR, I 👍 it, it's okay for now to bump up the fudge factor, but anything more involved should prefer switching to a tool made for the task, which in addition brings us reproducible filesystem images for free.

@flokli flokli merged commit 44d75ef into NixOS:master Jun 21, 2020
@flokli flokli deleted the make-ext4-fs-fudge-factor branch June 21, 2020 20:13
flokli added a commit to flokli/pynq that referenced this pull request Dec 5, 2022
This contains two more PRs currently staged for nixpkgs:

 - nixos/make-ext4-fs: increase fudge factor from 1.03 to 1.10
   NixOS/nixpkgs#91214
 - extlinux-conf-builder: allow a custom FDT to be specified
   NixOS/nixpkgs#91195

The former one is needed to have the initial sd image be big enough to
be able to resize itself, the latter to circumvent u-boot not properly
detecting our board, and trying to load the wrong FDT.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants