New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use zstd for ISO and SD images #85947
Conversation
I compressed a 2.7 GB aarch64 NixOS image with the following results:
It seems that zstd is even in the Debian stable, so this change is worth considering. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll add this to further drive the point of zstd
being the future:
https://www.archlinux.org/news/now-using-zstandard-instead-of-xz-for-package-compression/
The future is now!
I think you forgot the module arg:
|
@@ -277,6 +277,9 @@ environment.systemPackages = [ | |||
<title>Other Notable Changes</title> | |||
|
|||
<itemizedlist> | |||
<listitem> | |||
<para>SD and ISO images are now compressed by default using <literal>zstd</literal>.</para> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is wrong, only the SD image is now compressed by default using zstd
.
The ISO image needs to be configured to be compressed to be compressed, and then it would now be zstd
.
±0 for the SD image compression scheme. Hopefully no one is in a situation where using Now, I misunderstood the change when I was reading the code, and was building in parallel. The following was written under the assumption that the isos were now default to being compressed. This should now be read as a warning for someone that wants to change the -1 for the ISO image being compressed with Compressing the iso image needs changes at other places in the NixOS infrastructure, and documentation. I'm not sure what will happen with the mirror scripts, reading them makes me think it should end up copying the right thing. The website would need changes. The docs would need to be adjusted. And, furthermore, the iso is already compressed using squashfs. Sure, it's not as compressed as this fancy currently in vogue compression, but it is compressed already. The squashfs settings maybe can be looked at to change the compression settings, though. Note that I have no issues with |
@prusnak I wonder wether 8a67595 is necessary. The cost here is that this broke cross building of NixOS systems for me, as I guess it is needed at runtime of the system to decrypt the install ISO/image? Couldn't we include it conditionally in
like {
environment.systemPackages = lib.optional config.isoImage.compressImage [ pkgs.zstd ];
} ? |
@erictapen No strong opinion about 8a67595. If you think zstd does not belong to a base system, open a PR with a revert. The cross-compilation of zstd was fixed in #91177 |
Thanks for the link to the fix, missed that. I just realized that gzip, xz, bzip2 are all part of the base system, so maybe it makes sense to have zstd as well. Sorry for bothering you. |
Motivation for this change
While experimenting with sd-image builds I found out, that zstd is much superior to bzip2 when compressing SD/ISO images (95% reduction of time needed).
Because compression used to compress the distribution images is considered a public interface, I'd like to discuss this in the PR.
Things done
sandbox
innix.conf
on non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
./result/bin/
)nix path-info -S
before and after)