Motivation for this change

• Really obscure rust crash when compiling seemingly innocent library Really obscure rust crash when compiling seemingly innocent library #27370

And I believe this is also affecting

• librsvg: 2.40.18 -> 2.42.0 librsvg: 2.40.18 -> 2.42.0 #34083

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Fits CONTRIBUTING.md.

When running strip -S on the .rlib files produced as part of the rust build, the archive will become corrupted. In particular, the last entry will declare its size 2 bytes bigger than it actually is. When attempting to scan for the next header at lastHeader + size, the next header is located 2 bytes after the end of the archive, and hence the error message

offset to next archive member past the end of the archive

At a glance, the archive still looks coherent.

I wrote some throwaway code to introspect the archive, and learned a bit about what strip does to the archive structure. When strip processes the archive it rewrites all of the headers introducing additional padding to the names and sorts the symbol table.

Using llvm-ar as a test, one can quickly verify that strip -S causes the breakage:

# On the branch for this PR, in nix-shell -p llvm
$ std_rlib=$(nix-build ~/src/nixpkgs -A rustc)/lib/rustlib/x86_64-apple-darwin/lib/libstd-a5573c28baa2d270.rlib
$ llvm-ar t $std_rlib
std-a5573c28baa2d270.std0.rust-cgu.o
atomic.o
dwarf.o
fileline.o
posix.o
print.o
sort.o
state.o
backtrace.o
simple.o
macho.o
read.o
alloc.o
rust.metadata.bin
std-a5573c28baa2d270.std0.rust-cgu.bytecode.encoded


$ strip -S $std_rlib -o tmp.rlib
$ llvm-ar t tmp.rlib
std-a5573c28baa2d270.std0.rust-cgu.o
atomic.o
dwarf.o
fileline.o
posix.o
print.o
sort.o
state.o
backtrace.o
simple.o
macho.o
read.o
alloc.o
rust.metadata.bin
std-a5573c28baa2d270.std0.rust-cgu.bytecode.encoded
llvm-ar: truncated or malformed archive (offset to next archive member past the end of the archive after member std-a5573c28baa2d270.std0.rust-cgu.bytecode.encoded).

I don't know if it's valid to rewrite the contents of an .rlib file as if it were a regular archive using standard tools on any platform, since I'm assuming they're rust internal implementation detail. However I only disabled it on Darwin since that's where the problem is. Also I didn't see a way to exclude only .rlib files from the default stripping, which would be a more precise fix.

While we're here, when strip sorts the symbol table, it also includes some build-specific information that will make the build not bit-for-bit reproducible:

First header, not stripped:

            header = Container:
                header_offset = 8
                name = #1/12            (trailing, length: 12)
                date = 0            (total 12)
                uid = 0      (total 6)
                gid = 0      (total 6)
                mode = 0        (total 8)
                size = 109244     (total 10)
                fmag = 600a (total 4)
                optional_name = __.SYMDEF\x00\x00\x00 (total 12)

First header, stripped (from nixpkgs binary cache):

            header = Container:
                header_offset = 8
                name = #1/20            (trailing, length: 20)
                date = 1515850976   (total 12)
                uid = 30002  (total 6)
                gid = 30000  (total 6)
                mode = 100644   (total 8)
                size = 109252     (total 10)
                fmag = 600a (total 4)
                optional_name = __.SYMDEF SORTED\x00\x00\x00\x00 (total 20)