Thank you for your comments. On top of what you pointed out I realized that this code only goes 1 level deep for dependencies (oops). I rewrote the function after learning a few things and hope this is a better solution.

Here is a faster and more error-proof version of your function:

{
  makeFullPerlPath = deps: let
    getPropagated = pkg: if pkg == null then [] else
      [ pkg ] ++ concatMap getPropagated pkg.propagatedBuildInputs;
    closure = unique (concatMap getPropagated deps);
  in makePerlPath closure;
}

Notably it doesn't add everything twice (your ++ dep.propagatedBuildInputs) and it doesn't require flatten (I used concatMap instead).

However, a more powerful version of this function is actually already implemented in nixpkgs! Notably closePropagation defined in deprecated.nix (it should probably be moved out of that file, ping @Profpatsch @nbp):

With some quick testing, closePropagation is faster and more memory efficient than the function I defined above. Here are the details of my tests:

with import <nixpkgs> {};
with lib;
rec {


  makeFullPerlPath = deps: let
    getPropagated = pkg: if pkg == null then [] else
      [ pkg ] ++ concatMap getPropagated pkg.propagatedBuildInputs;
    closure = unique (concatMap getPropagated deps);
  in makePerlPath closure;

  makeFullPerlPath' = deps: makePerlPath (closePropagation deps);

  deps = with perlPackages; [
    AnyEventHTTP AnyEventRabbitMQ XMLSAXWriter X11XCB Appperlbrew
    CacheMemory CatalystAuthenticationStoreHtpasswd AlienWxWidgets
    ack AppCLI AppCmd FileTouch PerlIOeol pcscperl
  ];

  count = length (closePropagation deps);

  simple = makeFullPerlPath deps;
  closeProp = makeFullPerlPath' deps;

}

$ nix-instantiate --eval -A count
190
$ NIX_SHOW_STATS=1 nix-instantiate --eval -A simple
# Results below
$ NIX_SHOW_STATS=1 nix-instantiate --eval -A closeProp
# Results below

simple                                          closeProp

time elapsed: 0.482657                          time elapsed: 0.380315
size of a value: 24                             size of a value: 24
size of an attr: 24                             size of an attr: 24
environments allocated count: 658868            environments allocated count: 321723
environments allocated bytes: 17552664          environments allocated bytes: 9486104
list elements count: 649415                     list elements count: 687801
list elements bytes: 5195320                    list elements bytes: 5502408
list concatenations: 20738                      list concatenations: 25783
values allocated count: 1911160                 values allocated count: 870080
values allocated bytes: 45867840                values allocated bytes: 20881920
sets allocated: 89528 (32138080 bytes)          sets allocated: 89721 (32149632 bytes)
right-biased unions: 32350                      right-biased unions: 32350
values copied in right-biased unions: 965686    values copied in right-biased unions: 965686
symbols in symbol table: 49768                  symbols in symbol table: 49817
size of symbol table: 1212668                   size of symbol table: 1213172
number of thunks: 890635                        number of thunks: 636379
number of thunks avoided: 810321                number of thunks avoided: 554145
number of attr lookups: 236336                  number of attr lookups: 235995
number of primop calls: 524411                  number of primop calls: 266277
number of function calls: 619451                number of function calls: 279759
total allocations: 100753904 bytes              total allocations: 68020064 bytes
current Boehm heap size: 402915328 bytes        current Boehm heap size: 402915328 bytes
total Boehm heap allocations: 126684800 bytes   total Boehm heap allocations: 83332928 bytes

In short: For 190 dependencies, closePropagation is about 20% faster and needs about 35% less memory. And this will be better the more dependencies there are.

So this function should really be implemented with a simple

{
  makeFullPerlPath = deps: makePerlPath (closePropagation deps);
}

@infinisil Thank you, that is fantastic code.
@Profpatsch @nbp Which version of the code by @infinisil would you like me to adapt?

First of all, great contribution.

I’d wager the more general function, closePropagation is in deprecated.nix for a reason. I cannot find a comment or release note describing said reason, though. My guess is that it doesn’t play well with stuff like cross-compilation (cc @Ericson2314). We should document deprecation better.

I’d wait with introducing this until we know why closePropagation is deprecated in the first place. Maybe also cc @aszlig?

The deprecated.nix file got introduced as the result of the renaming of misc.nix 3 years ago (commit 0ae8b36). And misc.nix got created as a fact that it does not belong to any named category (commit 599015e). I think this function still behaves correctly with cross-compilation and multiple outputs.

Maybe we should just move it else-where as part of other changes. Also the closePropagation function is still use in 5 places in Nixpkgs.

@Profpatsch @nbp so is everyone happy with the rewrite by @infinisil that references the closePropagation function in deprecated.nix given the comment by @nbp ?

@volth Given the comment by @nbp I thought we would use the version by @infinisil which referenced closePropagation:

makeFullPerlPath = deps: makePerlPath (lib.misc.closePropagation deps);

@Profpatsch (Should have been squashed and fixed the commit message)