My reasoning for wanting data vs. derivations wasn't about space, as much as "implied space of variation". Having a file checked into source control with a comment at the top saying "this might look like a normal source file you can change, but it isn't, so please don't modify it; instead modify the thing that generated it" is a maintenance liability. I realize we do a lot of that and this is just another straw on the camel's back, but for example I now need to be aware of that sort of thing if I apply any broad refactoring to the repo. As a really simple example, if I rename buildGoPackage, I might think "oh, I'll use sed to automatically change every instance of buildGoPackage in .nix files to the new name". Indeed, with the current setup, that will work, for now, and I won't notice that I ran afoul of the little comment at the top of the file telling me not to change it. All will be well until someone goes and regenerates the file and produces a bogus nix file expecting buildGoPackage. Obviously this example would be a quick fix but autogenerated code (vs. data) is tricky like this and I'd rather stay away from it.

So tl;dr: I have knee-jerk reactions to all autogenerated code, especially when in a source repo, and especially when the language is capable of expressing the sort of factoring that's being autogenerated natively. So I'd have way less issues generating assembly (hah) or a language like tcsh than I do with something like Haskell or Nix.

Having said all that, I won't fight on this PR you if you think this is more maintainable, but I don't personally like it 😄 It seems like the folder-valid-interruption concern would still be addressed by putting data expressions in each folder rather than code. But I'm also not sure I get why you want it to be interruptible; I'd probably want to rerun the updater if it failed mid-run, or I'd end up with a set of half-updated plugins, right? When would I ever want to say "oh, it crashed, looks good, I guess I'll do something else"? Or am I misunderstanding?

Thanks, that kind of comment is exactly why I asked a review from you.

The premise was that it's easy to hit the github rate-limit, so I wanted a resumable way of doing things. Since I was generating separated files, that's where I came up with the importDirs method, which I think it quite nice (on a more general note) to avoid having to build and maintain indices unnecessarily.

That being said, I solved the issue later on by introducing the personal github token to the fetching method so I will revert to the old ways of doing this.

thank you ❤️