New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generated factorio mods #89799
base: master
Are you sure you want to change the base?
Generated factorio mods #89799
Conversation
@Baughn @elitak @erictapen: Is this the direction you wanted to go with factorio mod packaging? |
I don't have the resources to review this atm and honestly I'd rather step down from maintainership for factorio, as I haven't played it for a long time now. Unrelated: @priegger would you be interested in adding yourself as maintainer? I saw you submitting some updates for the factorio package and you seem to have an interest in the game. |
Hi, thanks for your continued contributions! I've recently gotten back into this game and I'm glad someone is enthusiastically updating the nix infrastructure. Some comments:
I don't see the harm in this, sounds like stable won't get updated to 0.18 until August when it becomes the 1.0 release. Can always change it again then.
In my opinion, this is much better than the trust-on-first-use model that most of nixpkgs is built around, so no need to change this.
I'm not opposed to having a massive JSON dump with all the mods, if there is a sufficiently nice programmatic way to get this data. Similar to how the Haskell module infrastructure works. In the grand scheme of things, several thousand mods aren't going to be an issue so long as we have an auto-update script like you have written :)
Sounds reasonable - what's the rationale for an optional dep to be hidden?
Probably not - I would suggest that this infrastructure works uniformly for every mod.
I think each mod should produce a derivation and then the license can be set in the meta attribute.
My suggestion is that you take the JSON API as the single source of truth and bug mod authors to fix their metadata on mods.factorio.com |
The function can be used to fetch factorio as well as factorio mods.
43cd650
to
9183f1b
Compare
This is based off what I committed to my branch years ago? I was close to finishing, but the major features that I wanted to request from the upstream devs was sha256 hashes in their JSON, and ideally a way to fetch the mod zips without authentication, maybe with some token they issued for all Nix users. Otherwise, I was going to create my own open mirror of the nixified mod zips that refreshed daily. I think my plan was to not store any json/nix in the nixpkgs tree, because it changed so often. Unlike executable dependencies (for example python packages, whose APIs remain largely the same between versions), the binary .zip factorio mods are highly volatile and have very strict dependencies. I haven't investigated Nix Flakes very much, but I suspect that mechanism would be ideal, as it addresses this very issue of having to "freeze" the dependency graph to get reproducible builds. |
Edit: I mixed up some things here. I removed the part that does not make sense. Hi @elitak, thanks for your reply! I did not know about your branch, I wrote this from scratch.
My current "workflow" when playing factorio is to use services.factorio and the client syncs with the server (so I think, we don't need to be able to provide mods for the non-headless version anymore in the factorio package, but that's another story). And with the mod sync it's less of a problem to have slightly outdated versions on the server, since it's fixed automatically when connecting to the game. At least that's my current observation and hope. I would prefer to get something merged rather sooner than later and base the flakes version on it. |
before making this ready for review, please squash all fixup commits, as per CONTRIBUTING.md |
I think I committed 80% of the important stuff to that mods.nix and functions.nix that are in the master branch of nixpkgs. Anything beyond that was a tangled mess, which is part of the reason I abandoned the effort. I maybe asked for sha256, but didn't follow up on it. If you post in the right forum it should get the lead dev's attention, they are pretty good at watching for requests. I think the best solution is to provide users with the means to turn the JSON they can fetch with their credentials into a mods.nix that they can pull form, e.g. run ( What follows is historical stuff I typed out that doesn't really matter anymore )Sha256 was a bit of a stopper to me, because the way I saw it was that I would have to mirror & sha256-hash has the entire set of mods, then furnish a binary nix cache to the end users. Maybe I was naive, or maybe hashes were completely absent from the json, but I suppose you could use whichever hashing algorithm they have now, if you can adapt the verification function. The other aspect is that rather than ship the mod.zip files I was creating derivations that may have involved unpacking them; I don't quite recall my reasoning on that. Maybe something to do with running the binary nix cache and having the sha256 hashes known. As far as legal mirroring goes, my plan was to make a nixos module so that anyone could enable a service and have their own mirror, using their credentials. Whether they choose to disseminate that URL to friends or the public is out of our hands, but ought to be discouraged. Again though, running a mirror is far less than ideal. I also had plans to parse the dependency format that the factorio mods uses to pull the proper versions as dependencies too. Simply using the HEAD of each is probably fine, in most cases. |
Factorio mods use sha1.
The Factorio devs recommend to not unzip the mods except if you are developing them to have a fast developing cycle.
Sharing Factorio mods unmodified is allowed as long as you don't proclaim that it is your work or modify them. The earlier MP versions uploaded mods quite often but I am not sure if MP still does that.
The version ranger in there are usually out of date and I would enjoy it more if you could just track latest or a specific version. No need to parse the info.json. |
I marked this as stale due to inactivity. → More info |
Motivation for this change
Simplify factorio mod management in NixOS.
Todos:
Future improvements (not in this PR):
Additional informaiton:
Things done
sandbox
innix.conf
on non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
./result/bin/
)nix path-info -S
before and after)