mindustry: build Arc from source #108274

ghost · 2021-01-03T00:06:07Z

Motivation for this change

Currently we use binary shared libraries from maven. These include statically linked variants of SDL2 and glew from the author's build machine. To run mindustry on wayland without Xwayland it is necessary to link against a different glew library.

Things done

pkgs/games/mindustry/default.nix

ghost · 2021-01-08T03:54:46Z

This works now, but the whole file definitely needs a cleanup because it is currently very messy.
There are way too many fixed-output derivation hashes that need to be redone on every update. We should automate that with an update script (think nix-universal-prefetch) and also add a check that ensures the hash isn't outdated (like buildRustPackage does with the cargo vendor hash).

ghost · 2021-01-08T04:35:55Z

fgaz · 2021-01-08T07:02:03Z

I wonder if this fixes the aarch64 build too

ghost · 2021-01-10T03:33:53Z

Partially: libsdl-arc64.so can be built, but libarcarm64.so, which is also used for sprite packing at build-time, is still missing. On x86_64-linux this is blob is downloaded from maven. All of these binary downloads that are not easily packagable all the way down are so ugly I can't even, but this is a generic Java problem.
I'm also wondering how this affects non-linux platforms, but it's difficult for me to test this.

ghost · 2021-01-10T03:35:40Z

Anuken/Mindustry#1704 (comment)

☹️

full build log for mindustry / mindustry-deps for reference: https://termbin.com/2igz

ghost · 2021-01-10T04:28:59Z

I can now build both libarc64.so (via the jnigenBuild task) and libsdl-arc64.so (as before via the sdlnatives task) on x86_64-linux and inject them into the downloaded maven deps before starting the build. This should also make some of the LD_LIBRARY_PATH hacks unnecessary, since libarc64.so now has the correct rpath with alsaLib and so on.
However, porting this to aarch64 will be a bit more difficult: Upstream has some hard-coded compiler flags that can not work with aarch64.

fgaz · 2021-01-10T13:45:00Z

That's unfortunate.

While you're at it then could you set meta.platforms to x86_64 and remove aarch64 from meta.broken, since it's unsupported upstream?

ghost · 2021-01-10T16:56:17Z

While you're at it then could you set meta.platforms to x86_64 and remove aarch64 from meta.broken, since it's unsupported upstream?

Done.

Okay, so this has turned into a complete refactor:

I didn't like how we had to build the whole project twice each time (once for the -deps), so I investigated a bit and found a solution to only prefetch all dependencies in the -deps derivation, without building anything.
By accident I discovered that the Mindustry gradle project will discover an Arc checkout if it's placed in ../Arc (probably mostly intended for development) and use that instead of attempting to download prebuilt versions. So I used this to combine the Arc and Mindustry builds into one derivation, going back to one -deps derivation as well.

This might have broken some more things on darwin, but since it didn't work before and it doesn't look like that problem will be solved soon, I would like to ignore that for now.

fgaz

I didn't like how we had to build the whole project twice each time (once for the -deps), so I investigated a bit and found a solution to only prefetch all dependencies in the -deps derivation, without building anything.

This is great! I also tried to do something like this some time ago but failed miserably...

By accident I discovered that the Mindustry gradle project will discover an Arc checkout if it's placed in ../Arc (probably mostly intended for development) and use that instead of attempting to download prebuilt versions. So I used this to combine the Arc and Mindustry builds into one derivation, going back to one -deps derivation as well.

Also good news! Only two hashes is definitely more manageable. Did you discover a way to find invalid hashes?

Thanks for the PR, +1 from me!

I left some minor comments here and there.

pkgs/games/mindustry/default.nix

fgaz · 2021-01-11T09:41:38Z

Conflict

ghost · 2021-01-11T13:10:47Z

Conflict

is resolved

fgaz

Weird. Github still shows it as conflicting. Oh well.

fgaz · 2021-01-11T14:43:51Z

pkgs/games/mindustry/default.nix

    install -Dm644 desktop/build/libs/Mindustry.jar $out/share/mindustry.jar
    mkdir -p $out/bin
-    makeWrapper ${jre}/bin/java $out/bin/mindustry \
-      ${stdenv.lib.optionalString stdenv.isLinux "--prefix LD_LIBRARY_PATH : ${alsaLib}/lib"} \
+    makeWrapper ${jdk}/bin/java $out/bin/mindustry \
      --add-flags "-jar $out/share/mindustry.jar"
    install -Dm644 core/assets/icons/icon_64.png $out/share/icons/hicolor/64x64/apps/mindustry.png
    install -Dm644 ${desktopItem}/share/applications/Mindustry.desktop $out/share/applications/Mindustry.desktop


wait I think you forgot to remove this since you use copyDesktopItems

Of course, you're right

- reduce the buildPhase of the fixed-output dependencies derivation to only download all dependencies, instead of building the full thing. - add wayland variant, which is linked against a different variant of the glew library to run natively using SDL_VIDEODRIVER=wayland - use jdk (Java 15) instead of jre (Java 8) at runtime - use new copyDesktopItems hook

ghost · 2021-01-11T14:49:52Z

Sorry I forgot to push earlier.

fgaz · 2021-01-11T14:57:56Z

thank you! one binary less in nixpkgs 🎉

ghost · 2021-01-11T15:00:04Z

There will probably be a followup PR with some minor refinements and aarch64 support in the next few days (also planning to add aarch64 support upstream)

ofborg bot requested a review from fgaz January 3, 2021 00:15

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 10.rebuild-linux: 1 labels Jan 3, 2021

SuperSandro2000 added the 2.status: blocked by pr/issue label Jan 3, 2021

nagy reviewed Jan 3, 2021

View reviewed changes

pkgs/games/mindustry/default.nix Outdated Show resolved Hide resolved

ofborg bot added 2.status: merge conflict and removed 2.status: merge conflict 10.rebuild-darwin: 1 10.rebuild-linux: 1 labels Jan 8, 2021

ofborg bot added the 8.has: package (new) label Jan 8, 2021

ghost marked this pull request as ready for review January 10, 2021 17:12

fgaz approved these changes Jan 10, 2021

View reviewed changes

ghost mentioned this pull request Jan 10, 2021

add resolveDependencies gradle task Anuken/Mindustry#4302

Merged

ghost changed the title ~~mindustry: use libs from nixpkgs~~ mindustry: build Arc from source Jan 10, 2021

ghost requested a review from fgaz January 10, 2021 19:43

ofborg bot added 8.has: clean-up 11.by: package-maintainer labels Jan 10, 2021

fgaz approved these changes Jan 11, 2021

View reviewed changes

fgaz reviewed Jan 11, 2021

View reviewed changes

ghost merged commit e614b5a into NixOS:master Jan 11, 2021

chkno mentioned this pull request Oct 28, 2022

mindustry: 126.2 → 139 #198323

Closed

13 tasks

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mindustry: build Arc from source #108274

mindustry: build Arc from source #108274

ghost commented Jan 3, 2021 •

edited by ghost

ghost commented Jan 8, 2021 •

edited by ghost

ghost commented Jan 8, 2021

fgaz commented Jan 8, 2021

ghost commented Jan 10, 2021

ghost commented Jan 10, 2021 •

edited by ghost

ghost commented Jan 10, 2021 •

edited by ghost

fgaz commented Jan 10, 2021

ghost commented Jan 10, 2021

fgaz left a comment

fgaz commented Jan 11, 2021

ghost commented Jan 11, 2021

fgaz left a comment

fgaz Jan 11, 2021

ghost Jan 11, 2021

ghost commented Jan 11, 2021

fgaz commented Jan 11, 2021

ghost commented Jan 11, 2021

mindustry: build Arc from source #108274

mindustry: build Arc from source #108274

Conversation

ghost commented Jan 3, 2021 • edited by ghost

Motivation for this change

Things done

ghost commented Jan 8, 2021 • edited by ghost

ghost commented Jan 8, 2021

fgaz commented Jan 8, 2021

ghost commented Jan 10, 2021

ghost commented Jan 10, 2021 • edited by ghost

ghost commented Jan 10, 2021 • edited by ghost

fgaz commented Jan 10, 2021

ghost commented Jan 10, 2021

fgaz left a comment

Choose a reason for hiding this comment

fgaz commented Jan 11, 2021

ghost commented Jan 11, 2021

fgaz left a comment

Choose a reason for hiding this comment

fgaz Jan 11, 2021

Choose a reason for hiding this comment

ghost Jan 11, 2021

Choose a reason for hiding this comment

ghost commented Jan 11, 2021

fgaz commented Jan 11, 2021

ghost commented Jan 11, 2021

ghost commented Jan 3, 2021 •

edited by ghost

ghost commented Jan 8, 2021 •

edited by ghost

ghost commented Jan 10, 2021 •

edited by ghost

ghost commented Jan 10, 2021 •

edited by ghost