Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

raylib: init at 3.5.0 #98030

Merged
merged 2 commits into from Mar 8, 2021
Merged

Conversation

adamlwgriffiths
Copy link
Contributor

@adamlwgriffiths adamlwgriffiths commented Sep 15, 2020

Motivation for this change

Add raylib 3.0.0 3.5.0
Fixes #98029

Execution was tested using the provided examples for both static (rlgl_standalone) and dynamic (other examples).

TODO:

  • Optionally supports Wayland, wasn't sure how to handle that in nix packages properly
  • Supports Android, again, unsure how to handle that.
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@adamlwgriffiths
Copy link
Contributor Author

Opened up for review to seek help for the audio issue.
Also any advice on the wayland / android packaging would be good. Otherwise I think it's ok to just merge with X11 support for now.

Copy link
Contributor

@OPNA2608 OPNA2608 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested example audio_module_playing. A fatal SIGSEGV aside, strace shows attempts to open libpulse and libasound, though these aren't declared dependencies of either the raylib ELF library nor the compiled binary. This will need abit more wrapping / ELF wrangling to work.

grafik

Additionally the pkgconfig file in lib/pkgconfig/raylib.pc has broken paths and will need some more attention as well.

prefix=/nix/store/kxlh5s1p5ld9bxk22wrl1zpglx493apw-raylib-3.0.0
exec_prefix=${prefix}
libdir=${exec_prefix}//nix/store/kxlh5s1p5ld9bxk22wrl1zpglx493apw-raylib-3.0.0/lib
includedir=${prefix}//nix/store/kxlh5s1p5ld9bxk22wrl1zpglx493apw-raylib-3.0.0/include

pkgs/development/libraries/raylib/default.nix Outdated Show resolved Hide resolved
pkgs/development/libraries/raylib/default.nix Outdated Show resolved Hide resolved
pkgs/development/libraries/raylib/default.nix Outdated Show resolved Hide resolved
maintainers/maintainer-list.nix Outdated Show resolved Hide resolved
@adamlwgriffiths
Copy link
Contributor Author

Thanks for the helpful comments. I'll address them tomorrow =)

@adamlwgriffiths
Copy link
Contributor Author

adamlwgriffiths commented Sep 16, 2020

Tested example audio_module_playing. A fatal SIGSEGV aside, strace shows attempts to open libpulse and libasound, though these aren't declared dependencies of either the raylib ELF library nor the compiled binary. This will need abit more wrapping / ELF wrangling to work.

grafik

Additionally the pkgconfig file in lib/pkgconfig/raylib.pc has broken paths and will need some more attention as well.

prefix=/nix/store/kxlh5s1p5ld9bxk22wrl1zpglx493apw-raylib-3.0.0
exec_prefix=${prefix}
libdir=${exec_prefix}//nix/store/kxlh5s1p5ld9bxk22wrl1zpglx493apw-raylib-3.0.0/lib
includedir=${prefix}//nix/store/kxlh5s1p5ld9bxk22wrl1zpglx493apw-raylib-3.0.0/include

I don't fully grok this one.
I've got alsaLib listed as a buildInput, I would've thought that would provide it at run-time.
I'll keep looking through the docs to see what I've misunderstood.

Regarding raylib.pc,
I can see that theres a double path (/nix/store/....//nix/store).
Aren't those paths generated by nix?

@adamlwgriffiths
Copy link
Contributor Author

adamlwgriffiths commented Sep 16, 2020

I've pushed the trivial changes you mentioned above.
Still todo (help would be great):

  • audio not working
  • pkgconfig/raylib.pc paths incorrect

I'd also like to expose the wayland option (Cmake flag USE_WAYLAND=ON).
Using this as a reference it's not too hard, I've got the option working locally. I don't however have wayland to test with.
Do you know if, when using wayland, you still require the libX* libraries as dependencies? Or does [ wayland wayland-protocols ] cover that?

@adamlwgriffiths
Copy link
Contributor Author

adamlwgriffiths commented Sep 16, 2020

Ok, took a further look and the USE_WAYLAND is just fed into the GLFW build, which makes sense. As I'm using the standard Nix package, I'll remove that option. Looks like the x11 libs are required regardless then.

OSX support uses CoreAudio and some other options. As I don't have ways to test these other platforms, I'm just going to constrain this to Linux / X11.

@adamlwgriffiths adamlwgriffiths force-pushed the raylib-3.0.0 branch 2 times, most recently from 858303c to aed811b Compare September 16, 2020 05:56
@adamlwgriffiths
Copy link
Contributor Author

adamlwgriffiths commented Sep 16, 2020

For some reason the lib paths that are being search don't include alsa. I'm not sure why because the other dependencies (glu, etc) are there.

If I add

LD_LIBRARY_PATH="${alsaLib}/lib";

to the definition, then audio plays - at least in the nix-shell used to build the library. But I have a feeling that's not the correct solution here.

@OPNA2608
Copy link
Contributor

OPNA2608 commented Sep 17, 2020

First things first, you prolly want to make alsaLib and libpulseaudio optional inputs to support non-Linux systems and Linux systems that, for example, don't want PulseAudio installed in any way. Enabling them by default should be fine though. E.g.

{ stdenv
# …
, alsaSupport ? stdenv.hostPlatform.isLinux
, alsaLib
, pulseSupport ? stdenv.hostPlatform.isLinux
, libpulseaudio
# …
}:

stdenv.mkDerivation rec {
  # …
  buildInputs = [
    # …
  ]
  ++ lib.optional alsaSupport alsaLib
  ++ lib.optional pulseSupport libpulseaudio;
  # …
}

I've got alsaLib listed as a buildInput, I would've thought that would provide it at run-time.
I'll keep looking through the docs to see what I've misunderstood.

I'm still confused as to why this is occuring to begin with. Surely Nix is aware it needs alsa, I've given it to the build process, but it's not in the LD_LIBRARY_PATH.

A dependency in buildInputs will be installed at runtime, there is no promise though that code will always be able to find it. Nix automagically fixes declared ELF dependencies on libraries during compilation - you can view these for example by running ldd on a binary / library. These are always required, and the dynamic loader will immediately abort if any of those listed dependencies cannot be found.

grafik

But that's not the only way a library can be loaded, and it's not always the preferred method. Presumably in the case of raylib (assuming both ALSA and PulseAudio are used at buildtime for their headers), the audio dependencies are just "enabled", not "required". This means that the code could use PulseAudio if it's available at runtime, but if it cannot be found then execution can continue and it may, for example, gracefully fall back to trying ALSA next, or continue without any audio. If these dependencies were handled via declared ELF dependencies, then execution would abort immediately the moment those libraries cannot be located. So such loading mechanisms require additional investigation and wrapping to function correctly.

There are multiple ways to resolve this. Generally, polluting LD_LIBRARY_PATH can lead to more complicated looking problems w/r/t dynamically loading other libraries / programs further down the line.

I think you could use patchelf or another mechanism to add alsaLib and libpulseaudio to raylib's rpath if support for them has been enabled, then those libraries should be discoverable by the dynamic linker.

patchelf --add-needed looks like the better way to go.

{
preFixup = if (stdenv.hostPlatform.isLinux && (alsaSupport || pulseSupport)) then ''
  ${lib.optionalString alsaSupport "patchelf --add-needed ${alsaLib}/lib/libasound.so $out/lib/libraylib.so"}
  ${lib.optionalString pulseSupport "patchelf --add-needed ${libpulseaudio}/lib/libpulse.so $out/lib/libraylib.so"}
'' else ""; # or a section for stdenv.hostPlatform.isDarwin w/ install_name_tool if needed
}

This can prolly be made to look prettier ofc and I didn't test this, stuck bisecting another PR's regressions rn sorry. Hope this more detailed explanation helps you understand the gist of the problem!

@adamlwgriffiths
Copy link
Contributor Author

No that's a very helpful response and makes sense.
I appreciate both of your time. This turned out to be a rougher introduction to nix packages than I had expected.
Again, I'll have another go with your suggestions tomorrow.
If I face any more issues I'll post on the IRC forums.

@adamlwgriffiths
Copy link
Contributor Author

adamlwgriffiths commented Sep 18, 2020

Great, that worked.
I'm just supporting Linux/X11 at the moment as I don't have easy access to the other targets. I may look into a BSD VM given time.

The only remaining issue is the pkconfig/raylib.pc file.
Is this a case where I need to re-create the raylib.pc file, like mesa does?

@OPNA2608
Copy link
Contributor

prefix=@CMAKE_INSTALL_PREFIX@
exec_prefix=${prefix}
libdir=${exec_prefix}/@CMAKE_INSTALL_LIBDIR@
includedir=${prefix}/@CMAKE_INSTALL_INCLUDEDIR@

You should notify upstream that they construct those paths in their pkg-config file improperly. You can link them to jtojnar/cmake-snips for a better explanation, specifically the sections "Assuming CMAKE_INSTALL_<dir> is relative path" and "Concatenating paths when building pkg-config files".

If you can't wait until upstream fixes this though, then there are patches in Nixpkgs you could adapt to it in the meantime, see pkgs/development/libraries/yder/fix-pkgconfig.patch. They're also improper but should fix this specific problem for now, just make sure to add a comment linking it to the upstream issue so it can be removed once a proper fix is in place and it's time to update this package. 🙂

@AndersonTorres
Copy link
Member

I really don't know a way to do this. Usually I test less graphical things, just text diffs.
But images are other beast. A pixel difference is way less important than one character difference.

@AndersonTorres
Copy link
Member

Just to know: the original tests contained in the upstream project aren't running, but the lib works nonetheless?

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/help-creating-lua-package-in-nix-shell/11754/1

@adamlwgriffiths
Copy link
Contributor Author

Sorry I didn't get notification of your subsequent messages =/

The lib works for me locally. I use it as part of a nix-shell environment and have used both nim and lua bindings with it.

There are no actual unit tests. It seems the closest thing to a test is the examples.

The examples in 3.5.0 have a bug that prevents them being used to demonstrate the library.
This PR fixes it, but there hasn't been a release that includes it.
I doubt there's any value in including the patch as the examples shouldn't be shipped with the lib.

If you want to test, you can apply the patch and re-enable compiling the examples by removing "-DBUILD_EXAMPLES=OFF" on line 30.

pkgs/development/libraries/raylib/default.nix Outdated Show resolved Hide resolved
pkgs/development/libraries/raylib/default.nix Outdated Show resolved Hide resolved
@AndersonTorres
Copy link
Member

About the tests, you can just include the patch explaining that it will be fixed the next version.

@adamlwgriffiths
Copy link
Contributor Author

Thanks for the helpful comments!
I'll look at getting the patch to fix the examples in tonight.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 3, 2021

@adamlwgriffiths please squash the suggestions into the original commit, maintaining only two (the one adding you to the maintainers and the other creating the expression).

@adamlwgriffiths
Copy link
Contributor Author

I know you edited your comment to remove the request to add the patch, but I went a long anyway (why else do anything eh?).
I'm having trouble though, and I can't seem to break through what's wrong.

I've added the following section

  patches = [
    (fetchpatch {
      # fixes examples not compiling in 3.5.0
      url = "https://patch-diff.githubusercontent.com/raw/raysan5/raylib/pull/1470.patch";
      sha256 = "be4d519ec2ac93bb6e7b18a50447a0900dc105182496deec69c3a2469b43ad53";
    })
  ];

But now I get the following error:

  wanted: sha256:0lxd8fdld8n3d7ndx5i4302w23chl13h998qgdpbp4xcqag52kdy
  got:    sha256:1ff5l839wl8dxwrs2bwky7kqa8kk9qmsflg31sk5vbil68dzbzg0
error: build of '/nix/store/d93f74mbshikx2chdx3zffnfhyr5ni9p-source.drv', '/nix/store/n9l2mk70cfl030b08dylb94483y9jxy7-1470.patch.drv' failed

It seems thats the resulting hash of the patched source and derivation file, as changing the derivation also changes the hash.
But I can't find any examples in nixpkgs or any documentation online that talks about this being a problem.
The hash for raylib hasn't changed, and the hash for the patch should be valid.

$ wget https://patch-diff.githubusercontent.com/raw/raysan5/raylib/pull/1470.patch
$ sha256sum 1470.patch 
be4d519ec2ac93bb6e7b18a50447a0900dc105182496deec69c3a2469b43ad53  1470.patch

Setting the patch's hash to another value still causes the same issue. I get a different "wanted" hash, but the same "got" hash.
Confused.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 3, 2021

@adamlwgriffiths, ah, this is a confusing history.

  1. fetchpatch is not identical to fetchurl. The fetchpatch function reads the patch and strips the irrelevant information contained in it. Only after that stripping it calculates the hash.

    This is similar to removing the comments of a Python source code: the program runs the exact same way, but the hash of the source code changes.

    Therefore, the results of an independent "sha256 my-patch.diff" will not be the same as the Nix expects.

  2. In Nixpkgs, we use the infamous Trust On First Use protocol. We put a fake hash sum and do a first try. The program will complain about a wrong hash and will provide you the "got" hash (tnis is the truly correct hash). Then you copy that got hash in the sha256 argument of fetchpatch.

@adamlwgriffiths
Copy link
Contributor Author

adamlwgriffiths commented Mar 3, 2021

That's where I'm confused.
The "wanted" doesn't match the sha I have listed, infact it's neither the src nor patch hashes.
If I set the patch hash to the "got" value nothing changes, it still lists the previous wanted and got hashes.
It seems the "wanted" is a derived hash?

Perhaps its how I'm trying to test it.
I'm using the following from a bash script
eval 'nix-shell -E "with import <nixpkgs> {}; callPackage ./nixpkgs/pkgs/development/libraries/raylib/default.nix {}"'

@AndersonTorres
Copy link
Member

Show me all the output, please.

Is the <nixpkgs> the most recent one?

@adamlwgriffiths
Copy link
Contributor Author

$ cat shell.sh 
set -e
eval 'nix-shell -E "with import <nixpkgs> {}; callPackage /home/adam/Workspace/nixpkgs/pkgs/development/libraries/raylib/default.nix {}"'

$ ./shell.sh 
these derivations will be built:
  /nix/store/d93f74mbshikx2chdx3zffnfhyr5ni9p-source.drv
  /nix/store/n9l2mk70cfl030b08dylb94483y9jxy7-1470.patch.drv
building '/nix/store/n9l2mk70cfl030b08dylb94483y9jxy7-1470.patch.drv'...
building '/nix/store/d93f74mbshikx2chdx3zffnfhyr5ni9p-source.drv'...

trying https://patch-diff.githubusercontent.com/raw/raysan5/raylib/pull/1470.patch
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

trying https://github.com/raysan5/raylib/archive/3.5.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   121  100   121    0     0    199      0 --:--:-- --:--:-- --:--:--   199
100  3626    0  3626    0     0   4893      0 --:--:-- --:--:-- --:--:--  4893
hash mismatch in fixed-output derivation '/nix/store/c2vnjrkabkz2wasqzr7nfsf0k9z2a0nm-1470.patch':
  wanted: sha256:0lxd8fdld8n3d7ndx5i4302w23chl13h998qgdpbp4xcqag52kdy
  got:    sha256:1ff5l839wl8dxwrs2bwky7kqa8kk9qmsflg31sk5vbil68dzbzg0
error: build of '/nix/store/d93f74mbshikx2chdx3zffnfhyr5ni9p-source.drv', '/nix/store/n9l2mk70cfl030b08dylb94483y9jxy7-1470.patch.drv' failed

No, nixpkgs is the version from this PR.

@AndersonTorres
Copy link
Member

Try to use this sha256 = 1ff5l839wl8dxwrs2bwky7kqa8kk9qmsflg31sk5vbil68dzbzg0; in fetchPatch

@adamlwgriffiths
Copy link
Contributor Author

adamlwgriffiths commented Mar 4, 2021

omg sorry, its just one of those days. I was editing the same file but in a different directory.
facepalm

Ill update that and do a squash later tonight.

@AndersonTorres
Copy link
Member

Oh, it haspens.

@adamlwgriffiths adamlwgriffiths force-pushed the raylib-3.0.0 branch 2 times, most recently from 931e41e to fd573e0 Compare March 4, 2021 05:19
Copy link
Member

@AndersonTorres AndersonTorres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A final change and everything will be OK.

pkgs/development/libraries/raylib/default.nix Outdated Show resolved Hide resolved
@adamlwgriffiths
Copy link
Contributor Author

Applied and squashed.
Testing locally in a nix-shell and the examples are working well.

@adamlwgriffiths
Copy link
Contributor Author

Ok, I think that covers everything then.
Thanks for your help Anderson, much appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Request] Raylib
6 participants