nvidia_x11_beta: reinit at 410.57 #45030

eadwu · 2018-08-14T16:07:13Z

Use i686 libs from the x86 package for > 391 (6ce6812)

Motivation for this change

I prefer to use the latest drivers. Also @nyanloutre.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

HappyEnt · 2018-08-22T12:28:09Z

Is it really needed to completely disable 32 bit support on 64 bit Systems? As far as i can see nvidia still provides 32 bit libraries. See for instance the arch pkgbuild. Currently Nix builds the i686 nvidia package and only links the 32 bit libraries from that package, one can still keep that, but for newer nvidia drivers the builder should probably be changed so we link the 32 bit libraries provided in the x86_64 package.

eadwu · 2018-08-23T16:41:01Z

Ah alright I'll look at that later when I have more time.

baracoder · 2018-08-26T23:23:28Z

I would like to try out the changes from this branch on my system but I am not able to get the configuration of my system right. I also fail to find any information on how to achieve this, so excuse me for asking here. What is the proper way to apply this to my system?

I have tried so far in the configuration.nixfile:

{ config, pkgs, ... }:                                                                                                                                                                        
let
  nvidia_beta_tar = fetchTarball https://github.com/eadwu/nixpkgs/archive/nvidia_x11_beta/396.51.tar.gz;
  nvidia_beta = import nvidia_beta_tar { config.allowUnfree = true; };
in
{
  imports = [
    (nvidia_beta_tar + "/nixos/modules/hardware/video/nvidia.nix")
    # ...
  ];
  disabledModules = [
    "hardware/video/nvidia.nix"
  ];

  # ...
  nixpkgs.config.packageOverrides = super: let self = super.pkgs; in {
    linuxPackages = super.linuxPackages // {
      nvidia_x11_beta = nvidia_beta.pkgs.linuxPackages.nvidia_x11_beta;
      nvidia_x11 = nvidia_beta.pkgs.linuxPackages.nvidia_x11_beta;
    };
  };
  services.xserver.videoDrivers = [ "nvidiaBeta" ];
  # ...
}

But it seams not to be enough.

eadwu · 2018-08-27T00:00:43Z

Personally I just use a local nixpkgs in which I rebase the changes and run nixos-rebuild like nixos-rebuild -I nixpkgs=/home/xps/Downloads/nixpkgs. As for overriding linuxPackages, I believe it should it should be

  nixpkgs = {
    overlays = [
      (self: super: {
        linuxPackages = super.linuxPackages.extend (linuxSelf: linuxSuper: {
          nvidia_x11 = nvidia_beta.linuxPackages.nvidia_x11_beta;
        });
      })
    ];
  };

The disabledModules and import shouldn't be necessary.

Also on 4.19-rc1 nvidia_x11 and nvidia_x11_beta both don't work.

baracoder · 2018-09-02T13:13:25Z

Thank you, for the suggestion. It works fine with overlays. I was confused that the /run/opengl-driver-32 contained 64bit libraries after building. Later I realized that it is because the libraries are precompiled and with the change in v391, the 32bit libraries are bundled in lib/32 folder. I managed to run steam, which requires the 32bit libs with your branch and the following changes:

diff --git a/pkgs/os-specific/linux/nvidia-x11/builder.sh b/pkgs/os-specific/linux/nvidia-x11/builder.sh
index 15c3e10e119..1e6d4c7b380 100755
--- a/pkgs/os-specific/linux/nvidia-x11/builder.sh
+++ b/pkgs/os-specific/linux/nvidia-x11/builder.sh
@@ -30,15 +30,19 @@ buildPhase() {
 installPhase() {
     # Install libGL and friends.
     mkdir -p "$out/lib"
-    cp -prd *.so.* tls "$out/lib/"
-    rm $out/lib/lib{glx,nvidia-wfb}.so.* # handled separately
+    if [ "$i686bundled" = "1" ]; then
+        cp -prd 32/*.so.* 32/tls "$out/lib/"
+    else
+        cp -prd *.so.* tls "$out/lib/"
+    fi
+    rm -f $out/lib/lib{glx,nvidia-wfb}.so.* # handled separately
     rm -f $out/lib/libnvidia-gtk* # built from source
     if [ "$useGLVND" = "1" ]; then
         # Pre-built libglvnd
         rm $out/lib/lib{GL,GLX,EGL,GLESv1_CM,GLESv2,OpenGL,GLdispatch}.so.*
     fi
     # Use ocl-icd instead
-    rm $out/lib/libOpenCL.so*
+    rm -f $out/lib/libOpenCL.so*
     # Move VDPAU libraries to their place
     mkdir $out/lib/vdpau
     mv $out/lib/libvdpau* $out/lib/vdpau
diff --git a/pkgs/os-specific/linux/nvidia-x11/generic.nix b/pkgs/os-specific/linux/nvidia-x11/generic.nix
index a6fd1ca1b93..29013eb561d 100644
--- a/pkgs/os-specific/linux/nvidia-x11/generic.nix
+++ b/pkgs/os-specific/linux/nvidia-x11/generic.nix
@@ -28,6 +28,8 @@ assert (versionOlder version "391") -> sha256_32bit != null;
 let
   nameSuffix = optionalString (!libsOnly) "-${kernel.version}";
   pkgSuffix = optionalString (versionOlder version "304") "-pkg0";
+  i686bundled = (stdenv.hostPlatform.system == "i686-linux" && versionAtLeast version "391");
+
 
   self = stdenv.mkDerivation {
     name = "nvidia-x11-${version}${nameSuffix}";
@@ -35,7 +37,7 @@ let
     builder = ./builder.sh;
 
     src =
-      if stdenv.hostPlatform.system == "x86_64-linux" || (stdenv.hostPlatform.system == "i686-linux" && versionAtLeast version "391") then
+      if stdenv.hostPlatform.system == "x86_64-linux" || i686bundled then
         fetchurl {
           url = "https://download.nvidia.com/XFree86/Linux-x86_64/${version}/NVIDIA-Linux-x86_64-${version}${pkgSuffix}.run";
           sha256 = sha256_64bit;
@@ -51,6 +53,7 @@ let
     inherit prePatch;
     inherit version useGLVND useProfiles;
     inherit (stdenv.hostPlatform) system;
+    inherit i686bundled;
 
     outputs = [ "out" ] ++ optional (!libsOnly) "bin";
     outputDev = if libsOnly then null else "bin";

jb55 · 2018-09-04T22:25:24Z

Without @baracoder's patch I get this when I launch steam:

after the patch steam seems to work!

HappyEnt · 2018-09-05T17:51:26Z

Looks really nice :)! Do we need an assertion in the xserver module to stop somebody on a 32 bit system from trying to install beta drivers? As far as I can see currently it would just install 64 bit beta drivers. Haven't tested it though, so maybe this case is already handled?

eadwu · 2018-09-05T20:39:47Z

Haven't really testing anything using i686 though from what was committed, baracoder's patch seems to take care of that (committed as 2f3e3e9).

HappyEnt · 2018-09-05T21:58:49Z

I think i686bundled is needed so the 64 bit package, instead of 32 bit, is used when building the i686-packages (Since we get the 32 bit libraries from there). Doesn't seem like there is anything stopping somebody on i686 from installing beta drivers which should not be possible.

eadwu · 2018-09-05T22:09:22Z

If I'm not wrong 90136bb should be the way to do this?

HappyEnt · 2018-09-05T22:22:51Z

I think we are nearly there :). The assertion should probably be moved over into the nvidia or xserver module, because we only want to prevent that somebody actually sets nvidiaBeta as their driver on i686. The problem now is, that i686bundled will never be true, because of the added assertion, but we still want to build the i686 with the x86_64 package, since we will only link lib and share from that anyways.

baracoder · 2018-09-05T23:26:54Z

My understanding of nixpkgs is very limited, and so my patch is not that good.

I did not change the build process:
Before version 391, the 32bit libraries were packages in a different archive, so they were copied from that package.
With my changes the 23bit libs are collected from the i686 "build" of the same x64 archive. See this line.
It depends on "building" for i686 to get the 32bit libraries. So I assume that the assert will break this.

I see now, that packages can have multiple outputs. So adding an output for lib32 would be better then using pkgs_i686 set.

I will try that later today

baracoder · 2018-09-07T00:16:08Z

I have added a commit here: master...baracoder:nvidiaBeta

i686 assertion is moved to the package file
32bit libraries are generated in an extra lib32 output instead of calling pkgsi686
I have tested with nvidia and nvidiaBeta as the selected driver both work as far as I can tell

eadwu · 2018-09-07T23:48:54Z

5cd9a35 should allow those who are on pkgs.linuxPackages_testing to use nvidia, using NVIDIA Optimus as specified #42846 or bumblebee (though the assertion causes some problems) has been working for me.

eadwu · 2018-09-23T15:24:34Z

Seems to compile fine on 4.19 now without any patches. Also rebased to master.

NVIDIA Optimus and Bumblebee work for me.

baracoder · 2018-10-04T23:24:25Z

pkgs/os-specific/linux/nvidia-x11/builder.sh

@@ -72,7 +72,7 @@ installPhase() {
        mkdir -p $bin/lib/xorg/modules/drivers
        cp -p nvidia_drv.so $bin/lib/xorg/modules/drivers
        mkdir -p $bin/lib/xorg/modules/extensions
-        cp -p libglx.so.* $bin/lib/xorg/modules/extensions
+        cp -p libglxserver_nvidia.so* $bin/lib/xorg/modules/extensions


Note: While this filename might have changed in the new version, it did not in 390, but the builder script has to work with both. Changing it to libglx*.so* works with both.

Sorry about that, I don't normally use 3xx drivers on my machine anymore so I kinda just forgot about that.

baracoder · 2018-10-14T15:17:43Z

(maintainer) @vcunat ping

pkgs/os-specific/linux/nvidia-x11/generic.nix

infinisil · 2018-10-14T15:28:33Z

pkgs/os-specific/linux/nvidia-x11/generic.nix

-          sha256 = sha256_32bit;
-        }
-      else if stdenv.hostPlatform.system == "x86_64-linux" then
+      if stdenv.hostPlatform.system == "x86_64-linux" then


Why was the order of i686 and x86_64 switched here?

If I remember what happened correctly, my system seemed to always evaluate i686 as true for some reason so I never got up to x86_64.

I think in some intermediate commits, there was an additional check, which made this order more logical. I have tested with the reverted order and it works as well.

Alright, I'll try changing it back tomorrow if the PR isn't merged by then.

pkgs/os-specific/linux/nvidia-x11/generic.nix

infinisil · 2018-10-14T16:43:54Z

@baracoder Unfortunately I still get the error mentioned by @jb55 when starting steam. I'm using a recent nixos-unstable (0a7e258) with the commits in this PR cherry-picked. Here's the full error when launched from the CLI:

cp: cannot create regular file '/home/infinisil/.local/share/Steam/bootstrap.tar.xz': Permission denied
Running Steam on nixos 19.03.git.4b6d3a7a779 64-bit
STEAM_RUNTIME has been set by the user to: /steamrt
Pins potentially out-of-date, rebuilding...
mkdir: cannot create directory ‘/nix/store/55zx8k453ji0iaxh0q97bjl084nsg0ya-steam-fhs/steamrt/pinned_libs_32’: Read-only file system
mkdir: cannot create directory ‘/nix/store/55zx8k453ji0iaxh0q97bjl084nsg0ya-steam-fhs/steamrt/pinned_libs_64’: Read-only file system
Installing breakpad exception handler for appid(steam)/version(1539393410)
Installing breakpad exception handler for appid(steam)/version(1539393410)
Installing breakpad exception handler for appid(steam)/version(1539393410)
Installing breakpad exception handler for appid(steam)/version(1539393410)
[2018-10-14 18:38:55] Startup - updater built Oct 13 2018 00:47:09
Looks like steam didn't shutdown cleanly, scheduling immediate update check
[2018-10-14 18:38:55] Checking for update on startup
[2018-10-14 18:38:55] Checking for available updates...
[2018-10-14 18:38:56] Download skipped: /client/steam_client_ubuntu12 version 1539393410, installed version 1539393410
[2018-10-14 18:38:56] Nothing to do
[2018-10-14 18:38:56] Verifying installation...
[2018-10-14 18:38:56] Performing checksum verification of executable files
[2018-10-14 18:38:56] Verification complete
glXChooseVisual failed
glXChooseVisual failedMain.cpp (326) : Assertion Failed: Fatal Error: glXChooseVisual failed
Main.cpp (326) : Assertion Failed: Fatal Error: glXChooseVisual failed
Installing breakpad exception handler for appid(steam)/version(1539393410)
[1014/183857.193658:INFO:crash_reporting.cc(216)] Crash reporting enabled for process: browser
[1014/183857.204353:WARNING:crash_reporting.cc(255)] Failed to set crash key: UserID with value: 0
[1014/183857.204391:WARNING:crash_reporting.cc(255)] Failed to set crash key: BuildID with value: 1539391679
[1014/183857.204405:WARNING:crash_reporting.cc(255)] Failed to set crash key: SteamUniverse with value: Public
[1014/183857.204408:WARNING:crash_reporting.cc(255)] Failed to set crash key: Vendor with value: Valve
[1014/183857.208732:ERROR:gpu_process_transport_factory.cc(1026)] Lost UI shared context.
assert_20181014183857_6.dmp[15586]: Uploading dump (out-of-process)
/tmp/dumps/assert_20181014183857_6.dmp
assert_20181014183857_6.dmp[15586]: Finished uploading minidump (out-of-process): success = yes
assert_20181014183857_6.dmp[15586]: response: CrashID=bp-5fbcce31-6fff-4d8f-ab12-4f5582181014
assert_20181014183857_6.dmp[15586]: file ''/tmp/dumps/assert_20181014183857_6.dmp'', upload yes: ''CrashID=bp-5fbcce31-6fff-4d8f-ab12-4f5582181014''

glxinfo output.

I'm using linuxPackages_latest, 4.18.12

Graphics card (from lspci): NVIDIA Corporation GM200 [GeForce GTX 980 Ti] (rev a1)

Do I need to do something special to have 32 bit stuff enabled?

baracoder · 2018-10-14T19:19:48Z

@infinisil other then hardware.opengl.driSupport32Bit = true; in your configuration.nix, there is nothing special required for 32bit libs. I am running nixpkgs master with this pr merged and steam it is working fine. I didn't test with linuxPackages_latest though. Can you check if the /run/opengl-driver-32/lib points to the lib32 output?

infinisil · 2018-10-14T19:29:11Z

other then hardware.opengl.driSupport32Bit = true; in your configuration.nix, there is nothing special required for 32bit libs.

Oh, that was it indeed, enabling that makes steam work. But it's a bit odd to now require this setting, because steam worked without it before.

Anyways, other than the minor things I mentioned above, this PR looks good to me. The only other thing is to fixup the commits a bit by squashing/renaming.

infinisil

See #45030 (comment)

The assertions should stay

baracoder · 2018-10-17T00:41:33Z

Recently a stable version 410.66 was released. I have cleaned up the accumulated changes and added the new stable version on my branch master...baracoder:nvidia-update
I did not test if the patch for kernel 4.19 still applies

infinisil · 2018-10-17T00:51:19Z

@baracoder Ah nice. I think this is better kept for a separate PR though.

@eadwu I think this looks good to merge now. Please squash the commits into some logical commits if possible.

eadwu · 2018-10-17T01:39:23Z

410.66 works on 4.19.0-rc8 through Optimus on my machine.

pkgs/os-specific/linux/nvidia-x11/generic.nix

infinisil

Awesome, thanks a lot for the PR <3

GrahamcOfBorg added 10.rebuild-darwin: 0 10.rebuild-linux: 11-100 labels Aug 14, 2018

pSub added the 2.status: work-in-progress label Aug 14, 2018

GrahamcOfBorg added 6.topic: nixos 8.has: module (update) labels Aug 15, 2018

GrahamcOfBorg removed 6.topic: nixos 8.has: module (update) labels Aug 23, 2018

eadwu force-pushed the nvidia_x11_beta/396.51 branch from 6c8c58b to 204105a Compare August 23, 2018 15:38

eadwu force-pushed the nvidia_x11_beta/396.51 branch from 204105a to 24acfb0 Compare August 23, 2018 22:20

eadwu force-pushed the nvidia_x11_beta/396.51 branch from 24acfb0 to 18389e3 Compare August 31, 2018 18:28

eadwu changed the title ~~[WIP] nvidia_x11_beta: reinit at 396.51~~ [WIP] nvidia_x11_beta: reinit at 396.54 Aug 31, 2018

eadwu force-pushed the nvidia_x11_beta/396.51 branch from 18389e3 to 37aa253 Compare August 31, 2018 18:54

baracoder mentioned this pull request Sep 4, 2018

Nvidia 396 Drivers #44461

Closed

jb55 mentioned this pull request Sep 4, 2018

Steam play doesn't work #45927

Closed

GrahamcOfBorg added the 8.has: clean-up label Sep 5, 2018

eadwu force-pushed the nvidia_x11_beta/396.51 branch from 90136bb to 1dfb439 Compare September 6, 2018 00:04

GrahamcOfBorg added 6.topic: nixos 8.has: module (update) labels Sep 6, 2018

eadwu changed the title ~~[WIP] nvidia_x11_beta: reinit at 396.54~~ nvidia_x11_beta: reinit at 396.54 Sep 13, 2018

eadwu changed the title ~~nvidia_x11_beta: reinit at 396.54~~ nvidia_x11_beta: reinit at 410.57 Sep 23, 2018

eadwu force-pushed the nvidia_x11_beta/396.51 branch 2 times, most recently from 59f76ae to 5bf5279 Compare September 23, 2018 16:21

baracoder reviewed Oct 4, 2018

View reviewed changes

infinisil requested changes Oct 14, 2018

View reviewed changes

infinisil requested changes Oct 15, 2018

View reviewed changes

eadwu force-pushed the nvidia_x11_beta/396.51 branch from bfd1d32 to 09e04fd Compare October 16, 2018 02:25

eadwu force-pushed the nvidia_x11_beta/396.51 branch from 09e04fd to 6faa340 Compare October 17, 2018 01:42

infinisil approved these changes Oct 17, 2018

View reviewed changes

infinisil reviewed Oct 17, 2018

View reviewed changes

pkgs/os-specific/linux/nvidia-x11/generic.nix Outdated Show resolved Hide resolved

infinisil reviewed Oct 17, 2018

View reviewed changes

pkgs/os-specific/linux/nvidia-x11/generic.nix Show resolved Hide resolved

eadwu force-pushed the nvidia_x11_beta/396.51 branch from 6faa340 to a81e27d Compare October 17, 2018 01:47

nvidia_x11_beta: reinit at 410.57

21bb1fa

eadwu force-pushed the nvidia_x11_beta/396.51 branch from a81e27d to 21bb1fa Compare October 17, 2018 23:31

infinisil approved these changes Oct 18, 2018

View reviewed changes

infinisil merged commit 77e90ef into NixOS:master Oct 18, 2018

baracoder mentioned this pull request Oct 18, 2018

nvidia_x11: 390.87 -> 410.66 #48661

Merged

11 tasks

qrilka mentioned this pull request Dec 14, 2018

steam: glXChooseVisual failed #47932

Closed

eadwu deleted the nvidia_x11_beta/396.51 branch November 17, 2020 23:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nvidia_x11_beta: reinit at 410.57 #45030

nvidia_x11_beta: reinit at 410.57 #45030

eadwu commented Aug 14, 2018 •

edited

HappyEnt commented Aug 22, 2018

eadwu commented Aug 23, 2018

baracoder commented Aug 26, 2018

eadwu commented Aug 27, 2018 •

edited

baracoder commented Sep 2, 2018

jb55 commented Sep 4, 2018

HappyEnt commented Sep 5, 2018

eadwu commented Sep 5, 2018

HappyEnt commented Sep 5, 2018 •

edited

eadwu commented Sep 5, 2018

HappyEnt commented Sep 5, 2018

baracoder commented Sep 5, 2018

baracoder commented Sep 7, 2018

eadwu commented Sep 7, 2018 •

edited

eadwu commented Sep 23, 2018 •

edited

baracoder Oct 4, 2018 •

edited

eadwu Oct 4, 2018

baracoder commented Oct 14, 2018

infinisil Oct 14, 2018

eadwu Oct 14, 2018

baracoder Oct 17, 2018

eadwu Oct 18, 2018

infinisil commented Oct 14, 2018

baracoder commented Oct 14, 2018

infinisil commented Oct 14, 2018

infinisil left a comment

baracoder commented Oct 17, 2018 •

edited

infinisil commented Oct 17, 2018

eadwu commented Oct 17, 2018

infinisil left a comment

nvidia_x11_beta: reinit at 410.57 #45030

nvidia_x11_beta: reinit at 410.57 #45030

Conversation

eadwu commented Aug 14, 2018 • edited

Motivation for this change

Things done

HappyEnt commented Aug 22, 2018

eadwu commented Aug 23, 2018

baracoder commented Aug 26, 2018

eadwu commented Aug 27, 2018 • edited

baracoder commented Sep 2, 2018

jb55 commented Sep 4, 2018

HappyEnt commented Sep 5, 2018

eadwu commented Sep 5, 2018

HappyEnt commented Sep 5, 2018 • edited

eadwu commented Sep 5, 2018

HappyEnt commented Sep 5, 2018

baracoder commented Sep 5, 2018

baracoder commented Sep 7, 2018

eadwu commented Sep 7, 2018 • edited

eadwu commented Sep 23, 2018 • edited

baracoder Oct 4, 2018 • edited

Choose a reason for hiding this comment

eadwu Oct 4, 2018

Choose a reason for hiding this comment

baracoder commented Oct 14, 2018

infinisil Oct 14, 2018

Choose a reason for hiding this comment

eadwu Oct 14, 2018

Choose a reason for hiding this comment

baracoder Oct 17, 2018

Choose a reason for hiding this comment

eadwu Oct 18, 2018

Choose a reason for hiding this comment

infinisil commented Oct 14, 2018

baracoder commented Oct 14, 2018

infinisil commented Oct 14, 2018

infinisil left a comment

Choose a reason for hiding this comment

baracoder commented Oct 17, 2018 • edited

infinisil commented Oct 17, 2018

eadwu commented Oct 17, 2018

infinisil left a comment

Choose a reason for hiding this comment

eadwu commented Aug 14, 2018 •

edited

eadwu commented Aug 27, 2018 •

edited

HappyEnt commented Sep 5, 2018 •

edited

eadwu commented Sep 7, 2018 •

edited

eadwu commented Sep 23, 2018 •

edited

baracoder Oct 4, 2018 •

edited

baracoder commented Oct 17, 2018 •

edited