@hpfr, this may be of-interest to you?

Would love to be able to add your https://github.com/hpfr/system/hosts/linux-surface.nix into the nixos-hardware/microsoft/surface upstream, so hopefully getting this merged will help -- if you're interested.

Can we fetch those binaries and patches from the upstream repo instead of embedding them in ours?
Maybe put the upstream repo in a derivation?

Yes. With so many patches, it is better to just pin the kernel version rather than apply them to our kernels.

I can try pulling them from upstream, as part of the build, yes.
(IIRC, they wouldn't apply cleanly when I first started on this last year which is why they're copied in)

Alternatively, in non-NixOS projects I've tended towards vendorising 3rd-party code to limit the impact of breaking changes from upstream.
Would something like submodules or git subrepo be acceptable, instead?

No. Please no git-submodules. If you need to rebase patches for some reason, I recommend if you maintain a fork of the repository and than just fetch them from their. However I would be interested why this is necessary? Did you update the patches for newer kernels?

At the time, the original patches were for a different kernel rev (and different distrib's) from NixOS' so I had to do a few manual fixes.

Last I tried, upstream patches applied cleanly.

But I offered the option because upstream is out of my control and therefore could diverge in an unfortunate way again -- and creating a patch to adjust a patch seems a supreme cludge.

FYI: I'm not a fan of git-submodules either, which is why I usually use the git-subrepo add-on to vendorise or run monorepos; but I didn't want to assume.

At the time, the original patches were for a different kernel rev (and different distrib's) from NixOS' so I had to do a few manual fixes.

Last I tried, upstream patches applied cleanly.

But I offered the option because upstream is out of my control and therefore could diverge in an unfortunate way again -- and creating a patch to adjust a patch seems a supreme cludge.

FYI: I'm not a fan of git-submodules either, which is why I usually use the git-subrepo add-on to vendorise or run monorepos; but I didn't want to assume.

Ever nixos-hardware user has to download this repository, that's why we want to keep it small. We can create a repository in https://github.com/nix-community, which can be fetched during the build. I don't think it is a good idea to complex software build in nixos-hardware since its used for both 20.03 and unstable it is hard to coordinate with kernel updates done in nixpkgs. Nixpkgs is probably a better place also given that hydra would actually built the kernel and provide a binary cache.

Ever nixos-hardware user has to download this repository, that's why we want to keep it small. We can create a repository in https://github.com/nix-community, which can be fetched during the build. I don't think it is a good idea to complex software build in nixos-hardware since its used for both 20.03 and unstable it is hard to coordinate with kernel updates done in nixpkgs. Nixpkgs is probably a better place also given that hydra would actually built the kernel and provide a binary cache.

It looks like I've got some time to work on this, again, so I'm going to try to revamp it to pull (appropriate) patches and apply them in-place, as you suggest.

@hpfr, this may be of-interest to you?
Would love to be able to add your config into the nixos-hardware/microsoft/surface upstream, so hopefully getting this merged will help -- if you're interested.

Can we fetch those binaries and patches from the upstream repo instead of embedding them in ours?
Maybe put the upstream repo in a derivation?

Yes. With so many patches, it is better to just pin the kernel version rather than apply them to our kernels.

It looks like I've got some time to work on this, again, so I'm going to try to revamp it to pull (appropriate) patches and apply them in-place, as you suggest.

Hi folks, I've also been using NixOS on a Surface device as @mexisme mentioned. Sorry for the long wait; I only update this stuff occasionally. In my configuration I used a big, ugly function to overlay the kernel with a patched version by pulling from upstream, and I pin packages by reading files generated by nix-prefetch-github. I've just updated it and it appears to work with the latest release from linux-surface/linux-surface. I also wrote a derivation for the new C-based iptsd. See the relevant commit to my configuration repo, and the files hosts/linux-surface.nix and pkgs/iptsd/default.nix. The reverse-engineered firmware for post-5.4 kernels is, as far as I can tell, more stable than the IPTS binaries that stopped working after 4.19. I wouldn't bother trying to mess with those binaries or pre-5.4 support.

Before you try to run NixOS on a Surface, you should know that building the kernel in the way I've implemented on the Surface itself will test your patience. It sometimes fails seemingly at random, possibly due to running out of memory for the build. It also takes a very long time (more than 3 hours in my experience, with only i3 and Alacritty running) and obviously superheats the device. Having a remote builder running NixOS makes things infinitely more bearable. Your remote builder will need the big-parallel feature to build the kernel. Some kind of Cachix (or equivalent) setup would probably be a better long-term solution. Also, packages like libwacom or the kernel itself are often very behind in Nixpkgs (Linux 5.8 took ages IIRC), meaning I was forever writing overlays and digging for old, compatible revisions of linux-surface or libwacom-surface. The new iptsd depends on inih which just made it into Nixpkgs.

I eventually grew tired of mucking with this nonsense and bought a Ryzen 2-in-1. I'll be selling my Surface in the next couple weeks, but I can still try to help here and in ##linux-surface on Freenode/Matrix. If you are curious about my approach, don't hesitate to ask. Maybe those of you who are much better at Nix conventions, pinning, etc can draw from what I have into this PR if it fits this repo's goals. Also, you'll probably want to set up options for different Surface models, as they share most tweaks but not all. I have no idea what the Surface Book's Nvidia GPU requires, for example.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/building-patched-4-19-kernel-fails-cryptically/4474/9

@hpfr, this may be of-interest to you?
Would love to be able to add your config into the nixos-hardware/microsoft/surface upstream, so hopefully getting this merged will help -- if you're interested.

Can we fetch those binaries and patches from the upstream repo instead of embedding them in ours?
Maybe put the upstream repo in a derivation?

Yes. With so many patches, it is better to just pin the kernel version rather than apply them to our kernels.

It looks like I've got some time to work on this, again, so I'm going to try to revamp it to pull (appropriate) patches and apply them in-place, as you suggest.

Hi folks, I've also been using NixOS on a Surface device as @mexisme mentioned. Sorry for the long wait; I only update this stuff occasionally. In my configuration I used a big, ugly function to overlay the kernel with a patched version by pulling from upstream, and I pin packages by reading files generated by nix-prefetch-github. I've just updated it and it appears to work with the latest release from linux-surface/linux-surface. I also wrote a derivation for the new C-based iptsd. See the relevant commit to my configuration repo, and the files hosts/linux-surface.nix and pkgs/iptsd/default.nix. The reverse-engineered firmware for post-5.4 kernels is, as far as I can tell, more stable than the IPTS binaries that stopped working after 4.19. I wouldn't bother trying to mess with those binaries or pre-5.4 support.

Before you try to run NixOS on a Surface, you should know that building the kernel in the way I've implemented on the Surface itself will test your patience. It sometimes fails seemingly at random, possibly due to running out of memory for the build. It also takes a very long time (more than 3 hours in my experience, with only i3 and Alacritty running) and obviously superheats the device. Having a remote builder running NixOS makes things infinitely more bearable. Your remote builder will need the big-parallel feature to build the kernel. Some kind of Cachix (or equivalent) setup would probably be a better long-term solution. Also, packages like libwacom or the kernel itself are often very behind in Nixpkgs (Linux 5.8 took ages IIRC), meaning I was forever writing overlays and digging for old, compatible revisions of linux-surface or libwacom-surface. The new iptsd depends on inih which just made it into Nixpkgs.

I eventually grew tired of mucking with this nonsense and bought a Ryzen 2-in-1. I'll be selling my Surface in the next couple weeks, but I can still try to help here and in ##linux-surface on Freenode/Matrix. If you are curious about my approach, don't hesitate to ask. Maybe those of you who are much better at Nix conventions, pinning, etc can draw from what I have into this PR if it fits this repo's goals. Also, you'll probably want to set up options for different Surface models, as they share most tweaks but not all. I have no idea what the Surface Book's Nvidia GPU requires, for example.

Thanks @hpfr!
Over last month or so, I managed to update mine to use a more recent kernel, and, as you say: things like the touch screen work significantly better.
I've got the Surface Go using a pre-i5 remote builder, and it builds in around an hour, so def. recommend doing that. Without a remote builder, I need to point a fan at the laptop to avoid throttling or even thermal shutdown...

I'm hoping to do a huge tidy-up over the holidays, and would love to leverage what you've done.
However, I've managed to side-track myself qith trying to get Calibre 5 working in Nix (whoops).

Can we fetch those binaries and patches from the upstream repo instead of embedding them in ours?
Maybe put the upstream repo in a derivation?

Hopefully this is what I've done, @JohnAZoidberg & @Mic92: replaced with #221