[RFC/RDY] kernelmptcp: 0.91.3 -> 0.92.1 #29226

teto · 2017-09-11T07:12:40Z

Motivation for this change

I would like to benefit from the many 0.92 additions (http://multipath-tcp.org/pmwiki.php?n=Main.Release92), mostly additional socket options.

Things done

I've just changed the version number/updated sha, yet when I want to build the kernel, I have some trouble upon the config generation (because it's a different kernel):

 warning: unused option: ARM_KPROBES_TEST
warning: unused option: B43_PCMCIA
warning: unused option: CPU_NOTIFIER_ERROR_INJECT
error: unused option: DRM_I915_KMS
builder for ‘/nix/store/fzkidgj6i9ia1v745wffm5dzl1vhmmlc-linux-config-4.1.38-mptcp_v0.92.1.drv’ failed with exit code 255

I am not sure what's the best course of action is to solve this kind of problems ?

Cheers

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- Linux
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

mention-bot · 2017-09-11T07:12:42Z

@teto, thanks for your PR! By analyzing the history of the files in this pull request, we identified @layus, @hSloan and @pSub to be potential reviewers.

teto · 2017-09-12T00:46:04Z

@layus any tip :) ?

layus · 2017-09-12T12:40:53Z

Hi @teto,

Thank you for pinging me. I lost interest on this because seemingly nobody was using it. I am glad to be wrong :-).

To update the mptcp kernel, you need to mimic the corresponding kernel in nixpkgs. linux 4.4 was recently removed, so you have to copy it from git history at

nixpkgs/pkgs/top-level/all-packages.nix

Lines 12089 to 12100 in 9917450

    
           linux_4_4 = callPackage ../os-specific/linux/kernel/linux-4.4.nix { 
        
             kernelPatches = 
        
               [ kernelPatches.bridge_stp_helper 
        
                 kernelPatches.p9_fixes 
        
                 kernelPatches.cpu-cgroup-v2."4.4" 
        
               ] 
        
               ++ lib.optionals ((platform.kernelArch or null) == "mips") 
        
               [ kernelPatches.mips_fpureg_emu 
        
                 kernelPatches.mips_fpu_sigill 
        
                 kernelPatches.mips_ext3_n32 
        
               ]; 
        
           };

and also resurrect the related patches.
In this case, mptcp 92.1 is a 4.4.83 kernel as per https://github.com/multipath-tcp/mptcp/blob/v0.92.1/Makefile#L1-L3. You also need to update the modDirVersion and branch accordingly.

In your case I guess the meta.branch attribute was missing. That one is used in picking the right config options in https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/linux/kernel/common-config.nix.

layus · 2017-09-12T12:44:23Z

See also teto#1 ;-)

teto · 2017-09-13T03:00:29Z

EDIT: setting boot.kernelPackages = pkgs.linuxPackages_mptcp; seems to solve the 2nd problem, now compiling

That improved the situation quite a bit, i.e., it now compiles

checking for references to /tmp/nix-build-linux-4.4.83-mptcp_v0.92.1.drv-0 in /nix/store/hiipbya20i6fmk2811vs9w6k5vn74pav-linux-4.4.83-mptcp_v0.92.1-dev...
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
wrong ELF type
cannot find section .dynamic

as it doesn't sound too dramatic, I tried booting it via boot.kernelPackages = pkgs.linux_mptcp; in my configuration.nix but it ends with attribute ‘extend’ missing, at /home/teto/nixpkgs/nixos/modules/system/boot/kernel.nix:26:31; full trace


sudo LANG=C nixos-rebuild switch --show-trace                ~/nixpkgsbis
building Nix...
building the system configuration...
error: while evaluating the attribute ‘config.system.build.toplevel’ at /home/teto/nixpkgs/nixos/modules/system/activation/top-level.nix:252:5:
while evaluating ‘foldr’ at /home/teto/nixpkgs/lib/lists.nix:34:20, called from /home/teto/nixpkgs/nixos/modules/system/activation/top-level.nix:132:12:
while evaluating ‘fold'’ at /home/teto/nixpkgs/lib/lists.nix:37:15, called from /home/teto/nixpkgs/lib/lists.nix:41:8:
while evaluating ‘showWarnings’ at /home/teto/nixpkgs/nixos/modules/system/activation/top-level.nix:98:18, called from /home/teto/nixpkgs/nixos/modules/system/activation/top-level.nix:105:16:
while evaluating ‘foldr’ at /home/teto/nixpkgs/lib/lists.nix:34:20, called from /home/teto/nixpkgs/nixos/modules/system/activation/top-level.nix:98:23:
while evaluating ‘fold'’ at /home/teto/nixpkgs/lib/lists.nix:37:15, called from /home/teto/nixpkgs/lib/lists.nix:41:8:
while evaluating the attribute ‘warnings’ at /home/teto/nixpkgs/lib/attrsets.nix:199:44:
while evaluating anonymous function at /home/teto/nixpkgs/lib/modules.nix:73:45, called from /home/teto/nixpkgs/lib/attrsets.nix:199:52:
while evaluating the attribute ‘value’ at /home/teto/nixpkgs/lib/modules.nix:310:9:
while evaluating the option `warnings':
while evaluating the attribute ‘isDefined’ at /home/teto/nixpkgs/lib/modules.nix:342:5:
while evaluating ‘filterOverrides’ at /home/teto/nixpkgs/lib/modules.nix:417:21, called from /home/teto/nixpkgs/lib/modules.nix:326:18:
while evaluating ‘concatMap’ at /home/teto/nixpkgs/lib/lists.nix:102:18, called from /home/teto/nixpkgs/lib/modules.nix:423:8:
while evaluating ‘concatMap’ at /home/teto/nixpkgs/lib/lists.nix:102:18, called from /home/teto/nixpkgs/lib/modules.nix:321:17:
while evaluating anonymous function at /home/teto/nixpkgs/lib/modules.nix:321:28, called from undefined position:
while evaluating ‘dischargeProperties’ at /home/teto/nixpkgs/lib/modules.nix:384:25, called from /home/teto/nixpkgs/lib/modules.nix:322:62:
while evaluating the attribute ‘value’ at /home/teto/nixpkgs/lib/modules.nix:214:48:
while evaluating the attribute ‘config.warnings’ at /home/teto/nixpkgs/nixos/modules/system/boot/systemd.nix:714:5:
while evaluating ‘mapAttrsToList’ at /home/teto/nixpkgs/lib/attrsets.nix:223:23, called from /home/teto/nixpkgs/nixos/modules/system/boot/systemd.nix:714:29:
while evaluating the attribute ‘services’ at /home/teto/nixpkgs/lib/attrsets.nix:199:44:
while evaluating anonymous function at /home/teto/nixpkgs/lib/modules.nix:73:45, called from /home/teto/nixpkgs/lib/attrsets.nix:199:52:
while evaluating the attribute ‘value’ at /home/teto/nixpkgs/lib/modules.nix:310:9:
while evaluating the option `systemd.services':
while evaluating the attribute ‘isDefined’ at /home/teto/nixpkgs/lib/modules.nix:342:5:
while evaluating ‘filterOverrides’ at /home/teto/nixpkgs/lib/modules.nix:417:21, called from /home/teto/nixpkgs/lib/modules.nix:326:18:
while evaluating ‘concatMap’ at /home/teto/nixpkgs/lib/lists.nix:102:18, called from /home/teto/nixpkgs/lib/modules.nix:423:8:
while evaluating ‘concatMap’ at /home/teto/nixpkgs/lib/lists.nix:102:18, called from /home/teto/nixpkgs/lib/modules.nix:321:17:
while evaluating anonymous function at /home/teto/nixpkgs/lib/modules.nix:321:28, called from undefined position:
while evaluating ‘dischargeProperties’ at /home/teto/nixpkgs/lib/modules.nix:384:25, called from /home/teto/nixpkgs/lib/modules.nix:322:62:
while evaluating the attribute ‘condition’ at /home/teto/nixpkgs/lib/modules.nix:465:14:
while evaluating the attribute ‘condition’ at /home/teto/nixpkgs/lib/modules.nix:465:14:
while evaluating the attribute ‘condition’ at /home/teto/nixpkgs/lib/modules.nix:465:14:
while evaluating the attribute ‘services.klogd.enable’ at /home/teto/nixpkgs/lib/attrsets.nix:199:44:
while evaluating anonymous function at /home/teto/nixpkgs/lib/modules.nix:73:45, called from /home/teto/nixpkgs/lib/attrsets.nix:199:52:
while evaluating the attribute ‘value’ at /home/teto/nixpkgs/lib/modules.nix:310:9:
while evaluating the option `services.klogd.enable':
while evaluating the attribute ‘isDefined’ at /home/teto/nixpkgs/lib/modules.nix:342:5:
while evaluating anonymous function at /home/teto/nixpkgs/lib/modules.nix:331:19, called from /home/teto/nixpkgs/lib/modules.nix:331:14:
while evaluating the attribute ‘value._type’ at /home/teto/nixpkgs/lib/modules.nix:422:73:
while evaluating the attribute ‘value.content’ at /home/teto/nixpkgs/lib/modules.nix:480:14:
while evaluating the attribute ‘default’ at /home/teto/nixpkgs/nixos/modules/services/logging/klogd.nix:12:7:
while evaluating ‘versionOlder’ at /home/teto/nixpkgs/lib/strings.nix:351:22, called from /home/teto/nixpkgs/nixos/modules/services/logging/klogd.nix:12:17:
while evaluating ‘getVersion’ at /home/teto/nixpkgs/lib/strings.nix:375:16, called from /home/teto/nixpkgs/nixos/modules/services/logging/klogd.nix:12:31:
while evaluating the attribute ‘boot.kernelPackages.kernel’ at /home/teto/nixpkgs/lib/attrsets.nix:199:44:
while evaluating anonymous function at /home/teto/nixpkgs/lib/modules.nix:73:45, called from /home/teto/nixpkgs/lib/attrsets.nix:199:52:
while evaluating the attribute ‘value’ at /home/teto/nixpkgs/lib/modules.nix:310:9:
while evaluating the option `boot.kernelPackages':
while evaluating ‘apply’ at /home/teto/nixpkgs/nixos/modules/system/boot/kernel.nix:26:15, called from /home/teto/nixpkgs/lib/modules.nix:305:11:
attribute ‘extend’ missing, at /home/teto/nixpkgs/nixos/modules/system/boot/kernel.nix:26:31

layus · 2017-09-13T08:39:16Z

@teto Part 1 does not seem too dramatic. It is mostly a failure to detect elf files that leads patchelf to fail verbosely in

nixpkgs/pkgs/build-support/setup-hooks/audit-tmpdir.sh

Lines 22 to 23 in e3b562e

    
           if isELF "$i"; then 
        
               if patchelf --print-rpath "$i" | grep -q -F "$TMPDIR"; then

.

And yes, you need boot.kernelPackages = pkgs.linuxPackages_mptcp;, boot.kernelPackages requires the attribute set that describes the kernel plus its packages/modules, not the raw kernel (as it name suggests ;)

Does it work for you now ?

teto · 2017-09-14T00:39:21Z

I booted it and it worked. I've rebased and updated the PR. Thanks for your help.

FRidh · 2017-09-14T07:23:58Z

Why am I requested for a review as code owner? I don't own these parts.

teto · 2017-09-14T07:28:27Z

I've not requested any special reviewer, I thought it was an automatic thing ?

peti

Can't say anything about this change. Not sure why my review was requested.

teto · 2017-09-14T07:47:37Z

I think I get why: my first rebase was wrong so it included many commits from other areas (python commits etc). I've force pusehd a fix but the bot didn't rescan it to fix the required reviews.

layus

It compiles and the required patches are applied. LGTM!

teto · 2017-10-24T06:58:35Z

Any update ? the old kernel is troublesome with setuid NixOS/nixops#745
The CI seems ok (2nd job got cancelled).

layus · 2017-10-24T07:55:01Z

@NeQuissimus You may be the best one to review this, if another review is still needed. As for me, I would merge this ASAP, as 1) It works and 2) @teto seems to be one of the very few to use it. If he thinks it's ready, it is.

NeQuissimus · 2017-10-24T12:27:49Z

LGTM from that I can tell. Never used this before...

teto · 2017-11-02T08:11:28Z

It's very possible that I write some sort of tests for this kernel for my personal usage. I wouldn't mind upstreaming it (or a subset) if there is any intereset for nixos. I could understand that testing a low interest package might be more a burden than anything for the nixos team anyway let me know. I confirm that rebooting with this kernel displayed yes on amiusingmptcp.de .
If you have any tip regarding kernel testing with nixos, let me know (I haven't checked the 'writing tests' documentation yet though, I guess it resorts to nixops).

globin · 2017-11-02T08:16:08Z

pkgs/top-level/all-packages.nix

  linux_mptcp = callPackage ../os-specific/linux/kernel/linux-mptcp.nix {
    kernelPatches =
      [ kernelPatches.bridge_stp_helper
        kernelPatches.p9_fixes
-        kernelPatches.DCCP_double_free_vulnerability_CVE-2017-6074


Could you remove the DCCP kernel patch, it is not referenced anywhere else.

it is already removed from pkgs/os-specific/linux/kernel/patches.nix I believe

Doesn't seem so:

$ rg DCCP_double_free pkgs/top-level/all-packages.nix 12402: kernelPatches.DCCP_double_free_vulnerability_CVE-2017-6074 pkgs/os-specific/linux/kernel/patches.nix 61: DCCP_double_free_vulnerability_CVE-2017-6074 = rec 62: { name = "DCCP_double_free_vulnerability_CVE-2017-6074.patch";

both locally and on github interface it looks gone
https://github.com/NixOS/nixpkgs/pull/29226/files#diff-608de687ae1a5f9a6b1796f184eca73bL61

In the pkgs/os-specific/linux/kernel/patches.nix file?
https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/linux/kernel/patches.nix#L61-L68

either I am tired or you are checking out the wrong branch ? I do see the DCCP lines in pkgs/os-specific/linux/kernel/patches.nix prefixed with a '-' in https://patch-diff.githubusercontent.com/raw/NixOS/nixpkgs/pull/29226.diff

Sorry not sure how I missed that

no pb thanks for merging

layus · 2017-11-02T22:25:31Z

@teto

It's very possible that I write some sort of tests for this kernel for my personal usage. I wouldn't mind upstreaming it (or a subset) if there is any intereset for nixos. I could understand that testing a low interest package might be more a burden than anything for the nixos team anyway let me know. I confirm that rebooting with this kernel displayed yes on amiusingmptcp.de .
If you have any tip regarding kernel testing with nixos, let me know (I haven't checked the 'writing tests' documentation yet though, I guess it resorts to nixops).

Usually, it is best to look at existing tests in nixos/tests :-). I would be glad to test your tests, and help writing them if need be.

teto force-pushed the mptcp_kernel branch from d865238 to 5bc672c Compare September 14, 2017 00:37

teto requested review from edolstra, FRidh and peti as code owners September 14, 2017 00:37

kernelmptcp: 0.91.3 -> 0.92.1

61ef15b

teto force-pushed the mptcp_kernel branch from 5bc672c to 61ef15b Compare September 14, 2017 00:43

teto changed the title ~~[WIP] kernelmptcp: 0.91.3 -> 0.92.1~~ [RFC/RDY] kernelmptcp: 0.91.3 -> 0.92.1 Sep 14, 2017

peti reviewed Sep 14, 2017

View reviewed changes

layus approved these changes Sep 14, 2017

View reviewed changes

teto mentioned this pull request Oct 24, 2017

setuid tshark prevents deployment (mkSetcapProgram related) NixOS/nixops#745

Closed

NeQuissimus approved these changes Oct 24, 2017

View reviewed changes

globin requested changes Nov 2, 2017

View reviewed changes

globin merged commit 7dce131 into NixOS:master Nov 2, 2017

teto deleted the mptcp_kernel branch November 2, 2017 12:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC/RDY] kernelmptcp: 0.91.3 -> 0.92.1 #29226

[RFC/RDY] kernelmptcp: 0.91.3 -> 0.92.1 #29226

teto commented Sep 11, 2017 •

edited

mention-bot commented Sep 11, 2017

teto commented Sep 12, 2017

layus commented Sep 12, 2017

layus commented Sep 12, 2017

teto commented Sep 13, 2017 •

edited

layus commented Sep 13, 2017

teto commented Sep 14, 2017

FRidh commented Sep 14, 2017

teto commented Sep 14, 2017

peti left a comment

teto commented Sep 14, 2017

layus left a comment

teto commented Oct 24, 2017 •

edited

layus commented Oct 24, 2017

NeQuissimus commented Oct 24, 2017

teto commented Nov 2, 2017

globin Nov 2, 2017

teto Nov 2, 2017

globin Nov 2, 2017

teto Nov 2, 2017

globin Nov 2, 2017

teto Nov 2, 2017

globin Nov 2, 2017

teto Nov 2, 2017

layus commented Nov 2, 2017

[RFC/RDY] kernelmptcp: 0.91.3 -> 0.92.1 #29226

[RFC/RDY] kernelmptcp: 0.91.3 -> 0.92.1 #29226

Conversation

teto commented Sep 11, 2017 • edited

Motivation for this change

Things done

mention-bot commented Sep 11, 2017

teto commented Sep 12, 2017

layus commented Sep 12, 2017

layus commented Sep 12, 2017

teto commented Sep 13, 2017 • edited

layus commented Sep 13, 2017

teto commented Sep 14, 2017

FRidh commented Sep 14, 2017

teto commented Sep 14, 2017

peti left a comment

Choose a reason for hiding this comment

teto commented Sep 14, 2017

layus left a comment

Choose a reason for hiding this comment

teto commented Oct 24, 2017 • edited

layus commented Oct 24, 2017

NeQuissimus commented Oct 24, 2017

teto commented Nov 2, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

layus commented Nov 2, 2017

teto commented Sep 11, 2017 •

edited

teto commented Sep 13, 2017 •

edited

teto commented Oct 24, 2017 •

edited