Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

makeModulesClosure: fixup firmware extraction #96008

Merged

Conversation

baloo
Copy link
Member

@baloo baloo commented Aug 22, 2020

Motivation for this change

When building on a non nixos platform, I would get error trying to lookup the firmware for unix module. modinfo -F firmware unix output looks wrong. This is an attempt to work around that. Also I'm not sure we were loading module information from the to-be-loaded kernel, but rather the currently running one.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@baloo
Copy link
Member Author

baloo commented Aug 22, 2020

somewhat related to #92081
cc @lheckemann @CrystalGamma @xaverdh

for i in $(modinfo -b $kernel --set-version "$version" -F firmware $module); do
# $ modinfo -F firmware unix
# name: unix
if test "$i" = "name:"; then break; fi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this really break, as opposed to skipping the line? Maybe piping modinfo into grep -v ^name: would make more sense?

That said, this seems to me like modinfo is behaving incorrectly, given its manpage ­— it should only print the value of the field if given a -F option.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the intention was probably to use continue instead of break.
Actually the man page of modinfo does say

modinfo by default lists each attribute of the module in form fieldname : value, for easy reading. The filename is listed the same way (although it's not really an attribute).

but I agree that it should probably not do so if -F is given..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, it only seems to do this for built-in modules. That confuses me some more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intention is to "continue" to the next module. But because it's two nested loops, I break the inner one, otherwise (using the example provided) we match on name: which we would continue/skip, then we're feeded unix (which is wrong too).

We should break the inner one, and continue on the outer one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I somewhat agree modinfo acts weird, piping it to | grep -v actually sounds better. Let me know.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, while the kmod fix is my main preference, my preferred workaround is the grep approach, since otherwise we'll fail to load firmware for any built-in modules that require it.

@xaverdh
Copy link
Contributor

xaverdh commented Aug 23, 2020

Ok, the following patch works for me: 0001-modinfo-dont-print-module-name-when-field-is-given.txt
Is linux-modules@vger.kernel.org the right place to post this?

(edit: updated patch)

xaverdh added a commit to xaverdh/nixpkgs that referenced this pull request Aug 23, 2020
This came up in NixOS#96008.
Without this patch modinfo always prints the module name for builtin
modules, even if an explicit --field is passed.
@baloo
Copy link
Member Author

baloo commented Aug 23, 2020

To note: There was also fixes on the kernel modules to read included. If #96123 is merged, we still need those.

Comment on lines 72 to 74
# $ modinfo -F firmware unix
# name: unix
if test "$i" = "name:"; then break; fi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once modinfo patches are merged, we still need this in:
-b $kernel --set-version "$version"

Suggested change
# $ modinfo -F firmware unix
# name: unix
if test "$i" = "name:"; then break; fi

xaverdh added a commit to xaverdh/nixpkgs that referenced this pull request Aug 24, 2020
This came up in NixOS#96008.
Without this patch modinfo always prints the module name for builtin
modules, even if an explicit --field is passed.
@xaverdh
Copy link
Contributor

xaverdh commented Aug 27, 2020

Since the patch for kmod is a mass rebuild and did not make it into the last staging round before branch-off for 20.09, something like this should probably go into master (before branch-off).

After a recent upgrade of modinfo, its output is now incorrect for
builtin modules. This commit filters out the output until a fix is made
available upstream
@baloo baloo force-pushed the baloo/bugfixes/make-modules-closure_firmware branch from ce5edfe to e417362 Compare August 27, 2020 17:47
Before this commit, the firmware information would be loaded from the
currently running kernel, not from the kernel to be loaded.

This commit ensures the correct kernel version and modules are read.
@baloo baloo force-pushed the baloo/bugfixes/make-modules-closure_firmware branch from e417362 to 70bc1a3 Compare August 27, 2020 17:48
@baloo
Copy link
Member Author

baloo commented Aug 27, 2020

Splitted the two commits, and rewrote the fix to use a | grep -v instead as discussed.

@lheckemann lheckemann merged commit 86fa610 into NixOS:master Aug 28, 2020
@baloo baloo deleted the baloo/bugfixes/make-modules-closure_firmware branch August 28, 2020 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants