Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildPythonPackage: add support for setupPyDistFlags #64682

Merged
merged 1 commit into from Jul 13, 2019

Conversation

aske
Copy link
Member

@aske aske commented Jul 12, 2019

Flags passed to the "python setup.py" command.

Motivation for this change

Allow to pass flags to "python setup.py", see #64627.

Things done

Added a new setupPyDistFlags option (not sure about the name).

cc @FRidh

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Flags passed to the "python setup.py" command.
@aske
Copy link
Member Author

aske commented Jul 12, 2019

By the way how to add labels? I think this one needs 1.severity: mass-rebuild.

@FRidh FRidh self-assigned this Jul 13, 2019
@FRidh FRidh changed the base branch from master to staging July 13, 2019 07:45
@FRidh FRidh changed the base branch from staging to staging-next July 13, 2019 07:46
@FRidh FRidh changed the base branch from staging-next to staging July 13, 2019 07:46
@FRidh FRidh merged commit 1e0ebdb into NixOS:staging Jul 13, 2019
in attrs // {
# we copy nix_run_setup over so it's executed relative to the root of the source
# many project make that assumption
buildPhase = attrs.buildPhase or ''
runHook preBuild
cp ${setuppy} nix_run_setup
${python.pythonForBuild.interpreter} nix_run_setup ${lib.optionalString (setupPyBuildFlags != []) ("build_ext " + (lib.concatStringsSep " " setupPyBuildFlags))} bdist_wheel
${python.pythonForBuild.interpreter} nix_run_setup ${setupPyDistFlagsString} ${setupPyBuildExtString} bdist_wheel
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I read nix_run_setup docs correctly setupPyDistFlags here will be applied to all commands, not just dist. In fact I need just that for Tensorflow but we need to either fix the option name to setupPyCommonFlags or force the arguments to be for bdist_wheel only by placing them after bdist_wheel.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you give a link to the docs you've used?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct the name is a bit too specific.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usage: nix_run_setup [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: nix_run_setup --help [cmd1 cmd2 ...]
   or: nix_run_setup --help-commands
   or: nix_run_setup cmd --help

From reading run_setup.py source code I assume this actually comes from setuptools.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, setup.py is distutils/setuptools and these options come from the setup function.

# Execute before shell hook
, preShellHook ? ""
# Execute after shell hook
, postShellHook ? ""
, ... } @ attrs:

let
installOptions = lib.concatMapStringsSep " " (option: "--install-option ${option}") setupPyDistFlags;
options = lib.concatMapStringsSep " " (option: "--global-option ${option}") setupPyBuildFlags;
Copy link
Member

@abbradar abbradar Jul 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also if we rename setupPyDistFlags option to pass global flags instead we also need to add them to options here as --global-options instead of --install-options.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speaking of which, is that right to use --global-option instead of some kind of --build-option here for setupPyBuildFlags?

@FRidh
Copy link
Member

FRidh commented Jul 13, 2019

I will revert this and we can have another go at it. There are indeed some mistakes regarding global-option vs build-option.

FRidh added a commit that referenced this pull request Jul 13, 2019
I merged this a bit too quick. We need to have a closer look at it.
See the points brought up in #64682.

This reverts commit 1e0ebdb.
@abbradar
Copy link
Member

@aske Given that you de facto used this as global options and I need it too for Tensorflow we could add setupPyGlobalFlags which is basically a renamed version of your previous argument. For pyproject we would need to:

  1. Use --global-option for setupPyGlobalFlags (so just use it in current options);
  2. Add buildOptions with --build-option for setupPyBuildFlags (move it from options).

@FRidh
Copy link
Member

FRidh commented Jul 13, 2019

Follow-up in #64701.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants