Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

imagemagick: add liquid rescale build option #83149

Merged
merged 1 commit into from Jun 25, 2021

Conversation

DieracDelta
Copy link
Member

Motivation for this change

It is not currently possible to build imagemagick with liquid rescaling on nix without an overlay.

Things done

This PR adds a optional dependency on liblqr1. If this library exists, imagemagick builds with with liquid rescale support.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@stale
Copy link

stale bot commented Sep 18, 2020

Hello, I'm a bot and I thank you in the name of the community for your contributions.

Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me". But there's a bit more you can do:

If you received an approval by an unprivileged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using Git blame on the relevant files, or via GitHub's web interface. You can see if someone's a member of the nixpkgs-committers team, by hovering with the mouse over their username on the web interface, or by searching them directly on the list.

If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use Git blame on the relevant files, or GitHub's web interface to find someone who touched the relevant files in the past.

If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always officially request them for a review, or just @ mention them and say you've addressed their comments.

Lastly, you can always ask for help at our Discourse Forum, or more specifically, at this thread or at #nixos' IRC channel.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Sep 18, 2020
Copy link
Member

@ajs124 ajs124 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"--with-lqr=yes" probably works just as "--with-lqr" without the =yes.

This seems like it might be useful to some people? Idk how the feature set of packages like this is decided on, tbh.

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Sep 18, 2020
@DieracDelta
Copy link
Member Author

You're right, I can definitely remove the --yes part. I ran into this as a feature that I wanted when running imagemagick. Maybe this is easy thing to add in via an overlay? On gentoo, I enabled this with a useflag. Nix doesn't look like it has an equivalent to useflags, though. In any case, it seems worthwhile to take note of it somewhere in case someone else runs into this problem.

@gebner
Copy link
Member

gebner commented Sep 19, 2020

This looks great! IMHO packages in nixpkgs should always be built with the maximum set of features enabled. (Because it's hard to substitute e.g. imagemagick with your own version if it is used as a dependency by something else.)

  1. This changes causes a lot of rebuilds (because many packages depend on imagemagick). Please make this PR to staging instead of master. (You can edit this PR, no need to make a new one.)
  2. Can we also add this dependency to imagemagick7?
  3. Please add liblqr1 = null; to the overrides in the imagemagick_light package. (This is in the pkgs/top-level/all-packages.nix file).

@SuperSandro2000
Copy link
Member

This changes causes a lot of rebuilds (because many packages depend on imagemagick). Please make this PR to staging instead of master. (You can edit this PR, no need to make a new one.)

100 are fine.

@DieracDelta

@DieracDelta
Copy link
Member Author

DieracDelta commented Jan 18, 2021

Thanks for the feedback! Per SuperSandro2000's suggestion, I've left this on master. I've made the recommended modifications. Sorry for the long delay on fixing this.

Changes:

  • rebased onto head master
  • copied changes to imagemagick 7.0 package
  • removed =yes from --with-lqr flag.
  • disabled liblqr when building imagemagick_light and imagemagick7_light packages.

Hopefully this passes CI.

Copy link
Member

@SuperSandro2000 SuperSandro2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diff LGTM

@SuperSandro2000
Copy link
Member

This is a semi-automatic executed nixpkgs-review which is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.

Result of nixpkgs-review pr 83149 run on x86_64-linux 1

1 package built:
  • imagemagick

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

imagemagick:

Please consider this feature to be alpha.

A substituteInPlace with an unused --replace got detected:

substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/Magick++-config'
substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/Magick++-config'
substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/Magick-config'
substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/Magick-config'
substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/MagickCore-config'
substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/MagickCore-config'
substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/MagickWand-config'
substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/MagickWand-config'
substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/Wand-config'
substituteStream(): WARNING: pattern '/nix/store/pjrkzmdm3p93r2la8j5bvksh5cq84535-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/ji7mdabnkd2ql7808ygbcll4cnmbznd3-imagemagick-6.9.11-29-dev/bin/Wand-config'

Please check the offending substituteInPlace for typos or changes in source.

@DieracDelta
Copy link
Member Author

DieracDelta commented Jan 19, 2021

Hmm... not really sure what to make of this error. I'm not able to reproduce it.

To try and reproduce, I cloned my branch of nixpkgs, changed NIXPKGS to point to it, then built imagemagick, imagemagick7, imagemagick7_light, and imagemagick_light. All built successfully. I also checked the outputs, and the list of features compiled in matched what I would expect. E.g. lqr for the non-light builds, and no lqr for the light builds.

It's worth noting I'm only on x86-64. Perhaps the bot built it on another arch?

I thought there might be a test in nixos/tests, but I'm not seeing anything.

@SuperSandro2000 any idea how to reproduce this error?

@SuperSandro2000
Copy link
Member

Hmm... not really sure what to make of this error. I'm not able to reproduce it.

This is not an error but additional feedback.

It's worth noting I'm only on x86-64. Perhaps the bot built it on another arch?

Only building on amd64.

When you take a look at https://github.com/DieracDelta/nixpkgs/blob/fix/imagemagick-default/pkgs/applications/graphics/ImageMagick/default.nix#L89 you will see the lines mentioned in the comment. They are not actually applied. Maybe they need an update or are outdated and can be removed.

@tomberek
Copy link
Contributor

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.

Result of nixpkgs-review pr 83149 run on x86_64-linux 1

6 packages built:
  • imagemagick
  • imagemagick7
  • imagemagick7Big
  • imagemagick7_light
  • imagemagickBig
  • imagemagick_light

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

imagemagick:

Please consider this feature to be alpha.

A substituteInPlace with an unmatched pattern got detected:

substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/Magick++-config'
substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/Magick++-config'
substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/Magick-config'
substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/Magick-config'
substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/MagickCore-config'
substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/MagickCore-config'
substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/MagickWand-config'
substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/MagickWand-config'
substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config -config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/Wand-config'
substituteStream(): WARNING: pattern '/nix/store/vlggxvpr2zfv54q89fs429nqrhzlngb4-pkg-config-wrapper-0.29.2/bin/pkg-config' doesn't match anything in file '/nix/store/hlagxfv4360ykxvkvpf2vdml79qg0rf0-imagemagick-6.9.11-60-dev/bin/Wand-config'

Please check the offending substituteInPlace for typos or changes in source.

@SuperSandro2000
Copy link
Member

I forgot this PR.

@DieracDelta Can you rebase it and resolve the merge conflict?

@DieracDelta
Copy link
Member Author

DieracDelta commented Jun 25, 2021

@SuperSandro2000 rebased. Built the packages on my system for a x86-64 linux target, and they seem to work fine. 👍

Note that if the substituteInPlace error shows up again for imagemagick 6, I can delete the substitute in place lines (this entire for-loop). I built the package with them deleted without a hitch, and it seems to run. I'm not sure why it's there in the first place, though so there may be unintended side effects. In either case that might be better as a separate PR.

@SuperSandro2000
Copy link
Member

Note that if the substituteInPlace error shows up again for imagemagick 6, I can delete the substitute in place lines (this entire for-loop). I built the package with them deleted without a hitch, and it seems to run. I'm not sure why it's there in the first place, though so there may be unintended side effects. In either case that might be better as a separate PR.

I think those are only doing something when you are consuming imagemagick as a library. It would be better to remove that in a separate PR to get this PR faster merged.

Also when I look at the amount of rebuilds we need to target staging here.

@DieracDelta DieracDelta changed the base branch from master to staging June 25, 2021 12:06
@DieracDelta
Copy link
Member Author

Switched to staging. (and I did not include that change in this PR, so we should be set there)

@SuperSandro2000 SuperSandro2000 merged commit 6b4b3de into NixOS:staging Jun 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants