Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PHP: Make the default package more sane [v3] #83896

Merged
merged 34 commits into from Apr 5, 2020

Conversation

etu
Copy link
Contributor

@etu etu commented Mar 31, 2020

Motivation for this change

Our old PHP derivation was crazy big with a crazy amount of enabled extensions that also had a crazy amount of defaults that we pulled in, yet excluded some extensions that would have been nice to have by default.

This is a follow up on: #82348 to utilize #79377 more. And also a follow-up on all of the work done by me and @talyz in #82794 and #83514

But this grew a bit more, and was more complex than I imagined when I started this. But it became a collaboration project with @talyz to finish everything up.

We have also written the initial documentation for PHP in NixOS.

The current PHP

The current closure size is:

/nix/store/4ga497cqvz04vcyw9lf7s74ymrc6r28g-php-7.4.3	234.3M

And this contains things (among others) like:

/nix/store/y5166rf2lkxkf3991qm390vn92czgfgh-openldap-2.4.49          	 53.6M
/nix/store/2zsgfrkp0hn878rnrp0azpmy76ljm2b0-linux-pam-1.3.1          	 53.6M

The new smaller PHP

Here we have a smaller closure size:

/nix/store/10867i6grp8w8p6p6k1w8rxiwza5nyyy-php-with-extensions-7.4.3	189.4M

And we have a smaller base package:

/nix/store/v5v6wn207ibpp6n90z05f11k7svq94wd-php-7.4.3	 85.5M
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

etu added 5 commits March 29, 2020 11:06
…ault

Also, add opcache to default extensions since it significantly
increases PHP's performance and is by default enabled on Debian based
distributions. Not having it enabled by default results in a puzzling
performance loss for anyone attempting to migrate from Debian/Ubuntu
to NixOS who is unaware of this. Therefore, enable it by default. /talyz
@etu
Copy link
Contributor Author

etu commented Mar 31, 2020

@GrahamcOfBorg build php72 php73 php74 php72Packages.exts php73Packages.exts php74Packages.exts nixosTests.nextcloud nixosTests.php

Copy link
Member

@aanderse aanderse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't been through all of this yet, but here are a few initial suggestions for wording in the documentation if anyone find them useful.

doc/languages-frameworks/php.section.md Outdated Show resolved Hide resolved
doc/languages-frameworks/php.section.md Outdated Show resolved Hide resolved
doc/languages-frameworks/php.section.md Outdated Show resolved Hide resolved
@talyz
Copy link
Contributor

talyz commented Apr 1, 2020

@GrahamcOfBorg build php72 php73 php74 php72Packages.exts php73Packages.exts php74Packages.exts nixosTests.nextcloud nixosTests.php

@aanderse
Copy link
Member

aanderse commented Apr 1, 2020

My only remaining comment is that I wonder if it becomes confusing (or overly verbose) to have php.packages.exts, especially for people newer to nixpkgs. Currently in my mind I wonder if php.packages and php.extensions (or php.pkgs and php.exts) is less confusing, but of course that would potentially end up making buildEnv more verbose. To alleviate that maybe we could have buildEnv operate on php instead of pp, like php.buildEnv { exts = with p; [ pkgs.imagick exts.opcache ]; }... but maybe I'm just rambling at this point and this doesn't make any sense... feel free to disregard 😄

@talyz
Copy link
Contributor

talyz commented Apr 1, 2020

@aanderse Yes, that's also something we discussed, but decided that the scope of the PR was big enough as is. Definitely something to think about, though :)

Copy link
Contributor

@flokli flokli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only read over the docs and skimmed over the code so far, haven't actually run anything yet.

nixos/doc/manual/release-notes/rl-2009.xml Outdated Show resolved Hide resolved
nixos/doc/manual/release-notes/rl-2009.xml Outdated Show resolved Hide resolved
nixos/doc/manual/release-notes/rl-2009.xml Outdated Show resolved Hide resolved
nixos/doc/manual/release-notes/rl-2009.xml Outdated Show resolved Hide resolved
nixos/doc/manual/release-notes/rl-2009.xml Outdated Show resolved Hide resolved
nixos/doc/manual/release-notes/rl-2009.xml Show resolved Hide resolved
pkgs/top-level/all-packages.nix Show resolved Hide resolved
@FRidh
Copy link
Member

FRidh commented Apr 1, 2020

Currently in my mind I wonder if php.packages and php.extensions (or php.pkgs and php.exts) is less confusing, but of course that would potentially end up making buildEnv more verbose.

I think we should typically go for the verbose options. Readability is important. Now, we already use pkgs everywhere, so maybe not change that one, but exts is...strange.

etu and others added 15 commits April 5, 2020 16:44
This test checks that we evaluate PHP properly and that certain
extensions are actually loaded.
So now we have only packages for human interaction in php.packages and
only extensions in php.extensions. With this php.packages.exts have
been merged into the same attribute set as all the other extensions to
make it flat and nice.

The nextcloud module have been updated to reflect this change as well
as the documentation.
This provides a means to build a PHP package based on a list of
extensions from another.

For example, to generate a package with all default extensions
enabled, except opcache, but with ImageMagick:

php.withExtensions (e:
  (lib.filter (e: e != php.extensions.opcache) php.enabledExtensions)
  ++ [ e.imagick ])
@talyz
Copy link
Contributor

talyz commented Apr 5, 2020

I've added an enabledExtensions attribute and documentation about its use, fixed a typo in the nextcloud module which broke evaluation and fixed a few other minor things. The nextcloud and php tests pass.

@aanderse
Copy link
Member

aanderse commented Apr 5, 2020

I've added an enabledExtensions attribute and documentation about its use, fixed a typo in the nextcloud module which broke evaluation and fixed a few other minor things. The nextcloud and php tests pass.

ping @FRidh @jtojnar @etu @Ma27 @flokli does enabledExtensions sound good to you?

@etu etu merged commit 3b65398 into NixOS:master Apr 5, 2020
@etu etu deleted the slim-down-default-php-v3 branch April 14, 2020 13:51
@chpio
Copy link

chpio commented Apr 15, 2020

hi, i have a custom extension, that i need for my script. The extension build seems to work, but it's not available in php.packages. Im relatively new to nix, so pardon me.

my nix shell:

let
	pkgs = import <nixpkgs> {};
	phpGeos = pkgs.stdenv.mkDerivation rec {
		pname = "php-geos";
		version = "1.0.0";
		meta = {
			description = "php binding for GEOS";
		};

		src = pkgs.fetchurl {
			url = "https://git.osgeo.org/gitea/geos/php-geos/archive/${version}.tar.gz";
			sha256 = "0d90z591s092ys0d9gww2xw8vsvcdl6jbc90cgc6avq27dx4xk89";
		};

		nativeBuildInputs = [ pkgs.autoconf pkgs.automake pkgs.php74base ];
		preConfigure = "phpize";

		buildInputs = [ pkgs.geos ];

		installPhase = ''
			mkdir -p $out/lib/php/extensions
			cp modules/geos.so $out/lib/php/extensions/geos.so
		'';
	};
	php = pkgs.php74.buildEnv {
		extensions = exts: pkgs.php74.enabledExtensions ++ [phpGeos];
	};
in
	pkgs.stdenv.mkDerivation {
		name = "...";
		buildInputs = [
			php
			php.packages.composer
		];
	}

i have assumed that all enabled extensions would be passed to packages. When i run php -i my ext shows up. But when running /nix/store/pdllmyi9bdk7hc0bxgr8x01z27wgrdi9-php-with-extensions-7.4.4/bin/php -i (the php bin that composer is run with) the ext is not listed.

My previous version had an let phpPkgs = pkgs.phpPackages.override {php=php}, this seems to not work anymore since this change (?).

@etu
Copy link
Contributor Author

etu commented Apr 15, 2020

Hi @chpio!

You're about right all the way there!

Composer get's it's own environments PHP, and currently on master it doesn't use the overriden php. But this should be totally doable when we get #85026 together.

With that PR you should be able to use customPhp.packages.composer for instance.

This isn't something that have been easily doable before in Nix to do across all packages in a convenient way, so this will be a very exciting development :-)

If you can it would be great if you can check out that PR and report back there if it works for you.

@talyz and me will probably team up and try to get that one shaped up some day as well so we can get it in.

If you want a faster workaround in the meanwhile you could make your own derivation for composer that just use your custom php since composer is only a phar anyways.

Something like this can probably work for your case:

let
  myphp = pkgs.php74; # customize here
  mycomposer = writeScriptBin "composer" ''
    #!/bin/sh
    ${myphp} ${myphp.packages.composer}/libexec/composer/composer.phar \"$@\"
  '';
in {}

The only thing I know is missing from that composer is to inject unzip into the $PATH in that script.

@talyz
Copy link
Contributor

talyz commented Apr 24, 2020

Hi @chpio! Can you try #85026 now? Your use case should work fine now :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants