[WIP] PHP: Make the default package more sane #82794

etu · 2020-03-17T17:20:28Z

Motivation for this change

Our old PHP derivation was crazy big with a crazy amount of enabled extensions that also had a crazy amount of defaults that we pulled in, yet excluded some extensions that would have been nice to have by default.

This is a follow up on: #82348 to utilize #79377 more.

I made a chart of extensions in org-mode: http://ix.io/2evY

We had several insane defaults such as ldap, imap, pdo_odbc, gd but still missed things that are quite sane such as opcache.

The current PHP

The current closure size is:

245697960    /nix/store/4ga497cqvz04vcyw9lf7s74ymrc6r28g-php-7.4.3

And this contains things (among very others) like:

56161288    /nix/store/2zsgfrkp0hn878rnrp0azpmy76ljm2b0-linux-pam-1.3.1
56167840    /nix/store/y5166rf2lkxkf3991qm390vn92czgfgh-openldap-2.4.49

The new smaller PHP

Here we have a much smaller closure size:

141377496    /nix/store/kij6n7rxzj9y54fg8ik44hhii3s824yr-php-7.4.3

Breaking changes

I've still have to track down packages in nixpkgs that will break from this, some examples are probably nextcloud among others due to the missing database extensions.

Things done

Ma27 · 2020-03-17T18:57:11Z

nixos/doc/manual/release-notes/rl-2009.xml

+         ];
+       </programlisting>
+
+       All native PHP extensions are available under <literal><![CDATA[phpPackages.exts.<name?>]]></literal>.


Does anybody know if there's a way to link packages from NixOS docs? (IIRC those are only available via the package search (or probably the nixpkgs-docs), but I may be wrong).

There is not documentation for packages except for the package list and that is not linkable. Docbook has <package> element though.

I've switched to the <package> element and mentioned that the listed extensions that aren't loaded as default are available as loadable extensions. That makes it at least a bit discoverable to the end user reading the release log. See 42b24cf064e

Ma27 · 2020-03-17T18:58:30Z

pkgs/development/interpreters/php/default.nix

-  , phpdbgSupport ? config.php.phpdbg or true
+
+  # Build a minimal php
+  , minimalBuild ? config.php.minimal or false


Should we add a few more usage instructions in the nixpkgs docs? (e.g. doc/builders/packages or doc/language-frameworks?)

We don't seem to have any doc at all for PHP... I looked into the language-frameworks directory and found python there for instance... And yeah. Nobody has even started a php document...

Nixpkgs docs are supposed to be for building packages (package build support). I guess section for PHP build support like composer2nix might make sense but user docs is better suited for NixOS docs.

composer2nix isn't in nixpkgs at all (:disappointed:) and isn't documented either because of that.

It would be great to document the php.buildEnv behavior somewhere other than the code and in pull requests.

pkgs/development/interpreters/php/default.nix

Ma27 · 2020-03-17T19:02:01Z

nixos/doc/manual/release-notes/rl-2009.xml

+             pdo_pgsql
+           ]; })
+         ];
+       </programlisting>


Would it make sense to provide e.g. a phpMinimal or a phpExtended (with opinionated extensions) or even a phpFull?

I would say that the one I provide in this PR is a good enough base, and if you want to build a minimal one you can do that. And a full one would be to add all the extensions to get a full one. And that shouldn't require any compiling so it's fast to build.

And I find the one I provide by default in this PR small enough so we don't need a minimal one in my opinion.

I was thinking about it before though, that's why the minimal flag is there at all :)

But the minimal one has some issues to build it completely minimal. That is if you have a minimal build without --enable-mysqlnd you're not able to build phpPackages.exts.pdo_mysql because the php used that provides phpize needs to be built with --enable-mysqlnd to have mysqlnd.h available.

This problem could be solved to use a less minimal php to build the extensions and then use a minimal as base for the package used. But then you would end up with both of those in the closure so I'd rather avoid that... :-)

nixos/doc/manual/release-notes/rl-2009.xml

etu · 2020-03-17T19:20:05Z

flokli · 2020-03-22T15:49:02Z

Recap: I had a very nice (remote) session with @etu hacking on this today.

We need to do some more thinking about how to make this a bit less obstrusive (it currently breaks every NixOS module shipping software requiring some sort of database access).

This could be solved by explicitly listing the required modules. However, the current approach istn't that nice. We could improve it by making the "default" php package configurable through system configuration, or by adding (some few) extensions.

There were also some cleanups (php-unit and php-embed), and the initial addition of some docs about the PHP package tooling.

This moves yet more extensions from the base build to phpPackages.ext. Some of the extensions are a bit quirky and need patching for this to work, most notably mysqlnd. Two new parameters are introduced for mkExtension - internalDeps and postPhpize. internalDeps is used to specify which other internal extensions the current extension depends on, in order to provide them at build time. postPhpize is for when patches and quirks need to be applied after running phpize.

The tests for many of the extensions run just fine, for some a small portion fail. This runs the tests by default and disables the tests extensions with any failing tests.

A slight rewrite of buildEnv which: 1. Makes buildEnv recursively add itself to its output, so that it can be accessed from any php derivation. 2. Orders the extension text strings according to their internalDeps attribute - dependencies have to be put before dependants in the php.ini or they will fail to load due to missing symbols.

Opcache significantly increases PHP's performance and is by default enabled on Debian based distributions. Not having it enabled by default results in a puzzling performance loss for anyone attempting to migrate from Debian/Ubuntu to NixOS who is unaware of this. Therefore, enable it by default.

…butes

Older versions of PHP have a bug where header files are included in the wrong order. This is only an issue when compiling the opcache extension as a separate module, like we do.

Also, document why the patch is needed.

etu · 2020-03-27T17:37:33Z

This PR has grown, not because of bad reasons. And things have gone quite back and forth a few times and I don't like that. But I still want to do this. So I made #83514 where I've cherry-picked out the things we need and resolved all the conflicts and made the history cleaner and nicer.

etu changed the title ~~PHP: Make the default package more saneSlim down default php~~ PHP: Make the default package more sane Mar 17, 2020

ofborg bot added 6.topic: nixos 8.has: changelog 8.has: documentation labels Mar 17, 2020

etu added the 8.has: clean-up label Mar 17, 2020

etu changed the title ~~PHP: Make the default package more sane~~ [WIP] PHP: Make the default package more sane Mar 17, 2020

ofborg bot requested review from jtojnar, offlinehacker, javaguirre, CMCDragonkai, Ma27, globin, dasJ and ajs124 March 17, 2020 17:50

ofborg bot added 10.rebuild-darwin: 11-100 10.rebuild-linux: 101-500 labels Mar 17, 2020

This comment has been minimized.

Sign in to view

Ma27 reviewed Mar 17, 2020

View reviewed changes

ofborg bot added the 2.status: merge conflict label Mar 17, 2020

etu force-pushed the slim-down-default-php branch from 52506ba to 6d36f6d Compare March 17, 2020 19:45

ofborg bot removed the 2.status: merge conflict label Mar 17, 2020

ofborg bot requested a review from Ma27 March 17, 2020 19:50

etu force-pushed the slim-down-default-php branch from 0d39249 to 903727a Compare March 21, 2020 11:56

etu added 2 commits March 22, 2020 13:00

php: Make a slimmer php as default

4dea09d

phpPackages.exts: Drop uneeded extensions

6de5e91

etu force-pushed the slim-down-default-php branch 3 times, most recently from 11f6350 to 428b55e Compare March 22, 2020 14:50

etu added 3 commits March 22, 2020 20:06

doc/php: Add initial documentation for PHP

d39a91e

phpPackages.exts.gmp: Enable on darwin

e4fa8ef

unit: Make unit use phpbase packages

91b7860

etu force-pushed the slim-down-default-php branch from a614f24 to 91b7860 Compare March 22, 2020 19:07

talyz and others added 7 commits March 25, 2020 16:33

phpPackages.exts: Run tests by default

a960074

The tests for many of the extensions run just fine, for some a small portion fail. This runs the tests by default and disables the tests extensions with any failing tests.

Merge branch 'talyz-slim-down-default-php' into slim-down-default-php

b9f910c

php: Drop phpbase packages since they aren't needed anymore

26eee55

Merge branch 'master' into slim-down-default-php

d45b758

ofborg bot added 2.status: merge conflict and removed 2.status: merge conflict labels Mar 25, 2020

docs/rl: Update release log to match the updates to the default attri…

57b200d

…butes

etu force-pushed the slim-down-default-php branch from a1ac835 to cdbda19 Compare March 25, 2020 16:47

php: Re-introduce phpbase for builds of packages to reduce complexity

b05c0af

etu force-pushed the slim-down-default-php branch from cdbda19 to b05c0af Compare March 25, 2020 17:03

etu and others added 8 commits March 25, 2020 19:52

phpPackages.exts: Enable more extensions to build with PHP less than 7.4

5ddbd59

php: Add json as default extension

bec0712

phpPackages.exts.mysqlnd: Fix building with PHP <7.4

a4d7827

Merge branch 'talyz-slim-down-default-php2' into slim-down-default-php

a123610

php: Make mkExtension pass on additional args to mkDerivation

e50e9c9

phpPackages.exts.opcache: Fix build for PHP <7.4

1798dfb

Older versions of PHP have a bug where header files are included in the wrong order. This is only an issue when compiling the opcache extension as a separate module, like we do.

phpPackages.exts.mysqlnd: Use a proper patch instead of sed

06aa5d0

Also, document why the patch is needed.

Merge branch 'talyz-slim-down-default-php3' into slim-down-default-php

345d491

etu mentioned this pull request Mar 27, 2020

[WIP] PHP: Make the default package more sane [v2] #83514

Closed

10 tasks

etu closed this Mar 27, 2020

etu mentioned this pull request Mar 31, 2020

PHP: Make the default package more sane [v3] #83896

Merged

10 tasks

etu deleted the slim-down-default-php branch April 5, 2020 18:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] PHP: Make the default package more sane #82794

[WIP] PHP: Make the default package more sane #82794

etu commented Mar 17, 2020 •

edited

This comment has been minimized.

Ma27 Mar 17, 2020

jtojnar Mar 17, 2020

etu Mar 17, 2020

Ma27 Mar 17, 2020

etu Mar 17, 2020

jtojnar Mar 17, 2020

etu Mar 17, 2020

Ma27 Mar 17, 2020

etu Mar 17, 2020

etu commented Mar 17, 2020

flokli commented Mar 22, 2020

etu commented Mar 27, 2020

[WIP] PHP: Make the default package more sane #82794

[WIP] PHP: Make the default package more sane #82794

Conversation

etu commented Mar 17, 2020 • edited

Motivation for this change

The current PHP

The new smaller PHP

Breaking changes

Things done

This comment has been minimized.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

etu commented Mar 17, 2020

flokli commented Mar 22, 2020

etu commented Mar 27, 2020

etu commented Mar 17, 2020 •

edited