Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/master' into openssl-1.1
Browse files Browse the repository at this point in the history
  • Loading branch information
globin committed Jun 2, 2017
2 parents dd43151 + 7eadf66 commit d2d8a02
Show file tree
Hide file tree
Showing 572 changed files with 20,063 additions and 14,291 deletions.
62 changes: 49 additions & 13 deletions doc/cross-compilation.xml
Expand Up @@ -30,7 +30,7 @@
<section>
<title>Platform parameters</title>
<para>
The three GNU Autoconf platforms, <wordasword>build</wordasword>, <wordasword>host</wordasword>, and <wordasword>cross</wordasword>, are historically the result of much confusion.
The three GNU Autoconf platforms, <wordasword>build</wordasword>, <wordasword>host</wordasword>, and <wordasword>target</wordasword>, are historically the result of much confusion.
<link xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html" /> clears this up somewhat but there is more to be said.
An important advice to get out the way is, unless you are packaging a compiler or other build tool, just worry about the build and host platforms.
Dealing with just two platforms usually better matches people's preconceptions, and in this case is completely correct.
Expand Down Expand Up @@ -62,8 +62,8 @@
The "target platform" is black sheep.
The other two intrinsically apply to all compiled software—or any build process with a notion of "build-time" followed by "run-time".
The target platform only applies to programming tools, and even then only is a good for for some of them.
Briefly, GCC, Binutils, GHC, and certain other tools are written in such a way such that a single build can only compiler code for a single platform.
Thus, when building them, one must think ahead about what platforms they wish to use the tool to produce machine code for, and build binaries for each.
Briefly, GCC, Binutils, GHC, and certain other tools are written in such a way such that a single build can only compile code for a single platform.
Thus, when building them, one must think ahead about which platforms they wish to use the tool to produce machine code for, and build binaries for each.
</para>
<para>
There is no fundamental need to think about the target ahead of time like this.
Expand All @@ -85,7 +85,8 @@
This field is obsolete and will soon disappear—please do not use it.
</para></note>
<para>
The exact scheme these fields is a bit ill-defined due to a long and convoluted evolution, but this is slowly being cleaned up.
The exact schema these fields follow is a bit ill-defined due to a long and convoluted evolution, but this is slowly being cleaned up.
You can see examples of ones used in practice in <literal>lib.systems.examples</literal>; note how they are not all very consistent.
For now, here are few fields can count on them containing:
</para>
<variablelist>
Expand Down Expand Up @@ -118,8 +119,27 @@
This is a nix representation of a parsed LLVM target triple with white-listed components.
This can be specified directly, or actually parsed from the <varname>config</varname>.
[Technically, only one need be specified and the others can be inferred, though the precision of inference may not be very good.]
See <literal>lib.systems.parse</literal> for the exact representation, along with some <literal>is*</literal>predicates.
These predicates are superior to the ones in <varname>stdenv</varname> as they aren't tied to the build platform (host, as previously discussed, would be a saner default).
See <literal>lib.systems.parse</literal> for the exact representation.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><varname>libc</varname></term>
<listitem>
<para>
This is a string identifying the standard C library used.
Valid identifiers include "glibc" for GNU libc, "libSystem" for Darwin's Libsystem, and "uclibc" for µClibc.
It should probably be refactored to use the module system, like <varname>parse</varname>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><varname>is*</varname></term>
<listitem>
<para>
These predicates are defined in <literal>lib.systems.inspect</literal>, and slapped on every platform.
They are superior to the ones in <varname>stdenv</varname> as they force the user to be explicit about which platform they are inspecting.
Please use these instead of those.
</para>
</listitem>
</varlistentry>
Expand All @@ -128,7 +148,7 @@
<listitem>
<para>
This is, quite frankly, a dumping ground of ad-hoc settings (it's an attribute set).
See <literal>lib.systems.platforms</literal> for examples—there's hopefully one in there that will work verbatim for each platform one is working.
See <literal>lib.systems.platforms</literal> for examples—there's hopefully one in there that will work verbatim for each platform that is working.
Please help us triage these flags and give them better homes!
</para>
</listitem>
Expand Down Expand Up @@ -158,10 +178,10 @@
The depending package's target platform is unconstrained by the sliding window principle, which makes sense in that one can in principle build cross compilers targeting arbitrary platforms.
</para></note>
<para>
How does this work in practice? Nixpkgs is now structured so that build-time dependencies are taken from from <varname>buildPackages</varname>, whereas run-time dependencies are taken from the top level attribute set.
How does this work in practice? Nixpkgs is now structured so that build-time dependencies are taken from <varname>buildPackages</varname>, whereas run-time dependencies are taken from the top level attribute set.
For example, <varname>buildPackages.gcc</varname> should be used at build time, while <varname>gcc</varname> should be used at run time.
Now, for most of Nixpkgs's history, there was no <varname>buildPackages</varname>, and most packages have not been refactored to use it explicitly.
Instead, one can use the four attributes used for specifying dependencies as documented in <link linkend="ssec-stdenv-attributes" />.
Instead, one can use the four attributes used for specifying dependencies as documented in <xref linkend="ssec-stdenv-attributes"/>.
We "splice" together the run-time and build-time package sets with <varname>callPackage</varname>, and then <varname>mkDerivation</varname> for each of four attributes pulls the right derivation out.
This splicing can be skipped when not cross compiling as the package sets are the same, but is a bit slow for cross compiling.
Because of this, a best-of-both-worlds solution is in the works with no splicing or explicit access of <varname>buildPackages</varname> needed.
Expand All @@ -184,11 +204,27 @@
More information needs to moved from the old wiki, especially <link xlink:href="https://nixos.org/wiki/CrossCompiling" />, for this section.
</para></note>
<para>
Many sources (manual, wiki, etc) probably mention passing <varname>system</varname>, <varname>platform</varname>, and, optionally, <varname>crossSystem</varname> to nixpkgs:
<literal>import &lt;nixpkgs&gt; { system = ..; platform = ..; crossSystem = ..; }</literal>.
<varname>system</varname> and <varname>platform</varname> together determine the system on which packages are built, and <varname>crossSystem</varname> specifies the platform on which packages are ultimately intended to run, if it is different.
This still works, but with more recent changes, one can alternatively pass <varname>localSystem</varname>, containing <varname>system</varname> and <varname>platform</varname>, for symmetry.
Nixpkgs can be instantiated with <varname>localSystem</varname> alone, in which case there is no cross compiling and everything is built by and for that system,
or also with <varname>crossSystem</varname>, in which case packages run on the latter, but all building happens on the former.
Both parameters take the same schema as the 3 (build, host, and target) platforms defined in the previous section.
As mentioned above, <literal>lib.systems.examples</literal> has some platforms which are used as arguments for these parameters in practice.
You can use them programmatically, or on the command line like <command>nix-build &lt;nixpkgs&gt; --arg crossSystem '(import &lt;nixpkgs/lib&gt;).systems.examples.fooBarBaz'</command>.
</para>
<para>
While one is free to pass both parameters in full, there's a lot of logic to fill in missing fields.
As discussed in the previous section, only one of <varname>system</varname>, <varname>config</varname>, and <varname>parsed</varname> is needed to infer the other two.
Additionally, <varname>libc</varname> will be inferred from <varname>parse</varname>.
Finally, <literal>localSystem.system</literal> is also <emphasis>impurely</emphasis> inferred based on the platform evaluation occurs.
This means it is often not necessary to pass <varname>localSystem</varname> at all, as in the command-line example in the previous paragraph.
</para>
<note>
<para>
Many sources (manual, wiki, etc) probably mention passing <varname>system</varname>, <varname>platform</varname>, along with the optional <varname>crossSystem</varname> to nixpkgs:
<literal>import &lt;nixpkgs&gt; { system = ..; platform = ..; crossSystem = ..; }</literal>.
Passing those two instead of <varname>localSystem</varname> is still supported for compatibility, but is discouraged.
Indeed, much of the inference we do for these parameters is motivated by compatibility as much as convenience.
</para>
</note>
<para>
One would think that <varname>localSystem</varname> and <varname>crossSystem</varname> overlap horribly with the three <varname>*Platforms</varname> (<varname>buildPlatform</varname>, <varname>hostPlatform,</varname> and <varname>targetPlatform</varname>; see <varname>stage.nix</varname> or the manual).
Actually, those identifiers are purposefully not used here to draw a subtle but important distinction:
Expand Down
2 changes: 1 addition & 1 deletion doc/languages-frameworks/python.md
Expand Up @@ -710,7 +710,7 @@ nix-env -if build.nix
```
Now you can use the Python interpreter, as well as the extra packages that you added to the environment.

#### Environment defined in `~/.nixpkgs/config.nix`
#### Environment defined in `~/.config/nixpkgs/config.nix`

If you prefer to, you could also add the environment as a package override to the Nixpkgs set.
```nix
Expand Down
8 changes: 5 additions & 3 deletions lib/default.nix
Expand Up @@ -5,8 +5,9 @@
*/
let

# trivial, often used functions
# often used, or depending on very little
trivial = import ./trivial.nix;
fixedPoints = import ./fixed-points.nix;

# datatypes
attrsets = import ./attrsets.nix;
Expand Down Expand Up @@ -42,7 +43,7 @@ let
filesystem = import ./filesystem.nix;

in
{ inherit trivial
{ inherit trivial fixedPoints
attrsets lists strings stringsWithDeps
customisation maintainers meta sources
modules options types
Expand All @@ -55,6 +56,7 @@ in
}
# !!! don't include everything at top-level; perhaps only the most
# commonly used functions.
// trivial // lists // strings // stringsWithDeps // attrsets // sources
// trivial // fixedPoints
// lists // strings // stringsWithDeps // attrsets // sources
// options // types // meta // debug // misc // modules
// customisation
78 changes: 78 additions & 0 deletions lib/fixed-points.nix
@@ -0,0 +1,78 @@
rec {
# Compute the fixed point of the given function `f`, which is usually an
# attribute set that expects its final, non-recursive representation as an
# argument:
#
# f = self: { foo = "foo"; bar = "bar"; foobar = self.foo + self.bar; }
#
# Nix evaluates this recursion until all references to `self` have been
# resolved. At that point, the final result is returned and `f x = x` holds:
#
# nix-repl> fix f
# { bar = "bar"; foo = "foo"; foobar = "foobar"; }
#
# Type: fix :: (a -> a) -> a
#
# See https://en.wikipedia.org/wiki/Fixed-point_combinator for further
# details.
fix = f: let x = f x; in x;

# A variant of `fix` that records the original recursive attribute set in the
# result. This is useful in combination with the `extends` function to
# implement deep overriding. See pkgs/development/haskell-modules/default.nix
# for a concrete example.
fix' = f: let x = f x // { __unfix__ = f; }; in x;

# Modify the contents of an explicitly recursive attribute set in a way that
# honors `self`-references. This is accomplished with a function
#
# g = self: super: { foo = super.foo + " + "; }
#
# that has access to the unmodified input (`super`) as well as the final
# non-recursive representation of the attribute set (`self`). `extends`
# differs from the native `//` operator insofar as that it's applied *before*
# references to `self` are resolved:
#
# nix-repl> fix (extends g f)
# { bar = "bar"; foo = "foo + "; foobar = "foo + bar"; }
#
# The name of the function is inspired by object-oriented inheritance, i.e.
# think of it as an infix operator `g extends f` that mimics the syntax from
# Java. It may seem counter-intuitive to have the "base class" as the second
# argument, but it's nice this way if several uses of `extends` are cascaded.
extends = f: rattrs: self: let super = rattrs self; in super // f self super;

# Compose two extending functions of the type expected by 'extends'
# into one where changes made in the first are available in the
# 'super' of the second
composeExtensions =
f: g: self: super:
let fApplied = f self super;
super' = super // fApplied;
in fApplied // g self super';

# Create an overridable, recursive attribute set. For example:
#
# nix-repl> obj = makeExtensible (self: { })
#
# nix-repl> obj
# { __unfix__ = «lambda»; extend = «lambda»; }
#
# nix-repl> obj = obj.extend (self: super: { foo = "foo"; })
#
# nix-repl> obj
# { __unfix__ = «lambda»; extend = «lambda»; foo = "foo"; }
#
# nix-repl> obj = obj.extend (self: super: { foo = super.foo + " + "; bar = "bar"; foobar = self.foo + self.bar; })
#
# nix-repl> obj
# { __unfix__ = «lambda»; bar = "bar"; extend = «lambda»; foo = "foo + "; foobar = "foo + bar"; }
makeExtensible = makeExtensibleWithCustomName "extend";

# Same as `makeExtensible` but the name of the extending attribute is
# customized.
makeExtensibleWithCustomName = extenderName: rattrs:
fix' rattrs // {
${extenderName} = f: makeExtensibleWithCustomName extenderName (extends f rattrs);
};
}
2 changes: 2 additions & 0 deletions lib/maintainers.nix
Expand Up @@ -132,6 +132,7 @@
deepfire = "Kosyrev Serge <_deepfire@feelingofgreen.ru>";
demin-dmitriy = "Dmitriy Demin <demindf@gmail.com>";
DerGuteMoritz = "Moritz Heidkamp <moritz@twoticketsplease.de>";
dermetfan = "Robin Stumm <serverkorken@gmail.com>";
DerTim1 = "Tim Digel <tim.digel@active-group.de>";
desiderius = "Didier J. Devroye <didier@devroye.name>";
devhell = "devhell <\"^\"@regexmail.net>";
Expand Down Expand Up @@ -161,6 +162,7 @@
ehegnes = "Eric Hegnes <eric.hegnes@gmail.com>";
ehmry = "Emery Hemingway <emery@vfemail.net>";
eikek = "Eike Kettner <eike.kettner@posteo.de>";
ekleog = "Leo Gaspard <leo@gaspard.io>";
elasticdog = "Aaron Bull Schaefer <aaron@elasticdog.com>";
eleanor = "Dejan Lukan <dejan@proteansec.com>";
elitak = "Eric Litak <elitak@gmail.com>";
Expand Down
7 changes: 6 additions & 1 deletion lib/strings.nix
Expand Up @@ -438,8 +438,13 @@ rec {
=> true
isStorePath pkgs.python
=> true
isStorePath [] || isStorePath 42 || isStorePath {} || …
=> false
*/
isStorePath = x: builtins.substring 0 1 (toString x) == "/" && dirOf (builtins.toPath x) == builtins.storeDir;
isStorePath = x:
builtins.isString x
&& builtins.substring 0 1 (toString x) == "/"
&& dirOf (builtins.toPath x) == builtins.storeDir;

/* Convert string to int
Obviously, it is a bit hacky to use fromJSON that way.
Expand Down
1 change: 1 addition & 0 deletions lib/systems/default.nix
Expand Up @@ -5,6 +5,7 @@ rec {
parse = import ./parse.nix;
inspect = import ./inspect.nix;
platforms = import ./platforms.nix;
examples = import ./examples.nix;

# Elaborate a `localSystem` or `crossSystem` so that it contains everything
# necessary.
Expand Down

0 comments on commit d2d8a02

Please sign in to comment.