|
| 1 | +<chapter xmlns="http://docbook.org/ns/docbook" |
| 2 | + xmlns:xlink="http://www.w3.org/1999/xlink" |
| 3 | + xml:id="chap-cross"> |
| 4 | + |
| 5 | +<title>Cross-compilation</title> |
| 6 | + |
| 7 | +<section xml:id="sec-cross-intro"> |
| 8 | + <title>Introduction</title> |
| 9 | + <para> |
| 10 | + "Cross-compilation" means compiling a program on one machine for another type of machine. |
| 11 | + For example, a typical use of cross compilation is to compile programs for embedded devices. |
| 12 | + These devices often don't have the computing power and memory to compile their own programs. |
| 13 | + One might think that cross-compilation is a fairly niche concern, but there are advantages to being rigorous about distinguishing build-time vs run-time environments even when one is developing and deploying on the same machine. |
| 14 | + Nixpkgs is increasingly adopting this opinion in that packages should be written with cross-compilation in mind, and nixpkgs should evaluate in a similar way (by minimizing cross-compilation-specific special cases) whether or not one is cross-compiling. |
| 15 | + </para> |
| 16 | + |
| 17 | + <para> |
| 18 | + This chapter will be organized in three parts. |
| 19 | + First, it will describe the basics of how to package software in a way that supports cross-compilation. |
| 20 | + Second, it will describe how to use Nixpkgs when cross-compiling. |
| 21 | + Third, it will describe the internal infrastructure supporting cross-compilation. |
| 22 | + </para> |
| 23 | +</section> |
| 24 | + |
| 25 | +<!--============================================================--> |
| 26 | + |
| 27 | +<section xml:id="sec-cross-packaging"> |
| 28 | + <title>Packing in a cross-friendly manner</title> |
| 29 | + |
| 30 | + <section> |
| 31 | + <title>Platform parameters</title> |
| 32 | + <para> |
| 33 | + The three GNU Autoconf platforms, <wordasword>build</wordasword>, <wordasword>host</wordasword>, and <wordasword>cross</wordasword>, are historically the result of much confusion. |
| 34 | + <link xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html" /> clears this up somewhat but there is more to be said. |
| 35 | + An important advice to get out the way is, unless you are packaging a compiler or other build tool, just worry about the build and host platforms. |
| 36 | + Dealing with just two platforms usually better matches people's preconceptions, and in this case is completely correct. |
| 37 | + </para> |
| 38 | + <para> |
| 39 | + In Nixpkgs, these three platforms are defined as attribute sets under the names <literal>buildPlatform</literal>, <literal>hostPlatform</literal>, and <literal>targetPlatform</literal>. |
| 40 | + All are guaranteed to contain at least a <varname>platform</varname> field, which contains detailed information on the platform. |
| 41 | + All three are always defined at the top level, so one can get at them just like a dependency in a function that is imported with <literal>callPackage</literal>: |
| 42 | + <programlisting>{ stdenv, buildPlatform, hostPlatform, fooDep, barDep, .. }: ...</programlisting> |
| 43 | + </para> |
| 44 | + <warning><para> |
| 45 | + These platforms should all have the same structure in all scenarios, but that is currently not the case. |
| 46 | + When not cross-compiling, they will each contain a <literal>system</literal> field with a short 2-part, hyphen-separated summering string name for the platform. |
| 47 | + But, when when cross compiling, <literal>hostPlatform</literal> and <literal>targetPlatform</literal> may instead contain <literal>config</literal> with a fuller 3- or 4-part string in the manner of LLVM. |
| 48 | + We should have all 3 platforms always contain both, and maybe give <literal>config</literal> a better name while we are at it. |
| 49 | + </para></warning> |
| 50 | + <variablelist> |
| 51 | + <varlistentry> |
| 52 | + <term><varname>buildPlatform</varname></term> |
| 53 | + <listitem><para> |
| 54 | + The "build platform" is the platform on which a package is built. |
| 55 | + Once someone has a built package, or pre-built binary package, the build platform should not matter and be safe to ignore. |
| 56 | + </para></listitem> |
| 57 | + </varlistentry> |
| 58 | + <varlistentry> |
| 59 | + <term><varname>hostPlatform</varname></term> |
| 60 | + <listitem><para> |
| 61 | + The "host platform" is the platform on which a package is run. |
| 62 | + This is the simplest platform to understand, but also the one with the worst name. |
| 63 | + </para></listitem> |
| 64 | + </varlistentry> |
| 65 | + <varlistentry> |
| 66 | + <term><varname>targetPlatform</varname></term> |
| 67 | + <listitem> |
| 68 | + <para> |
| 69 | + The "target platform" is black sheep. |
| 70 | + The other two intrinsically apply to all compiled software—or any build process with a notion of "build-time" followed by "run-time". |
| 71 | + The target platform only applies to programming tools, and even then only is a good for for some of them. |
| 72 | + Briefly, GCC, Binutils, GHC, and certain other tools are written in such a way such that a single build can only compiler code for a single platform. |
| 73 | + Thus, when building them, one must think ahead about what platforms they wish to use the tool to produce machine code for, and build binaries for each. |
| 74 | + </para> |
| 75 | + <para> |
| 76 | + There is no fundamental need to think about the target ahead of time like this. |
| 77 | + LLVM, for example, was designed from the beginning with cross-compilation in mind, and so a normal LLVM binary will support every architecture that LLVM supports. |
| 78 | + If the tool supports modular or pluggable backends, one might imagine specifying a <emphasis>set</emphasis> of target platforms / backends one wishes to support, rather than a single one. |
| 79 | + </para> |
| 80 | + <para> |
| 81 | + The biggest reason for mess, if there is one, is that many compilers have the bad habit a build process that builds the compiler and standard library/runtime together. |
| 82 | + Then the specifying target platform is essential, because it determines the host platform of the standard library/runtime. |
| 83 | + Nixpkgs tries to avoid this where possible too, but still, because the concept of a target platform is so ingrained now in Autoconf and other tools, it is best to support it as is. |
| 84 | + Tools like LLVM that don't need up-front target platforms can safely ignore it like normal packages, and it will do no harm. |
| 85 | + </para> |
| 86 | + </listitem> |
| 87 | + </varlistentry> |
| 88 | + </variablelist> |
| 89 | + <note><para> |
| 90 | + If you dig around nixpkgs, you may notice there is also <varname>stdenv.cross</varname>. |
| 91 | + This field defined as <varname>hostPlatform</varname> when the host and build platforms differ, but otherwise not defined at all. |
| 92 | + This field is obsolete and will soon disappear—please do not use it. |
| 93 | + </para></note> |
| 94 | + </section> |
| 95 | + |
| 96 | + <section> |
| 97 | + <title>Specifying Dependencies</title> |
| 98 | + <para> |
| 99 | + As mentioned in the introduction to this chapter, one can think about a build time vs run time distinction whether cross-compiling or not. |
| 100 | + In the case of cross-compilation, this corresponds with whether a derivation running on the native or foreign platform is produced. |
| 101 | + An interesting thing to think about is how this corresponds with the three Autoconf platforms. |
| 102 | + In the run-time case, the depending and depended-on package simply have matching build, host, and target platforms. |
| 103 | + But in the build-time case, one can imagine "sliding" the platforms one over. |
| 104 | + The depended-on package's host and target platforms (respectively) become the depending package's build and host platforms. |
| 105 | + This is the most important guiding principle behind cross-compilation with Nixpkgs, and will be called the <wordasword>sliding window principle</wordasword>. |
| 106 | + In this manner, given the 3 platforms for one package, we can determine the three platforms for all its transitive dependencies. |
| 107 | + </para> |
| 108 | + <note><para> |
| 109 | + The depending package's target platform is unconstrained by the sliding window principle, which makes sense in that one can in principle build cross compilers targeting arbitrary platforms. |
| 110 | + </para></note> |
| 111 | + <warning><para> |
| 112 | + From the above, one would surmise that if a package is being built with a <literal>(build, host, target)</literal> platform triple of <literal>(foo, bar, bar)</literal>, then its build-time dependencies would have a triple of <literal>(foo, foo, bar)</literal>, and <emphasis>those packages'</emphasis> build-time dependencies would have triple of <literal>(foo, foo, foo)</literal>. |
| 113 | + In other words, it should take two "rounds" of following build-time dependency edges before one reaches a fixed point where, by the sliding window principle, the platform triple no longer changes. |
| 114 | + Unfortunately, at the moment, we do <emphasis>not</emphasis> implement this correctly, and after only one round of following build-time dependencies is the fixed point reached, with target incorrectly kept different than the others. |
| 115 | + </para></warning> |
| 116 | + <para> |
| 117 | + How does this work in practice? Nixpkgs is now structured so that build-time dependencies are taken from from <varname>buildPackages</varname>, whereas run-time dependencies are taken from the top level attribute set. |
| 118 | + For example, <varname>buildPackages.gcc</varname> should be used at build time, while <varname>gcc</varname> should be used at run time. |
| 119 | + Now, for most of Nixpkgs's history, there was no <varname>buildPackages</varname>, and most packages have not been refactored to use it explicitly. |
| 120 | + Instead, one can use the four attributes used for specifying dependencies as documented in <link linkend="ssec-stdenv-attributes" />. |
| 121 | + We "splice" together the run-time and build-time package sets with <varname>callPackage</varname>, and then <varname>mkDerivation</varname> for each of four attributes pulls the right derivation out. |
| 122 | + This splicing can be skipped when not cross compiling as the package sets are the same, but is a bit slow for cross compiling. |
| 123 | + Because of this, a best-of-both-worlds solution is in the works with no splicing or explicit access of <varname>buildPackages</varname> needed. |
| 124 | + For now, feel free to use either method. |
| 125 | + </para> |
| 126 | + </section> |
| 127 | + |
| 128 | +</section> |
| 129 | + |
| 130 | +<!--============================================================--> |
| 131 | + |
| 132 | +<section xml:id="sec-cross-usage"> |
| 133 | + <title>Cross-building packages</title> |
| 134 | + <para> |
| 135 | + To be written. |
| 136 | + This is basically unchanged so see the old wiki for now. |
| 137 | + </para> |
| 138 | +</section> |
| 139 | + |
| 140 | +<!--============================================================--> |
| 141 | + |
| 142 | +<section xml:id="sec-cross-infra"> |
| 143 | + <title>Cross-compilation infrastructure</title> |
| 144 | + <para>To be written.</para> |
| 145 | + <note><para> |
| 146 | + If one explores nixpkgs, they will see derivations with names like <literal>gccCross</literal>. |
| 147 | + Such <literal>*Cross</literal> derivations is a holdover from before we properly distinguished between the host and target platforms |
| 148 | + —the derivation with "Cross" in the name covered the <literal>build = host != target</literal> case, while the other covered the <literal>host = target</literal>, with build platform the same or not based on whether one was using its <literal>.nativeDrv</literal> or <literal>.crossDrv</literal>. |
| 149 | + This ugliness will disappear soon. |
| 150 | + </para></note> |
| 151 | +</section> |
| 152 | + |
| 153 | +</chapter> |
0 commit comments