Skip to content
This repository was archived by the owner on Apr 12, 2021. It is now read-only.
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: NixOS/nixpkgs-channels
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: e89b1063fb58
Choose a base ref
...
head repository: NixOS/nixpkgs-channels
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 4dc8447c55fe
Choose a head ref

Commits on Mar 28, 2020

  1. buildRustCrate: remap the current build dir to / for (more) reproduci…

    …ble builds
    
    (cherry picked from commit 56e11bc)
    andir committed Mar 28, 2020
    Copy the full SHA
    1febd68 View commit details
  2. buildRustCrate: remove superfluous dependency overrides

    By overriding each dependency on every level of the dependency tree we
    are creating a lot of unnecessary instances of the same derivation
    
    Looking at the output size of `nix-instantiate --trace-function-calls
    -vvvv …` and the execution time I got about a 10x improvement after
    applying this change.
    
    It was probably good intentions that lead to these overrides but in
    practice no tooling (that I know of) really needs this. `carnix` and
    `crate2nix` are fine without those overrides. Furthermore I believe that
    it is the job of the tooling around `buildRustCrate` to provide a
    coherent set of overrides. By not enforcing all of the overrides, debug
    flags, verbosity, … to be the same throughout the closure we also allow
    consumers to override specific aspects of the crates. Some (older?)
    crates might need different `crateOverrides` then newer crates with the
    same name. Currently such situations can not (easily) be implemented
    with the override in-place.
    
    (cherry picked from commit be5597f)
    andir committed Mar 28, 2020
    Copy the full SHA
    be561c7 View commit details
  3. buildRustCrate: refactor colored logging

    * Make errors include the crate name and make them much more prominent.
    * Move more code into lib.sh
    * Already source generated logging code and lib.sh in configure
    
    (cherry picked from commit 04e7462)
    kolloch authored and andir committed Mar 28, 2020
    Copy the full SHA
    7d8d24e View commit details
  4. build-support/rust/buildRustCrate: Search for matching Cargo.toml in …

    …sub directories
    
    This is what cargo does for git repositories.
    
    See related issues:
    
    * nix-community/crate2nix#53
    * nix-community/crate2nix#33
    
    (cherry picked from commit 8a6638d)
    kolloch authored and andir committed Mar 28, 2020
    Copy the full SHA
    0965206 View commit details
  5. buildRustCrate: only link build deps into build script

    According to the Cargo documentation:
    
    > The build script does not have access to the dependencies listed in
    > the dependencies or dev-dependencies section (they’re not built
    > yet!). Also, build dependencies are not available to the package
    > itself unless also explicitly added in the [dependencies] table.
    
    https://doc.rust-lang.org/cargo/reference/build-scripts.html
    
    This change separates linkage of regular dependencies and build
    dependencies.
    
    (cherry picked from commit ea6e048)
    danieldk authored and andir committed Mar 28, 2020
    Copy the full SHA
    ed3dc87 View commit details
  6. buildRustCrate: sort linker options in-place

    (cherry picked from commit 412c72d)
    danieldk authored and andir committed Mar 28, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    fcb8dc7 View commit details
  7. buildRustCrate: fewer backslashes

    This is a slight readability boost, I think.
    
    (cherry picked from commit 7533876)
    alyssais authored and andir committed Mar 28, 2020
    Copy the full SHA
    eb7d450 View commit details
  8. buildRustCrate: don't sort link flags

    Linkage order is significant and sorting can result in link errors.
    
    (cherry picked from commit d8b8537)
    symphorien authored and andir committed Mar 28, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    2597875 View commit details
  9. buildRustCrateTests: add regression test for link order

    (cherry picked from commit 2f7fb1c)
    symphorien authored and andir committed Mar 28, 2020
    Copy the full SHA
    9408b0c View commit details
  10. mongodb: 3.4.10 -> 4.0.4

    fix: Adding libtool to allow darwin compiles
    
    Libtool seems to be required for mongodb to compile on darwin.
    
    fix: Marking MongoDB as broken on aarch64
    
    fix: Adding libtools to the pkg imports
    
    Update mongodb to 4.0.4
    
    (cherry picked from commit e9bec1a)
    jarjee authored and Ma27 committed Mar 28, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    ac23e50 View commit details
  11. mongodb: split packages to expose 3.4, 3.6 and 4.0

    (cherry picked from commit 97c4dff)
    jarjee authored and Ma27 committed Mar 28, 2020

    Verified

    This commit was signed with the committer’s verified signature.
    primeos Michael Weiss
    Copy the full SHA
    54d2944 View commit details
  12. mongodb: 3.4.20 -> 3.4.22, 3.6.12 -> 3.6.13, 4.0.9 -> 4.0.11

    (cherry picked from commit 165d8bd)
    jarjee authored and Ma27 committed Mar 28, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    83d7124 View commit details
  13. nixos/tests/mongodb: test against mongodb versions 3.4, 3.6, 4.0

    Now has tests for 3.4, 3.6, 4.0. Has some duplication, but it appears to
    work on my machine.
    
    (cherry picked from commit 44641ed)
    jarjee authored and Ma27 committed Mar 28, 2020
    Copy the full SHA
    2cba56e View commit details
  14. mongodb-4_0: 4.0.11 -> 4.0.12

    (cherry picked from commit de69821)
    jarjee authored and Ma27 committed Mar 28, 2020

    Verified

    This commit was signed with the committer’s verified signature. The key has expired.
    NeQuissimus Tim Steinbach
    Copy the full SHA
    ef9bd8e View commit details
  15. mongodb: 3.4.22 -> 3.4.24 & fix ssl

    Reverts previous ssl fix commit and updates the mongodb version which fixes the ssl compile problem on darwin.
    
    (cherry picked from commit c2eee6e)
    Thibault Gagnaux authored and Ma27 committed Mar 28, 2020

    Verified

    This commit was signed with the committer’s verified signature. The key has expired.
    NeQuissimus Tim Steinbach
    Copy the full SHA
    00d69b8 View commit details
  16. mongodb: builds on aarch64 as well

    (cherry picked from commit 80e6da7)
    Ma27 committed Mar 28, 2020

    Verified

    This commit was signed with the committer’s verified signature. The key has expired.
    NeQuissimus Tim Steinbach
    Copy the full SHA
    56bfb0d View commit details
  17. nixos/tests/mongodb: rewrite with python

    perl-based VM tests are deprecated.
    
    (cherry picked from commit 2934f04)
    Ma27 committed Mar 28, 2020

    Verified

    This commit was signed with the committer’s verified signature. The key has expired.
    NeQuissimus Tim Steinbach
    Copy the full SHA
    07fb019 View commit details
  18. nixos/release-notes: mention mongodb update

    (cherry picked from commit b65ff5d)
    Ma27 committed Mar 28, 2020
    Copy the full SHA
    7cf4906 View commit details
  19. nixos/release-notes: mention that mongodb is unfree now

    (cherry picked from commit 2712152)
    Ma27 committed Mar 28, 2020
    Copy the full SHA
    cd36c1d View commit details

Commits on Mar 29, 2020

  1. nixos/tests/mongodb: also test mongodb-3_4

    (cherry picked from commit 2c133fb)
    Ma27 committed Mar 29, 2020
    Copy the full SHA
    834fb02 View commit details
  2. buildRustCrateTests: Fix link order test on darwin

    As it turns out Darwin does most of the things differently then "normal"
    systems. They are using a different shared library extension and require
    an obscure commandline parameter that has to be added to every build
    system out there. That issue seems to be with clang on Darwin as on
    Linux that flag isn't required to build the very same tests (when using
    clang).
    
    After adjusting these two details the tests are running fine on the
    darwin box that I was able to obtain.
    
    (cherry picked from commit c8de31b)
    andir committed Mar 29, 2020
    Copy the full SHA
    32cbd89 View commit details
  3. buildRustCrate: Add tests for checking files in outputs.

    ...and remove superfluous dependency files (*.d).
    ...and copy dSYM directories on Mac OS when in release=false mode.
    
    (cherry picked from commit 782b304)
    kolloch authored and andir committed Mar 29, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    040f611 View commit details

Commits on Apr 3, 2020

  1. freeoffice: 973 -> 974

    Cherry picked from PR #82038
    
    Changelog:
    
    https://www.freeoffice.com/en/download/servicepacks
    (cherry picked from commit 44085e3)
    danieldk authored and romildo committed Apr 3, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    a1d49d6 View commit details
  2. freeoffice: 974 -> 976

    Cherry picked from PR #83495
    
    (cherry picked from commit 55b6146)
    romildo committed Apr 3, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    26df648 View commit details
  3. sympa: 6.2.52 -> 6.2.54

    (cherry picked from commit adc7388)
    mmilata committed Apr 3, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    769a5cc View commit details
  4. sympa: build with --enable-fhs

    Update module accordingly.
    
    (cherry picked from commit 8f632b4)
    mmilata committed Apr 3, 2020
    Copy the full SHA
    b95cf62 View commit details
  5. nixos/sympa: fix outgoing messaging

    Because ProtectKernelModules implies NoNewPrivileges, postfix's sendmail
    executable, which is setgid, wasn't able to send mail.
    
    (cherry picked from commit fdc36e2)
    mmilata committed Apr 3, 2020

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    5edcadc View commit details
  6. Merge pull request #83628 from Ma27/mongodb-20.03

    [20.03] mongodb: update
    Ma27 authored Apr 3, 2020
    Copy the full SHA
    0abd4fd View commit details
  7. grocy: 2.6.1 -> 2.6.2

    Ma27 committed Apr 3, 2020
    Copy the full SHA
    995229d View commit details
  8. lutris: custom wine installations require libkrb5

    fixes the error: symbol k5_os_mutex_destroy version krb5support_0_MIT not defined in file libkrb5support.so.0
    
    (cherry picked from commit 41fe7db)
    RohanHart authored and Jon committed Apr 3, 2020
    Copy the full SHA
    449c060 View commit details
  9. plan9port: fix linker flags for macosx

    (cherry picked from commit 4d2a825)
    (cherry picked from commit 7724ef7)
    eraserhd authored and LnL7 committed Apr 3, 2020
    Copy the full SHA
    6693266 View commit details
  10. bazel: fix linker flags for darwin

    (cherry picked from commit d1ee615)
    marsam authored and LnL7 committed Apr 3, 2020
    Copy the full SHA
    f941714 View commit details
  11. Merge pull request #80890 from LnL7/darwin-ldflags

    darwin: fix NIX_LDFLAGS usages
    LnL7 authored Apr 3, 2020
    Copy the full SHA
    1c32029 View commit details
  12. fix wireguard service as well after it got upstreamed.

    (cherry picked from commit 50fb52d)
    ikervagyok authored and rnhmjoj committed Apr 3, 2020
    Copy the full SHA
    8224137 View commit details
  13. nixos/wg-quick: Fix after wireguard got upstreamed

    (cherry picked from commit 644d643)
    bkchr authored and rnhmjoj committed Apr 3, 2020
    Copy the full SHA
    ef33b9f View commit details
  14. pythonPackages.pint: 0.9 -> 0.11

    (cherry picked from commit 4b822bb)
    drewrisinger committed Apr 3, 2020
    Copy the full SHA
    029276f View commit details
  15. datasette: 0.35 -> 0.39

    (cherry picked from commit fb25290)
    drewrisinger committed Apr 3, 2020
    Copy the full SHA
    a1d1b7a View commit details
  16. Merge pull request #84216 from drewrisinger/dr-pr-backport-20.03-84197

    [20.03] python3Packages.pint: 0.9 -> 0.11
    bhipple authored Apr 3, 2020
    Copy the full SHA
    e8610d7 View commit details
  17. Merge pull request #84184 from romildo/upd.freeoffice

    [20.03] freeoffice: 973 -> 976
    bhipple authored Apr 3, 2020
    Copy the full SHA
    aa3c504 View commit details
  18. Merge pull request #83601 from andir/20.03-buildRustCrate-fixes

    [20.03] buildRustCrate fixes backports
    flokli authored Apr 3, 2020
    Copy the full SHA
    e46f456 View commit details

Commits on Apr 4, 2020

  1. Merge pull request #84202 from mmilata/sympa-outgoing-20.03

    [20.03] nixos/sympa: fix outgoing emails, update package version
    bhipple authored Apr 4, 2020
    Copy the full SHA
    4dc8447 View commit details
Showing with 782 additions and 213 deletions.
  1. +7 −0 lib/licenses.nix
  2. +14 −0 nixos/doc/manual/release-notes/rl-2003.xml
  3. +5 −7 nixos/modules/services/mail/sympa.nix
  4. +1 −1 nixos/modules/services/networking/wg-quick.nix
  5. +1 −1 nixos/modules/services/networking/wireguard.nix
  6. +47 −37 nixos/tests/mongodb.nix
  7. +1 −1 pkgs/applications/misc/lutris/chrootenv.nix
  8. +2 −2 pkgs/applications/office/softmaker/freeoffice.nix
  9. +3 −5 pkgs/build-support/rust/build-rust-crate/build-crate.nix
  10. +65 −20 pkgs/build-support/rust/build-rust-crate/configure-crate.nix
  11. +10 −11 pkgs/build-support/rust/build-rust-crate/default.nix
  12. +2 −2 pkgs/build-support/rust/build-rust-crate/install-crate.nix
  13. +44 −8 pkgs/build-support/rust/build-rust-crate/lib.sh
  14. +48 −22 pkgs/build-support/rust/build-rust-crate/log.nix
  15. +268 −3 pkgs/build-support/rust/build-rust-crate/test/default.nix
  16. +32 −21 pkgs/development/python-modules/datasette/default.nix
  17. +23 −3 pkgs/development/python-modules/pint/default.nix
  18. +10 −2 pkgs/development/tools/build-managers/bazel/bazel_2/default.nix
  19. +2 −2 pkgs/servers/grocy/default.nix
  20. +8 −4 pkgs/servers/mail/sympa/default.nix
  21. +20 −0 pkgs/servers/nosql/mongodb/asio-no-experimental-string-view.patch
  22. +17 −0 pkgs/servers/nosql/mongodb/forget-build-dependencies-3-4.patch
  23. +3 −7 pkgs/servers/nosql/mongodb/forget-build-dependencies.patch
  24. +52 −41 pkgs/servers/nosql/mongodb/{default.nix → mongodb.nix}
  25. +27 −0 pkgs/servers/nosql/mongodb/mozjs-45_fix-3-byte-opcode.patch
  26. +15 −0 pkgs/servers/nosql/mongodb/v3_4.nix
  27. +16 −0 pkgs/servers/nosql/mongodb/v3_6.nix
  28. +17 −0 pkgs/servers/nosql/mongodb/v4_0.nix
  29. +3 −10 pkgs/tools/system/plan9port/builder.sh
  30. +19 −3 pkgs/top-level/all-packages.nix
7 changes: 7 additions & 0 deletions lib/licenses.nix
Original file line number Diff line number Diff line change
@@ -649,6 +649,13 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
url = http://metadata.ftp-master.debian.org/changelogs/main/d/debianutils/debianutils_4.8.1_copyright;
};

sspl = {
shortName = "SSPL";
fullName = "Server Side Public License";
url = https://www.mongodb.com/licensing/server-side-public-license;
free = false;
};

tcltk = spdx {
spdxId = "TCL";
fullName = "TCL/TK License";
14 changes: 14 additions & 0 deletions nixos/doc/manual/release-notes/rl-2003.xml
Original file line number Diff line number Diff line change
@@ -1141,6 +1141,20 @@ auth required pam_succeed_if.so uid >= 1000 quiet
not <command>systemd-networkd</command>.
</para>
</listitem>
<listitem>
<para>
<package>mongodb</package> has been updated to version <literal>3.4.24</literal>.
<warning>
<para>
Please note that <package>mongodb</package> has been relicensed under their own
<link xlink:href="https://www.mongodb.com/licensing/server-side-public-license/faq"><literal>
sspl</literal></link>-license. Since it's not entirely free and not OSI-approved,
it's listed as non-free. This means that Hydra doesn't provide prebuilt
<package>mongodb</package>-packages and needs to be built locally.
</para>
</warning>
</para>
</listitem>
</itemizedlist>
</section>
</section>
12 changes: 5 additions & 7 deletions nixos/modules/services/mail/sympa.nix
Original file line number Diff line number Diff line change
@@ -25,8 +25,6 @@ let
StateDirectory = "sympa";
ProtectHome = true;
ProtectSystem = "full";
ProtectKernelTunables = true;
ProtectKernelModules = true;
ProtectControlGroups = true;
};

@@ -415,7 +413,7 @@ in
# force-copy static_content so it's up to date with package
# set permissions for wwsympa which needs write access (...)
"R ${dataDir}/static_content - - - - -"
"C ${dataDir}/static_content 0711 ${user} ${group} - ${pkg}/static_content"
"C ${dataDir}/static_content 0711 ${user} ${group} - ${pkg}/var/lib/sympa/static_content"
"e ${dataDir}/static_content/* 0711 ${user} ${group} - -"

"d /run/sympa 0755 ${user} ${group} - -"
@@ -497,7 +495,7 @@ in
-F ${toString cfg.web.fcgiProcs} \
-P /run/sympa/wwsympa.pid \
-s /run/sympa/wwsympa.socket \
-- ${pkg}/bin/wwsympa.fcgi
-- ${pkg}/lib/sympa/cgi/wwsympa.fcgi
'';

} // commonServiceConfig;
@@ -518,7 +516,7 @@ in
fastcgi_split_path_info ^(${loc})(.*)$;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_param SCRIPT_FILENAME ${pkg}/bin/wwsympa.fcgi;
fastcgi_param SCRIPT_FILENAME ${pkg}/lib/sympa/cgi/wwsympa.fcgi;
'';
}) // {
"/static-sympa/".alias = "${dataDir}/static_content/";
@@ -550,7 +548,7 @@ in
args = [
"flags=hqRu"
"user=${user}"
"argv=${pkg}/bin/queue"
"argv=${pkg}/libexec/queue"
"\${nexthop}"
];
};
@@ -562,7 +560,7 @@ in
args = [
"flags=hqRu"
"user=${user}"
"argv=${pkg}/bin/bouncequeue"
"argv=${pkg}/libexec/bouncequeue"
"\${nexthop}"
];
};
2 changes: 1 addition & 1 deletion nixos/modules/services/networking/wg-quick.nix
Original file line number Diff line number Diff line change
@@ -302,7 +302,7 @@ in {
###### implementation

config = mkIf (cfg.interfaces != {}) {
boot.extraModulePackages = [ kernel.wireguard ];
boot.extraModulePackages = optional (versionOlder kernel.kernel.version "5.6") kernel.wireguard;
environment.systemPackages = [ pkgs.wireguard-tools ];
# This is forced to false for now because the default "--validmark" rpfilter we apply on reverse path filtering
# breaks the wg-quick routing because wireguard packets leave with a fwmark from wireguard.
2 changes: 1 addition & 1 deletion nixos/modules/services/networking/wireguard.nix
Original file line number Diff line number Diff line change
@@ -435,7 +435,7 @@ in
message = "networking.wireguard.interfaces.${interfaceName} peer «${peer.publicKey}» has both presharedKey and presharedKeyFile set, but only one can be used.";
}) all_peers;

boot.extraModulePackages = [ kernel.wireguard ];
boot.extraModulePackages = optional (versionOlder kernel.kernel.version "5.6") kernel.wireguard;
environment.systemPackages = [ pkgs.wireguard-tools ];

systemd.services =
84 changes: 47 additions & 37 deletions nixos/tests/mongodb.nix
Original file line number Diff line number Diff line change
@@ -1,42 +1,52 @@
# This test start mongodb, runs a query using mongo shell

import ./make-test-python.nix ({ pkgs, ...} : let
testQuery = pkgs.writeScript "nixtest.js" ''
db.greetings.insert({ "greeting": "hello" });
print(db.greetings.findOne().greeting);
'';
in {
name = "mongodb";
meta = with pkgs.stdenv.lib.maintainers; {
maintainers = [ bluescreen303 offline cstrahan rvl phile314 ];
};
import ./make-test-python.nix ({ pkgs, ... }:
let
testQuery = pkgs.writeScript "nixtest.js" ''
db.greetings.insert({ "greeting": "hello" });
print(db.greetings.findOne().greeting);
'';

nodes = {
one =
{ ... }:
{
services = {
mongodb.enable = true;
mongodb.enableAuth = true;
mongodb.initialRootPassword = "root";
mongodb.initialScript = pkgs.writeText "mongodb_initial.js" ''
db = db.getSiblingDB("nixtest");
db.createUser({user:"nixtest",pwd:"nixtest",roles:[{role:"readWrite",db:"nixtest"}]});
'';
mongodb.extraConfig = ''
# Allow starting engine with only a small virtual disk
storage.journal.enabled: false
storage.mmapv1.smallFiles: true
'';
};
};
runMongoDBTest = pkg: ''
node.execute("(rm -rf data || true) && mkdir data")
node.execute(
"${pkg}/bin/mongod --fork --logpath logs --dbpath data"
)
node.wait_for_open_port(27017)
assert "hello" in node.succeed(
"mongo ${testQuery}"
)
node.execute(
"${pkg}/bin/mongod --shutdown --dbpath data"
)
node.wait_for_closed_port(27017)
'';

in {
name = "mongodb";
meta = with pkgs.stdenv.lib.maintainers; {
maintainers = [ bluescreen303 offline cstrahan rvl phile314 ];
};

nodes = {
node = {...}: {
environment.systemPackages = with pkgs; [
mongodb-3_4
mongodb-3_6
mongodb-4_0
];
};
};

testScript = ''
start_all()
one.wait_for_unit("mongodb.service")
one.succeed(
"mongo -u nixtest -p nixtest nixtest ${testQuery} | grep -q hello"
)
'';
})
testScript = ''
node.start()
''
+ runMongoDBTest pkgs.mongodb-3_4
+ runMongoDBTest pkgs.mongodb-3_6
+ runMongoDBTest pkgs.mongodb-4_0
+ ''
node.shutdown()
'';
})
2 changes: 1 addition & 1 deletion pkgs/applications/misc/lutris/chrootenv.nix
Original file line number Diff line number Diff line change
@@ -103,7 +103,7 @@ in buildFHSUserEnv {

# WINE
cups lcms2 mpg123 cairo unixODBC samba4 sane-backends openldap
ocl-icd utillinux
ocl-icd utillinux libkrb5

# Winetricks
fribidi
4 changes: 2 additions & 2 deletions pkgs/applications/office/softmaker/freeoffice.nix
Original file line number Diff line number Diff line change
@@ -2,13 +2,13 @@

callPackage ./generic.nix (args // rec {
pname = "freeoffice";
version = "973";
version = "976";
edition = "2018";
suiteName = "FreeOffice";

src = fetchurl {
url = "https://www.softmaker.net/down/softmaker-freeoffice-${version}-amd64.tgz";
sha256 = "0xac4ynf1lfh8qmni5bhp4ybaamdfngva4bqaq21n1m4pgrx1ba5";
sha256 = "13yh4lyqakbdqf4r8vw8imy5gwpfva697iqfd85qmp3wimqvzskl";
};

archive = "freeoffice${edition}.tar.lzma";
8 changes: 3 additions & 5 deletions pkgs/build-support/rust/build-rust-crate/build-crate.nix
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{ lib, stdenv, echo_build_heading, noisily, mkRustcDepArgs, rust }:
{ lib, stdenv, mkRustcDepArgs, rust }:
{ crateName,
dependencies,
crateFeatures, crateRenames, libName, release, libPath,
@@ -11,6 +11,7 @@
baseRustcOpts =
[(if release then "-C opt-level=3" else "-C debuginfo=2")]
++ ["-C codegen-units=$NIX_BUILD_CORES"]
++ ["--remap-path-prefix=$NIX_BUILD_TOP=/" ]
++ [(mkRustcDepArgs dependencies crateRenames)]
++ [crateFeatures]
++ extraRustcOpts
@@ -34,16 +35,13 @@
build_bin = if buildTests then "build_bin_test" else "build_bin";
in ''
runHook preBuild
${echo_build_heading colors}
${noisily colors verbose}
# configure & source common build functions
LIB_RUSTC_OPTS="${libRustcOpts}"
BIN_RUSTC_OPTS="${binRustcOpts}"
LIB_EXT="${stdenv.hostPlatform.extensions.sharedLibrary}"
LIB_PATH="${libPath}"
LIB_NAME="${libName}"
source ${./lib.sh}
CRATE_NAME='${lib.replaceStrings ["-"] ["_"] libName}'
85 changes: 65 additions & 20 deletions pkgs/build-support/rust/build-rust-crate/configure-crate.nix
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{ lib, stdenv, echo_build_heading, noisily, mkRustcDepArgs }:
{ lib, stdenv, echo_colored, noisily, mkRustcDepArgs }:
{ build
, buildDependencies
, colors
@@ -31,42 +31,88 @@ let version_ = lib.splitString "-" crateVersion;
completeDepsDir = lib.concatStringsSep " " completeDeps;
completeBuildDepsDir = lib.concatStringsSep " " completeBuildDeps;
in ''
cd ${workspace_member}
runHook preConfigure
${echo_build_heading colors}
${echo_colored colors}
${noisily colors verbose}
source ${./lib.sh}
${lib.optionalString (workspace_member != null) ''
noisily cd "${workspace_member}"
''}
${lib.optionalString (workspace_member == null) ''
echo_colored "Searching for matching Cargo.toml (${crateName})"
local cargo_toml_dir=$(matching_cargo_toml_dir "${crateName}")
if [ -z "$cargo_toml_dir" ]; then
echo_error "ERROR configuring ${crateName}: No matching Cargo.toml in $(pwd) found." >&2
exit 23
fi
noisily cd "$cargo_toml_dir"
''}
runHook preConfigure
symlink_dependency() {
# $1 is the nix-store path of a dependency
# $2 is the target path
i=$1
ln -s -f $i/lib/*.rlib $2 #*/
ln -s -f $i/lib/*.so $i/lib/*.dylib $2 #*/
if [ -e "$i/lib/link" ]; then
cat $i/lib/link >> target/link
cat $i/lib/link >> target/link.final
fi
if [ -e $i/env ]; then
source $i/env
fi
}
# The following steps set up the dependencies of the crate. Two
# kinds of dependencies are distinguished: build dependencies
# (used by the build script) and crate dependencies. For each
# dependency we have to:
#
# - Make its Rust library available to rustc. This is done by
# symlinking all library dependencies into a directory that
# can be provided to rustc.
# - Accumulate linking flags. These flags are largely used for
# linking native libraries.
#
# The crate link flags are added to the `link` and `link.final`
# files. The `link` file is used for linkage in the current
# crate. The `link.final` file will be copied to the output and can
# be used by downstream crates to get the linker flags of this
# crate.
mkdir -p target/{deps,lib,build,buildDeps}
chmod uga+w target -R
echo ${extraLinkFlags} > target/link
echo ${extraLinkFlags} > target/link.final
# Prepare crate dependencies
for i in ${completeDepsDir}; do
symlink_dependency $i target/deps
if [ -e "$i/lib/link" ]; then
cat $i/lib/link >> target/link
cat $i/lib/link >> target/link.final
fi
done
# Prepare crate build dependencies that are used for the build script.
for i in ${completeBuildDepsDir}; do
symlink_dependency $i target/buildDeps
symlink_dependency $i target/buildDeps
if [ -e "$i/lib/link" ]; then
cat $i/lib/link >> target/link.build
fi
done
if [[ -e target/link ]]; then
sort -u target/link > target/link.sorted
mv target/link.sorted target/link
sort -u target/link.final > target/link.final.sorted
mv target/link.final.sorted target/link.final
tr '\n' ' ' < target/link > target/link_
# Remove duplicate linker flags from the build dependencies.
if [[ -e target/link.build ]]; then
sort -uo target/link.build target/link.build
fi
# Remove duplicate linker flags from the dependencies.
sort -uo target/link target/link
tr '\n' ' ' < target/link > target/link_
# Remove duplicate linker flags from the that are written
# to the derivation's output.
sort -uo target/link.final target/link.final
EXTRA_BUILD=""
BUILD_OUT_DIR=""
export CARGO_PKG_NAME=${crateName}
@@ -105,15 +151,14 @@ in ''
elif [[ -e "build.rs" ]]; then
BUILD="build.rs"
fi
# Compile and run the build script, when available.
if [[ ! -z "$BUILD" ]] ; then
echo_build_heading "$BUILD" ${libName}
mkdir -p target/build/${crateName}
EXTRA_BUILD_FLAGS=""
if [ -e target/link_ ]; then
EXTRA_BUILD_FLAGS=$(cat target/link_)
fi
if [ -e target/link.build ]; then
EXTRA_BUILD_FLAGS="$EXTRA_BUILD_FLAGS $(cat target/link.build)"
EXTRA_BUILD_FLAGS="$EXTRA_BUILD_FLAGS $(tr '\n' ' ' < target/link.build)"
fi
noisily rustc --crate-name build_script_build $BUILD --crate-type bin ${rustcOpts} \
${crateFeatures} --out-dir target/build/${crateName} --emit=dep-info,link \
@@ -127,7 +172,7 @@ in ''
set +e
EXTRA_BUILD=$(sed -n "s/^cargo:rustc-flags=\(.*\)/\1/p" target/build/${crateName}.opt | tr '\n' ' ' | sort -u)
EXTRA_FEATURES=$(sed -n "s/^cargo:rustc-cfg=\(.*\)/--cfg \1/p" target/build/${crateName}.opt | tr '\n' ' ')
EXTRA_LINK=$(sed -n "s/^cargo:rustc-link-lib=\(.*\)/\1/p" target/build/${crateName}.opt | tr '\n' ' ' | sort -u)
EXTRA_LINK=$(sed -n "s/^cargo:rustc-link-lib=\(.*\)/\1/p" target/build/${crateName}.opt | tr '\n' ' ')
EXTRA_LINK_SEARCH=$(sed -n "s/^cargo:rustc-link-search=\(.*\)/\1/p" target/build/${crateName}.opt | tr '\n' ' ' | sort -u)
for env in $(sed -n "s/^cargo:rustc-env=\(.*\)/\1/p" target/build/${crateName}.opt); do
21 changes: 10 additions & 11 deletions pkgs/build-support/rust/build-rust-crate/default.nix
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@
# This can be useful for deploying packages with NixOps, and to share
# binary dependencies between projects.

{ lib, stdenv, defaultCrateOverrides, fetchCrate, rustc, rust }:
{ lib, stdenv, defaultCrateOverrides, fetchCrate, rustc, rust, cargo, jq }:

let
# This doesn't appear to be officially documented anywhere yet.
@@ -29,14 +29,14 @@ let
" --extern ${name}=${dep.lib}/lib/lib${extern}-${dep.metadata}${stdenv.hostPlatform.extensions.sharedLibrary}")
) dependencies;

inherit (import ./log.nix { inherit lib; }) noisily echo_build_heading;
inherit (import ./log.nix { inherit lib; }) noisily echo_colored;

configureCrate = import ./configure-crate.nix {
inherit lib stdenv echo_build_heading noisily mkRustcDepArgs;
inherit lib stdenv echo_colored noisily mkRustcDepArgs;
};

buildCrate = import ./build-crate.nix {
inherit lib stdenv echo_build_heading noisily mkRustcDepArgs rust;
inherit lib stdenv mkRustcDepArgs rust;
};

installCrate = import ./install-crate.nix { inherit stdenv; };
@@ -62,9 +62,6 @@ let crate = crate_ // (lib.attrByPath [ crate_.crateName ] (attr: {}) crateOverr
extraRustcOpts_ = extraRustcOpts;
buildTests_ = buildTests;

# take a list of crates that we depend on and override them to fit our overrides, rustc, release, …
makeDependencies = map (dep: lib.getLib (dep.override { inherit release verbose crateOverrides; }));

# crate2nix has a hack for the old bash based build script that did split
# entries at `,`. No we have to work around that hack.
# https://github.com/kolloch/crate2nix/blame/5b19c1b14e1b0e5522c3e44e300d0b332dc939e7/crate2nix/templates/build.nix.tera#L89
@@ -91,10 +88,10 @@ stdenv.mkDerivation (rec {

src = crate.src or (fetchCrate { inherit (crate) crateName version sha256; });
name = "rust_${crate.crateName}-${crate.version}${lib.optionalString buildTests_ "-test"}";
depsBuildBuild = [ rust stdenv.cc ];
depsBuildBuild = [ rust stdenv.cc cargo jq ];
buildInputs = (crate.buildInputs or []) ++ buildInputs_;
dependencies = makeDependencies dependencies_;
buildDependencies = makeDependencies buildDependencies_;
dependencies = map lib.getLib dependencies_;
buildDependencies = map lib.getLib buildDependencies_;

completeDeps = lib.unique (dependencies ++ lib.concatMap (dep: dep.completeDeps) dependencies);
completeBuildDeps = lib.unique (
@@ -103,7 +100,7 @@ stdenv.mkDerivation (rec {
);

crateFeatures = lib.optionalString (crate ? features)
(lib.concatMapStringsSep " " (f: "--cfg feature=\\\"${f}\\\"") (crate.features ++ features));
(lib.concatMapStringsSep " " (f: ''--cfg feature=\"${f}\"'') (crate.features ++ features));

libName = if crate ? libName then crate.libName else crate.crateName;
libPath = if crate ? libPath then crate.libPath else "";
@@ -117,6 +114,8 @@ stdenv.mkDerivation (rec {
in lib.substring 0 10 hashedMetadata;

build = crate.build or "";
# Either set to a concrete sub path to the crate root
# or use `null` for auto-detect.
workspace_member = crate.workspace_member or ".";
crateVersion = crate.version;
crateDescription = crate.description or "";
4 changes: 2 additions & 2 deletions pkgs/build-support/rust/build-rust-crate/install-crate.nix
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ if !buildTests then ''
fi
if [[ "$(ls -A target/lib)" ]]; then
mkdir -p $lib/lib
cp target/lib/* $lib/lib #*/
cp -r target/lib/* $lib/lib #*/
for library in $lib/lib/*.so $lib/lib/*.dylib; do #*/
ln -s $library $(echo $library | sed -e "s/-${metadata}//")
done
@@ -26,7 +26,7 @@ if !buildTests then ''
if [[ -d target/bin ]]; then
if [[ "$(ls -A target/bin)" ]]; then
mkdir -p $out/bin
cp -P target/bin/* $out/bin # */
cp -rP target/bin/* $out/bin # */
fi
fi
runHook postInstall
52 changes: 44 additions & 8 deletions pkgs/build-support/rust/build-rust-crate/lib.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
echo_build_heading() {
if (( $# == 1 )); then
echo_colored "Building $1"
else
echo_colored "Building $1 ($2)"
fi
}

build_lib() {
lib_src=$1
echo_build_heading $lib_src ${libName}
@@ -6,7 +14,6 @@ build_lib() {
--crate-name $CRATE_NAME \
$lib_src \
--out-dir target/lib \
--emit=dep-info,link \
-L dependency=target/deps \
--cap-lints allow \
$LIB_RUSTC_OPTS \
@@ -37,7 +44,6 @@ build_bin() {
--crate-type bin \
$BIN_RUSTC_OPTS \
--out-dir target/bin \
--emit=dep-info,link \
-L dependency=target/deps \
$LINK \
$EXTRA_LIB \
@@ -71,6 +77,7 @@ build_bin_test_file() {
build_bin_test "$derived_crate_name" "$file"
}

# Add additional link options that were provided by the build script.
setup_link_paths() {
EXTRA_LIB=""
if [[ -e target/link_ ]]; then
@@ -96,11 +103,6 @@ setup_link_paths() {
done

if [[ -e target/link ]]; then
sort -u target/link.final > target/link.final.sorted
mv target/link.final.sorted target/link.final
sort -u target/link > target/link.sorted
mv target/link.sorted target/link

tr '\n' ' ' < target/link > target/link_
LINK=$(cat target/link_)
fi
@@ -132,7 +134,41 @@ search_for_bin_path() {
done

if [[ -z "$BIN_PATH" ]]; then
echo "failed to find file for binary target: $BIN_NAME" >&2
echo_error "ERROR: failed to find file for binary target: $BIN_NAME" >&2
exit 1
fi
}

# Extracts cargo_toml_path of the matching crate.
matching_cargo_toml_path() {
local manifest_path="$1"
local expected_crate_name="$2"

# If the Cargo.toml is not a workspace root,
# it will only contain one package in ".packages"
# because "--no-deps" suppressed dependency resolution.
#
# But to make it more general, we search for a matching
# crate in all packages and use the manifest path that
# is referenced there.
cargo metadata --no-deps --format-version 1 \
--manifest-path "$manifest_path" \
| jq -r '.packages[]
| select( .name == "'$expected_crate_name'")
| .manifest_path'
}

# Find a Cargo.toml in the current or any sub directory
# with a matching crate name.
matching_cargo_toml_dir() {
local expected_crate_name="$1"

find -L -name Cargo.toml | sort | while read manifest_path; do
echo "...checking manifest_path $manifest_path" >&2
local matching_path="$(matching_cargo_toml_path "$manifest_path" "$expected_crate_name")"
if [ -n "${matching_path}" ]; then
echo "$(dirname $matching_path)"
break
fi
done
}
70 changes: 48 additions & 22 deletions pkgs/build-support/rust/build-rust-crate/log.nix
Original file line number Diff line number Diff line change
@@ -1,30 +1,56 @@
{ lib }:
{
echo_build_heading = colors: ''
echo_build_heading() {
start=""
end=""
${lib.optionalString (colors == "always") ''
start="$(printf '\033[0;1;32m')" #set bold, and set green.
end="$(printf '\033[0m')" #returns to "normal"
''}
if (( $# == 1 )); then
echo "$start""Building $1""$end"
else
echo "$start""Building $1 ($2)""$end"
fi

let echo_colored_body = start_escape:
# Body of a function that behaves like "echo" but
# has the output colored by the given start_escape
# sequence. E.g.
#
# * echo_x "Building ..."
# * echo_x -n "Running "
#
# This is more complicated than apparent at first sight
# because:
# * The color markers and the text must be print
# in the same echo statement. Otherise, other
# intermingled text from concurrent builds will
# be colored as well.
# * We need to preserve the trailing newline of the
# echo if and only if it is present. Bash likes
# to strip those if we capture the output of echo
# in a variable.
# * Leading "-" will be interpreted by test as an
# option for itself. Therefore, we prefix it with
# an x in `[[ "x$1" =~ ^x- ]]`.
''
local echo_args="";
while [[ "x$1" =~ ^x- ]]; do
echo_args+=" $1"
shift
done
local start_escape="$(printf '${start_escape}')"
local reset="$(printf '\033[0m')"
echo $echo_args $start_escape"$@"$reset
'';
echo_conditional_colored_body = colors: start_escape:
if colors == "always"
then (echo_colored_body start_escape)
else ''echo "$@"'';
in {
echo_colored = colors: ''
echo_colored() {
${echo_conditional_colored_body colors ''\033[0;1;32m''}
}
'';
echo_error() {
${echo_conditional_colored_body colors ''\033[0;1;31m''}
}
'';

noisily = colors: verbose: ''
noisily() {
start=""
end=""
${lib.optionalString (colors == "always") ''
start="$(printf '\033[0;1;32m')" #set bold, and set green.
end="$(printf '\033[0m')" #returns to "normal"
''}
${lib.optionalString verbose ''
echo -n "$start"Running "$end"
echo_colored -n "Running "
echo $@
''}
$@
271 changes: 268 additions & 3 deletions pkgs/build-support/rust/build-rust-crate/test/default.nix
Original file line number Diff line number Diff line change
@@ -1,4 +1,14 @@
{ lib, buildRustCrate, runCommand, writeTextFile, symlinkJoin, callPackage, releaseTools }:
{ lib
, buildRustCrate
, callPackage
, releaseTools
, runCommand
, runCommandCC
, stdenv
, symlinkJoin
, writeTextFile
}:

let
mkCrate = args: let
p = {
@@ -8,6 +18,14 @@ let
} // args;
in buildRustCrate p;

mkCargoToml =
{ name, crateVersion ? "0.1.0", path ? "Cargo.toml" }:
mkFile path ''
[package]
name = ${builtins.toJSON name}
version = ${builtins.toJSON crateVersion}
'';

mkFile = destination: text: writeTextFile {
name = "src";
destination = "/${destination}";
@@ -86,10 +104,62 @@ let
''
);

/* Returns a derivation that asserts that the crate specified by `crateArgs`
has the specified files as output.
`name` is used as part of the derivation name that performs the checking.
`crateArgs` is passed to `mkCrate` to build the crate with `buildRustCrate`.
`expectedFiles` contains a list of expected file paths in the output. E.g.
`[ "./bin/my_binary" ]`.
`output` specifies the name of the output to use. By default, the default
output is used but e.g. `output = "lib";` will cause the lib output
to be checked instead. You do not need to specify any directories.
*/
assertOutputs = { name, crateArgs, expectedFiles, output? null }:
assert (builtins.isString name);
assert (builtins.isAttrs crateArgs);
assert (builtins.isList expectedFiles);

let
crate = mkCrate (builtins.removeAttrs crateArgs ["expectedTestOutput"]);
crateOutput = if output == null then crate else crate."${output}";
expectedFilesFile = writeTextFile {
name = "expected-files-${name}";
text =
let sorted = builtins.sort (a: b: a<b) expectedFiles;
concatenated = builtins.concatStringsSep "\n" sorted;
in "${concatenated}\n";
};
in
runCommand "assert-outputs-${name}" {
} ''
local actualFiles=$(mktemp)
cd "${crateOutput}"
find . -type f | sort >$actualFiles
diff -q ${expectedFilesFile} $actualFiles >/dev/null || {
echo -e "\033[0;1;31mERROR: Difference in expected output files in ${crateOutput} \033[0m" >&2
echo === Got:
sed -e 's/^/ /' $actualFiles
echo === Expected:
sed -e 's/^/ /' ${expectedFilesFile}
echo === Diff:
diff -u ${expectedFilesFile} $actualFiles |\
tail -n +3 |\
sed -e 's/^/ /'
exit 1
}
touch $out
''
;

in rec {

tests = let
cases = {
cases = rec {
libPath = { libPath = "src/my_lib.rs"; src = mkLib "src/my_lib.rs"; };
srcLib = { src = mkLib "src/lib.rs"; };

@@ -199,6 +269,36 @@ let
})
];
};
buildScriptDeps = let
depCrate = boolVal: mkCrate {
crateName = "bar";
src = mkFile "src/lib.rs" ''
pub const baz: bool = ${boolVal};
'';
};
in {
crateName = "foo";
src = symlinkJoin {
name = "build-script-and-main";
paths = [
(mkFile "src/main.rs" ''
extern crate bar;
#[cfg(test)]
#[test]
fn baz_false() { assert!(!bar::baz); }
fn main() { }
'')
(mkFile "build.rs" ''
extern crate bar;
fn main() { assert!(bar::baz); }
'')
];
};
buildDependencies = [ (depCrate "true") ];
dependencies = [ (depCrate "false") ];
buildTests = true;
expectedTestOutputs = [ "test baz_false ... ok" ];
};
# Regression test for https://github.com/NixOS/nixpkgs/issues/74071
# Whenevever a build.rs file is generating files those should not be overlayed onto the actual source dir
buildRsOutDirOverlay = {
@@ -220,9 +320,174 @@ let
];
};
};
# Regression test for https://github.com/NixOS/nixpkgs/pull/83379
# link flag order should be preserved
linkOrder = {
src = symlinkJoin {
name = "buildrs-out-dir-overlay";
paths = [
(mkFile "build.rs" ''
fn main() {
// in the other order, linkage will fail
println!("cargo:rustc-link-lib=b");
println!("cargo:rustc-link-lib=a");
}
'')
(mkFile "src/main.rs" ''
extern "C" {
fn hello_world();
}
fn main() {
unsafe {
hello_world();
}
}
'')
];
};
buildInputs = let
compile = name: text: let
src = writeTextFile {
name = "${name}-src.c";
inherit text;
};
in runCommandCC name {} ''
mkdir -p $out/lib
# Note: On darwin (which defaults to clang) we have to add
# `-undefined dynamic_lookup` as otherwise the compilation fails.
cc -shared \
${lib.optionalString stdenv.isDarwin "-undefined dynamic_lookup"} \
-o $out/lib/${name}${stdenv.hostPlatform.extensions.sharedLibrary} ${src}
'';
b = compile "libb" ''
#include <stdio.h>
void hello();
void hello_world() {
hello();
printf(" world!\n");
}
'';
a = compile "liba" ''
#include <stdio.h>
void hello() {
printf("hello");
}
'';
in [ a b ];
};
rustCargoTomlInSubDir = {
# The "workspace_member" can be set to the sub directory with the crate to build.
# By default ".", meaning the top level directory is assumed.
# Using null will trigger a search.
workspace_member = null;
src = symlinkJoin rec {
name = "find-cargo-toml";
paths = [
(mkCargoToml { name = "ignoreMe"; })
(mkTestFileWithMain "src/main.rs" "ignore_main")

(mkCargoToml { name = "rustCargoTomlInSubDir"; path = "subdir/Cargo.toml"; })
(mkTestFileWithMain "subdir/src/main.rs" "src_main")
(mkTestFile "subdir/tests/foo/main.rs" "tests_foo")
(mkTestFile "subdir/tests/bar/main.rs" "tests_bar")
];
};
buildTests = true;
expectedTestOutputs = [
"test src_main ... ok"
"test tests_foo ... ok"
"test tests_bar ... ok"
];
};

rustCargoTomlInTopDir =
let
withoutCargoTomlSearch = builtins.removeAttrs rustCargoTomlInSubDir [ "workspace_member" ];
in
withoutCargoTomlSearch // {
expectedTestOutputs = [
"test ignore_main ... ok"
];
};
};
brotliCrates = (callPackage ./brotli-crates.nix {});
in lib.mapAttrs (key: value: mkTest (value // lib.optionalAttrs (!value?crateName) { crateName = key; })) cases // {
tests = lib.mapAttrs (key: value: mkTest (value // lib.optionalAttrs (!value?crateName) { crateName = key; })) cases;
in tests // rec {

crateBinWithPathOutputs = assertOutputs {
name="crateBinWithPath";
crateArgs = {
crateBin = [{ name = "test_binary1"; path = "src/foobar.rs"; }];
src = mkBin "src/foobar.rs";
};
expectedFiles = [
"./bin/test_binary1"
];
};

crateBinWithPathOutputsDebug = assertOutputs {
name="crateBinWithPath";
crateArgs = {
release = false;
crateBin = [{ name = "test_binary1"; path = "src/foobar.rs"; }];
src = mkBin "src/foobar.rs";
};
expectedFiles = [
"./bin/test_binary1"
] ++ lib.optionals stdenv.isDarwin [
# On Darwin, the debug symbols are in a seperate directory.
"./bin/test_binary1.dSYM/Contents/Info.plist"
"./bin/test_binary1.dSYM/Contents/Resources/DWARF/test_binary1"
];
};

crateBinNoPath1Outputs = assertOutputs {
name="crateBinNoPath1";
crateArgs = {
crateBin = [{ name = "my-binary2"; }];
src = mkBin "src/my_binary2.rs";
};
expectedFiles = [
"./bin/my-binary2"
];
};

crateLibOutputs = assertOutputs {
name="crateLib";
output="lib";
crateArgs = {
libName = "test_lib";
type = [ "rlib" ];
libPath = "src/lib.rs";
src = mkLib "src/lib.rs";
};
expectedFiles = [
"./nix-support/propagated-build-inputs"
"./lib/libtest_lib-042a1fdbef.rlib"
"./lib/link"
];
};

crateLibOutputsDebug = assertOutputs {
name="crateLib";
output="lib";
crateArgs = {
release = false;
libName = "test_lib";
type = [ "rlib" ];
libPath = "src/lib.rs";
src = mkLib "src/lib.rs";
};
expectedFiles = [
"./nix-support/propagated-build-inputs"
"./lib/libtest_lib-042a1fdbef.rlib"
"./lib/link"
];
};

brotliTest = let
pkg = brotliCrates.brotli_2_5_0 {};
in runCommand "run-brotli-test-cmd" {
53 changes: 32 additions & 21 deletions pkgs/development/python-modules/datasette/default.nix
Original file line number Diff line number Diff line change
@@ -1,49 +1,52 @@
{ lib
, buildPythonPackage
, fetchFromGitHub
, aiofiles
, click
, click-default-group
, janus
, jinja2
, hupper
, pint
, pluggy
, pytest
, uvicorn
# Check Inputs
, pytestCheckHook
, pytestrunner
, pytest-asyncio
, black
, aiohttp
, beautifulsoup4
, uvicorn
, asgiref
, aiofiles
}:

buildPythonPackage rec {
pname = "datasette";
version = "0.35";
version = "0.39";

src = fetchFromGitHub {
owner = "simonw";
repo = "datasette";
rev = version;
sha256 = "0v6af7agg27lapz1nbab07595v4hl2x5wm2f03drj81f7pm8y7hc";
sha256 = "07d46512bc9sdan9lv39sf1bwlf7vf1bfhcsm825vk7sv7g9kczd";
};

nativeBuildInputs = [ pytestrunner ];

propagatedBuildInputs = [
aiofiles
click
click-default-group
janus
jinja2
hupper
pint
pluggy
uvicorn
aiofiles
];

checkInputs = [
pytest
pytestCheckHook
pytest-asyncio
aiohttp
beautifulsoup4
@@ -53,24 +56,32 @@ buildPythonPackage rec {

postConfigure = ''
substituteInPlace setup.py \
--replace "click-default-group==1.2" "click-default-group" \
--replace "Sanic==0.7.0" "Sanic" \
--replace "hupper==1.0" "hupper" \
--replace "pint~=0.8.1" "pint" \
--replace "pluggy~=0.12.0" "pint" \
--replace "Jinja2==2.10.1" "Jinja2" \
--replace "uvicorn~=0.8.4" "uvicorn"
--replace "click~=7.1.1" "click" \
--replace "click-default-group~=1.2.2" "click-default-group" \
--replace "Jinja2~=2.10.3" "Jinja2" \
--replace "hupper~=1.9" "hupper" \
--replace "pint~=0.9" "pint" \
--replace "pluggy~=0.13.0" "pint" \
--replace "uvicorn~=0.11" "uvicorn" \
--replace "aiofiles~=0.4.0" "aiofiles" \
--replace "janus~=0.4.0" "janus" \
--replace "PyYAML~=5.3" "PyYAML"
'';

# many tests require network access
# test_black fails on darwin
checkPhase = ''
pytest --ignore tests/test_api.py \
--ignore tests/test_csv.py \
--ignore tests/test_html.py \
--ignore tests/test_black.py \
-k 'not facet'
'';
dontUseSetuptoolsCheck = true;
pytestFlagsArray = [
"--ignore=tests/test_api.py"
"--ignore=tests/test_csv.py"
"--ignore=tests/test_html.py"
"--ignore=tests/test_docs.py"
"--ignore=tests/test_black.py"
];
disabledTests = [
"facet"
"_invalid_database" # checks error message when connecting to invalid database
];

meta = with lib; {
description = "An instant JSON API for your SQLite databases";
26 changes: 23 additions & 3 deletions pkgs/development/python-modules/pint/default.nix
Original file line number Diff line number Diff line change
@@ -2,20 +2,40 @@
, buildPythonPackage
, fetchPypi
, isPy27
, pythonOlder
, funcsigs
, setuptools_scm
# Check Inputs
, pytestCheckHook
, numpy
, matplotlib
, uncertainties
}:

buildPythonPackage rec {
pname = "pint";
version = "0.9";
version = "0.11";

src = fetchPypi {
inherit version;
pname = "Pint";
sha256 = "32d8a9a9d63f4f81194c0014b3b742679dce81a26d45127d9810a68a561fe4e2";
sha256 = "0kfgnmcs6z9ndhzvwg2xzhpwxgyyagdsdz5dns1jy40fa1q113rh";
};

propagatedBuildInputs = lib.optional isPy27 funcsigs;
disabled = pythonOlder "3.6";

propagatedBuildInputs = [
setuptools_scm
] ++ lib.optional isPy27 funcsigs;

# Test suite explicitly requires pytest
checkInputs = [
pytestCheckHook
numpy
matplotlib
uncertainties
];
dontUseSetuptoolsCheck = true;

meta = with lib; {
description = "Physical quantities module";
12 changes: 10 additions & 2 deletions pkgs/development/tools/build-managers/bazel/bazel_2/default.nix
Original file line number Diff line number Diff line change
@@ -326,6 +326,14 @@ stdenv.mkDerivation rec {
postPatch = let

darwinPatches = ''
bazelLinkFlags () {
eval set -- "$NIX_LDFLAGS"
local flag
for flag in "$@"; do
printf ' -Wl,%s' "$flag"
done
}
# Disable Bazel's Xcode toolchain detection which would configure compilers
# and linkers from Xcode instead of from PATH
export BAZEL_USE_CPP_ONLY_TOOLCHAIN=1
@@ -335,14 +343,14 @@ stdenv.mkDerivation rec {
# Framework search paths aren't added by bintools hook
# https://github.com/NixOS/nixpkgs/pull/41914
export NIX_LDFLAGS="$NIX_LDFLAGS -F${CoreFoundation}/Library/Frameworks -F${CoreServices}/Library/Frameworks -F${Foundation}/Library/Frameworks"
export NIX_LDFLAGS+=" -F${CoreFoundation}/Library/Frameworks -F${CoreServices}/Library/Frameworks -F${Foundation}/Library/Frameworks"
# libcxx includes aren't added by libcxx hook
# https://github.com/NixOS/nixpkgs/pull/41589
export NIX_CFLAGS_COMPILE="$NIX_CFLAGS_COMPILE -isystem ${libcxx}/include/c++/v1"
# don't use system installed Xcode to run clang, use Nix clang instead
sed -i -E "s;/usr/bin/xcrun (--sdk macosx )?clang;${stdenv.cc}/bin/clang $NIX_CFLAGS_COMPILE $NIX_LDFLAGS -framework CoreFoundation;g" \
sed -i -E "s;/usr/bin/xcrun (--sdk macosx )?clang;${stdenv.cc}/bin/clang $NIX_CFLAGS_COMPILE $(bazelLinkFlags) -framework CoreFoundation;g" \
scripts/bootstrap/compile.sh \
src/tools/xcode/realpath/BUILD \
src/tools/xcode/stdredirect/BUILD \
4 changes: 2 additions & 2 deletions pkgs/servers/grocy/default.nix
Original file line number Diff line number Diff line change
@@ -2,11 +2,11 @@

stdenv.mkDerivation rec {
pname = "grocy";
version = "2.6.1";
version = "2.6.2";

src = fetchurl {
url = "https://github.com/grocy/grocy/releases/download/v${version}/grocy_${version}.zip";
sha256 = "1fq1zlxxhpcxj67xxlgf20dia95xcimgnm13cr56sy9f2vjx58m6";
sha256 = "1cjkyv50vwx24xb1mxgy51mr4qqsqgixjww06rql77d9czmmd94k";
};

nativeBuildInputs = [ unzip ];
12 changes: 8 additions & 4 deletions pkgs/servers/mail/sympa/default.nix
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
{ stdenv, perl, fetchFromGitHub, autoreconfHook
}:
{ stdenv, perl, fetchFromGitHub, autoreconfHook, nixosTests }:

let
dataDir = "/var/lib/sympa";
@@ -64,16 +63,17 @@ let
in
stdenv.mkDerivation rec {
pname = "sympa";
version = "6.2.52";
version = "6.2.54";

src = fetchFromGitHub {
owner = "sympa-community";
repo = pname;
rev = version;
sha256 = "071kx6ryifs2f6fhfky9g297frzp5584kn444af1vb2imzydsbnh";
sha256 = "07wfvr8rrg7pwkl2zglrdri7n42rl9gwrjbaffb8m37wq67s7fca";
};

configureFlags = [
"--enable-fhs"
"--without-initdir"
"--without-unitsdir"
"--without-smrshdir"
@@ -106,6 +106,10 @@ stdenv.mkDerivation rec {
rm -rf "$TMP/bin"
'';

passthru.tests = {
inherit (nixosTests) sympa;
};

meta = with stdenv.lib; {
description = "Open source mailing list manager";
homepage = "https://www.sympa.org";
20 changes: 20 additions & 0 deletions pkgs/servers/nosql/mongodb/asio-no-experimental-string-view.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
diff --git a/src/third_party/asio-master/asio/include/asio/detail/config.hpp b/src/third_party/asio-master/asio/include/asio/detail/config.hpp
index 7fe6a95a..ff4cc56b 100644
--- a/src/third_party/asio-master/asio/include/asio/detail/config.hpp
+++ b/src/third_party/asio-master/asio/include/asio/detail/config.hpp
@@ -786,7 +786,6 @@
# if (__cplusplus >= 201402)
# if __has_include(<experimental/string_view>)
# define ASIO_HAS_STD_STRING_VIEW 1
-# define ASIO_HAS_STD_EXPERIMENTAL_STRING_VIEW 1
# endif // __has_include(<experimental/string_view>)
# endif // (__cplusplus >= 201402)
# endif // defined(__clang__)
@@ -794,7 +793,6 @@
# if ((__GNUC__ == 4) && (__GNUC_MINOR__ >= 9)) || (__GNUC__ > 4)
# if (__cplusplus >= 201402)
# define ASIO_HAS_STD_STRING_VIEW 1
-# define ASIO_HAS_STD_EXPERIMENTAL_STRING_VIEW 1
# endif // (__cplusplus >= 201402)
# endif // ((__GNUC__ == 4) && (__GNUC_MINOR__ >= 7)) || (__GNUC__ > 4)
# endif // defined(__GNUC__)
17 changes: 17 additions & 0 deletions pkgs/servers/nosql/mongodb/forget-build-dependencies-3-4.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
--- a/site_scons/mongo_scons_utils.py
+++ b/site_scons/mongo_scons_utils.py
@@ -84,14 +84,11 @@
def default_buildinfo_environment_data():
return (
('distmod', '$MONGO_DISTMOD', True, True,),
('distarch', '$MONGO_DISTARCH', True, True,),
('cc', '$CC_VERSION', True, False,),
- ('ccflags', '$CCFLAGS', True, False,),
('cxx', '$CXX_VERSION', True, False,),
- ('cxxflags', '$CXXFLAGS', True, False,),
- ('linkflags', '$LINKFLAGS', True, False,),
('target_arch', '$TARGET_ARCH', True, True,),
('target_os', '$TARGET_OS', True, False,),
)

# If you want buildInfo and --version to be relatively empty, set
10 changes: 3 additions & 7 deletions pkgs/servers/nosql/mongodb/forget-build-dependencies.patch
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
--- a/site_scons/mongo_scons_utils.py
+++ b/site_scons/mongo_scons_utils.py
@@ -84,14 +84,11 @@
def default_buildinfo_environment_data():
return (
--- a/site_scons/mongo/generators.py
+++ b/site_scons/mongo/generators.py
@@ -18,10 +18,7 @@ def default_buildinfo_environment_data():
('distmod', '$MONGO_DISTMOD', True, True,),
('distarch', '$MONGO_DISTARCH', True, True,),
('cc', '$CC_VERSION', True, False,),
@@ -13,5 +11,3 @@
('target_arch', '$TARGET_ARCH', True, True,),
('target_os', '$TARGET_OS', True, False,),
)

# If you want buildInfo and --version to be relatively empty, set
Original file line number Diff line number Diff line change
@@ -1,58 +1,63 @@
{ stdenv, fetchurl, fetchpatch, scons, boost, gperftools, pcre-cpp, snappy
, zlib, libyamlcpp, sasl, openssl, libpcap, Security
}:
{ stdenv, fetchurl, scons, boost, gperftools, pcre-cpp, snappy, zlib, libyamlcpp
, sasl, openssl, libpcap, python27, curl, Security, CoreFoundation, cctools }:

# Note:
# The command line tools are written in Go as part of a different package (mongodb-tools)

with stdenv.lib;

let version = "3.4.10";
system-libraries = [
"pcre"
#"asio" -- XXX use package?
#"wiredtiger"
"boost"
"snappy"
"zlib"
#"valgrind" -- mongodb only requires valgrind.h, which is vendored in the source.
#"stemmer" -- not nice to package yet (no versioning, no makefile, no shared libs).
"yaml"
] ++ optionals stdenv.isLinux [ "tcmalloc" ];

in stdenv.mkDerivation {
pname = "mongodb";
{ version, sha256, patches ? [] }@args:

let
python = python27.withPackages (ps: with ps; [ pyyaml typing cheetah ]);
system-libraries = [
"boost"
"pcre"
"snappy"
"yaml"
"zlib"
#"asio" -- XXX use package?
#"stemmer" -- not nice to package yet (no versioning, no makefile, no shared libs).
#"valgrind" -- mongodb only requires valgrind.h, which is vendored in the source.
#"wiredtiger"
] ++ optionals stdenv.isLinux [ "tcmalloc" ];
inherit (stdenv.lib) systems subtractLists;

in stdenv.mkDerivation rec {
inherit version;
name = "mongodb-${version}";

src = fetchurl {
url = "https://fastdl.mongodb.org/src/mongodb-src-r${version}.tar.gz";
sha256 = "1wz2mhl9z0b1bdkg6m8v8mvw9k60mdv5ybq554xn3yjj9z500f24";
inherit sha256;
};

nativeBuildInputs = [ scons ];
buildInputs = [
sasl boost gperftools pcre-cpp snappy
zlib libyamlcpp sasl openssl.dev openssl.out libpcap
] ++ stdenv.lib.optionals stdenv.isDarwin [ Security ];

patches =
[
# MongoDB keeps track of its build parameters, which tricks nix into
# keeping dependencies to build inputs in the final output.
# We remove the build flags from buildInfo data.
./forget-build-dependencies.patch
(fetchpatch {
url = https://projects.archlinux.org/svntogit/community.git/plain/trunk/boost160.patch?h=packages/mongodb;
name = "boost160.patch";
sha256 = "0bvsf3499zj55pzamwjmsssr6x63w434944w76273fr5rxwzcmh8";
})
];
boost
curl
gperftools
libpcap
libyamlcpp
openssl
pcre-cpp
python
sasl
snappy
zlib
] ++ stdenv.lib.optionals stdenv.isDarwin [ Security CoreFoundation cctools ];

# MongoDB keeps track of its build parameters, which tricks nix into
# keeping dependencies to build inputs in the final output.
# We remove the build flags from buildInfo data.
inherit patches;

postPatch = ''
# fix environment variable reading
substituteInPlace SConstruct \
--replace "env = Environment(" "env = Environment(ENV = os.environ,"
'' + stdenv.lib.optionalString stdenv.isDarwin ''
substituteInPlace src/third_party/mozjs-45/extract/js/src/jsmath.cpp --replace 'defined(HAVE_SINCOS)' 0
substituteInPlace src/third_party/s2/s1angle.cc --replace drem remainder
substituteInPlace src/third_party/s2/s1interval.cc --replace drem remainder
@@ -66,13 +71,14 @@ in stdenv.mkDerivation {
--replace 'engine("wiredTiger")' 'engine("mmapv1")'
'';

NIX_CFLAGS_COMPILE = stdenv.lib.optionalString stdenv.cc.isClang "-Wno-unused-command-line-argument";
NIX_CFLAGS_COMPILE = stdenv.lib.optionalString stdenv.cc.isClang
"-Wno-unused-command-line-argument";

sconsFlags = [
"--release"
"--ssl"
#"--rocksdb" # Don't have this packaged yet
"--wiredtiger=${if stdenv.is64bit then "on" else "off"}"
"--wiredtiger=on"
"--js-engine=mozjs"
"--use-sasl-client"
"--disable-warnings-as-errors"
@@ -87,8 +93,13 @@ in stdenv.mkDerivation {
'';

preInstall = ''
mkdir -p $out/lib
mkdir -p "$out/lib"
'';

postInstall = ''
rm -f "$out/bin/install_compass" || true
'';

prefixKey = "--prefix=";

enableParallelBuilding = true;
@@ -97,10 +108,10 @@ in stdenv.mkDerivation {

meta = {
description = "A scalable, high-performance, open source NoSQL database";
homepage = http://www.mongodb.org;
license = licenses.agpl3;
homepage = "http://www.mongodb.org";
license = licenses.sspl;

maintainers = with maintainers; [ bluescreen303 offline cstrahan ];
platforms = platforms.unix;
platforms = subtractLists systems.doubles.i686 systems.doubles.unix;
};
}
27 changes: 27 additions & 0 deletions pkgs/servers/nosql/mongodb/mozjs-45_fix-3-byte-opcode.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# HG changeset patch
# User Dan Gohman <sunfish@mozilla.com>
# Parent d9b405d82cffb07343a5f2fd941e029298c7f6c4
# Bug 1390214 - IonMonkey: Don't test for a 3-byte opcode in a 2-byte opcode predicate.
# https://bug1390214.bmoattachments.org/attachment.cgi?id=8902972

diff --git a/src/third_party/mozjs-45/extract/js/src/jit/x86-shared/Encoding-x86-shared.h b/src/third_party/mozjs-45/extract/js/src/jit/x86-shared/Encoding-x86-shared.h
--- a/src/third_party/mozjs-45/extract/js/src/jit/x86-shared/Encoding-x86-shared.h
+++ b/src/third_party/mozjs-45/extract/js/src/jit/x86-shared/Encoding-x86-shared.h
@@ -310,17 +310,16 @@ enum ThreeByteOpcodeID {

// Test whether the given opcode should be printed with its operands reversed.
inline bool IsXMMReversedOperands(TwoByteOpcodeID opcode)
{
switch (opcode) {
case OP2_MOVSD_WsdVsd: // also OP2_MOVPS_WpsVps
case OP2_MOVAPS_WsdVsd:
case OP2_MOVDQ_WdqVdq:
- case OP3_PEXTRD_EdVdqIb:
return true;
default:
break;
}
return false;
}

enum ThreeByteEscape {
15 changes: 15 additions & 0 deletions pkgs/servers/nosql/mongodb/v3_4.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{ stdenv, callPackage, lib, sasl, boost, Security, CoreFoundation, cctools }:

let
buildMongoDB = callPackage ./mongodb.nix {
inherit sasl;
inherit boost;
inherit Security;
inherit CoreFoundation;
inherit cctools;
};
in buildMongoDB {
version = "3.4.24";
sha256 = "0j6mvgv0jnsnvgkl8505bl88kbxkba66qijlpi1la0dd5pd1imfr";
patches = [ ./forget-build-dependencies-3-4.patch ];
}
16 changes: 16 additions & 0 deletions pkgs/servers/nosql/mongodb/v3_6.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{ stdenv, callPackage, lib, sasl, boost, Security, CoreFoundation, cctools }:

let
buildMongoDB = callPackage ./mongodb.nix {
inherit sasl;
inherit boost;
inherit Security;
inherit CoreFoundation;
inherit cctools;
};
in buildMongoDB {
version = "3.6.13";
sha256 = "1mbvk4bmabrswjdm01jssxcygjpq5799zqyx901nsi12vlcymwg4";
patches = [ ./forget-build-dependencies.patch ]
++ stdenv.lib.optionals stdenv.isDarwin [ ./asio-no-experimental-string-view.patch ];
}
17 changes: 17 additions & 0 deletions pkgs/servers/nosql/mongodb/v4_0.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{ stdenv, callPackage, lib, sasl, boost, Security, CoreFoundation, cctools }:

let
buildMongoDB = callPackage ./mongodb.nix {
inherit sasl;
inherit boost;
inherit Security;
inherit CoreFoundation;
inherit cctools;
};
in buildMongoDB {
version = "4.0.12";
sha256 = "1j8dqa4jr623y87jrdanyib9r7x18srrvdx952q4azcc8zrdwci1";
patches =
[ ./forget-build-dependencies.patch ./mozjs-45_fix-3-byte-opcode.patch ]
++ stdenv.lib.optionals stdenv.isDarwin [ ./asio-no-experimental-string-view.patch ];
}
13 changes: 3 additions & 10 deletions pkgs/tools/system/plan9port/builder.sh
Original file line number Diff line number Diff line change
@@ -5,18 +5,11 @@ export PLAN9_TARGET=$PLAN9

plan9portLinkFlags()
{
local -a linkFlags=()
eval set -- "$NIX_LDFLAGS"
while (( $# > 0 )); do
if [[ $1 = -rpath ]]; then
linkFlags+=( "-Wl,-rpath,$2" )
shift 2
else
linkFlags+=( "$1" )
shift
fi
local flag
for flag in "$@"; do
printf ' -Wl,%s' "$flag"
done
echo "${linkFlags[*]}"
}

configurePhase()
22 changes: 19 additions & 3 deletions pkgs/top-level/all-packages.nix
Original file line number Diff line number Diff line change
@@ -15553,11 +15553,27 @@ in
};
mysql = mariadb; # TODO: move to aliases.nix

mongodb = callPackage ../servers/nosql/mongodb {
mongodb = hiPrio mongodb-3_4;

mongodb-3_4 = callPackage ../servers/nosql/mongodb/v3_4.nix {
sasl = cyrus_sasl;
boost = boost160;
openssl = openssl_1_0_2;
inherit (darwin.apple_sdk.frameworks) Security;
inherit (darwin) cctools;
inherit (darwin.apple_sdk.frameworks) CoreFoundation Security;
};

mongodb-3_6 = callPackage ../servers/nosql/mongodb/v3_6.nix {
sasl = cyrus_sasl;
boost = boost160;
inherit (darwin) cctools;
inherit (darwin.apple_sdk.frameworks) CoreFoundation Security;
};

mongodb-4_0 = callPackage ../servers/nosql/mongodb/v4_0.nix {
sasl = cyrus_sasl;
boost = boost169;
inherit (darwin) cctools;
inherit (darwin.apple_sdk.frameworks) CoreFoundation Security;
};

nginx-sso = callPackage ../servers/nginx-sso { };