Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: NixOS/nix
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: c7866733d7ce
Choose a base ref
...
head repository: NixOS/nix
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 1dc29df1d384
Choose a head ref
  • 17 commits
  • 9 files changed
  • 7 contributors

Commits on Dec 20, 2019

  1. Disable use-sqlite-wal under WSL

    Before:
    
        $ nix-channel --update
        unpacking channels...
        warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
        warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
        warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
        warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
        warning: SQLite database '/nix/var/nix/db/db.sqlite' is busy (SQLITE_PROTOCOL)
    
    After:
    
        $ inst/bin/nix-channel --update
        unpacking channels...
        created 1 symlinks in user environment
    
    I've seen complaints that "sandbox" caused problems under WSL but I'm
    having no problems. I think recent changes could have fixed the issue.
    puffnfresh committed Dec 20, 2019
    Copy the full SHA
    d259232 View commit details

Commits on Dec 22, 2019

  1. Pass -P to cp to preserve symlinks

    This is commonly the default behavior with -R, but POSIX leaves the
    default unspecified.
    michaelforney committed Dec 22, 2019
    Copy the full SHA
    10414d4 View commit details
  2. Merge pull request #3287 from michaelforney/cp-flag

    Pass -P to cp to preserve symlinks
    edolstra authored Dec 22, 2019
    Copy the full SHA
    7dcfa80 View commit details
  3. Merge pull request #3284 from puffnfresh/wsl

    Disable use-sqlite-wal under WSL
    edolstra authored Dec 22, 2019
    Copy the full SHA
    aaf57c9 View commit details

Commits on Dec 23, 2019

  1. Pass -J to tar for xz decompression

    Some tar implementations can't auto-detect compression formats, so
    they must be specified explicitly.
    michaelforney committed Dec 23, 2019
    Copy the full SHA
    43eb7b6 View commit details

Commits on Dec 25, 2019

  1. Merge pull request #3289 from michaelforney/tar-J

    Pass -J to tar for xz decompression
    edolstra authored Dec 25, 2019
    Copy the full SHA
    b0cadf5 View commit details

Commits on Jan 2, 2020

  1. exportReferencesGraph: support working

    Before, we would get:
    
        [deploy@bastion:~]$ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
        these derivations will be built:
          /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
          /nix/store/ssxwmll7v21did1c8j027q0m8w6pg41i-unit-prometheus-alertmanager-irc-notifier.service.drv
          /nix/store/mvyvkj46ay7pp7b1znqbkck2mq98k0qd-unit-script-network-local-commands-start.drv
          /nix/store/vsl1y9mz38qfk6pyirjwnfzfggz5akg6-unit-network-local-commands.service.drv
          /nix/store/wi5ighfwwb83fdmav6z6n2fw6npm9ffl-unit-prometheus-hydra-exporter.service.drv
          /nix/store/x0qkv535n75pbl3xn6nn1w7qkrg9wwyg-unit-prometheus-packet-sd.service.drv
          /nix/store/lv491znsjxdf51xnfxh9ld7r1zg14d52-unit-script-packet-sd-env-key-pre-start.drv
          /nix/store/nw4nzlca49agsajvpibx7zg5b873gk9f-unit-script-packet-sd-env-key-start.drv
          /nix/store/x674wwabdwjrkhnykair4c8mpxa9532w-unit-packet-sd-env-key.service.drv
          /nix/store/ywivz64ilb1ywlv652pkixw3vxzfvgv8-unit-wireguard-wg0.service.drv
          /nix/store/v3b648293g3zl8pnn0m1345nvmyd8dwb-unit-script-acme-selfsigned-status.nixos.org-start.drv
          /nix/store/zci5d3zvr6fgdicz6k7jjka6lmx0v3g4-unit-acme-selfsigned-status.nixos.org.service.drv
          /nix/store/f6pwvnm63d0kw5df0v7sipd1rkhqxk5g-system-units.drv
          /nix/store/iax8071knxk9c7krpm9jqg0lcrawf4lc-etc.drv
          /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
        error: invalid file name 'closure-init-0' in 'exportReferencesGraph'
    
    This was tough to debug, I didn't figure out which one was broken until I did:
    
        nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv 2>&1 | grep  nix/store | xargs -n1 nix-store -r
    
    and then looking at the remaining build graph:
    
        $ nix-store -r /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
        these derivations will be built:
          /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
          /nix/store/grfnl76cahwls0igd2by2pqv0dimi8h2-nixos-system-eris-19.09.20191213.03f3def.drv
        error: invalid file name 'closure-init-0' in 'exportReferencesGraph'
    
    and knowing the initrd build is before the system, then:
    
        $ nix show-derivation /nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv
        {
          "/nix/store/3ka4ihvwh6wsyhpd2qa9f59506mnxvx1-initrd-linux-4.19.88.drv": {
            [...]
            "exportReferencesGraph": "closure-init-0 /nix/store/...-stage-1-init.sh closure-mdadm.conf-1 /nix/store/...-mdadm.conf closure-ubuntu.conf-2 ...",
            [...]
          }
        }
    
    I then searched the repo for "in 'exportReferencesGraph'", found this
    recently updated regex, and realized it was missing a "-".
    grahamc committed Jan 2, 2020
    Copy the full SHA
    c502831 View commit details
  2. Merge pull request #3296 from grahamc/export-reference-graph

    exportReferencesGraph: support working
    edolstra authored Jan 2, 2020
    Copy the full SHA
    3469062 View commit details
  3. passAsFile: hash the attribute name instead of numbering sequentially

    This makes the paths consistent without relying on ordering.
    
    Co-authored-by: edef <edef@edef.eu>
    puckipedia and edef1c committed Jan 2, 2020
    Copy the full SHA
    515c0a2 View commit details
  4. Merge pull request #3297 from edef1c/passasfile-hash

    passAsFile: hash the attribute name instead of numbering sequentially
    edolstra authored Jan 2, 2020
    Copy the full SHA
    3ad4a33 View commit details
  5. passAsFile: leave out the hash prefix

    Having a colon in the path may cause issues, and having the hash
    function indicated isn't actually necessary. We now verify the path 
    format in the tests to prevent regressions.
    edef1c committed Jan 2, 2020
    Copy the full SHA
    c65a6fa View commit details

Commits on Jan 3, 2020

  1. Merge pull request #3298 from edef1c/passasfile-noprefix

    passAsFile: leave out the hash prefix
    edolstra authored Jan 3, 2020
    Copy the full SHA
    0de33cc View commit details

Commits on Jan 4, 2020

  1. build: recover store path when replacing fails

    This shouldn't happen in normal circumstances, but just in case
    attempt to move the temporary path back if possible.
    LnL7 committed Jan 4, 2020
    Copy the full SHA
    b33fefc View commit details
  2. build: fix path repairing when hash rewriting is required

    Handle store path repairing on darwin when sandboxing is enabled. Unlike
    on linux sandboxing on darwin still requires hash rewriting.
    LnL7 committed Jan 4, 2020
    Copy the full SHA
    7d448bc View commit details

Commits on Jan 5, 2020

  1. Hide FunctionCallTrace constructor/destructor

    This prevents them from being inlined. On gcc 9, this reduces the
    stack size needed for
    
      nix-instantiate '<nixpkgs>' -A texlive.combined.scheme-full --dry-run
    
    from 12.9 MiB to 4.8 MiB.
    edolstra committed Jan 5, 2020
    2
    Copy the full SHA
    cb90e38 View commit details
  2. Merge pull request #3302 from LnL7/darwin-repair-with-sandbox

    build: fix path repairing with the darwin sandbox
    edolstra authored Jan 5, 2020
    Copy the full SHA
    0486e87 View commit details

Commits on Jan 6, 2020

  1. Copy the full SHA
    1dc29df View commit details
2 changes: 1 addition & 1 deletion scripts/install-nix-from-closure.sh
Original file line number Diff line number Diff line change
@@ -102,7 +102,7 @@ for i in $(cd "$self/store" >/dev/null && echo ./*); do
rm -rf "$i_tmp"
fi
if ! [ -e "$dest/store/$i" ]; then
cp -Rp "$self/store/$i" "$i_tmp"
cp -RPp "$self/store/$i" "$i_tmp"
chmod -R a-w "$i_tmp"
chmod +w "$i_tmp"
mv "$i_tmp" "$dest/store/$i"
2 changes: 1 addition & 1 deletion scripts/install.in
Original file line number Diff line number Diff line change
@@ -56,7 +56,7 @@ fi

unpack=$tmpDir/unpack
mkdir -p "$unpack"
tar -xf "$tarball" -C "$unpack" || oops "failed to unpack '$url'"
tar -xJf "$tarball" -C "$unpack" || oops "failed to unpack '$url'"

script=$(echo "$unpack"/*/install)

4 changes: 1 addition & 3 deletions src/libexpr/eval.cc
Original file line number Diff line number Diff line change
@@ -1146,9 +1146,7 @@ void EvalState::callPrimOp(Value & fun, Value & arg, Value & v, const Pos & pos)

void EvalState::callFunction(Value & fun, Value & arg, Value & v, const Pos & pos)
{
std::unique_ptr<FunctionCallTrace> trace;
if (evalSettings.traceFunctionCalls)
trace = std::make_unique<FunctionCallTrace>(pos);
auto trace = evalSettings.traceFunctionCalls ? std::make_unique<FunctionCallTrace>(pos) : nullptr;

forceValue(fun, pos);

17 changes: 17 additions & 0 deletions src/libexpr/function-trace.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#include "function-trace.hh"

namespace nix {

FunctionCallTrace::FunctionCallTrace(const Pos & pos) : pos(pos) {
auto duration = std::chrono::high_resolution_clock::now().time_since_epoch();
auto ns = std::chrono::duration_cast<std::chrono::nanoseconds>(duration);
printMsg(lvlInfo, "function-trace entered %1% at %2%", pos, ns.count());
}

FunctionCallTrace::~FunctionCallTrace() {
auto duration = std::chrono::high_resolution_clock::now().time_since_epoch();
auto ns = std::chrono::duration_cast<std::chrono::nanoseconds>(duration);
printMsg(lvlInfo, "function-trace exited %1% at %2%", pos, ns.count());
}

}
14 changes: 2 additions & 12 deletions src/libexpr/function-trace.hh
Original file line number Diff line number Diff line change
@@ -9,17 +9,7 @@ namespace nix {
struct FunctionCallTrace
{
const Pos & pos;

FunctionCallTrace(const Pos & pos) : pos(pos) {
auto duration = std::chrono::high_resolution_clock::now().time_since_epoch();
auto ns = std::chrono::duration_cast<std::chrono::nanoseconds>(duration);
printMsg(lvlInfo, "function-trace entered %1% at %2%", pos, ns.count());
}

~FunctionCallTrace() {
auto duration = std::chrono::high_resolution_clock::now().time_since_epoch();
auto ns = std::chrono::duration_cast<std::chrono::nanoseconds>(duration);
printMsg(lvlInfo, "function-trace exited %1% at %2%", pos, ns.count());
}
FunctionCallTrace(const Pos & pos);
~FunctionCallTrace();
};
}
34 changes: 17 additions & 17 deletions src/libstore/build.cc
Original file line number Diff line number Diff line change
@@ -1514,8 +1514,10 @@ void replaceValidPath(const Path & storePath, const Path tmpPath)
Path oldPath = (format("%1%.old-%2%-%3%") % storePath % getpid() % random()).str();
if (pathExists(storePath))
rename(storePath.c_str(), oldPath.c_str());
if (rename(tmpPath.c_str(), storePath.c_str()) == -1)
if (rename(tmpPath.c_str(), storePath.c_str()) == -1) {
rename(oldPath.c_str(), storePath.c_str()); // attempt to recover
throw SysError("moving '%s' to '%s'", tmpPath, storePath);
}
deletePath(oldPath);
}

@@ -1986,7 +1988,7 @@ void DerivationGoal::startBuilder()
throw BuildError(format("odd number of tokens in 'exportReferencesGraph': '%1%'") % s);
for (Strings::iterator i = ss.begin(); i != ss.end(); ) {
string fileName = *i++;
static std::regex regex("[A-Za-z_][A-Za-z0-9_.]*");
static std::regex regex("[A-Za-z_][A-Za-z0-9_.-]*");
if (!std::regex_match(fileName, regex))
throw Error("invalid file name '%s' in 'exportReferencesGraph'", fileName);

@@ -2456,12 +2458,12 @@ void DerivationGoal::initTmpDir() {
if (!parsedDrv->getStructuredAttrs()) {

StringSet passAsFile = tokenizeString<StringSet>(get(drv->env, "passAsFile").value_or(""));
int fileNr = 0;
for (auto & i : drv->env) {
if (passAsFile.find(i.first) == passAsFile.end()) {
env[i.first] = i.second;
} else {
string fn = ".attr-" + std::to_string(fileNr++);
auto hash = hashString(htSHA256, i.first);
string fn = ".attr-" + hash.to_string(Base32, false);
Path p = tmpDir + "/" + fn;
writeFile(p, i.second);
chownToBuilder(p);
@@ -3566,19 +3568,6 @@ void DerivationGoal::registerOutputs()
if (!missingPaths.count(i.second.path)) continue;

Path actualPath = path;
if (useChroot) {
actualPath = chrootRootDir + path;
if (pathExists(actualPath)) {
/* Move output paths from the chroot to the Nix store. */
if (buildMode == bmRepair)
replaceValidPath(path, actualPath);
else
if (buildMode != bmCheck && rename(actualPath.c_str(), worker.store.toRealPath(path).c_str()) == -1)
throw SysError(format("moving build output '%1%' from the sandbox to the Nix store") % path);
}
if (buildMode != bmCheck) actualPath = worker.store.toRealPath(path);
}

if (needsHashRewrite()) {
auto r = redirectedOutputs.find(i.second.path);
if (r != redirectedOutputs.end()) {
@@ -3590,6 +3579,17 @@ void DerivationGoal::registerOutputs()
if (buildMode == bmCheck)
actualPath = redirected;
}
} else if (useChroot) {
actualPath = chrootRootDir + path;
if (pathExists(actualPath)) {
/* Move output paths from the chroot to the Nix store. */
if (buildMode == bmRepair)
replaceValidPath(path, actualPath);
else
if (buildMode != bmCheck && rename(actualPath.c_str(), worker.store.toRealPath(path).c_str()) == -1)
throw SysError(format("moving build output '%1%' from the sandbox to the Nix store") % path);
}
if (buildMode != bmCheck) actualPath = worker.store.toRealPath(path);
}

struct stat st;
9 changes: 9 additions & 0 deletions src/libstore/globals.cc
Original file line number Diff line number Diff line change
@@ -117,6 +117,15 @@ void Settings::requireExperimentalFeature(const std::string & name)
throw Error("experimental Nix feature '%s' is disabled", name);
}

bool Settings::isWSL1()
{
struct utsname utsbuf;
uname(&utsbuf);
// WSL1 uses -Microsoft suffix
// WSL2 uses -microsoft-standard suffix
return hasSuffix(utsbuf.release, "-Microsoft");
}

const string nixVersion = PACKAGE_VERSION;

template<> void BaseSetting<SandboxMode>::set(const std::string & str)
4 changes: 3 additions & 1 deletion src/libstore/globals.hh
Original file line number Diff line number Diff line change
@@ -34,6 +34,8 @@ class Settings : public Config {

StringSet getDefaultSystemFeatures();

bool isWSL1();

public:

Settings();
@@ -130,7 +132,7 @@ public:
Setting<bool> fsyncMetadata{this, true, "fsync-metadata",
"Whether SQLite should use fsync()."};

Setting<bool> useSQLiteWAL{this, true, "use-sqlite-wal",
Setting<bool> useSQLiteWAL{this, !isWSL1(), "use-sqlite-wal",
"Whether SQLite should use WAL mode."};

Setting<bool> syncBeforeRegistering{this, false, "sync-before-registering",
1 change: 1 addition & 0 deletions tests/pass-as-file.sh
Original file line number Diff line number Diff line change
@@ -10,6 +10,7 @@ mkDerivation {
passAsFile = [ \"foo\" ];
foo = [ \"xyzzy\" ];
builder = builtins.toFile \"builder.sh\" ''
[ \"\$(basename \$fooPath)\" = .attr-1bp7cri8hplaz6hbz0v4f0nl44rl84q1sg25kgwqzipzd1mv89ic ]
[ \"\$(cat \$fooPath)\" = xyzzy ]
touch \$out
'';