Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flake support #730

Merged
merged 91 commits into from Apr 7, 2020
Merged

Flake support #730

merged 91 commits into from Apr 7, 2020

Conversation

edolstra
Copy link
Member

This PR adds flake support to Hydra. This means that it can jobsets can be specified by a flake URL such as github:NixOS/nixos-homepage (see https://hydra.nixos.org/jobset/flakes/nixos-homepage#tabs-configuration). Hydra will build the hydraJobs output of the flake if defined, and otherwise the checks output.

The flake.nix and flake.lock files of the flake capture all the dependencies, so it's no longer necessary to specify any jobset inputs in the web interface. Note that Hydra never generates or updates flake.lock, so it can't dynamically integrate different repositories.

It also removes the need for the "Reproduce locally" script. Clicking on "Reproduce locally" will just tell you to run a command like nix build github:NixOS/nixos-homepage/c5780d9cff360d5a0af9a7d0dac2e12b991386ef#hydraJobs.x86_64-linux.build (taken from https://hydra.nixos.org/build/115654567).

This PR also turns Hydra into a flake itself. For backwards compatibility there is a default.nix and shell.nix that use https://github.com/edolstra/flake-compat.

edolstra added 30 commits May 8, 2019 13:57
E.g. 'hydra-eval-jobs ~/Dev/patchelf' is enough to evaluate patchelf -
no need to set up a $NIX_PATH, pass arguments, etc.
No more need for a reproduction script! It just says something like

  If you have Nix installed, you can reproduce this build on your own
  machine by running the following command:

  # nix build github:edolstra/dwarffs/09c823e977946668b63ad6c88ed358b48220f124:hydraJobs.build.x86_64-linux
May 15 09:20:10 chef hydra-queue-runner[27523]: Hydra::Plugin::GitlabStatus=HASH(0x519a7b8)->buildFinished: Can't call method "value" on an undefined value at /nix/store/858hinflxcl2jd12wv1r3a8j11ybsf6w-hydra-0.1.2629.89fa829/libexec/hydra/lib/Hydra/Plugin/GitlabStatus.pm line 57.
The hydra flake now exports a Nixpkgs overlay and a NixOS module.
edolstra and others added 23 commits February 15, 2020 21:55
Flake input changes:

* Updated 'nix': 'github:NixOS/nix/eb19ff3b82240326fc0e999e09f81b6c8ed98640' -> 'github:NixOS/nix/144bb3ef7ddca6502ea3e1878f7fc0a3f9013aba'
This is now propagated by Nix.
Flake input changes:

* Updated 'nix': 'github:NixOS/nix/144bb3ef7ddca6502ea3e1878f7fc0a3f9013aba' -> 'github:NixOS/nix/6529490cc10018d5191e50c482ac1180b96b1a3c'
Flake input changes:

* Updated 'nix': 'github:NixOS/nix/6529490cc10018d5191e50c482ac1180b96b1a3c' -> 'github:NixOS/nix/4ad5826a182f8eeb4cbc61e27d2128b08050b0d4'
This got lost when I converted from releaseTools.nixBuild to
stdenv.mkDerivation. Doh!
If we don't see machine that supports a build step for
'max_unsupported_time' seconds, the step is aborted. The default is 0,
which is appropriate for Hydra installations that don't provision
missing machines dynamically.
 Abort unsupported build steps
Otherwise, errors will not be shown to end-users, which makes debugging
long evals pretty much impossible.

(cherry picked from commit 76299b9)
Flake input changes:

* Updated 'nix': 'github:NixOS/nix/4ad5826a182f8eeb4cbc61e27d2128b08050b0d4' -> 'github:NixOS/nix/3e7aab81ce6787e01ea4ced5af1fc6a84e523762'
* Updated 'nix/nixpkgs': 'github:edolstra/nixpkgs/43e283790fbe330b723ac44b584824cf6d85c1ee' -> 'github:NixOS/nixpkgs/b88ff468e9850410070d4e0ccd68c7011f15b2be'
* Updated 'nixpkgs': 'github:edolstra/nixpkgs/d959778e0934c4c93896d509a65de798909c7cdd' -> 'github:NixOS/nixpkgs/b0c285807d6a9f1b7562ec417c24fa1a30ecc31a'
Apparentely, buildEnv in 20.03 no longer respects
propagated-build-inputs.

Note that the use of a library function (closePropagation) seems
fundamentally wrong to me - propagated-build-inputs should be used at
runtime, not at evaluation time.
In Nixpkgs 20.03, Mercurial fails if PYTHONPATH is set:

  $ hg
  Traceback (most recent call last):
    File "/nix/store/q7s856v6nw4dffdrm9k3w38qs35i8kr3-mercurial-5.2.2/bin/..hg-wrapped-wrapped", line 37, in <module>
      dispatch.run()
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/importlib/util.py", line 245, in __getattribute__
      self.__spec__.loader.exec_module(self)
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/nix/store/q7s856v6nw4dffdrm9k3w38qs35i8kr3-mercurial-5.2.2/lib/python3.7/site-packages/mercurial/dispatch.py", line 10, in <module>
      import difflib
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/difflib.py", line 1084, in <module>
      import re
    File "/nix/store/bffdy7q3wi3qinflnvbdkigqj39zzynd-python3-3.7.6/lib/python3.7/re.py", line 143, in <module>
      class RegexFlag(enum.IntFlag):
  AttributeError: module 'enum' has no attribute 'IntFlag'
@edolstra edolstra merged commit 4cabb37 into master Apr 7, 2020
@edolstra edolstra deleted the flake branch April 7, 2020 09:18
Ma27 added a commit to Ma27/nixpkgs that referenced this pull request Apr 7, 2020
Also removed `pkgs.hydra-flakes` since flake-support has been merged
into master[1]. Because of that, `pkgs.hydra-unstable` is now compiled
against `pkgs.nixFlakes` and currently requires a patch since Hydra's
master doesn't compile[2] atm.

[1] NixOS/hydra#730
[2] NixOS/hydra#732
Ma27 added a commit to NixOS/nixpkgs that referenced this pull request Apr 9, 2020
Also removed `pkgs.hydra-flakes` since flake-support has been merged
into master[1]. Because of that, `pkgs.hydra-unstable` is now compiled
against `pkgs.nixFlakes` and currently requires a patch since Hydra's
master doesn't compile[2] atm.

[1] NixOS/hydra#730
[2] NixOS/hydra#732

(cherry picked from commit 0f5c38f)
stigok pushed a commit to stigok/nixpkgs that referenced this pull request Jun 12, 2020
Also removed `pkgs.hydra-flakes` since flake-support has been merged
into master[1]. Because of that, `pkgs.hydra-unstable` is now compiled
against `pkgs.nixFlakes` and currently requires a patch since Hydra's
master doesn't compile[2] atm.

[1] NixOS/hydra#730
[2] NixOS/hydra#732

(cherry picked from commit 0f5c38f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants