Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul quicklisp-to-nix #28810

Merged
merged 6 commits into from Sep 2, 2017
Merged

Conversation

SquircleSpace
Copy link
Contributor

Motivation for this change

The quicklisp-to-nix system misses many dependencies and is awkward to run. Some lisp systems cannot be loaded because they weren't compiled before the nix package directory was made readonly.

Things done
  • Tested using sandboxing
    (nix.useSandbox on NixOS,
    or option build-use-sandbox in nix.conf
    on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • Linux
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@mention-bot
Copy link

@bradleyjensen, thanks for your PR! By analyzing the history of the files in this pull request, we identified @7c6f434c to be a potential reviewer.

@SquircleSpace
Copy link
Contributor Author

Note: I took the (potentially?) contentious stance that lispPackages should be about lisp code and we shouldn't be building the executables provided by lisp systems. I believe that executables provided by lispPackages (such as stumpwm) should be in their own nix package.

Brad Jensen added 6 commits August 31, 2017 20:10
1. Detect (and automatically handle) parasitic systems.
2. Each nix package has only one asd, and (almost) every parasitic
   package inside it builds.
3. Ensure that parasitic systems are compiled.
4. Remove unnecessary testnames lisp override mechanism (the
   testnae/testSystem is replaced by parasites/buildSystems).
5. Parasitic systems (if included in the system closure) become
   aliases to their host package.
6. Support caching fasl files in a known directory (for faster
   re-generation after modifying quicklisp-to-nix-system-info).
7. Eliminate unnecessary overrides.  We're going to determine ALL
   lisp dependencies correctly.
8. Don't try to "build" lisp packages with make.  lispPackages should
   be about bringing in a lisp library.
9. Eliminate the hand-maintained list of aliases.  Parasites should
   become aliases.  Everything else should be a real package.
Note: Changes to overrides were necessary
@Mic92
Copy link
Member

Mic92 commented Sep 1, 2017

Makes sense. We also move out applications out of pythonPackages.

@Mic92 Mic92 requested a review from 7c6f434c September 1, 2017 06:00
@7c6f434c
Copy link
Member

7c6f434c commented Sep 1, 2017

We don't have any Lisp binaries that you are expected to run without writing additional Lisp code, as far as I know…

@SquircleSpace
Copy link
Contributor Author

I can't reproduce the build failure for 7a9dxzcii40616l2dlv1vkjnkf21n501-lisp-alexandria-20170630-git on my NixOS box. I did the following.

  1. Added "nix.useSandbox = true;" to my system's configuration.nix
  2. Ran "nixos-rebuild test"
  3. Removed all gc roots related to my local clone of nixpkgs
  4. Merged my branch with top-of-tree master
  5. Built alexandria. It built cleanly with the same hash.

Am I missing something?

KEEP_THIS_ASD=0
for ALLOWED_ASD in $asdFilesToKeep; do
ALLOWED_ASD="/$ALLOWED_ASD"
ALLOWED_ASD_LENGTH=${"$"}{#ALLOWED_ASD}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, do you know about escaping by '', i.e. '' x=''${y}''?

if [ "$(expr substr "$ASD_FILE" "$ASD_FILE_SUFFIX_INDEX" "$ASD_FILE_LENGTH")" == "$ALLOWED_ASD" ]; then
KEEP_THIS_ASD=1
break
fi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is all the computation to check if test "$ALLOWED_ASD" = "$(basename "ASD_FILE")"?

@@ -23,9 +22,6 @@ in
cp "$out/lib/common-lisp/stumpwm/stumpwm" "$out/bin"
'';
};
propagatedBuildInputs = (x.propagatedBuildInputs or []) ++ (with qlnp; [
alexandria cl-ppcre clx
]);
};
iterate = skipBuildPhase;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand your changes correctly, skipBuildPhase is now the default

@7c6f434c
Copy link
Member

7c6f434c commented Sep 2, 2017

On the whole great, thanks.

@7c6f434c 7c6f434c merged commit 522a03c into NixOS:master Sep 2, 2017
@7c6f434c
Copy link
Member

7c6f434c commented Sep 4, 2017

Did you run the package update without using sandboxing?

@SquircleSpace
Copy link
Contributor Author

I did not know about that quoting behavior. Thanks!

I don’t know why I didn’t think to use basename. Brain fart. Sorry about that.

Correct. Skipping Rhee build is now the default. I suppose I should have removed the unnecessary overrides.

@7c6f434c
Copy link
Member

7c6f434c commented Sep 6, 2017

I dunno if it would make sense to keep going to list all the systems that couldn't be processed. Unfortunately, even among Quicklisp-provided systems, that is not always the empty set.

@SquircleSpace
Copy link
Contributor Author

I’m not sure if I follow. Are you saying we should try to get every buildable quicklisp system into lispPackages? Last time I checked, there were about a thousand releases and about 3 thousand quicklispable systems provided by those releases.

@7c6f434c
Copy link
Member

7c6f434c commented Sep 6, 2017

No, I mean that when I add multiple systems (or even try to switch to the next upstream version) before regenerating, handling some systems fails, and not only because of the missing dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants