Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix vim-plugin dependencies #52767

Merged
merged 8 commits into from Dec 28, 2018
Merged

Fix vim-plugin dependencies #52767

merged 8 commits into from Dec 28, 2018

Conversation

timokau
Copy link
Member

@timokau timokau commented Dec 24, 2018

Motivation for this change

I noticed that plugin dependencies simply did not work with the native vim plugin manager. It simply was not implemented, which was surprising.

To implement it, I first had to refactor vim-utils.nix. Now vim plugins are derivations by default, although the "name + knownPlugins" variant is still possible for backwards compatibility. I can't really see a reason why it was strings in the first place.

I also added a new ncm2 completion backend and updated the plugins (same PR since it would conflict otherwise).

Because of the autogenerated file this will bitrot fast, so I would like to get this merged relatively quickly.

Based on git-shortlog: @andsild, @jagajaga, @Mic92

I have build and tested the provided test instances with the different plugin managers. While doing that I also fixed the pathogen implementation that was already broken. I verified that the installed plugins shows up under :scriptnames.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@timokau
Copy link
Member Author

timokau commented Dec 24, 2018

@GrahamcOfBorg build vimUtils.test_vim_with_vim_nix vimUtils.test_vim_with_vim_nix_using_pathogen vimUtils.test_vim_with_vim_nix_using_plug vimPackages.vim-nix vimPackages.ncm2-jedi

@timokau
Copy link
Member Author

timokau commented Dec 24, 2018

Finally fixed the eval issues.
@GrahamcOfBorg build vimUtils.test_vim_with_vim_nix vimUtils.test_vim_with_vim_nix_using_pathogen vimUtils.test_vim_with_vim_nix_using_plug vimPackages.vim-nix vimPackages.ncm2-jedi``

@Mic92
Copy link
Member

Mic92 commented Dec 24, 2018

Dependencies are strings for historically reason when the packages where generated by a hacky vim script.

Vim plugins were previously represented as strings by default,
necessitating a `knownPlugins` set. Backwards compatibility is kept
(strings are still accepted), so vam plugins should continue to work.
nativeImpl previously simply ignored dependency information.
@timokau
Copy link
Member Author

timokau commented Dec 26, 2018

If nobody objects I'll merge this on friday (2018-12-28, Berlin time).

@timokau
Copy link
Member Author

timokau commented Dec 26, 2018

Just noticed that the other two people I pinged haven't been very active the past year, so probably no point in waiting for a response from them. Are you happy with the current version @Mic92?

@@ -310,6 +310,7 @@ def generate_nix(plugins: List[Tuple[str, str, Plugin]]):
f"""
{plugin.normalized_name} = buildVimPluginFrom2Nix {{
name = "{plugin.normalized_name}-{plugin.version}";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would that work?

Suggested change
name = "{plugin.normalized_name}-{plugin.version}";
version = plugin.version;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because stdenv is computing name anyway.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite as easily, because buildVimPlugin does its own name building. But I have pushed a new commit to add that functionality. I've also refactored the buildVimPlugin related functions into its own file, since vim-utils.nix is doing too much.

@timokau
Copy link
Member Author

timokau commented Dec 27, 2018

I also figured out how to use fixed points (by copying what python is doing) so its way less hacky now.

@timokau
Copy link
Member Author

timokau commented Dec 27, 2018

Sorry for all the force-pushes, I don't know how to test eval locally so I'm dependent on ofBorg for that. Should be good to go now.

@FRidh
Copy link
Member

FRidh commented Dec 27, 2018

@timokau I use the following pre-commit hook:

#!/bin/sh

echo "Checking evaluation before committing";
exec nix-env -f . -qaP --meta --json --drv-path --show-trace >/dev/null

@timokau
Copy link
Member Author

timokau commented Dec 27, 2018

Thanks, that's very useful.

The override actually broke the build, no idea why it was added
originally. The build works fine without it.
@timokau timokau merged commit 22d0f32 into NixOS:master Dec 28, 2018
@timokau timokau deleted the vim-plugin-updates branch December 28, 2018 12:48
timokau added a commit to timokau/nixpkgs that referenced this pull request Dec 30, 2018
I messed that up in my refactoring in NixOS#52767, since I moved the relevant
bash code out of a function and failed to adjust the shell variable
name. Now the plugin build will fail loudly when help tag generation
fails to make sure this doesn't happen again.
timokau added a commit that referenced this pull request Dec 30, 2018
I messed that up in my refactoring in #52767, since I moved the relevant
bash code out of a function and failed to adjust the shell variable
name. Now the plugin build will fail loudly when help tag generation
fails to make sure this doesn't happen again.
@rvolosatovs rvolosatovs mentioned this pull request Dec 30, 2018
10 tasks
@disassembler
Copy link
Member

Can we get a release notes update? My neovim plugins are all broken now with the following error, which I'm fine with breaking changes, but lets make sure we have a note in the release notes:

building Nix...
building the system configuration...
error: while evaluating the attribute 'activationScript' of the derivation 'nixos-system-sarov-19.03.git.2f218c64ea9' at /home/sam/nixpkgs/custom/pkgs/stdenv/generic/make-derivation.nix:185:11:
while evaluating the attribute 'system.activationScripts.script' at /home/sam/nixpkgs/custom/nixos/modules/system/activation/activation-script.nix:68:9:
while evaluating 'textClosureMap' at /home/sam/nixpkgs/custom/lib/strings-with-deps.nix:70:35, called from /home/sam/nixpkgs/custom/nixos/modules/system/activation/activation-script.nix:89:18:
while evaluating 'id' at /home/sam/nixpkgs/custom/lib/trivial.nix:14:5, called from undefined position:
while evaluating the attribute 'text' at /home/sam/nixpkgs/custom/nixos/modules/system/activation/activation-script.nix:9:5:
while evaluating the attribute 'text' at /home/sam/nixpkgs/custom/lib/strings-with-deps.nix:77:38:
while evaluating the attribute 'sources' of the derivation 'etc' at /home/sam/nixpkgs/custom/pkgs/stdenv/generic/make-derivation.nix:185:11:
while evaluating anonymous function at /home/sam/nixpkgs/custom/nixos/modules/system/etc/etc.nix:20:20, called from undefined position:
while evaluating the attribute 'source' at undefined position:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/modules.nix:75:45, called from undefined position:
while evaluating the attribute 'value' at /home/sam/nixpkgs/custom/lib/modules.nix:336:9:
while evaluating the option `environment.etc.systemd/system.source':
while evaluating the attribute 'mergedValue' at /home/sam/nixpkgs/custom/lib/modules.nix:369:5:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/modules.nix:369:32, called from /home/sam/nixpkgs/custom/lib/modules.nix:369:19:
while evaluating 'check' at /home/sam/nixpkgs/custom/lib/types.nix:245:15, called from /home/sam/nixpkgs/custom/lib/modules.nix:370:10:
while evaluating the attribute 'buildCommand' of the derivation 'system-units' at /home/sam/nixpkgs/custom/pkgs/stdenv/generic/make-derivation.nix:185:11:
while evaluating the attribute 'text' of the derivation 'unit-dbus.service' at /home/sam/nixpkgs/custom/pkgs/stdenv/generic/make-derivation.nix:185:11:
while evaluating the attribute 'text' at undefined position:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/modules.nix:75:45, called from undefined position:
while evaluating the attribute 'value' at /home/sam/nixpkgs/custom/lib/modules.nix:336:9:
while evaluating the option `systemd.units.dbus.service.text':
while evaluating the attribute 'isDefined' at /home/sam/nixpkgs/custom/lib/modules.nix:374:5:
while evaluating the attribute 'values' at /home/sam/nixpkgs/custom/lib/modules.nix:362:9:
while evaluating the attribute 'values' at /home/sam/nixpkgs/custom/lib/modules.nix:458:7:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/modules.nix:348:28, called from /home/sam/nixpkgs/custom/lib/modules.nix:348:17:
while evaluating 'dischargeProperties' at /home/sam/nixpkgs/custom/lib/modules.nix:416:25, called from /home/sam/nixpkgs/custom/lib/modules.nix:349:62:
while evaluating the attribute 'value' at /home/sam/nixpkgs/custom/lib/modules.nix:232:58:
while evaluating the attribute 'config.text' at /home/sam/nixpkgs/custom/nixos/modules/system/boot/systemd.nix:315:7:
while evaluating 'commonUnitText' at /home/sam/nixpkgs/custom/nixos/modules/system/boot/systemd.nix:299:20, called from /home/sam/nixpkgs/custom/nixos/modules/system/boot/systemd.nix:315:14:
while evaluating 'attrsToSection' at /home/sam/nixpkgs/custom/nixos/modules/system/boot/systemd-lib.nix:105:20, called from /home/sam/nixpkgs/custom/nixos/modules/system/boot/systemd.nix:301:9:
while evaluating 'mapAttrsToList' at /home/sam/nixpkgs/custom/lib/attrsets.nix:233:23, called from /home/sam/nixpkgs/custom/nixos/modules/system/boot/systemd-lib.nix:106:33:
while evaluating the attribute 'unitConfig' at undefined position:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/modules.nix:75:45, called from undefined position:
while evaluating the attribute 'value' at /home/sam/nixpkgs/custom/lib/modules.nix:336:9:
while evaluating the option `systemd.services.dbus.unitConfig':
while evaluating the attribute 'mergedValue' at /home/sam/nixpkgs/custom/lib/modules.nix:369:5:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/modules.nix:369:32, called from /home/sam/nixpkgs/custom/lib/modules.nix:369:19:
while evaluating 'merge' at /home/sam/nixpkgs/custom/lib/types.nix:282:20, called from /home/sam/nixpkgs/custom/lib/modules.nix:372:8:
while evaluating 'filterAttrs' at /home/sam/nixpkgs/custom/lib/attrsets.nix:124:23, called from /home/sam/nixpkgs/custom/lib/types.nix:283:35:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/attrsets.nix:125:29, called from /home/sam/nixpkgs/custom/lib/attrsets.nix:125:18:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/types.nix:283:51, called from /home/sam/nixpkgs/custom/lib/attrsets.nix:125:62:
while evaluating the attribute 'X-Restart-Triggers' at /home/sam/nixpkgs/custom/lib/attrsets.nix:344:7:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/types.nix:283:86, called from /home/sam/nixpkgs/custom/lib/attrsets.nix:344:15:
while evaluating the attribute 'optionalValue' at /home/sam/nixpkgs/custom/lib/modules.nix:376:5:
while evaluating the attribute 'values' at /home/sam/nixpkgs/custom/lib/modules.nix:362:9:
while evaluating the attribute 'values' at /home/sam/nixpkgs/custom/lib/modules.nix:458:7:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/modules.nix:348:28, called from /home/sam/nixpkgs/custom/lib/modules.nix:348:17:
while evaluating 'dischargeProperties' at /home/sam/nixpkgs/custom/lib/modules.nix:416:25, called from /home/sam/nixpkgs/custom/lib/modules.nix:349:62:
while evaluating the attribute 'value' at /home/sam/nixpkgs/custom/lib/types.nix:288:55:
while evaluating the attribute 'X-Restart-Triggers' at /home/sam/nixpkgs/custom/nixos/modules/system/boot/systemd.nix:216:13:
while evaluating the attribute 'serviceDirectories' of the derivation 'dbus-1' at /home/sam/nixpkgs/custom/pkgs/stdenv/generic/make-derivation.nix:185:11:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/types.nix:257:14, called from undefined position:
while evaluating the attribute 'value' at /home/sam/nixpkgs/custom/lib/modules.nix:377:27:
while evaluating anonymous function at /home/sam/nixpkgs/custom/lib/modules.nix:369:32, called from /home/sam/nixpkgs/custom/lib/modules.nix:369:19:
while evaluating 'check' at /home/sam/nixpkgs/custom/lib/types.nix:245:15, called from /home/sam/nixpkgs/custom/lib/modules.nix:370:10:
while evaluating the attribute 'passAsFile' of the derivation 'system-path' at /home/sam/nixpkgs/custom/pkgs/stdenv/generic/make-derivation.nix:185:11:
while evaluating the attribute 'buildCommand' of the derivation 'neovim-0.3.1' at /home/sam/nixpkgs/custom/pkgs/stdenv/generic/make-derivation.nix:185:11:
while evaluating 'optionalString' at /home/sam/nixpkgs/custom/lib/strings.nix:200:5, called from /home/sam/nixpkgs/custom/pkgs/applications/editors/neovim/wrapper.nix:84:9:
while evaluating the attribute 'passAsFile' of the derivation 'python-2.7.15-env' at /home/sam/nixpkgs/custom/pkgs/stdenv/generic/make-derivation.nix:185:11:
while evaluating 'requiredPythonModules' at /home/sam/nixpkgs/custom/pkgs/top-level/python-packages.nix:94:27, called from /home/sam/nixpkgs/custom/pkgs/development/interpreters/python/wrapper.nix:14:13:
while evaluating 'unique' at /home/sam/nixpkgs/custom/lib/lists.nix:630:12, called from /home/sam/nixpkgs/custom/pkgs/top-level/python-packages.nix:96:6:
while evaluating anonymous function at /home/sam/nixpkgs/custom/pkgs/applications/editors/neovim/wrapper.nix:44:50, called from /home/sam/nixpkgs/custom/pkgs/development/interpreters/python/with-packages.nix:3:19:
while evaluating anonymous function at /home/sam/nixpkgs/custom/pkgs/applications/editors/neovim/wrapper.nix:47:24, called from /home/sam/nixpkgs/custom/pkgs/applications/editors/neovim/wrapper.nix:47:13:
while evaluating anonymous function at /home/sam/nixpkgs/custom/pkgs/applications/editors/neovim/wrapper.nix:41:28, called from undefined position:
while evaluating anonymous function at /home/sam/nixpkgs/custom/pkgs/misc/vim-plugins/vim-utils.nix:427:31, called from undefined position:
value is a set while a string was expected

@Mic92
Copy link
Member

Mic92 commented Dec 31, 2018

@disassembler breaking the API was not the intention, you should open an issue on that with the code you tried.

@timokau timokau mentioned this pull request Dec 31, 2018
10 tasks
@timokau
Copy link
Member Author

timokau commented Dec 31, 2018

@disassembler yes it was very much not my intention to break the API, sorry about that. I think I figured out the issue from your trace and it should be fixed in #53113. The issue was in requiredPlugins, which is only used by neovim while I only tested with vim. I've also added another testcase for the future.

timokau added a commit to timokau/nixpkgs that referenced this pull request Dec 31, 2018
I broke this in NixOS#52767 and didn't notice because I only tested with vim
and `requiredPlugins` is only used by neovim. This would break setups
that use string-plugins (like pathogen) with neovim.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants