vcvrack: init at 0.5.1 #33132

mrVanDalo · 2017-12-28T11:50:44Z

Motivation for this change

VCVRack is an aweseome tool and a joy to play around with.
I had to update rtmidi and glew to make it work (they are separate commits though)

I'm not sure about the version handling of rtmidi this is a major version bump (2.1.1 -> 3.0.0).
Somebody should check this thing with the nox I'm not that deep into Nix to understand what is going on there.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS (using my overlay )
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

lukateras · 2017-12-29T03:54:38Z

pkgs/applications/audio/vcvrack/default.nix

@@ -0,0 +1,115 @@
+{ lib, stdenv, fetchFromGitHub, fetchgit, makeWrapper, gcc,
+  # buildInputs


I think this was left here by accident :-)

lukateras · 2017-12-29T03:55:48Z

pkgs/applications/audio/vcvrack/default.nix

+  # fetch source
+  # ------------
+  # need to use fetchgit here to pull submodules as well
+  src = fetchgit {


It would be preferable to use fetchFromGitHub, it's much faster than fetchgit. But, then, it would involve quite a bit of hash recalculation...

I did not get fetchFromGitHub to pull also submodules this is why I used fetchgit.

Oh, I didn't expect so many repositories with submodules. Sorry!

@mrVanDalo, you can fetch the submodules by adding fetchSubmodules = true; to fetchFromGitHub.

lukateras · 2017-12-29T03:56:41Z

pkgs/applications/audio/vcvrack/default.nix

+
+  githubUser = "VCVRack";
+
+  # fetch source


I think these comments are redundant.

lukateras · 2017-12-29T03:57:20Z

pkgs/applications/audio/vcvrack/default.nix

+  name = "vcvrack-${version}";
+  version = "0.5.1";
+
+  githubUser = "VCVRack";


"githubUser" is longer than "VCVRack", inlining it would be more straightforward.

lukateras · 2017-12-29T03:59:58Z

pkgs/applications/audio/vcvrack/default.nix

+
+  # fetch Plugins
+  # -------------
+  pluginFundamental = fetchgit {


If you define this in a single attrset plugins, you can easily iterate through it later in prePatch.

I tried that, but I couldn't make it work. For example I need the pluginFundamental variable later in the patch phase, I don't know how to dynamically create variables and such. If this would be Haskell I would know how to do it.

lukateras · 2017-12-29T04:03:01Z

pkgs/applications/audio/vcvrack/default.nix

+
+    # otherwise we will not find the fonts and pulgins
+    sed src/asset.cpp -i -e "46s@.*@dir = \"$out/share\"\;@"
+    sed src/asset.cpp -i -e "49s@.*@dir = \"$out/share\"\;@"


I replace a specific line here. Not sure how to do that with substituteInPlace.

I couldn't find something like substituteInPlace --line 46 "dir = \"$out/share\"\;"

lukateras · 2017-12-29T04:03:27Z

pkgs/applications/audio/vcvrack/default.nix

+
+
+  buildPhase = ''
+    make allplugins && make


Use makeFlags instead of overriding buildPhase. In this case it would be:

makeFlags = [ "allplugins" "all" ];

where all is the name of the first target in Makefile, which is probably all.

lukateras · 2017-12-29T04:08:03Z

pkgs/applications/audio/vcvrack/default.nix

+    ln -s ../share/Rack $out/bin/Rack
+    ln -s ../share/Rack $out/bin/VCVRack
+
+    cp -r res     $out/share/


Packaging binaries that look for assets in their dir is best solved by having a separate derivation that installs everything to $out, and a wrapper that links binaries from the "bare" derivation to $out/bin.

At least, this shouldn't pollute $out/share, it should go to its own subdirectory within it, but approach described above is better.

I don't understand what you mean by

having a separate derivation that installs everything to $out and a wrapper that links binaries from the "bare" derivation to $out/bin.

Do you mean I should install everything in $out/share/vcvrack/ ?

lukateras · 2017-12-29T04:09:45Z

pkgs/applications/audio/vcvrack/default.nix

+      Rack is the engine for the VCV open-source virtual Eurorack DAW.
+    '';
+    homepage     = https://vcvrack.com/;
+    downloadPage = https://vcvrack.com/;


downloadPage is de facto deprecated.

lukateras · 2017-12-29T04:12:06Z

pkgs/applications/audio/vcvrack/default.nix

+    cp -r plugins $out/share/ # todo : only copy libs
+  '';
+
+  meta = {


stdenv.lib can be moved to meta definition:

meta = with stdenv.lib; { license = licenses.gpl3; };

mrVanDalo · 2017-12-30T01:38:00Z

@yegortimoshenko I fixed most of the things you commented, and add 2 more plugins (which bring in some good features, like recording or sending fixed numbers).

I would like to separate the plugins, but it is expected they are compiled via make allplugins in the plugins folder. Did not have time to check out if that is really necessary.

I'm using fetchgit for all, because some of them need to pull submodules. I thought it might help to create an attrset loop later. Btw I could need some help on this topic. We just need to put the plugin sources in the $src/plugins/ folder.

I put everything in $out/share/vcvrack and create a link to $out/bin if that is ok. I did not understand the other comment of yours, sorry :(

Hope you are satisfied. (my nick on freenode is palo btw, if you want to chat, but I'm in NewZealand so I have a different time zone :D )

lukateras · 2017-12-30T02:09:03Z

Thank you a lot for your work! I'll review it now.

For loop, here is an inspiration for future packages if need arises: https://github.com/yegortimoshenko/nixpkgs/blob/05dc1bd798d1a445b9d2459d41229ea50e349564/pkgs/tools/inputmethods/ibus/default.nix#L38

But of course it's not required for merge.

lukateras · 2017-12-30T02:10:13Z

By the way, it's 5 AM in my time zone and I usually go to bed at around 12 PM so I probably align with New Zealand time zone pretty well :-)

lukateras

Looks good! Let me build this and then I'll merge.

lukateras · 2017-12-30T02:31:32Z

pkgs/development/libraries/audio/rtmidi/default.nix

@@ -1,14 +1,14 @@
 { stdenv, fetchFromGitHub, autoconf, automake, libtool, libjack2, alsaLib, pkgconfig }:

 stdenv.mkDerivation rec {
-  version = "2.1.1";
+  version = "3.0.0";


From RtMidi page:

The version number has been bumped to 3.0.0 because of the past API change concerning the renaming of the RtError class to RtMidiError.

That might be problematic, I'll check if this bump breaks other packages and if it does, I'll add a separate derivation for v3...

I don't know how to do that by now. Not a long term NixOS user. But I create a ticket to learn that too.

This can be done via nix-shell -p nox --run "nox-review pr 33132", but it will involve a lot of rebuilding.

I tried that, but I get "merge conflicts" (dont know why, and where) I'm using nixos stable (17.09 I think). So I can't do these checks I guess

mrVanDalo · 2017-12-31T02:36:53Z

@yegortimoshenko I had some time yesterday, and I improved the plugin part using map.
I also put some more plugins. There are (plugin manager proposed on the website is not working as expected, so it is a good idea to deliver as much plugins as possible I would say)

sorki · 2018-01-01T22:27:06Z

Sorry for not publishing this earlier as I totally forgot about it but here's my hackish attempt - I've packaged the dependencies as well (if it makes sense to add them) sorki@2508164

Regarding plugins I really like what you did. To make the built-in plugin-manager working too you need to pass VERSION to make, see sorki@5264512#diff-0a96701bbd982f577aab66458a659b5eR59

mrVanDalo · 2018-01-02T03:03:19Z

Sorry for not publishing this earlier as I totally forgot about it but here's my hackish attempt - I've packaged the dependencies as well (if it makes sense to add them) sorki/nixpkgs@2508164

I just pull the repo with all submodules this makes stuff much simpler, of course if we reuse the dependencies some time in future, it make sense to package them as well. (not sure what is best practice here)

Regarding plugins I really like what you did. To make the built-in plugin-manager working too you need to pass VERSION to make, see sorki/nixpkgs@5264512#diff-0a96701bbd982f577aab66458a659b5eR59

That is a very good advice! thanks I will include this.

mrVanDalo · 2018-01-02T03:49:52Z

@sorki @yegortimoshenko I'm not sure about the prepackaged plugins. I made this actually because I didn't had the plugin-manager. But because the plugin-manager is there, we might only ship the default plugins, what do you think?

lukateras · 2018-01-02T05:37:18Z

I think just shipping all plugins is OK, but if you can make the plugin manager work, then it would probably be better to have it, because in theory it allows to install paid plugins. I'm not sure if it's flexible enough for Nix, perhaps it tries to write in Nix store which would mean it's not an option.

The only reason I haven't merged this yet is that I still need to rebuild rtmidi/glut dependents in order to ensure that updates don't break anything. I should receive access to aarch64-build-box pretty soon, then I'll be able to do that.

sorki · 2018-01-02T05:40:20Z

I like the idea of being able to add plugins via nix conf but I don't really mind if it's not exposed. Now you need to do a bit of hunting as not all the plugins are available via plugin manager. @yegortimoshenko got a point about immutable plugins directory though - I'm currently running my hacked build from ~/Downloads/VCVRack so it is working as expected. Think it's quite reasonable to suggest this to upstream if not already there.

mrVanDalo · 2018-01-02T05:49:59Z

The only reason I haven't merged this yet is that I still need to rebuild rtmidi/glut dependents in order to ensure that updates don't break anything. I should receive access to aarch64-build-box pretty soon, then I'll be able to do that.

Great ! If you need anything to be done, which helps you please let me know. But I must mention I'm using the stable channel and I'm pretty new to everything in NixOS.

I'm not sure if it's flexible enough for Nix, perhaps it tries to write in Nix store which would mean it's not an option.

I just checked this again and it install the packages into ~/.Rack so this should not be a problem.

Now you need to do a bit of hunting as not all the plugins are available via plugin manager.

That is true.

I suggest we put the vcvrack upstream as it is for now. And I create a ticket for me

separate the plugins into vcvrackPlugins.<name> nix-packages.

sorki · 2018-01-10T12:39:56Z

According to IRC - "plugin manager disabled now for 3rd parties :/"

mrVanDalo · 2018-02-12T00:05:02Z

I bumped some of the plugin versions, because there was something happening the last month.

@yegortimoshenko I guess you have been busy the last time, or forgot about this pull-request.

svenkeidel · 2018-03-04T15:53:01Z

Hi, @mrVanDalo, I also created a PR for the same package (I was not aware of this PR at first). Maybe we can compare our packages. I created the package for the current master of VCVRack. I leave some comments inline in your code if this is ok for you.

svenkeidel · 2018-03-04T15:59:12Z

pkgs/applications/audio/vcvrack/default.nix

+with stdenv.lib;
+
+let
+  plugins = {


I put the plugins into a separate plugin.nix file. I turned this file then into a function from plugins to derivation:
vcvrack/default.nix:

{ stdenv, ... }: plugins: stdenv.mkDerivation rec { name = "Rack-${version}"; ... }

all-packages.nix:

vcvrackWithPlugins = callPackage ../applications/audio/vcvrack { }; vcvrack = pkgs.vcvrackWithPlugins (import ../applications/audio/vcvrack/plugins.nix { inherit fetchFromGitHub; });

This allows people to add their own plugins by calling vcvrackWithPlugins.

Good idea, I would like to be able to pick plugins like for vim or something.
Bad thing is that some of the plugins need the Rack repo as well.
I don't think a vcvrack without modules makes any sense. I think the vcvrack (withoutPlugins) should at least deliver the default repos.
I would like to deliver all the other modules as separate plugins. Something like

nixos.vcvrack nixos.vcvrackPlugins.AS nixos.vcvrackPlugins.Vult ...

what do you think ?

I don't think a vcvrack without modules makes any sense.

Yes this makes sense.

I would like to deliver all the other modules as separate plugins.

How would vcvrack find the installed plugins? So far I have not found an easy way to compile plugins separate from vcvrack.

How would vcvrack find the installed plugins? So far I have not found an easy way to compile plugins separate from vcvrack.

I did not accomplish that task (of course).
But I thought of links. (of course we can't change the vcvrack, but maybe we use /etc/ or something)

something like ${pkgs.vcvrack}/usr/share/vcvrack/plugins -> /etc/vcvrack/plugins and
than every plugin (e.g. AS plugin) creates /etc/vcvrack/AS -> ${pkgs.vcvrackPlugins.AS}/usr/share/vcvrack/plugins/AS

Yes this makes sense.

What do you want to do with a vcvrack without any plugins ? You can only look at the background image.

@svenkeidel Would it be Ok, if we merge this pull-request with as few further changes, because @yegortimoshenko would have to spend time on reviewing it. And when it's merged you create a pull-request ? We could also talk on irc (i'm palo in the #nixos channel)

Yes this makes sense.

What do you want to do with a vcvrack without any plugins ? You can only look at the
background image.

This was a misunderstanding 😄, I was agreeing with you that vcvrack without any plugins does not make any sense.

About the plugin problem. I don't know if vcvrack finding plugins over the global filesystem is the most Nix way of doing things. I think that for packages that can be extended with plugins it is acceptable to take these as an input argument, see for example emacsWithPackages. This gives the users more flexibility to instantiate this package multiple times with different sets of plugins. This flexibility probably will not be needed in case of vcvrack, however, I think it is good practice.

Based this plea, I would like to refine my proposal from above to support a standard set of plugins:

vcvrack/default.nix:

{ stdenv, ... }: plugins: stdenv.mkDerivation rec { name = "Rack-${version}"; ... }

all-packages.nix:

vcvrackWithPlugins = callPackage ../applications/audio/vcvrack { }; vcvrack = userPlugins: let corePlugins = import ../applications/audio/vcvrack/core-plugins.nix { inherit fetchFromGitHub; }; in pkgs.vcvrackWithPlugins (corePlugins // userPlugins);

@mrVanDalo, what do you think?

@svenkeidel : Yeah that sounds also good. But I still would like to "just" get this pr done (also because it is so old) and solve your issues in another pull-request right after this thing got merged.

svenkeidel · 2018-03-04T18:21:39Z

As far as I'm concerned this PR should be merged as soon as possible, that people can stop packaging VCVRack 😄.

mrVanDalo · 2018-03-06T02:10:00Z

@yegortimoshenko I guess you had a lot of other troubles to do and had time (or other problems) figuring out what if the rtmidi version bump breaks anything.

I could create a rtmidi3 package so we'll be on the safe side, and can merge this pull-request, this was your last issue as far as I read (and remember).
(just give me a small go and I'll do it)

svenkeidel · 2018-03-06T17:43:58Z

I could create a rtmidi3 package so we'll be on the safe side, and can merge this pull-request.

Yes I think that is the best solution. I think newer major versions of rtmidi and rtaudio are not backwards compatible with older versions. Hence, if you update these libraries, you break all projects that still depend on these older major versions. Making the new versions of these libraries available while defaulting to the existing version is the best way to go.

mrVanDalo · 2018-03-09T10:27:21Z

@yegortimoshenko I created a new packages (rtmidi3) to avoid dependency problems with other packages. This was the last issue here, so this pull-request should be ready to merge.

mrVanDalo · 2018-05-01T01:10:16Z

I'm closing this Pull-Request, because VCVRack 0.6.0 is out and has to be build in a different way. Maybe one of you (@svanderburg @sorki) are quicker the next time. Please ping me when you created a pull-request, so I wont.

mrVanDalo · 2018-05-06T18:58:21Z

@svanderburg @sorki @yegortimoshenko If you want to use version 0.6 I created a build in my overlay. This is the commit The pain point are the dependencies. Also compiling the Plugins did not work for me, but the Plugin Manager works just fine.

mrVanDalo force-pushed the feature/vcvrack branch from 4761e2e to bb27bf0 Compare December 28, 2017 11:59

GrahamcOfBorg added 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Dec 28, 2017

lukateras requested changes Dec 29, 2017

View reviewed changes

mrVanDalo force-pushed the feature/vcvrack branch 2 times, most recently from 753f75b to d6e9ddc Compare December 30, 2017 01:29

lukateras approved these changes Dec 30, 2017

View reviewed changes

lukateras reviewed Dec 30, 2017

View reviewed changes

mrVanDalo force-pushed the feature/vcvrack branch from d6e9ddc to 0936a32 Compare December 31, 2017 02:33

mrVanDalo force-pushed the feature/vcvrack branch from 0936a32 to 7c5ed2d Compare January 2, 2018 03:46

mrVanDalo force-pushed the feature/vcvrack branch 2 times, most recently from f3160e4 to ae6a702 Compare February 12, 2018 00:02

GrahamcOfBorg added the 8.has: package (new) label Feb 12, 2018

pbogdan mentioned this pull request Mar 4, 2018

vcvrack: init at 0.5.1 #36291

Closed

8 tasks

svenkeidel reviewed Mar 4, 2018

View reviewed changes

mrVanDalo force-pushed the feature/vcvrack branch 2 times, most recently from 3d3ec95 to 3e524ab Compare March 9, 2018 10:22

mrVanDalo force-pushed the feature/vcvrack branch 6 times, most recently from a524d6a to fc3360a Compare March 9, 2018 11:16

mrVanDalo added 3 commits March 9, 2018 19:25

rtmidi: 2.1.1 -> 3.0.0 (as rtmidi3)

5db4ef4

glew: 2.0.0 -> 2.1.0

09acb3e

vcvrack: init at 0.5.1

7794ef2

mrVanDalo force-pushed the feature/vcvrack branch from fc3360a to 7794ef2 Compare March 9, 2018 11:25

mrVanDalo closed this May 1, 2018

maxkrieger mentioned this pull request Feb 25, 2020

vcv-rack: 1.1.5 -> 1.1.6 #72851

Merged

10 tasks

		@@ -0,0 +1,115 @@
		{ lib, stdenv, fetchFromGitHub, fetchgit, makeWrapper, gcc,
		# buildInputs

vcvrack: init at 0.5.1 #33132

vcvrack: init at 0.5.1 #33132

Conversation

mrVanDalo commented Dec 28, 2017

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrVanDalo Dec 29, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lukateras Dec 29, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrVanDalo Dec 29, 2017 • edited

Choose a reason for hiding this comment

lukateras Dec 29, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrVanDalo commented Dec 30, 2017 • edited

lukateras commented Dec 30, 2017

lukateras commented Dec 30, 2017 • edited

lukateras left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lukateras Dec 30, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrVanDalo commented Dec 31, 2017

sorki commented Jan 1, 2018

mrVanDalo commented Jan 2, 2018

mrVanDalo commented Jan 2, 2018

lukateras commented Jan 2, 2018

sorki commented Jan 2, 2018

mrVanDalo commented Jan 2, 2018 • edited

sorki commented Jan 10, 2018

mrVanDalo commented Feb 12, 2018

svenkeidel commented Mar 4, 2018 • edited

svenkeidel Mar 4, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

svenkeidel Mar 5, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

svenkeidel Mar 6, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

svenkeidel commented Mar 4, 2018

mrVanDalo commented Mar 6, 2018

svenkeidel commented Mar 6, 2018

mrVanDalo commented Mar 9, 2018

mrVanDalo commented May 1, 2018 • edited

mrVanDalo commented May 6, 2018 • edited

mrVanDalo Dec 29, 2017 •

edited

lukateras Dec 29, 2017 •

edited

mrVanDalo Dec 29, 2017 •

edited

lukateras Dec 29, 2017 •

edited

mrVanDalo commented Dec 30, 2017 •

edited

lukateras commented Dec 30, 2017 •

edited

lukateras Dec 30, 2017 •

edited

mrVanDalo commented Jan 2, 2018 •

edited

svenkeidel commented Mar 4, 2018 •

edited

svenkeidel Mar 4, 2018 •

edited

svenkeidel Mar 5, 2018 •

edited

svenkeidel Mar 6, 2018 •

edited

mrVanDalo commented May 1, 2018 •

edited

mrVanDalo commented May 6, 2018 •

edited