vscode-utils/vscodeEnv: add vscodeWithConfiguration, vscodeExts2nix a… #76427

countoren · 2019-12-24T18:18:47Z

based on this project

Motivation for this change

Declarative configuration of vscode settings and extensions in a per project basis.

And I was trying to solve these problems:

some extensions mutate the contents of their folder. Therefore, the nix store is not the best place for those extensions.
being able to automatically get the extensions' information that nix needs to fetch them. Also, use vscode interface to manage the extensions while having a nix expression defining the extensions that are used for a project.
VSCode Global configuration folder and globally enabled extensions.

Things done

created :

vsCodeWithConfiguration - wrapper for vscode that:
- copy extensions to local extensions dir. mutable extensions will get copied while nix extensions will get symlinked.
- set user-data folder to the template path of the nix-shell in order to avoid global settings(those could be configured in nix) and symlink global settings.json and keybindings.json to the local one in the project's .vscode folder.
updateSettings - shell script that creates or overrides all settings files in .vscode based on the nix expression.
settings.json will be initialized with :

{  
   "extensions.autoCheckUpdates": false,
  "extensions.autoUpdate": false,
  "update.mode": "none"
}

vscodeExtensions2nix - with a vscode given as parameter, it will run with "--list-extensions" to output a valid nix expression that returns a list of sets of this expression (including sha).
vscodeEnv - puts together vscodeWithConfiguration, vscodeExtensions2nix and updateSettings. Creates another wrapper above vscodeWithConfiguration which adds "--wait" flag in order to run vscodeExtensions2nix after vscode exits. vscodeExtensions2nix will update an expression file with the mutable extensions currently installed.

Notify maintainers

cc @zimbatm

…nd vscodeEnv move mktplcExtRefToFetchArgs to file in order to be shared with the new derivations(privately)

zimbatm · 2019-12-27T16:39:37Z

pkgs/misc/vscode-extensions/vscode-utils.nix

-in
+  vscodeWithConfiguration = (userParams : import ./vscodeWithConfiguration.nix {
+   inherit lib vscode extensionsFromVscodeMarketplace writeShellScriptBin;
+  } // userParams);


the parenthesis are not needed here

zimbatm · 2019-12-27T16:39:46Z

pkgs/misc/vscode-extensions/vscode-utils.nix

+
+  vscodeExts2nix = (userParams : import ./vscodeExts2nix.nix {
+    inherit lib vscode;
+  } // userParams);


zimbatm · 2019-12-27T16:39:52Z

pkgs/misc/vscode-extensions/vscode-utils.nix

+
+  vscodeEnv = (userParams : import ./vscodeEnv.nix {
+    inherit lib writeShellScriptBin extensionsFromVscodeMarketplace vscode;
+  } // userParams );


zimbatm · 2019-12-27T16:43:48Z

pkgs/misc/vscode-extensions/vscodeEnv.nix

+  };
+  code = writeShellScriptBin "code" ''
+    ${vscodeWithConfiguration}/bin/code --wait "$@" 
+    echo 'running vscodeExts2nix to update ${mutableExtensionsFilePath}...'


Suggested change

echo 'running vscodeExts2nix to update ${mutableExtensionsFilePath}...'

echo 'running vscodeExts2nix to update ${toString mutableExtensionsFilePath}...'

zimbatm · 2019-12-27T16:44:18Z

pkgs/misc/vscode-extensions/vscodeEnv.nix

+  code = writeShellScriptBin "code" ''
+    ${vscodeWithConfiguration}/bin/code --wait "$@" 
+    echo 'running vscodeExts2nix to update ${mutableExtensionsFilePath}...'
+    ${vscodeExts2nix}/bin/vscodeExts2nix > ${mutableExtensionsFilePath}


Suggested change

${vscodeExts2nix}/bin/vscodeExts2nix > ${mutableExtensionsFilePath}

${vscodeExts2nix}/bin/vscodeExts2nix > ${toString mutableExtensionsFilePath}

this is to avoid having the path be added to the /nix/store

zimbatm · 2019-12-27T16:44:31Z

pkgs/misc/vscode-extensions/vscodeEnv.nix

+, vscode                          ? pkgs.vscode
+}:
+let  
+  mutableExtensionsFilePath = builtins.toPath mutableExtensionsFile;


this is not needed. or use toString instead, it does effectively the same thing.

zimbatm · 2019-12-27T16:51:07Z

pkgs/misc/vscode-extensions/mktplcExtRefToFetchArgs.nix

@@ -0,0 +1,8 @@
+ext: 


Suggested change

ext:

{ publisher, name, version }:

you can extract attributes

zimbatm · 2019-12-27T16:54:49Z

pkgs/misc/vscode-extensions/vscodeEnv.nix

+, nixExtensions                   ? []
+# if file exists will use it and import the extensions in it into this dervation else will use empty extensions list
+# this file will be created/updated by vscodeExts2nix when vscode exists
+, mutableExtensionsFile           ? ./extensions.nix


this argument should be provided by the user

zimbatm · 2019-12-27T16:56:21Z

pkgs/misc/vscode-extensions/vscodeEnv.nix

+, lib                             ? pkgs.lib
+, writeShellScriptBin             ? pkgs.writeShellScriptBin
+, extensionsFromVscodeMarketplace ? pkgs.vscode-utils.extensionsFromVscodeMarketplace
+


I would return a second function with the user attributes below. These are the dependencies and below the function that the user wants to call.

zimbatm · 2019-12-27T16:58:31Z

pkgs/misc/vscode-extensions/vscodeExts2nix.nix

+, lib                 ? pkgs.lib
+, vscode              ? pkgs.vscode
+, writeShellScriptBin ? pkgs.writeShellScriptBin
+


same, split in two functions

zimbatm · 2019-12-27T16:59:01Z

pkgs/misc/vscode-extensions/vscodeWithConfiguration.nix

+, lib                             ? pkgs.lib
+, writeShellScriptBin             ? pkgs.writeShellScriptBin
+, extensionsFromVscodeMarketplace ? pkgs.vscode-utils.extensionsFromVscodeMarketplace
+


split in two function

Suggested change

}@orig:

{

zimbatm · 2019-12-27T17:00:05Z

pkgs/misc/vscode-extensions/vscodeWithConfiguration.nix

+{ pkgs                            ? import <nixpkgs> {}
+, lib                             ? pkgs.lib
+, writeShellScriptBin             ? pkgs.writeShellScriptBin
+, extensionsFromVscodeMarketplace ? pkgs.vscode-utils.extensionsFromVscodeMarketplace


Suggested change

, extensionsFromVscodeMarketplace ? pkgs.vscode-utils.extensionsFromVscodeMarketplace

, extensionsFromVscodeMarketplace ? pkgs.vscode-utils.extensionsFromVscodeMarketplace

, vscode

zimbatm · 2019-12-27T17:00:22Z

pkgs/misc/vscode-extensions/vscodeWithConfiguration.nix

+
+##User input
+
+, vscode                          ? pkgs.vscode


Suggested change

, vscode ? pkgs.vscode

, vscode ? orig.vscode

… vscodeExts2nix, vscodeEnv change usage of toPath with toString

countoren · 2020-01-06T20:39:56Z

@zimbatm updated base on your comments.

pkgs/misc/vscode-extensions/vscodeEnv.nix

strager · 2020-01-09T00:30:43Z

pkgs/misc/vscode-extensions/vscodeWithConfiguration.nix

+  cpExtensions = ''
+    ${lib.concatMapStringsSep "\n" (e : ''ln -sfn ${e}/share/vscode/extensions/* ${vscodeExtsFolderName}/'') nixExtsDrvs}
+    ${lib.concatMapStringsSep "\n" (e : ''
+      cp -a ${e}/share/vscode/extensions/${e.vscodeExtUniqueId} ${vscodeExtsFolderName}/${e.vscodeExtUniqueId}-${(lib.findSingle (ext: ''${ext.publisher}.${ext.name}'' == e.vscodeExtUniqueId) "" "m" mutableExtensions ).version}


Which extensions have problems when their directory is immutable? Could we fix these extensions to avoid this copying?

using this expression:

{ pkgs ? import <nixpkgs>{}}: pkgs.vscode-with-extensions.override ({ vscodeExtensions = pkgs.vscode-utils.extensionsFromVscodeMarketplace ([ { name = "csharp"; publisher = "ms-vscode"; version = "1.21.5"; sha256 = "031yx2kpjrw1b86y5nq110k9ywjgl26730ljddly05d8fxxham05"; } ]); })

will cuase this error in vscode:

Failed at stage: touchBeginFile Error: EACCES: permission denied, mkdir '/nix/store/vnz8ja4qzmg4qpbr5am647djadclswsm-vscode-extensions-1.40.1/share/vscode/extensions/ms-vscode.csharp/.omnisharp'

and this error when trying to use nix-collect-garbage :

deleting '/nix/store/trash' 12428 store paths deleted, 664.94 MiB freed error: cannot unlink '/nix/store/trash/4mnbcnp5fb8495pjlzwrfy24c9yk122n-vscode/extension-ms-vscode-csharp-1.21.5/share/vscode/extensions/ms-vscode.csharp/.omnisharp/1.34.5/license.md': Permission denied

im using darwin.

countoren · 2020-01-19T19:12:49Z

@jonringer, what do you think about this PR?

peti · 2020-04-17T20:15:42Z

@zimbatm, are you still interested in this PR? It's been sitting in the queue for quite a while now and it does look pretty useful to me. IMHO, the submitter has addressed your comments so I wonder whether there's a path forward for this PR to get merged?

zimbatm · 2020-04-19T13:11:56Z

@peti: it seems like a good idea and @countoren put quite a bit of effort in it. It's a mix between the pure and impure universes which is quite a new approach.

The current complexity of the code is what prevented me from properly evaluating the PR. It would require me to block 1-2h to really look into it and see if it could be simplified. Which is what is slowing this PR down really.

countoren · 2020-04-19T16:17:33Z

@peti , @zimbatm thanks guys for looking at it :).
while this PR was waiting to the evaluation I copied the code as a new repo (https://github.com/countoren/VSCodeEnv) and updated a bit more one of the main things was to be able to "ignore" global configuration of VSCode (this is to have more guaranties of the configurations) by moving vscode data folder to the tmp folder (instead of user folder) and symlink the user configuration files there to the project's one(which are overwritten from nix).
I am not sure if I should bring this feature before we merge or should we just look at this version first.

Another thing that I would like to solve here is the sudo that is done in order to keep the .vscode-exts folder in sync by removing non-nix-declared extensions,
I'm not 100% sure it is needed because the folder should be created by the user that runs the wrapper (code executable) but I had some problem with it so I added it but it could be due to something else I was missing.

…nd vscodeEnv move mktplcExtRefToFetchArgs to file in order to be shared with the new derivations(privately)

… vscodeExts2nix, vscodeEnv change usage of toPath with toString

…nto the a vscode settings file

…ings.json, keybindings.json

… parameter type from path to string

Treat workspace setting files as global for that user-data-dir with symlink. Add updateLaunchCmd to update .vscode/launch.json.

moinessim · 2020-05-25T23:15:55Z

This is a use example:

    let  
        personal-extensions-file = ./personal-extensions.nix;
    in
    vscodeEnv {
        nixExtensions = import ./extensions.nix 
          ++ (
                    if builtins.pathExists personal-extensions-file 
                    then import personal-extensions-file else []
                );
        mutableExtensionsFile = ./mutable-extensions.nix; 
        settings = 
        {
          "FSharp.useSdkScripts" = true;
        };
      }

… delete them Do not use sudo.

…/nixpkgs-1 into vscode-utils/vscodeEnv * 'vscode-utils/vscodeEnv' of https://github.com/countoren/nixpkgs-1: (178391 commits) Use a different vscode user-data-dir for every project. Treat workspace setting files as global for that user-data-dir with symlink. Add updateLaunchCmd to update .vscode/launch.json. fix permission problem of mutableExtensions vscodeEnv updateSettings for keybindings change keybindings file name parameter type from path to string change vscodeSettingsFile parameter type from path to string add missing jq to vscodeEnv use updateSettings in vscodeEnv in order to create and/or update settings.json, keybindings.json add updateSettings drv which will union nix settings configurations into the a vscode settings file vscode-utils/vscodeEnv: fix typo and grammer in the description comment vscode-utils/vscodeEnv: split to 2 functions vscodeWithConfiguration, vscodeExts2nix, vscodeEnv vsode-utils: extracting attributes to limit input range vscode-utils/vscodeEnv: add vscodeWithConfiguration, vscodeExts2nix and vscodeEnv xfce.xfce4-icon-theme: update gtk dependency and add license xfce.xfce4-hardware-monitor-plugin: update meta xfce.xfce4-embed-plugin: update meta and buildInputs xfce.xfce4-mailwatch-plugin: remove broken status xfce.exo: add support for gtk2, besides gtk3 xfce.xfdesktop: 4.14.1 -> 4.14.2 xfce.xfce4-weather-plugin: 0.8.10 -> 0.10.1 xfce.xfdashboard: 0.7.5 -> 0.7.7 xfce.xfce4-timer-plugin: 1.6.0 -> 1.7.0 ...

peti · 2020-05-29T19:33:26Z

I believe this PR has been waiting long enough and I'm not aware of any controversy around it, i.e. no-one seems to have anything against merging it. So ... I'm merging it. :-)

vscode: fix #76427 fallout

vscode-utils/vscodeEnv: add vscodeWithConfiguration, vscodeExts2nix a…

4b25c6a

…nd vscodeEnv move mktplcExtRefToFetchArgs to file in order to be shared with the new derivations(privately)

countoren requested a review from jonringer as a code owner December 24, 2019 18:18

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Dec 24, 2019

zimbatm reviewed Dec 27, 2019

View reviewed changes

countoren added 2 commits December 28, 2019 16:40

vsode-utils: extracting attributes to limit input range

2d59425

vscode-utils/vscodeEnv: split to 2 functions vscodeWithConfiguration,…

7fc2b2a

… vscodeExts2nix, vscodeEnv change usage of toPath with toString

countoren mentioned this pull request Dec 28, 2019

add vscodeWithConfiguration, vscodeExts2nix, vscodeEnv #76624

Closed

6 tasks

eraserhd mentioned this pull request Jan 3, 2020

mercurial_4: init at 4.9.1 #76903

Merged

10 tasks

strager reviewed Jan 9, 2020

View reviewed changes

vscode-utils/vscodeEnv: fix typo and grammer in the description comment

e708176

countoren and others added 12 commits April 19, 2020 21:32

vscode-utils/vscodeEnv: add vscodeWithConfiguration, vscodeExts2nix a…

666d708

…nd vscodeEnv move mktplcExtRefToFetchArgs to file in order to be shared with the new derivations(privately)

vsode-utils: extracting attributes to limit input range

e2c8033

vscode-utils/vscodeEnv: split to 2 functions vscodeWithConfiguration,…

6d31311

… vscodeExts2nix, vscodeEnv change usage of toPath with toString

vscode-utils/vscodeEnv: fix typo and grammer in the description comment

3d102d1

add updateSettings drv which will union nix settings configurations i…

a535e7a

…nto the a vscode settings file

use updateSettings in vscodeEnv in order to create and/or update sett…

0e4ac8b

…ings.json, keybindings.json

add missing jq to vscodeEnv

02105da

change vscodeSettingsFile parameter type from path to string

7e929f5

vscodeEnv updateSettings for keybindings change keybindings file name…

88b9721

… parameter type from path to string

fix permission problem of mutableExtensions

3816b9d

Use a different vscode user-data-dir for every project.

673b122

Treat workspace setting files as global for that user-data-dir with symlink. Add updateLaunchCmd to update .vscode/launch.json.

Merge branch 'updateSettings' into vscode-utils/vscodeEnv

4d58f00

countoren mentioned this pull request May 25, 2020

Users can't run "nix-collect-garbage -d" any more #6244

Closed

moinessim added 2 commits May 26, 2020 13:14

vscode-extensions: Find mutable extensions by version in order not to…

97df561

… delete them Do not use sudo.

peti merged commit e080ab1 into NixOS:master May 29, 2020

eadwu mentioned this pull request May 30, 2020

vscode: fix #76427 fallout #89196

Merged

10 tasks

veprbl added a commit that referenced this pull request Jun 4, 2020

Merge pull request #89196 from eadwu/vscode/76427-fallout

dd8f077

vscode: fix #76427 fallout

Janik-Haag added the 12. first-time contribution label Jun 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vscode-utils/vscodeEnv: add vscodeWithConfiguration, vscodeExts2nix a… #76427

vscode-utils/vscodeEnv: add vscodeWithConfiguration, vscodeExts2nix a… #76427

countoren commented Dec 24, 2019 •

edited

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019 •

edited

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

zimbatm Dec 27, 2019

countoren commented Jan 6, 2020

strager Jan 9, 2020

countoren Jan 11, 2020 •

edited

countoren commented Jan 19, 2020

peti commented Apr 17, 2020

zimbatm commented Apr 19, 2020

countoren commented Apr 19, 2020

moinessim commented May 25, 2020 •

edited

peti commented May 29, 2020

	echo 'running vscodeExts2nix to update ${mutableExtensionsFilePath}...'
	echo 'running vscodeExts2nix to update ${toString mutableExtensionsFilePath}...'

	${vscodeExts2nix}/bin/vscodeExts2nix > ${mutableExtensionsFilePath}
	${vscodeExts2nix}/bin/vscodeExts2nix > ${toString mutableExtensionsFilePath}

	, extensionsFromVscodeMarketplace ? pkgs.vscode-utils.extensionsFromVscodeMarketplace
	, extensionsFromVscodeMarketplace ? pkgs.vscode-utils.extensionsFromVscodeMarketplace
	, vscode

vscode-utils/vscodeEnv: add vscodeWithConfiguration, vscodeExts2nix a… #76427

vscode-utils/vscodeEnv: add vscodeWithConfiguration, vscodeExts2nix a… #76427

Conversation

countoren commented Dec 24, 2019 • edited

Motivation for this change

Things done

Notify maintainers

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zimbatm Dec 27, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

countoren commented Jan 6, 2020

Choose a reason for hiding this comment

countoren Jan 11, 2020 • edited

Choose a reason for hiding this comment

countoren commented Jan 19, 2020

peti commented Apr 17, 2020

zimbatm commented Apr 19, 2020

countoren commented Apr 19, 2020

moinessim commented May 25, 2020 • edited

peti commented May 29, 2020

countoren commented Dec 24, 2019 •

edited

zimbatm Dec 27, 2019 •

edited

countoren Jan 11, 2020 •

edited

moinessim commented May 25, 2020 •

edited