Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PowerShell: init at 6.0.2 #41242

Merged
merged 1 commit into from Jun 5, 2018
Merged

PowerShell: init at 6.0.2 #41242

merged 1 commit into from Jun 5, 2018

Conversation

yrashk
Copy link
Contributor

@yrashk yrashk commented May 30, 2018

This is my first draft of the derivation for PowerShell. Most notable deficiency: does not (yet) support OSX.

I'd like to gather feedback on this to see if there would be any other improvement suggestions.

Motivation for this change

Not present in nixpkgs

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

Further plan:

  • OSX support
  • Update-Help doesn't work (Update-Help : Read-only file system)

buildInputs = [ autoPatchelfHook stdenv.cc.cc ];

installPhase = ''
mkdir -p $out/bin
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: We use 4 Spaces in shell scripts and 2 in Nix code as indentation.
https://nixos.org/nixpkgs/manual/#sec-syntax

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it's addressed now, too.

sha256 = "0hhh33cdfs29gy73p6bp56nqqv3cr3x9r3j7ib6141arp5yn0kxf";
leaveDotGit = true;
fetchSubmodules = true;
deepClone = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really need a deep clone here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without deepClone, I get this error during the build:

/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/PowerShell.Common.props(17,5): error MSB3073: The command "git describe --abbrev=60 --long" exited with code 128. [/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/src/Microsoft.PowerShell.Commands.Management/Microsoft.PowerShell.Commands.Management.csproj]
/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/PowerShell.Common.props(17,5): error MSB3073: The command "git describe --abbrev=60 --long" exited with code 128. [/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/src/Microsoft.PowerShell.Commands.Utility/Microsoft.PowerShell.Commands.Utility.csproj]
/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/PowerShell.Common.props(17,5): error MSB3073: The command "git describe --abbrev=60 --long" exited with code 128. [/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/src/Microsoft.PowerShell.ConsoleHost/Microsoft.PowerShell.ConsoleHost.csproj]
/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/PowerShell.Common.props(17,5): error MSB3073: The command "git describe --abbrev=60 --long" exited with code 128. [/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/src/Microsoft.PowerShell.Security/Microsoft.PowerShell.Security.csproj]
/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/PowerShell.Common.props(17,5): error MSB3073: The command "git describe --abbrev=60 --long" exited with code 128. [/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/src/System.Management.Automation/System.Management.Automation.csproj]
/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/PowerShell.Common.props(17,5): error MSB3073: The command "git describe --abbrev=60 --long" exited with code 128. [/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/src/Microsoft.PowerShell.CoreCLR.Eventing/Microsoft.PowerShell.CoreCLR.Eventing.csproj]
/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/PowerShell.Common.props(17,5): error MSB3073: The command "git describe --abbrev=60 --long" exited with code 128. [/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/src/Microsoft.PowerShell.SDK/Microsoft.PowerShell.SDK.csproj]
/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/PowerShell.Common.props(17,5): error MSB3073: The command "git describe --abbrev=60 --long" exited with code 128. [/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/src/Microsoft.PowerShell.PSReadLine/Microsoft.PowerShell.PSReadLine.csproj]
/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/PowerShell.Common.props(17,5): error MSB3073: The command "git describe --abbrev=60 --long" exited with code 128. [/tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/src/powershell-unix/powershell-unix.csproj]
Execution of { dotnet restore $_ $RestoreArguments } by build.psm1: line 536 failed with exit code 1
At /tmp/nix-build-powershell-6.0.2.drv-0/PowerShell/build.psm1:2054 char:17
+                 throw $errorMessage
+                 ~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : OperationStopped: (Execution of { ...ith exit code 1:String) [], RuntimeException
+ FullyQualifiedErrorId : Execution of { dotnet restore $_ $RestoreArguments } by build.psm1: line 536 failed with exit code 1

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to patch the build system to not depend on .git? deepClone tends to break checksums quite often.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mic92 I pushed an update to avoid this dependence on deepClone -- but I still depend on .git to pick the correct version with git rev-parse HEAD to make everything look perfect to powershell build system just in case. Would this work?

dontStrip = true;

meta = with stdenv.lib; {
description = "PowerShell Core";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description should not contain the name. C.f. https://nixos.org/nixpkgs/manual/#sec-standard-meta-attributes

@xeji
Copy link
Contributor

xeji commented May 30, 2018

@GrahamcOfBorg eval

@yrashk yrashk force-pushed the powershell branch 2 times, most recently from 4aa05b5 to 47b2734 Compare May 31, 2018 08:03
@Mic92
Copy link
Member

Mic92 commented May 31, 2018

@GrahamcOfBorg build powershell

@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: powershell

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.


@GrahamcOfBorg
Copy link

No attempt on x86_64-darwin (full log)

The following builds were skipped because they don't evaluate on x86_64-darwin: powershell

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.


@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: powershell

Partial log (click to expand)

/nix/store/x1802vh4p62vr0dqpmy4x1ch8bd2djxm-dotnet-sdk-2.0.3/sdk/2.0.3-servicing-007037/NuGet.targets(102,5): error :   Couldn't resolve host name [/build/PowerShell/src/powershell-unix/powershell-unix.csproj]
Execution of { dotnet restore $_ $RestoreArguments } by build.psm1: line 536 failed with exit code 1
At /build/PowerShell/build.psm1:2054 char:17
+                 throw $errorMessage
+                 ~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : OperationStopped: (Execution of { ...ith exit code 1:String) [], RuntimeException
+ FullyQualifiedErrorId : Execution of { dotnet restore $_ $RestoreArguments } by build.psm1: line 536 failed with exit code 1

builder for '/nix/store/vnr857bxrwqg8pawsgcp6mc19rs16qf4-powershell-6.0.2.drv' failed with exit code 1
error: build of '/nix/store/vnr857bxrwqg8pawsgcp6mc19rs16qf4-powershell-6.0.2.drv' failed

@Mic92
Copy link
Member

Mic92 commented May 31, 2018

It still seems to break because it tries to download packages in the nix sandbox.
This means it won't build on hydra as it is.

Welcome to .NET Core!
---------------------
Learn more about .NET Core @ https://aka.ms/dotnet-docs. Use dotnet --help to see available commands or go to https://aka.ms/dotnet-cli-docs.
Telemetry
--------------
The .NET Core tools collect usage data in order to improve your experience. The data is anonymous and does not include command-line arguments. The data is collected by Microsoft and shared with the community.
You can opt out of telemetry by setting a DOTNET_CLI_TELEMETRY_OPTOUT environment variable to 1 using your favorite shell.
You can read more about .NET Core tools telemetry @ https://aka.ms/dotnet-cli-telemetry.
Configuring...
-------------------
A command is running to initially populate your local package cache, to improve restore speed and enable offline access. This command will take up to a minute to complete and will only happen once.
Decompressing 0%...1%...2%...3%...4%...5%...6%...7%...8%...9%...10%...11%...12%...13%...14%...15%...16%...17%...18%...19%...20%...21%...22%...23%...24%...25%...26%...27%...28%...29%...30%...31%...32%...33%...34%...35%...36%...37%...38%...39%...40%...41%...42%...43%...44%...45%...46%...47%...48%...49%...50%...51%...52%...53%...54%...55%...56%...57%...58%...59%...60%...61%...62%...63%...64%...65%...66%...67%...68%...69%...70%...71%...72%...73%...74%...75%...76%...77%...78%...79%...80%...81%...82%...83%...84%...85%...86%...87%...88%...89%...90%...91%...92%...93%...94%...95%...96%...97%...98%...99%...100% 5882 ms
Expanding 0%...1%...2%...3%...4%...5%...6%...7%...8%...9%...10%...11%...12%...13%...14%...15%...16%...17%...18%...19%...20%...21%...22%...23%...24%...25%...26%...27%...28%...29%...30%...31%...32%...33%...34%...35%...36%...37%...38%...39%...40%...41%...42%...43%...44%...45%...46%...47%...48%...49%...50%...51%...52%...53%...54%...55%...56%...57%...58%...59%...60%...61%...62%...63%...64%...65%...66%...67%...68%...69%...70%...71%...72%...73%...74%...75%...76%...77%...78%...79%...80%...81%...82%...83%...84%...85%...86%...87%...88%...89%...90%...91%...92%...93%...94%...95%...96%...97%...98%...99%...100% 12634 ms
/nix/store/x1802vh4p62vr0dqpmy4x1ch8bd2djxm-dotnet-sdk-2.0.3/sdk/2.0.3-servicing-007037/NuGet.targets(102,5): error : Unable to load the service index for source https://api.nuget.org/v3/index.json. [/build/PowerShell/src/powershell-unix/powershell-unix.csproj]
/nix/store/x1802vh4p62vr0dqpmy4x1ch8bd2djxm-dotnet-sdk-2.0.3/sdk/2.0.3-servicing-007037/NuGet.targets(102,5): error :   An error occurred while sending the request. [/build/PowerShell/src/powershell-unix/powershell-unix.csproj]
/nix/store/x1802vh4p62vr0dqpmy4x1ch8bd2djxm-dotnet-sdk-2.0.3/sdk/2.0.3-servicing-007037/NuGet.targets(102,5): error :   Couldn't resolve host name [/build/PowerShell/src/powershell-unix/powershell-unix.csproj]
Execution of { dotnet restore $_ $RestoreArguments } by build.psm1: line 536 failed with exit code 1
At /build/PowerShell/build.psm1:2054 char:17
+                 throw $errorMessage
+                 ~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : OperationStopped: (Execution of { ...ith exit code 1:String) [], RuntimeException
+ FullyQualifiedErrorId : Execution of { dotnet restore $_ $RestoreArguments } by build.psm1: line 536 failed with exit code 1
builder for '/nix/store/3bjjgfv07fr6xpsq8haarw0f9jz0pi00-powershell-6.0.2.drv' failed with exit code 1

@Mic92
Copy link
Member

Mic92 commented May 31, 2018

There would be two options how to proceed:

  1. Use the pre-build version of powershell.
  2. Figure out how to pre-download nuget packages in a reproducible way (there is some dotnet infrastructure in nixpkgs, but I am not sure what state of that is... we can build stuff like fsharp and dotnetcore)

@yrashk
Copy link
Contributor Author

yrashk commented May 31, 2018 via email

@yrashk
Copy link
Contributor Author

yrashk commented May 31, 2018 via email

@Mic92
Copy link
Member

Mic92 commented May 31, 2018

If pre-downloading generates a reproducible cache, it could be replaced by a fixed-output derivation.
We have some examples for doing that with mvn: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/exhibitor/default.nix#L19

However then nuget most not update packages if newer versions are available remotely for this to work.

@Mic92
Copy link
Member

Mic92 commented May 31, 2018

Both options are fine. Ideally we could build from source but we often also use the prebuild version for example for many java applications and for basically all electron applications.

@yrashk
Copy link
Contributor Author

yrashk commented May 31, 2018

I decided to go with the prebuilt version. The reasons are:

  1. The pre-built version includes more modules than the we one I got from the source code (most notably, [Find|Install]-Package (note: for installation to work, current user scope should be used: Install-Package -Scope CurrentUser <PackageName>)
  2. The pre-built version is faster at startup:

Pre-built:

$ time ./result/bin/pwsh -c Get-Uptime


Days              : 4
Hours             : 2
Minutes           : 43
Seconds           : 12
Milliseconds      : 0
Ticks             : 3553920000000
TotalDays         : 4.11333333333333
TotalHours        : 98.72
TotalMinutes      : 5923.2
TotalSeconds      : 355392
TotalMilliseconds : 355392000



0.45user 0.06system 0:00.43elapsed 117%CPU (0avgtext+0avgdata 92684maxresident)k

Built:

 $ time ./result/bin/pwsh -c Get-Uptime


Days              : 4
Hours             : 2
Minutes           : 43
Seconds           : 0
Milliseconds      : 0
Ticks             : 3553800000000
TotalDays         : 4.11319444444444
TotalHours        : 98.7166666666667
TotalMinutes      : 5923
TotalSeconds      : 355380
TotalMilliseconds : 355380000



1.37user 0.17system 0:01.26elapsed 121%CPU (0avgtext+0avgdata 88052maxresident)k

@yrashk yrashk force-pushed the powershell branch 3 times, most recently from 9afccd7 to 045ff1a Compare May 31, 2018 17:52
@Mic92
Copy link
Member

Mic92 commented May 31, 2018

hash sum of src is too short.

@yrashk
Copy link
Contributor Author

yrashk commented May 31, 2018

@Mic92 my bad, updated


installPhase = ''
mkdir -p $out/bin
cp -r * $out
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move the content to $out/share/powershell instead? Otherwise it may clutter nix profiles with useless files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@yrashk yrashk force-pushed the powershell branch 2 times, most recently from 43e6f99 to 2727979 Compare June 1, 2018 18:58
cp -r * $out/share/powershell
rm $out/share/powershell/DELETE_ME_TO_DISABLE_CONSOLEHOST_TELEMETRY
makeWrapper $out/share/powershell/pwsh $out/bin/pwsh --prefix ${platformLdLibraryPath} : "${stdenv.lib.makeLibraryPath [ libunwind libuuid icu openssl curl ]}" \
--set PAGER ${less}/bin/less --set TERM linux
Copy link
Member

@Mic92 Mic92 Jun 1, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the incremental review, but I just noticed that TERM is set for linux. It currently does not like screen as a terminal, however if we add workaround should it not better be xterm, because I suppose most people are using a graphical terminal that can be resized rather a virtual console.

Is less as pager strictly required? An alternative could be to only set less if no other PAGER is specified.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for this PAGER workaround is that if I omit this, this is what I'll get:


PS /home/yrashk/Projects/nix/nixpkgs> help                                                                                                                                                                          
Get-Command : The term 'less -R' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Any other ideas on how to work around this?

I've changed the terminal to be xterm, it works fine.

Copy link
Member

@Mic92 Mic92 Jun 1, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be PAGER not an executable rather then a command with arguments? There is the LESS variable to pass arguments to PAGER.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mic92 I'm not sure what you mean exactly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for the pager issue, it appears to be fixed in a preview version of PowerShell (v6.1.0-preview.1 - 2018-03-23) as per their changelog (search for 6144) PowerShell/PowerShell#6144

I suppose we have two options:

  • Use a preview build instead of the release
  • Keep PAGER set as it is right now but add a TODO: "remove after upgrading to v6.1.0-preview.1 or later" or similar.

Suggestions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Please add the TODO then.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mic92 added TODO, pushed

Anything else?

@yrashk yrashk force-pushed the powershell branch 2 times, most recently from ef83e17 to 3aa71ba Compare June 4, 2018 20:02
@Mic92 Mic92 merged commit 1d1a4bf into NixOS:master Jun 5, 2018
@pSub
Copy link
Member

pSub commented Jun 5, 2018

👍

@RokeJulianLockhart
Copy link

Does support for macOS exist yet?

@charles-dyfis-net
Copy link
Contributor

@yrashk, do you have the older version of this PR where you were attempting to build from source sitting around somewhere? I'd like to try to pick up where you left off (probably with building Nix code to assemble a nuget cache).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants