updating jetbrains channels; made unknown channel error non-fatal #29312

bbarker · 2017-09-13T13:39:12Z

Motivation for this change

Needed to update Jetbrains channels; newer builds are available from JetBrains, some older builds are not.

Things done

Changed update.pl for the case when an existing channel no longer appears as a JetBrains channel. This should not prevent updates to existing channels that do not have a problem.
Added support for new download URL format as introduced for channel rider_2017_2_eap.
Ran update.pl

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- Linux
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

bbarker · 2017-09-13T13:47:38Z

I guess I can understand leaving the script as fatal if there are errors updating a block, but it seems a bit too unfriendly for someone running the script the first time and just wanting to get one of the more commonly used IDEs up-to-date.

Mic92 · 2017-09-13T21:12:55Z

pkgs/applications/editors/jetbrains/update.pl

+      die "no version in $block" unless $version;
+      if ($version eq $latest_versions{$channel}) {
+	print("$channel is up to date at $version\n");
+      } else {


Did you mix spaces and tabs here?

👍 removing tabs now

Mic92 · 2017-09-13T21:14:15Z

cc @volth

bbarker · 2017-09-13T21:36:35Z

Sorry if I did, was editing on a system I don't use very much. Will try to fix soon.

…

On Wed, Sep 13, 2017 at 5:14 PM, Jörg Thalheim ***@***.***> wrote: cc @volth <https://github.com/volth> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#29312 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA37jqojMDyz45UHxV5Q5YY4qFUB3zWdks5siEWsgaJpZM4PWIFX> .

-- Brandon Barker brandon.barker@gmail.com

edwtjo · 2017-09-14T13:02:38Z

@bbarker or @volth: minor nit, since I missed #26769, please make sure that the outputted hash from the script is in base32.

Mic92 · 2017-09-15T08:21:08Z

pkgs/applications/editors/jetbrains/default.nix

    description = "A cross-platform .NET IDE based on the IntelliJ platform and ReSharper";
    license = stdenv.lib.licenses.unfree;
    src = fetchurl {
-      url = "https://download.jetbrains.com/resharper/Rider-RC-${version}.tar.gz";
-      sha256 = "37bad69cdfcc4f297b2500a7bb673af7ef8f1fd45baa4eb2fa388d2c4bcb41ee"; /* updated by script */
+      url = "https://download.jetbrains.com/resharper/JetBrains.Rider-${version}.tar.gz";


I get an 403 for this url: https://download.jetbrains.com/resharper/JetBrains.Rider-171.4456.3271.tar.gz

Oops, thanks for catching that, fixed

Mic92 · 2017-09-15T17:39:02Z

pkgs/applications/editors/jetbrains/update.pl

 	  ($sha256) = get("$url.sha256") =~ /^([0-9a-f]{64})/;
+	  $version_string = $full_version;


indentation :)

Mic92 · 2017-09-15T17:53:29Z

Can please also make one commit per update. I want to backport some of the updates to 17.03 and most updates to 17.09.

bbarker · 2017-09-15T17:59:01Z

@Mic92 Not sure I follow on how exactly you want the commits done, as there are already multiple commits - do you mean you to squash them into one update for this PR?

Mic92 · 2017-09-15T19:02:24Z

You can use git-reset to undo the commits and then add one commit per package update, with commit messages in the following style:

clion: 2017.2.1 -> 2017.2.2
gogland: 172.3757.46 -> 172.3968.45

The perl script update can be just one commit. And then just do git-push force.
This helps me with git-cherry-pick.

bbarker · 2017-09-18T22:03:07Z

Ah, I see. Normally I'd say yes but I'm super busy this week with some work deadlines and won't get a chance - please feel free to do this. I just used a local intelliJ derivation in my nix-env as a failsafe to get something going in my nix environment.

…

On Fri, Sep 15, 2017 at 3:02 PM, Jörg Thalheim ***@***.***> wrote: You can use git-reset to undo the commits and then add one commit per package update, with commit messages in the following style: - clion: 2017.2.1 -> 2017.2.2 - gogland: 172.3757.46 -> 172.3968.45 The perl script update can be just one commit. And then just do git-push force. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29312 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA37jmre6AB3q8mW8aBl9IVE-9KuzKZkks5sisnLgaJpZM4PWIFX> .

-- Brandon Barker brandon.barker@gmail.com

bbarker · 2017-09-18T22:03:49Z

A quick question/suggestion - would it be possible to utilized the nix CI to do some of this manually, and generate pull requests automatically using the github API? On Mon, Sep 18, 2017 at 6:02 PM, Brandon Barker <brandon.barker@gmail.com> wrote:

…

Ah, I see. Normally I'd say yes but I'm super busy this week with some work deadlines and won't get a chance - please feel free to do this. I just used a local intelliJ derivation in my nix-env as a failsafe to get something going in my nix environment. On Fri, Sep 15, 2017 at 3:02 PM, Jörg Thalheim ***@***.***> wrote: > You can use git-reset to undo the commits and then add one commit per > package update, with commit messages in the following style: > > - clion: 2017.2.1 -> 2017.2.2 > - gogland: 172.3757.46 -> 172.3968.45 > > The perl script update can be just one commit. And then just do git-push > force. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#29312 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AA37jmre6AB3q8mW8aBl9IVE-9KuzKZkks5sisnLgaJpZM4PWIFX> > . > -- Brandon Barker ***@***.***

-- Brandon Barker brandon.barker@gmail.com

edwtjo · 2017-09-19T06:06:51Z

@volth AFAICT the chromium stuff is in base32, as is the majority of hashes in nixpkgs, as well as previous jetbrains/idea expressions. The reason is simple, base32 is more compact for manual edits and we're not really supposed to add generator scripts such as this into the tree, even though we continue to do so. This is also why we have nix-hash to help with the conversion. Regarding the academic discussion surrounding nix-lint: sure when there is such a thing that would be great but there isn't so lets make the generator follow majority rule. Right now AFAIK we only have nixpkgs-lint which doesn't check the hash, perhaps it should.

edwtjo · 2017-09-19T07:25:15Z

@volth that stuff gets merged without proper vetting, or by slip up, isn't an excuse for not doing the right thing when called upon. Neither is it psychologically pleasant to argue silly stuff like hash bases with contributors.

Mic92 · 2017-09-19T08:09:34Z

~~Using base16 has the advantage, that we have the right checksum before downloading.~~

edwtjo · 2017-09-19T08:30:07Z

@Mic92 Sorry I don't understand what you mean, can you elaborate. I've just updated the generator to encode hashes as base32.

edwtjo · 2017-09-22T11:51:37Z

Cherry-picked appropriate changes in 14f2e0c

Mic92 · 2017-09-22T12:44:19Z

done in e4a859a

updating jetbrains channels; made unknown channel error non-fatal

d923aff

handling new download url format (for e.g. Rider)

d2a9df1

Mic92 reviewed Sep 13, 2017

View reviewed changes

whitespace (tab) and minor fixes in jetbrains update.pl

6ec1432

Mic92 reviewed Sep 15, 2017

View reviewed changes

fixing url handling for alternative version names

0892235

bbarker force-pushed the jetbrains_update branch from bd1a057 to 0892235 Compare September 15, 2017 12:39

Mic92 reviewed Sep 15, 2017

View reviewed changes

jetbrains: fix indentation in update.pl

0cb16de

edwtjo closed this Sep 22, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

updating jetbrains channels; made unknown channel error non-fatal #29312

updating jetbrains channels; made unknown channel error non-fatal #29312

bbarker commented Sep 13, 2017 •

edited

bbarker commented Sep 13, 2017

Mic92 Sep 13, 2017

bbarker Sep 14, 2017

Mic92 commented Sep 13, 2017

bbarker commented Sep 13, 2017 via email

edwtjo commented Sep 14, 2017

Mic92 Sep 15, 2017

bbarker Sep 15, 2017

Mic92 Sep 15, 2017

Mic92 commented Sep 15, 2017

bbarker commented Sep 15, 2017

Mic92 commented Sep 15, 2017 •

edited

bbarker commented Sep 18, 2017 via email

bbarker commented Sep 18, 2017 via email

edwtjo commented Sep 19, 2017

edwtjo commented Sep 19, 2017

Mic92 commented Sep 19, 2017 •

edited

edwtjo commented Sep 19, 2017

edwtjo commented Sep 22, 2017

Mic92 commented Sep 22, 2017

		($sha256) = get("$url.sha256") =~ /^([0-9a-f]{64})/;
		$version_string = $full_version;

updating jetbrains channels; made unknown channel error non-fatal #29312

updating jetbrains channels; made unknown channel error non-fatal #29312

Conversation

bbarker commented Sep 13, 2017 • edited

Motivation for this change

Things done

bbarker commented Sep 13, 2017

Mic92 Sep 13, 2017

Choose a reason for hiding this comment

bbarker Sep 14, 2017

Choose a reason for hiding this comment

Mic92 commented Sep 13, 2017

bbarker commented Sep 13, 2017 via email

edwtjo commented Sep 14, 2017

Mic92 Sep 15, 2017

Choose a reason for hiding this comment

bbarker Sep 15, 2017

Choose a reason for hiding this comment

Mic92 Sep 15, 2017

Choose a reason for hiding this comment

Mic92 commented Sep 15, 2017

bbarker commented Sep 15, 2017

Mic92 commented Sep 15, 2017 • edited

bbarker commented Sep 18, 2017 via email

bbarker commented Sep 18, 2017 via email

edwtjo commented Sep 19, 2017

edwtjo commented Sep 19, 2017

Mic92 commented Sep 19, 2017 • edited

edwtjo commented Sep 19, 2017

edwtjo commented Sep 22, 2017

Mic92 commented Sep 22, 2017

bbarker commented Sep 13, 2017 •

edited

Mic92 commented Sep 15, 2017 •

edited

Mic92 commented Sep 19, 2017 •

edited