grafana: 7.3.3 -> 7.3.4 #104794

WilliButz · 2020-11-24T19:01:10Z

Motivation for this change

https://github.com/grafana/grafana/releases/tag/v7.3.4

Things done

Ma27 · 2020-11-25T09:06:20Z

@WilliButz do you have an idea why ofborg is failing like this (https://logs.nix.ci/?key=nixos/nixpkgs.104794&attempt_id=07f48470-5654-4b14-8c2f-2757606a823a):

hash mismatch in fixed-output derivation '/nix/store/sg9dnqkz9xlkx99ciw8ax5m09sgwkv2h-grafana-7.3.4.linux-amd64.tar.gz':
  wanted: sha256:0icwxikan1bmvj0a708g34lky6cvl0mvmcgx3ihc9fpj2qgxlrgv
  got:    sha256:1fmfscfbdg2wdzf5fbad83pln4azy2y3fcghcanafkvqr2vv4hwk

When I run nix-build /nix/store/20s301v70807fq8cs1ig243qsw9539cy-grafana-7.3.4.linux-amd64.tar.gz.drv --check I don't have a hash-mismatch though.

WilliButz · 2020-11-25T09:13:03Z

@Ma27 I have no idea. I am also unable to reproduce the mismatch on two other hosts 🤔

@GrahamcOfBorg test grafana

WilliButz · 2020-11-25T09:19:31Z

I just managed to get the same hash as ofborg on a hetzner server in helsinki:

$ nix-prefetch-url https://dl.grafana.com/oss/release/grafana-7.3.4.linux-amd64.tar.gz
[51.5 MiB DL]
path is '/nix/store/sg9dnqkz9xlkx99ciw8ax5m09sgwkv2h-grafana-7.3.4.linux-amd64.tar.gz'
1fmfscfbdg2wdzf5fbad83pln4azy2y3fcghcanafkvqr2vv4hwk

On five other hosts I tested, it's the same hash we both get:

$ nix-prefetch-url https://dl.grafana.com/oss/release/grafana-7.3.4.linux-amd64.tar.gz
[51.5 MiB DL]
path is '/nix/store/gbrhas64dm6xzj95hhhay6g1mp3xhp5q-grafana-7.3.4.linux-amd64.tar.gz'
0icwxikan1bmvj0a708g34lky6cvl0mvmcgx3ihc9fpj2qgxlrgv

WilliButz · 2020-11-25T09:39:29Z

Even though a colleague just confirmed the other hash on their server in Finland as well, to me the issue does not seem to be CDN-related - or at least not DNS-related, because all hosts I tested resolve the FQDN to the same IPs.
I'll look into it later, unless the mystery magically solves until then :)

Ma27 · 2020-11-25T09:55:27Z

@GrahamcOfBorg build grafana

cc @grahamc @cole-h would you mind taking a look? (tl;dr we have a hash mismatch, see comment #104794 (comment))

(maybe a rebuild helps :p)

WilliButz · 2020-11-25T10:02:36Z

I diffed the tarball contents 🙃
"-ger" is with the hash ending in ...hwk
"-fin" in ...rgv

Binary files grafana-7.3.4-ger/bin/grafana-cli and grafana-7.3.4-fin/bin/grafana-cli differ
diff -u '--color=always' -u --color -r grafana-7.3.4-ger/bin/grafana-cli.md5 grafana-7.3.4-fin/bin/grafana-cli.md5
--- grafana-7.3.4-ger/bin/grafana-cli.md5	2020-11-24 13:26:53.000000000 +0100
+++ grafana-7.3.4-fin/bin/grafana-cli.md5	2020-11-24 15:31:01.000000000 +0100
@@ -1 +1 @@
-0e8949cbb19ae99265846e4d1fc1b0cc
+07da47b961b5d07ff4bdf012c600dde9
Binary files grafana-7.3.4-ger/bin/grafana-server and grafana-7.3.4-fin/bin/grafana-server differ
diff -u '--color=always' -u --color -r grafana-7.3.4-ger/bin/grafana-server.md5 grafana-7.3.4-fin/bin/grafana-server.md5
--- grafana-7.3.4-ger/bin/grafana-server.md5	2020-11-24 13:26:53.000000000 +0100
+++ grafana-7.3.4-fin/bin/grafana-server.md5	2020-11-24 15:31:01.000000000 +0100
@@ -1 +1 @@
-ae712e79b5544ec3de6103f0feb63c64
+038659c3a97473575e79fdadde9ee15e
diff -u '--color=always' -u --color -r grafana-7.3.4-ger/plugins-bundled/gel-0.6.0/MANIFEST.txt grafana-7.3.4-fin/plugins-bundled/gel-0.6.0/MANIFEST.txt
--- grafana-7.3.4-ger/plugins-bundled/gel-0.6.0/MANIFEST.txt	2020-11-24 13:26:54.000000000 +0100
+++ grafana-7.3.4-fin/plugins-bundled/gel-0.6.0/MANIFEST.txt	2020-11-24 15:31:01.000000000 +0100
@@ -9,7 +9,7 @@
   "signedByOrgName": "Grafana Labs",
   "plugin": "gel",
   "version": "0.6.0",
-  "time": 1606220813528,
+  "time": 1606228261061,
   "keyId": "7e4d0c6a708866e7",
   "files": {
     "README.md": "84ddf7143f83dec00d70cb19f64df9dd34ddbadedf05b8c243ef0f512ca46de5",
@@ -23,9 +23,9 @@
 Version: OpenPGP.js v4.10.1
 Comment: https://openpgpjs.org
 
-wqAEARMKAAYFAl+8/A0ACgkQfk0ManCIZucJ2gIGLCdaXsXDCHTBTyBxHZ4i
-amUpd9A06WNlJIfuZGECr76Bh4SEJDK4mHOc6zNtbBj+Qx45i/1F42RbkjAY
-Wh/hW6ECCOJVvc1nX6l07Vrt5Az5TjvXIvKB1ePi2uNvW4yVhpMFrNkKvf3u
-CO8eJGoWEwJ9ZIrZBfnCn+xr6n7ymGX1yOhf
-=C1Vw
+wqEEARMKAAYFAl+9GSUACgkQfk0ManCIZueJPQIJASLx+jt8IbjSCCQn6Cdl
+NBvK+pCPgl72H+HDjcIAypENm7MNB4cLB9MCzO5zXbP7jTTlOiB9Mahgsnag
+d2LcehB1AgiVXYWTFW1GXGemCcKa6IcPK9Nj7po4QWMsmEhRYPzgii6LDkFK
+xI8OxwGF7ue8zVLfIg431XF9sL5ckiNz0LJPKg==
+=fS0K
 -----END PGP SIGNATURE-----
diff -u '--color=always' -u --color -r grafana-7.3.4-ger/plugins-bundled/internal/input-datasource/MANIFEST.txt grafana-7.3.4-fin/plugins-bundled/internal/input-datasource/MANIFEST.txt
--- grafana-7.3.4-ger/plugins-bundled/internal/input-datasource/MANIFEST.txt	2020-11-24 13:26:54.000000000 +0100
+++ grafana-7.3.4-fin/plugins-bundled/internal/input-datasource/MANIFEST.txt	2020-11-24 15:31:01.000000000 +0100
@@ -9,7 +9,7 @@
   "signedByOrgName": "Grafana Labs",
   "plugin": "input",
   "version": "1.0.0",
-  "time": 1606220358744,
+  "time": 1606227787023,
   "keyId": "7e4d0c6a708866e7",
   "files": {
     "LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30",
@@ -25,9 +25,9 @@
 Version: OpenPGP.js v4.10.1
 Comment: https://openpgpjs.org
 
-wqAEARMKAAYFAl+8+kYACgkQfk0ManCIZufZCQIHZD2ma6mUfEm8tEZnyu/H
-lc1p+2mhO8TtXQVTr0RO+usccNBjagE10D/Ju0lVc1cp4lCbejilHUYXygsH
-GqJwdcsCB3haRkb8W4azHdCdv/qjysmwDhyJMxGAgJpxZARBeTZSMsaUaG6X
-8pYYhNHee66QRXYUO++HCiGaLYKHNAihLszK
-=QfaJ
+wqAEARMKAAYFAl+9F0sACgkQfk0ManCIZueqfQIIi+EC0/NUx4pMu2qQu9O0
+u0dm42DBDHrIfl8uUZXS3o4g/MHCOTj25yb87Duqkpl2KD3tG04Kul75HKoE
+GGrT2fECCNgM2Ketw9H/BSeoNvri7utlOX9n2u1Y+HNG3EQH9N0wpi6HCpJA
+5QEcrVMcMGVVgm7ogaK2BorwySi1n39xaYdz
+=kvlm
 -----END PGP SIGNATURE-----
diff -u '--color=always' -u --color -r grafana-7.3.4-ger/scripts/lib.star grafana-7.3.4-fin/scripts/lib.star
--- grafana-7.3.4-ger/scripts/lib.star	2020-11-24 13:26:53.000000000 +0100
+++ grafana-7.3.4-fin/scripts/lib.star	2020-11-24 15:31:01.000000000 +0100
@@ -301,6 +301,7 @@
         'image': publish_image,
         'depends_on': [
             'build-storybook',
+            'end-to-end-tests',
         ],
         'environment': {
             'GCP_KEY': {
diff -u '--color=always' -u --color -r grafana-7.3.4-ger/scripts/release.star grafana-7.3.4-fin/scripts/release.star
--- grafana-7.3.4-ger/scripts/release.star	2020-11-24 13:26:53.000000000 +0100
+++ grafana-7.3.4-fin/scripts/release.star	2020-11-24 15:31:01.000000000 +0100
@@ -45,7 +45,8 @@
         'name': 'release-npm-packages',
         'image': build_image,
         'depends_on': [
-            'end-to-end-tests',
+            # Has to run after publish-storybook since this step cleans the files publish-storybook depends on
+            'publish-storybook',
         ],
         'environment': {
             'NPM_TOKEN': {
@@ -78,8 +79,8 @@
     if publish:
         steps.extend([
             upload_packages_step(edition=edition, ver_mode=ver_mode),
-            release_npm_packages_step(edition=edition, ver_mode=ver_mode),
             publish_storybook_step(edition=edition, ver_mode=ver_mode),
+            release_npm_packages_step(edition=edition, ver_mode=ver_mode),
         ])
     windows_steps = get_windows_steps(edition=edition, ver_mode=ver_mode)

cole-h · 2020-11-25T18:03:58Z

FWIW, I also get the 1fm... hash:

vin@scadrial ~> nix-prefetch-url https://dl.grafana.com/oss/release/grafana-7.3.4.linux-amd64.tar.gz
path is '/nix/store/sg9dnqkz9xlkx99ciw8ax5m09sgwkv2h-grafana-7.3.4.linux-amd64.tar.gz'
1fmfscfbdg2wdzf5fbad83pln4azy2y3fcghcanafkvqr2vv4hwk

That's pretty annoying, but I'm not sure what can be done...

Ma27 · 2020-11-25T18:17:01Z

@WilliButz already contacted Grafana's support. Might be an issue with slightly different tarballs delivered on different CDN locations.

WilliButz · 2020-11-25T18:32:53Z

A bit over an hour ago I received good news from the Grafana support:

We had some release issues yesterday and mistakenly multiple version of grafana was pushed to the dl servers. Someone apparently downloaded these versions before the were publicly listed which resulted in them being cached on 3 CDN nodes.

The CDN nodes have been refreshed and all nodes should now have the correct version. Thanks for reporting this issue, we are going to improve our release pipeline to minimize the risk of this happening again in th
e future.

So hopefully ofborg should be fine now :)

@GrahamcOfBorg test grafana

WilliButz · 2020-11-25T18:36:16Z

Oh, okay the other hash is the one of the more recent version %)

https://github.com/grafana/grafana/releases/tag/v7.3.4

WilliButz · 2020-11-25T18:39:38Z

Hopefully the last time:

@GrahamcOfBorg test grafana

Ma27 · 2020-11-25T19:59:56Z

Ported to 20.09 as debc958!

Up until now, the frontend was taken from `srcStatic`, i.e. prebuilt from upstream. I recall at least three cases[1][2][3] where we got a hash mismatch eventually. Rather than spending time finding out whether or not it's a supply-chain attack or just a build issue, I decided to implement a source-build now with the following benefits: * It's now actually possible to apply patches for Grafana's frontend. * We rely a little less on third-party build systems. Of course, patching potential vulnerabilities in transitive frontend dependencies is still hard (let alone discovering that this package is affected!), but that's a fundamental issue we have in nixpkgs and I won't invent a half-baked solution just for this package, I still consider this a step into the right direction. The build itself mainly orients on the `yarn` commands used in the upstream Makefile[4]. However, we can't use `fetchYarnDeps` here because yarn v2 (a.k.a. `berry`) is in use which is why the same was done as in `hedgedoc`, writing a custom FoD that downloads all dependencies and writes the offline cache into `$out`[5]. Additionally there are two more notable differences to upstream: * We patch out every dependency to `@grafana/e2e` and `cypress`. The first is a dependency on the latter in another version and the latter downloads random blobs from the Internet in postInstall. Since it's a testing framework (and the `e2e` package apparently a testing library), I decided it's not worth the effort and patched it out everywhere. * There was a `zoneinfo.zip` in `$out/share/grafana/tools` that was installed from `srcStatic`. This only seems to be used on Windows[6] and that's not supported by this package, so I decided to drop it. [1] NixOS#251479 [2] NixOS#130201 [3] NixOS#104794 [4] https://github.com/grafana/grafana/blob/v10.3.1/Makefile [5] NixOS#245170 [6] https://github.com/grafana/grafana/blob/v10.3.1/pkg/setting/setting.go#L1012-L1014 (cherry picked from commit 282c93a7ad12a5d7df590b18c56253d4289ffd3a)

Up until now, the frontend was taken from `srcStatic`, i.e. prebuilt from upstream. I recall at least three cases[1][2][3] where we got a hash mismatch eventually. Rather than spending time finding out whether or not it's a supply-chain attack or just a build issue, I decided to implement a source-build now with the following benefits: * It's now actually possible to apply patches for Grafana's frontend. * We rely a little less on third-party build systems. Of course, patching potential vulnerabilities in transitive frontend dependencies is still hard (let alone discovering that this package is affected!), but that's a fundamental issue we have in nixpkgs and I won't invent a half-baked solution just for this package, I still consider this a step into the right direction. The build itself mainly orients on the `yarn` commands used in the upstream Makefile[4]. However, we can't use `fetchYarnDeps` here because yarn v2 (a.k.a. `berry`) is in use which is why the same was done as in `hedgedoc`, writing a custom FoD that downloads all dependencies and writes the offline cache into `$out`[5]. Additionally there are two more notable differences to upstream: * We patch out every dependency to `@grafana/e2e` and `cypress`. The first is a dependency on the latter in another version and the latter downloads random blobs from the Internet in postInstall. Since it's a testing framework (and the `e2e` package apparently a testing library), I decided it's not worth the effort and patched it out everywhere. * There was a `zoneinfo.zip` in `$out/share/grafana/tools` that was installed from `srcStatic`. This only seems to be used on Windows[6] and that's not supported by this package, so I decided to drop it. [1] NixOS#251479 [2] NixOS#130201 [3] NixOS#104794 [4] https://github.com/grafana/grafana/blob/v10.3.1/Makefile [5] NixOS#245170 [6] https://github.com/grafana/grafana/blob/v10.3.1/pkg/setting/setting.go#L1012-L1014 (cherry picked from commit 608db26)

Up until now, the frontend was taken from `srcStatic`, i.e. prebuilt from upstream. I recall at least three cases[1][2][3] where we got a hash mismatch eventually. Rather than spending time finding out whether or not it's a supply-chain attack or just a build issue, I decided to implement a source-build now with the following benefits: * It's now actually possible to apply patches for Grafana's frontend. * We rely a little less on third-party build systems. Of course, patching potential vulnerabilities in transitive frontend dependencies is still hard (let alone discovering that this package is affected!), but that's a fundamental issue we have in nixpkgs and I won't invent a half-baked solution just for this package, I still consider this a step into the right direction. The build itself mainly orients on the `yarn` commands used in the upstream Makefile[4]. However, we can't use `fetchYarnDeps` here because yarn v2 (a.k.a. `berry`) is in use which is why the same was done as in `hedgedoc`, writing a custom FoD that downloads all dependencies and writes the offline cache into `$out`[5]. Additionally there are two more notable differences to upstream: * We patch out every dependency to `@grafana/e2e` and `cypress`. The first is a dependency on the latter in another version and the latter downloads random blobs from the Internet in postInstall. Since it's a testing framework (and the `e2e` package apparently a testing library), I decided it's not worth the effort and patched it out everywhere. * There was a `zoneinfo.zip` in `$out/share/grafana/tools` that was installed from `srcStatic`. This only seems to be used on Windows[6] and that's not supported by this package, so I decided to drop it. [1] NixOS#251479 [2] NixOS#130201 [3] NixOS#104794 [4] https://github.com/grafana/grafana/blob/v10.3.1/Makefile [5] NixOS#245170 [6] https://github.com/grafana/grafana/blob/v10.3.1/pkg/setting/setting.go#L1012-L1014

Up until now, the frontend was taken from `srcStatic`, i.e. prebuilt from upstream. I recall at least three cases[1][2][3] where we got a hash mismatch eventually. Rather than spending time finding out whether or not it's a supply-chain attack or just a build issue, I decided to implement a source-build now with the following benefits: * It's now actually possible to apply patches for Grafana's frontend. * We rely a little less on third-party build systems. Of course, patching potential vulnerabilities in transitive frontend dependencies is still hard (let alone discovering that this package is affected!), but that's a fundamental issue we have in nixpkgs and I won't invent a half-baked solution just for this package, I still consider this a step into the right direction. The build itself mainly orients on the `yarn` commands used in the upstream Makefile[4]. However, we can't use `fetchYarnDeps` here because yarn v2 (a.k.a. `berry`) is in use which is why the same was done as in `hedgedoc`, writing a custom FoD that downloads all dependencies and writes the offline cache into `$out`[5]. Additionally there are two more notable differences to upstream: * We patch out every dependency to `@grafana/e2e` and `cypress`. The first is a dependency on the latter in another version and the latter downloads random blobs from the Internet in postInstall. Since it's a testing framework (and the `e2e` package apparently a testing library), I decided it's not worth the effort and patched it out everywhere. * There was a `zoneinfo.zip` in `$out/share/grafana/tools` that was installed from `srcStatic`. This only seems to be used on Windows[6] and that's not supported by this package, so I decided to drop it. [1] NixOS#251479 [2] NixOS#130201 [3] NixOS#104794 [4] https://github.com/grafana/grafana/blob/v10.3.1/Makefile [5] NixOS#245170 [6] https://github.com/grafana/grafana/blob/v10.3.1/pkg/setting/setting.go#L1012-L1014 (cherry picked from commit 608db26)

WilliButz added the 9.needs: port to stable A PR needs a backport to the stable release. label Nov 24, 2020

ofborg bot requested review from fpletz, Frostman, globin, offlinehacker, kalbasit and Ma27 November 24, 2020 19:58

ofborg bot added 11.by: package-maintainer 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 10.rebuild-linux: 1 labels Nov 24, 2020

Ma27 approved these changes Nov 25, 2020

View reviewed changes

grafana: 7.3.3 -> 7.3.4

4e4d498

https://github.com/grafana/grafana/releases/tag/v7.3.4

WilliButz force-pushed the update/grafana/v7.3.4 branch from 2e5bc36 to 4e4d498 Compare November 25, 2020 18:38

ofborg bot requested a review from Ma27 November 25, 2020 18:49

Ma27 approved these changes Nov 25, 2020

View reviewed changes

Ma27 merged commit 107750e into NixOS:master Nov 25, 2020

Ma27 removed the 9.needs: port to stable A PR needs a backport to the stable release. label Nov 25, 2020

Ma27 added the 8.has: port to stable A PR already has a backport to the stable release. label Nov 25, 2020

WilliButz deleted the update/grafana/v7.3.4 branch November 28, 2020 14:10

Ma27 mentioned this pull request Jul 16, 2021

grafana: 8.0.5 -> 8.0.6 #130201

Merged

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grafana: 7.3.3 -> 7.3.4 #104794

grafana: 7.3.3 -> 7.3.4 #104794

WilliButz commented Nov 24, 2020

Ma27 commented Nov 25, 2020 •

edited

WilliButz commented Nov 25, 2020

WilliButz commented Nov 25, 2020

WilliButz commented Nov 25, 2020

Ma27 commented Nov 25, 2020

WilliButz commented Nov 25, 2020 •

edited

cole-h commented Nov 25, 2020

Ma27 commented Nov 25, 2020

WilliButz commented Nov 25, 2020

WilliButz commented Nov 25, 2020

WilliButz commented Nov 25, 2020

Ma27 commented Nov 25, 2020

grafana: 7.3.3 -> 7.3.4 #104794

grafana: 7.3.3 -> 7.3.4 #104794

Conversation

WilliButz commented Nov 24, 2020

Motivation for this change

Things done

Ma27 commented Nov 25, 2020 • edited

WilliButz commented Nov 25, 2020

WilliButz commented Nov 25, 2020

WilliButz commented Nov 25, 2020

Ma27 commented Nov 25, 2020

WilliButz commented Nov 25, 2020 • edited

cole-h commented Nov 25, 2020

Ma27 commented Nov 25, 2020

WilliButz commented Nov 25, 2020

WilliButz commented Nov 25, 2020

WilliButz commented Nov 25, 2020

Ma27 commented Nov 25, 2020

Ma27 commented Nov 25, 2020 •

edited

WilliButz commented Nov 25, 2020 •

edited