Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gitea: 1.5.1 -> 1.5.2 #48128

Merged
merged 1 commit into from Oct 15, 2018
Merged

gitea: 1.5.1 -> 1.5.2 #48128

merged 1 commit into from Oct 15, 2018

Conversation

florianjacob
Copy link
Contributor

@florianjacob florianjacob commented Oct 9, 2018

Motivation for this change

https://blog.gitea.io/2018/10/gitea-1.5.2-is-released/

“This is a smaller release, with only 12 merged PRs, but it contains several security related fixes and so we recommend upgrading whenever possible.”

Should therefore be safe to backport to 18.09, and required to get the security fixes.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@florianjacob
Copy link
Contributor Author

Maintainer: @disassembler

@xeji
Copy link
Contributor

xeji commented Oct 10, 2018

@GrahamcOfBorg build gitea

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: gitea

Partial log (click to expand)

code.gitea.io/gitea/models
code.gitea.io/gitea/modules/notification
code.gitea.io/gitea/modules/templates
code.gitea.io/gitea/modules/auth
code.gitea.io/gitea/modules/context
code.gitea.io/gitea/modules/lfs
code.gitea.io/gitea/routers/repo
code.gitea.io/gitea/routers/user
builder for '/nix/store/rz4y8f8pppi5k194gj7k89cskiapy8hf-gitea-1.5.2.drv' failed with exit code 47
error: build of '/nix/store/rz4y8f8pppi5k194gj7k89cskiapy8hf-gitea-1.5.2.drv' failed

@GrahamcOfBorg
Copy link

Unexpected error: command failed with exit code 1 on x86_64-darwin (full log)

Attempted: gitea

Partial log (click to expand)


trying https://github.com/go-gitea/gitea/archive/v1.5.2.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   122    0   122    0     0    203      0 --:--:-- --:--:-- --:--:--   202
100 17.9M    0 17.9M    0     0  4643k      0 --:--:--  0:00:03 --:--:-- 6856k
unpacking source archive /private/tmp/nix-build-source.drv-0/v1.5.2.tar.gz
fixed-output derivation produced path '/nix/store/4gydlwklm9v77jk8zj3n03f7csmkq18z-source' with sha256 hash '1dr7pxb60y215625nndmisxl6n5zlq1aicaswsg3xfzlk7a93k7k' instead of the expected hash '049f4gs1lw14l0ybz6rx6sibwpqdihqrx1qgvs6ghy6asrc2blif'
cannot build derivation '/nix/store/jsm9n5l3qsr8n7filwzmcbxvkvpgf3xa-gitea-1.5.2.drv': 1 dependencies couldn't be built
error: build of '/nix/store/jsm9n5l3qsr8n7filwzmcbxvkvpgf3xa-gitea-1.5.2.drv' failed

@xeji
Copy link
Contributor

xeji commented Oct 10, 2018

aarch64 failure looks like a builder issue, we can try again later.
Please fix the hash mismatch on darwin.

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: gitea

Partial log (click to expand)

checking for references to /build in /nix/store/fz05imgv890j71v5jp8bc8j54wqvx9is-gitea-1.5.2-bin...
shrinking RPATHs of ELF executables and libraries in /nix/store/hypb926rcw4dw6df3pf9pshrj54mnxvy-gitea-1.5.2
strip is /nix/store/dxf1m7dhc4qb655bdljc1fsd74v1nag3-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/hypb926rcw4dw6df3pf9pshrj54mnxvy-gitea-1.5.2
checking for references to /build in /nix/store/hypb926rcw4dw6df3pf9pshrj54mnxvy-gitea-1.5.2...
shrinking RPATHs of ELF executables and libraries in /nix/store/mj23a6lwgj105m2z58rh6llrn3mjymh8-gitea-1.5.2-data
strip is /nix/store/dxf1m7dhc4qb655bdljc1fsd74v1nag3-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/mj23a6lwgj105m2z58rh6llrn3mjymh8-gitea-1.5.2-data
checking for references to /build in /nix/store/mj23a6lwgj105m2z58rh6llrn3mjymh8-gitea-1.5.2-data...
/nix/store/fz05imgv890j71v5jp8bc8j54wqvx9is-gitea-1.5.2-bin

@xeji xeji mentioned this pull request Oct 11, 2018
9 tasks
@c0bw3b c0bw3b added 1.severity: security 9.needs: port to stable A PR needs a backport to the stable release. labels Oct 11, 2018
@andir
Copy link
Member

andir commented Oct 11, 2018

@GrahamcOfBorg build gitea

@GrahamcOfBorg
Copy link

Unexpected error: command failed with exit code 1 on x86_64-darwin (full log)

Attempted: gitea

Partial log (click to expand)


trying https://github.com/go-gitea/gitea/archive/v1.5.2.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   122    0   122    0     0    652      0 --:--:-- --:--:-- --:--:--   655
100 17.9M    0 17.9M    0     0  5791k      0 --:--:--  0:00:03 --:--:-- 6654k
unpacking source archive /private/tmp/nix-build-source.drv-0/v1.5.2.tar.gz
fixed-output derivation produced path '/nix/store/4gydlwklm9v77jk8zj3n03f7csmkq18z-source' with sha256 hash '1dr7pxb60y215625nndmisxl6n5zlq1aicaswsg3xfzlk7a93k7k' instead of the expected hash '049f4gs1lw14l0ybz6rx6sibwpqdihqrx1qgvs6ghy6asrc2blif'
cannot build derivation '/nix/store/jsm9n5l3qsr8n7filwzmcbxvkvpgf3xa-gitea-1.5.2.drv': 1 dependencies couldn't be built
error: build of '/nix/store/jsm9n5l3qsr8n7filwzmcbxvkvpgf3xa-gitea-1.5.2.drv' failed

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: gitea

Partial log (click to expand)

/nix/store/fz05imgv890j71v5jp8bc8j54wqvx9is-gitea-1.5.2-bin

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: gitea

Partial log (click to expand)

code.gitea.io/gitea/routers/api/v1/repo
code.gitea.io/gitea/routers/api/v1/user
code.gitea.io/gitea/routers
code.gitea.io/gitea/routers/api/v1/org
code.gitea.io/gitea/routers/api/v1/admin
code.gitea.io/gitea/routers/admin
code.gitea.io/gitea/routers/api/v1
code.gitea.io/gitea/routers/routes
builder for '/nix/store/rz4y8f8pppi5k194gj7k89cskiapy8hf-gitea-1.5.2.drv' failed with exit code 45
error: build of '/nix/store/rz4y8f8pppi5k194gj7k89cskiapy8hf-gitea-1.5.2.drv' failed

@andir
Copy link
Member

andir commented Oct 11, 2018

builds fine on aarch64 when issued with --option cores 2. Probably some limitation with the default go runtime settings. I vaguely recall some default thread limits.

@NixOS/darwin-maintainers Any idea why fetchFromGitHub expects a different hash on darwin?

@ElvishJerricco posted the following on IRC:

$ nix-prefetch-url --unpack https://github.com/go-gitea/gitea/archive/v1.5.2.tar.gz
unpacking...
path is '/nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz'
049f4gs1lw14l0ybz6rx6sibwpqdihqrx1qgvs6ghy6asrc2blif

$ ssh my-mac nix-prefetch-url --unpack https://github.com/go-gitea/gitea/archive/v1.5.2.tar.gz
unpacking...
path is '/nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz'
1dr7pxb60y215625nndmisxl6n5zlq1aicaswsg3xfzlk7a93k7k


```

@ElvishJerricco
Copy link
Contributor

My first guess is that it's due to macOS's case sensitive file system, but I haven't confirmed that the repo actually has any collisions.

@copumpkin
Copy link
Member

Doesn't appear to have obvious case collisions on this macOS, so I'm not sure why it fails.

Furthermore, I'm on macOS, and:

$ nix-prefetch-url --unpack https://github.com/go-gitea/gitea/archive/v1.5.2.tar.gz
unpacking...
path is '/nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz'
049f4gs1lw14l0ybz6rx6sibwpqdihqrx1qgvs6ghy6asrc2blif

I'm not running a case-sensitive filesystem so I'm puzzled.

@copumpkin
Copy link
Member

copumpkin commented Oct 12, 2018

@ElvishJerricco I can't get the other store path, but perhaps you can unpack and diff the two trees on your linux box?

@ElvishJerricco
Copy link
Contributor

@copumpkin:

$ linux=/nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz
$ mac=/nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz
$ diff -r $linux $mac 
Only in /nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz/integrations/gitea-repositories-meta/user2/utf8.git/refs/heads: ブランチ
Only in /nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz/integrations/gitea-repositories-meta/user2/utf8.git/refs/heads: ブランチ
Only in /nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz/integrations/gitea-repositories-meta/user2/utf8.git/refs/heads: Grüßen
Only in /nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz/integrations/gitea-repositories-meta/user2/utf8.git/refs/heads: Grüßen
Only in /nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz/integrations/gitea-repositories-meta/user2/utf8.git/refs/tags: タグ
Only in /nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz/integrations/gitea-repositories-meta/user2/utf8.git/refs/tags: タグ
Only in /nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz/integrations/gitea-repositories-meta/user2/utf8.git/refs/tags: Ё
Only in /nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz/integrations/gitea-repositories-meta/user2/utf8.git/refs/tags: Ё
diff: /nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz/vendor/github.com/Unknwon/cae/tz/testdata/testdir.lnk: No such file or directory
diff: /nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz/vendor/github.com/Unknwon/cae/tz/testdata/testdir.lnk: No such file or directory
diff: /nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz/vendor/github.com/Unknwon/cae/tz/testdata/test.lnk: No such file or directory
diff: /nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz/vendor/github.com/Unknwon/cae/tz/testdata/test.lnk: No such file or directory
diff: /nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz/vendor/github.com/Unknwon/cae/zip/testdata/testdir.lnk: No such file or directory
diff: /nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz/vendor/github.com/Unknwon/cae/zip/testdata/testdir.lnk: No such file or directory
diff: /nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz/vendor/github.com/Unknwon/cae/zip/testdata/test.lnk: No such file or directory
diff: /nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz/vendor/github.com/Unknwon/cae/zip/testdata/test.lnk: No such file or directory
diff: /nix/store/n3bg2vhxv8yfc631la9gfykglg3f55b3-v1.5.2.tar.gz/vendor/gopkg.in/macaron.v1/fixtures/symlink: No such file or directory
diff: /nix/store/jq8p6y5zjzhaipsb5wg6bqziw7i45q7g-v1.5.2.tar.gz/vendor/gopkg.in/macaron.v1/fixtures/symlink: No such file or directory

Maybe something unicode related? I'm on sierra, if that matters.

@copumpkin
Copy link
Member

Ah yes, some of the unicode normalization of combining pairs (whether to store them as two characters or one) behavior can be weird. Not really sure what to do about that, other than to add a postFetch step to our fetchFromGitHub that strips that junk out

@andir
Copy link
Member

andir commented Oct 12, 2018

@copumpkin @ElvishJerricco does any of you have the time to implement test it?

I do not have a Mac to test it and never used one. Would probably take me 10x just to figure that out with always going through ofBorg 😕

@c0bw3b
Copy link
Contributor

c0bw3b commented Oct 12, 2018

Here is a previous occurrence of the same issue.

If subdirs vendor and integrations are not necessary to the build process, then you can do:

  src = fetchFromGitHub {
    owner = "go-gitea";
    repo = "gitea";
    rev = "v${version}";
    sha256 = "<hash>";
    extraPostFetch = "rm -rf $out/integrations && rm -rf $out/vendor";
  };

@ElvishJerricco
Copy link
Contributor

@c0bw3b you'll need to update the sha256 there.

@c0bw3b
Copy link
Contributor

c0bw3b commented Oct 12, 2018

Yes indeed that hash needs to be re-computed. I edited the code block.

@florianjacob
Copy link
Contributor Author

Thank you all for taking a look at this, I was really lost at how it could possibly be the case that MacOS computes a different checksum than on Linux.

As one could expect from their names, I just verified that integrations contains integration tests and is not required for build, while vendor contains various third-party library dependencies and is indeed required, and needs selective manual / automatic filtering.

For further testing / discussion, I pushed a new version with a extraPostFetch that deletes offending files and still seems to work for me.

Some kind of automatic mechanism in the fetch function would still help though, so that this won't return on the next release, and nix-prefetch-git gives the correct sha256 sum. This would lead to hard-to-detect problems though if some project uses / starts to use affected unicode regions in file names that are relevant for the build process.

@florianjacob
Copy link
Contributor Author

Could someone with the appropriate permissions please trigger a @GrahamcOfBorg build gitea?

@andir
Copy link
Member

andir commented Oct 15, 2018

@GrahamcOfBorg build gitea

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: gitea

Partial log (click to expand)

code.gitea.io/gitea/vendor/github.com/Unknwon/cae/zip
code.gitea.io/gitea/vendor/github.com/glycerine/go-unsnap-stream
code.gitea.io/gitea/vendor/github.com/couchbase/vellum
code.gitea.io/gitea/vendor/code.gitea.io/git
code.gitea.io/gitea/vendor/github.com/Unknwon/i18n
code.gitea.io/gitea/vendor/gopkg.in/editorconfig/editorconfig-core-go.v1
code.gitea.io/gitea/vendor/github.com/go-xorm/xorm
code.gitea.io/gitea/vendor/gopkg.in/testfixtures.v2
builder for '/nix/store/8jijwi2w49hsyfcyhb0mgana4hxp6sbd-gitea-1.5.2.drv' failed with exit code 43
error: build of '/nix/store/8jijwi2w49hsyfcyhb0mgana4hxp6sbd-gitea-1.5.2.drv' failed

@ElvishJerricco
Copy link
Contributor

Works fine on my mac.

$ nix build -f https://github.com/NixOS/nixpkgs/archive/0a5c7cf512fcefb4b337186f0aee69ae8dd1df2c.tar.gz gitea.src
[1 built, 55 copied (111.7 MiB), 37.0 MiB DL]
/nix/store/cg1gbwbc68ilw81djc13hp78bhgfsqq1-source

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: gitea

Partial log (click to expand)

installing
post-installation fixup
strip is /nix/store/53nysl8bqwxihwzs1pgwka20nf8mbvlp-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/8ld4g6s4v1w0mqw0h8qrjxxgv7bls7yh-gitea-1.5.2-bin/bin
patching script interpreter paths in /nix/store/8ld4g6s4v1w0mqw0h8qrjxxgv7bls7yh-gitea-1.5.2-bin
strip is /nix/store/53nysl8bqwxihwzs1pgwka20nf8mbvlp-cctools-binutils-darwin/bin/strip
patching script interpreter paths in /nix/store/3rg9lqwgksn39bg2qw54dzdypa4jjwmj-gitea-1.5.2
strip is /nix/store/53nysl8bqwxihwzs1pgwka20nf8mbvlp-cctools-binutils-darwin/bin/strip
patching script interpreter paths in /nix/store/zjndjg1yzmphlmafws14kzhckb7pd5a5-gitea-1.5.2-data
/nix/store/8ld4g6s4v1w0mqw0h8qrjxxgv7bls7yh-gitea-1.5.2-bin

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: gitea

Partial log (click to expand)

checking for references to /build in /nix/store/7jk9mp702a9ry6qnv2rc40vhl8pxrzl8-gitea-1.5.2-bin...
shrinking RPATHs of ELF executables and libraries in /nix/store/w5h82yifgv8ifcpjm73v288k65n49766-gitea-1.5.2
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/w5h82yifgv8ifcpjm73v288k65n49766-gitea-1.5.2
checking for references to /build in /nix/store/w5h82yifgv8ifcpjm73v288k65n49766-gitea-1.5.2...
shrinking RPATHs of ELF executables and libraries in /nix/store/h51jpsy5mwx25h73qx3jj4vbs6gfll6z-gitea-1.5.2-data
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/h51jpsy5mwx25h73qx3jj4vbs6gfll6z-gitea-1.5.2-data
checking for references to /build in /nix/store/h51jpsy5mwx25h73qx3jj4vbs6gfll6z-gitea-1.5.2-data...
/nix/store/7jk9mp702a9ry6qnv2rc40vhl8pxrzl8-gitea-1.5.2-bin

@andir andir merged commit 81593e3 into NixOS:master Oct 15, 2018
@andir
Copy link
Member

andir commented Oct 15, 2018

18.03: 8ccf5b6
18.09: 38f5b07

@florianjacob florianjacob deleted the gitea branch October 15, 2018 19:46
@florianjacob
Copy link
Contributor Author

Thanks! 😃

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants