Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Java 7 and Java 9, bootstrap JDK 10 via JDK 10 #38097

Closed
wants to merge 11 commits into from

Conversation

thoughtpolice
Copy link
Member

@thoughtpolice thoughtpolice commented Mar 30, 2018

Motivation for this change

This bootstraps the JDK 10 builds using a pre-existing JDK 10 binary bootstrap tarball, created using pkgs/development/compilers/make-bootstrap.nix. The bootstrapping images for i686 and x86_64 are both compiled to a fixed point: I compiled the original images with JDK 9, producing a minimal JDK 10 bootstrap image, and used it to build a minimal JDK 10 again, and packaged that as the final image. So this is JDK10 compiled by JDK10.

With this in place, we can remove the constraint on Java9 as the JDK 10 bootstrap method, and proceed to eliminate it.

I also took the liberty to take this time to remove some ancient packages that were still relying on JDK 7, and eliminate that too. Technically I believe Red Hat is still supporting it (it was EOL'd years ago otherwise by Oracle), but honestly I see little reason to keep building it; less packages for Hydra are always better.

To do this however, I had to eliminate some older packages nobody is maintaining anymore it seems: ECJ (the Eclipse Java Compiler), GNU classpath, and an ancient version of processing 2.0. I hope this is OK; I have no need for these packages and nobody seems to have touched them in years, really, other than tree-wide changes. So I think they should go, too.

With this, Nixpkgs only offers JDK 8 and JDK 10, which is a much easier support window (especially for my upcoming plans, such as the Shenandoah previews).

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.
  • Binary tarballs must be moved to tarballs.nixos.org and off Dropbox and my personal server; @shlevy will handle this.

./bin/java now apparently requires zlib.so, otherwise the whole
thing is busted. This is even required in the minimal configuration.

Unfortunately this impiles a rebuild of *all* OpenJDK packages and
their downstream dependencies.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
This is now required to build JDK 10 with JDK 10, so we're a bit
stuck on this one.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
JDK 10 can only be bootstrapped by version 9 or 10, but we want to
now get rid of version 9.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
Java 9 is EOL, and we no longer need it with JDK 10 builds bootstrapped via JDK
10.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
This is the only package that still needs ECJ and has no dependencies.
It's ancient and unmaintained and should just be removed.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
Nobody has stepped up to keep maintaining this and it's several
years old, and the last strict Java 7 dependency, as it won't work
with newer versions without an update.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Signed-off-by: Austin Seipp <aseipp@pobox.com>
JDK 7 was technically EOL'd a while ago, although RedHat etc are still
doing updates I believe. However, JDK 8 is the default in the tree and
really used everywhere, and JDK 7 isn't seeing many updates by current maintainers, so dropping it seems appropriate.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: diffoscope

The following builds were skipped because they don't evaluate on aarch64-linux: openjdk10

Partial log (click to expand)

make: Leaving directory '/build/diffoscope/doc'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91
gzipping man pages under /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91/share/man/
strip is /nix/store/3zq400fri5dv7d30lpxlqm2v9y1iis6j-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91/lib  /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91/bin
patching script interpreter paths in /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91
checking for references to /build in /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91...
wrapping `/nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91/bin/diffoscope'...
/nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91

Signed-off-by: Austin Seipp <aseipp@pobox.com>
Signed-off-by: Austin Seipp <aseipp@pobox.com>
@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: diffoscope

The following builds were skipped because they don't evaluate on aarch64-linux: openjdk, openjdk10, openjdk8

Partial log (click to expand)

make: Leaving directory '/build/diffoscope/doc'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91
gzipping man pages under /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91/share/man/
strip is /nix/store/3zq400fri5dv7d30lpxlqm2v9y1iis6j-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91/lib  /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91/bin
patching script interpreter paths in /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91
checking for references to /build in /nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91...
wrapping `/nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91/bin/diffoscope'...
/nix/store/2yrjbjzmm4hhj5l2rxvb8b32dfdl5kyj-diffoscope-91

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: diffoscope, openjdk10

Partial log (click to expand)

strange: no string table
strange: no string table
strange: no string table
strange: no string table
strange: no string table
strange: no string table
strange: no string table
strange: no string table
/nix/store/w2p9iiwz31rjm8w79hi2zrvs4jpam1yb-diffoscope-91
/nix/store/qkq74f2lzr4jxqilgh8akyqh55csnhag-openjdk-10-b46

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: diffoscope, openjdk, openjdk10, openjdk8

Partial log (click to expand)

strange: no string table
strange: no string table
strange: no string table
strange: no string table
strange: no string table
strange: no string table
strange: no string table
strange: no string table
strange: no string table
�[31;1merror:�[0m build of '/nix/store/jg0sa9l8305p1801a6ymplfqmrlg32rr-openjdk-8u172b02.drv' failed

@shlevy
Copy link
Member

shlevy commented Mar 30, 2018

@thoughtpolice Any idea about the build failure here?

Copy link
Member

@shlevy shlevy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM. Will upload tarballs when this is otherwise completely ready, would like to understand the build failure and give some time for people to object to the removals first.

@thoughtpolice
Copy link
Member Author

Apparently the doc building failed or something? But I have no clue how!

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007ffff7469872, pid=18114, tid=140736692631296
#
# JRE version: OpenJDK Runtime Environment (8.0_40) (build 1.8.0_40-27)
# Java VM: OpenJDK 64-Bit Server VM (25.40-b25 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libc.so.6+0x7c872]  _int_free+0x5c2
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /build/jdk8u-jdk8u172-b02/make/hs_err_pid18114.log
Compiled method (c1)   16742 2487       3       com.sun.tools.javadoc.DocImpl::comment (104 bytes)
 total in heap  [0x00007fffe1578490,0x00007fffe1579f28] = 6808
 relocation     [0x00007fffe15785b8,0x00007fffe15786f8] = 320
 main code      [0x00007fffe1578700,0x00007fffe15795c0] = 3776
 stub code      [0x00007fffe15795c0,0x00007fffe15796d8] = 280
 metadata       [0x00007fffe15796d8,0x00007fffe1579748] = 112
 scopes data    [0x00007fffe1579748,0x00007fffe1579c08] = 1216
 scopes pcs     [0x00007fffe1579c08,0x00007fffe1579e88] = 640
 dependencies   [0x00007fffe1579e88,0x00007fffe1579ea0] = 24
 nul chk table  [0x00007fffe1579ea0,0x00007fffe1579f28] = 136
Compiled method (c1)   16742 2487       3       com.sun.tools.javadoc.DocImpl::comment (104 bytes)
 total in heap  [0x00007fffe1578490,0x00007fffe1579f28] = 6808
 relocation     [0x00007fffe15785b8,0x00007fffe15786f8] = 320
 main code      [0x00007fffe1578700,0x00007fffe15795c0] = 3776
 stub code      [0x00007fffe15795c0,0x00007fffe15796d8] = 280
 metadata       [0x00007fffe15796d8,0x00007fffe1579748] = 112
 scopes data    [0x00007fffe1579748,0x00007fffe1579c08] = 1216
 scopes pcs     [0x00007fffe1579c08,0x00007fffe1579e88] = 640
 dependencies   [0x00007fffe1579e88,0x00007fffe1579ea0] = 24
 nul chk table  [0x00007fffe1579ea0,0x00007fffe1579f28] = 136
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

The only thing I can think of is if b2b98b8 somehow has changed something, but I can't really imagine how, since otherwise I don't think any JDK 8 rebuild would have happened at all!

Maybe it had an OOM?

@shlevy
Copy link
Member

shlevy commented Mar 30, 2018

@GrahamcOfBorg build openjdk8

@GrahamcOfBorg
Copy link

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: openjdk8

Partial log (click to expand)

Cannot nix-instantiate `openjdk8' because:
�[31;1merror:�[0m while evaluating the attribute 'buildInputs' of the derivation 'openjdk-8u172b02' at �[1m/var/lib/gc-of-borg/nix-test-rs-19/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-19/pkgs/stdenv/generic/make-derivation.nix�[0m:148:11:
while evaluating the attribute 'buildCommand' of the derivation 'openjdk-bootstrap' at �[1m/var/lib/gc-of-borg/nix-test-rs-19/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-19/pkgs/stdenv/generic/make-derivation.nix�[0m:148:11:
No bootstrap for system

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: openjdk8

Partial log (click to expand)

shrinking /nix/store/nqkgz6s57fpjgsmwch9ghijck5nf9gn5-openjdk-8u172b02-jre/lib/openjdk/jre/lib/amd64/libsctp.so
shrinking /nix/store/nqkgz6s57fpjgsmwch9ghijck5nf9gn5-openjdk-8u172b02-jre/lib/openjdk/jre/lib/amd64/liblcms.so
strip is /nix/store/fzcs0fn6bb04m82frhlb78nc03ny3w55-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/nqkgz6s57fpjgsmwch9ghijck5nf9gn5-openjdk-8u172b02-jre/lib  /nix/store/nqkgz6s57fpjgsmwch9ghijck5nf9gn5-openjdk-8u172b02-jre/bin
patching script interpreter paths in /nix/store/nqkgz6s57fpjgsmwch9ghijck5nf9gn5-openjdk-8u172b02-jre
checking for references to /build in /nix/store/nqkgz6s57fpjgsmwch9ghijck5nf9gn5-openjdk-8u172b02-jre...
not an ELF executable
not an ELF executable
not an ELF executable
/nix/store/xpjj0j2r340rf2mk6pj5y90l0rsxy968-openjdk-8u172b02

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: openjdk8

Partial log (click to expand)

these paths will be fetched (56.31 MiB download, 168.63 MiB unpacked):
  /nix/store/p417q2rl7dfpjc98rbn1r8hlg30b18hi-zulu1.8.0_121-8.20.0.5
  /nix/store/qv1jbhrfj7wi614kgwbh4aw39xinyjii-hook
copying path '/nix/store/qv1jbhrfj7wi614kgwbh4aw39xinyjii-hook' from 'https://cache.nixos.org'...
copying path '/nix/store/p417q2rl7dfpjc98rbn1r8hlg30b18hi-zulu1.8.0_121-8.20.0.5' from 'https://cache.nixos.org'...
/nix/store/p417q2rl7dfpjc98rbn1r8hlg30b18hi-zulu1.8.0_121-8.20.0.5

@thoughtpolice thoughtpolice mentioned this pull request Mar 30, 2018
8 tasks
@shlevy
Copy link
Member

shlevy commented Mar 31, 2018

@thoughtpolice OK, looks good to me. Any more changes pending besides the URL update?

@vcunat
Copy link
Member

vcunat commented Mar 31, 2018

I suppose at least the removals are suitable for 18.03 before we announce it – but you surely know the java stuff better than me...

@jerith666
Copy link
Contributor

I agree, probably best to backport these removals to 18.03 as well.

@thoughtpolice
Copy link
Member Author

thoughtpolice commented Mar 31, 2018

Yes, I think we should backport this; I can do that. @shlevy I think this is otherwise ready to go if you're willing to upload the tarballs. I can adjust the URLs and merge.

@NeQuissimus
Copy link
Member

👍 Java 9, the one nobody ever used xD

@shlevy
Copy link
Member

shlevy commented Apr 1, 2018

@thoughtpolice http://tarballs.nixos.org/openjdk/2018-03-31/{8,10}/{x86_64,i686}-linux.tar.xz now exist. You probably want to use the name attribute to fetchurl to avoid changing all the downstream hashes.

@thoughtpolice
Copy link
Member Author

I rebased this into master directly but it didn't close this. :/ Thanks @shlevy for the help, and everyone else!

I'll be porting these to 18.03 soon.

thoughtpolice added a commit to thoughtpolice/nixpkgs that referenced this pull request Apr 1, 2018
Many thanks to @shlevy for doing this migration of the old and new tarballs.

Closes NixOS#38097.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
@thoughtpolice thoughtpolice deleted the rm-rf-jdk7-and-9 branch April 1, 2018 20:25
@vcunat vcunat mentioned this pull request Jun 24, 2018
@vcunat
Copy link
Member

vcunat commented Jun 24, 2018

@thoughtpolice: you haven't got to look at this for 18.03, right? (I don't know the Java ecosystem.)

@thoughtpolice
Copy link
Member Author

Sorry I missed this @vcunat -- yes, I thought I had pushed these changes a while back, but had not it seems! I'll look into fixing this...

@lopsided98 lopsided98 mentioned this pull request Oct 12, 2019
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants