hadoop: fixing builds #68370
hadoop: fixing builds #68370
Conversation
|
@volth: Yeah, probably better. I was wondering about that but didn't look into the other corners yet. Do you already have a workable fix in mind? I don't know the java dev packages ecosystem in nixos too well yet. |
|
@volth: Thanks for the pointer - I expected it to be a bit more involved. I'll try a fix from that angle and will report back. |
For space optimization reasons, the JRE was linked into the JDK from a separate output. While this saves some space, it breaks certain assumptions made in some software, expecting the JRE to reside within the JDK tree and to be able to reach JDK libraries by traversing up the folders and similar. This approach should reduce the need for workarounds in jvm based builds and other issues at the cost of some storage space. JDK colosure size delta: +24% JRE colosure size delta: +4% (this is likely for other reasons) This addresses NixOS#37364 and might also supersede NixOS#68370 I tested this building jogl (removing the workaround) and hadoop.
Some annotation processing depends on tools.jar, which is usually shipped with JDKs. Because we symlink JRE into the JDK, a traversal from the JRE back to the JDK becomes impossible, so some assumptions of relative paths break. For ZHF: NixOS#68361 Tested build for 2.7, 3.0 and 3.1
d-goldin
force-pushed
the
release-19.09/fix-hadoop
branch
from
c4dc570
to
7d4eff9
Compare
|
@volth: Alright, I made another PR to copy instead of link the JRE but also removed the unneeded addition to nativeBuildInputs. Lets see which one is favoured. |
|
I'm guessing this should be targetting release-19.09, given the source branch name? |
|
@lheckemann: Initially this was targeted at 19.09, but then I tried to fix the issue at the root and if #68382 is merged, this PR should become obsolete, as @volth said. |
|
Right. 19.09 has been branched off already, but #68382 should imho be backported when it's merged, so I guess this PR doesn't need to go in either of them? |
|
@lheckemann: If we think the other one has a good chance at being merged, then we can close off this one for now and see what happens. Regarding release-19.09 being branched off - I have been basing all of my fixes off master at the moment, at least that's how I interpreted #68361 . Should I do them against release-19.09 directly going forward? |
|
Closing this one in favour of #68382 - if it for some reason doesn't go through or is otherwise problematic, we can still come back to this one. |
No, you've been doing everything right :) the only case where a PR should only go against release-19.09 is if the bug doesn't exist in master. |
|
@lheckemann: Thanks for explaining :) |
Motivation for this change
Some annotation processing depends on tools.jar, which is usually
shipped with JDKs. Because we symlink JRE into the JDK, a traversal
from the JRE back to the JDK becomes impossible, so some assumptions
of relative paths break.
For ZHF: #68361
Overall, the impact of this change should only be visible during the build and have no further consequences.
Some logs of recently failed builds:
https://hydra.nixos.org/build/100037784/nixlog/1 (2.7)
https://hydra.nixos.org/build/100064140/nixlog/1 (2.8)
https://hydra.nixos.org/build/100022159/nixlog/2 (2.9)
https://hydra.nixos.org/build/100055839/nixlog/1 (3.0)
https://hydra.nixos.org/build/100059593/nixlog/1 (3.1)
Things done
sandboxinnix.confon non-NixOS)nix-shell -p nix-review --run "nix-review wip"./result/bin/)nix path-info -Sbefore and after)Notify maintainers
cc @volth
cc @NixOS/backports