Tests still fail on OfBorg. I'm a little leery of disabling tests in a PR that doesn't include that in the title.

cc @twhitehead who was working on the Julia tests in #79174 for thoughts on this disabling of tests.

Tests still fail on OfBorg. I'm a little leery of disabling tests in a PR that doesn't include that in the title.

yeh, we had a discussing in julia_13 PR. #79174 (comment) about disable the test file. In expected testing that is

You either have to do some text substitution on the test file (e.g., use sed to put a # in front of the line that runs the test) or remove or zero the test file itself. I'll submit my suggested cleanups when I get the chance (currently just having problems with one more test that I'm tracking down).

but, Until this time the testing problem still in here. that is why the Julia didn't have binary cache in hydra CI.

Tests still fail on OfBorg.

@drewrisinger my own hydra CI test for julia_13 at here. http://221.4.35.244:8300/build/6#tabs-summary

The OfBorg evaluation of this PR with tests fails: https://github.com/NixOS/nixpkgs/pull/89270/checks?check_run_id=725656788. Looks OfBorg build was against julia, not julia_13. Is that the one you meant to build/fix?
@GrahamcOfBorg build julia_13

looks like title should be changed to julia_13: ...

@GrahamcOfBorg eval
@GrahamcOfBorg build julia_13

@drewrisinger sorry for the delay in following up with the prior one. I've followed opened up the following upstream bug reports with respect to a variety of issues that are tripping up in nix

• test12806 breaks later exception handling tests (Exception stack not cleaned up after return inside of a catch handler with a finally clause JuliaLang/julia#34579)
• keywordargs test breaks precompile test set (keywordargs testset breaks precompile testset JuliaLang/julia#34921)
• REPL completions is broken by shared global state (Base.runtests("REPL") fails, but not when run together with some other testset JuliaLang/julia#32377)
• SIGINT stress test has issues ([0.7/1.0] Failed to catch SIGINT during stress.jl test JuliaLang/julia#28580)

I also discovered you need to set HOME and USER to something reasonable or a bunch of tests break

preCheck = ''
  HOME=$NIX_BUILD_TOP
  USER=$(whoami)
  JULIA_CPU_THREADS=''${NIX_BUILD_CORES:-1}
''

@twhitehead thanks for the response, I think fixing Julia tests is now out of scope for this PR, though I think it's still very important.

The underlying issue is that the build runs multiple tests in a single julia instance and not all tests leave behind a clean state (i.e., some of the earlier tests break things for later tests).

Why this is showing up under nix and not most other systems is because building without a network connection causes all the test to run serially in one julia process (where all bad test interactions surface) compared to the standard way of running tests in parallel across multiple julia process (where bad test interactions only surface if they happen to occur in the same julia process).

So, yeah, the Julia test process as it stands is reasonably broken. At least now upstream has a reproducible way to trigger it (test with only one process) and will hopefully get stuff cleaned up.

@twhitehead. @cstich I bumped up julia_13 to julia1.4.2. also, I did a build test and patched you have mentioned code on hydra CI. there are my test logs.http://221.4.35.244:8300/build/36/nixlog/1
that will help you to debug in testall process.