Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for target aarch64-linux-musl #5861

Merged
merged 1 commit into from Mar 26, 2018

Conversation

jirutka
Copy link
Contributor

@jirutka jirutka commented Mar 25, 2018

@RX14, could you please verify if it’s correct? Is there some way how to automatically check it or even generate? I’ve done this by hand, comparing existing x86_64-linux-gnu vs. aarch64-linux-gnu and musl arch-specific sources.

Related to #5467

@RX14
Copy link
Contributor

RX14 commented Mar 25, 2018

@jirutka really the only way to test it would be to run the spec suite and compile the compiler with a compiled compiler.

There used to be an automated tool for generating these but it's out of date :(

@@ -6,7 +6,7 @@ LLVM.init_aarch64
{% end %}

private def abi
triple = "aarch64-unknown-linux-gnu"
triple = LLVM.default_target_triple.gsub(/^(.+?)-/, "aarch64-")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not 100% on this change, why does it need to happen?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not sure about it too, but hard-coded aarch64-unknown-linux-gnu is not very useful for testing aarch64 with musl, isn’t it?

Copy link
Contributor Author

@jirutka jirutka Mar 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ve copied this from:

triple = LLVM.default_target_triple.gsub(/^(.+?)-/, "x86_64-")

Copy link
Contributor

@RX14 RX14 Mar 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these specs should be changed to always test all the triples we support... Specs run shouldn't depend on the config of the host.

However, leaving it as-is is fine for now, since there's no difference between the calling convention and struct layout for musl and glibc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I’ve reverted this change.

@jirutka
Copy link
Contributor Author

jirutka commented Mar 25, 2018

There used to be an automated tool for generating these but it's out of date :(

Does it at least partially work? Where can I find it?

@Sija
Copy link
Contributor

Sija commented Mar 25, 2018

@jirutka There are several, see crystal-lang/crystal_lib, olbat/libgen and TechMagister/bindgencr.

@RX14
Copy link
Contributor

RX14 commented Mar 25, 2018

@Sija no, the one specifically used for generating the current bindings was https://github.com/ysbaddaden/posix. But the definition files are incomplete and I never got it to work. It might be able to get you most of the way there though?

@Sija
Copy link
Contributor

Sija commented Mar 25, 2018

@RX14 thx, I had a feeling I missed one from @ysbaddaden.

@jirutka
Copy link
Contributor Author

jirutka commented Mar 25, 2018

make spec fails on:

Code gen: asm
  codegens without inputs<inline asm>:1:6: error: invalid token in expression
        mov $1234, x11
            ^
<inline asm>:1:6: error: invalid operand
        mov $1234, x11
            ^
LLVM ERROR: Error parsing inline asm

And it stops on this, even when --fail-fast is not specified. :(

@jirutka
Copy link
Contributor Author

jirutka commented Mar 25, 2018

$ ./bin/crystal spec -p -s --error-trace --verbose spec/compiler
...
Failures:

  1) Code gen: sizeof gets sizeof union
     Failure/Error: {% if flag?(:x86_64) %}

       Expected: 12
            got: 16_u64

     # spec/compiler/codegen/sizeof_spec.cr:51

  2) Code gen: sizeof can use sizeof of virtual type
     Failure/Error: {% if flag?(:x86_64) %}

       Expected: 4
            got: 8_u64

     # spec/compiler/codegen/sizeof_spec.cr:140

Finished in 39:08 minutes
7374 examples, 2 failures, 0 errors, 9 pending

These are just broken tests, the result is correct.


$ ./bin/crystal spec -p -s --error-trace --verbose spec/std
Failures:

  1) IO encoding encode raises on invalid byte sequence
     Failure/Error: expect_raises ArgumentError, "Invalid multibyte sequence" do

       Expected ArgumentError with "Invalid multibyte sequence", got #<ArgumentError: Invalid encoding: GB2312> with backtrace:
         # src/iconv.cr:0:11 in 'initialize'
         # src/iconv.cr:7:3 in 'new'
         # src/io/encoding.cr:20:16 in 'initialize'
         # src/io/encoding.cr:19:5 in 'new'
         # src/io.cr:494:7 in 'encoder'
         # src/io.cr:484:8 in 'write_utf8'
         # src/string.cr:4139:5 in 'to_s'
         # src/io.cr:192:5 in '<<'
         # src/io.cr:206:5 in 'print'
         # src/spec/methods.cr:255:3 in 'it'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # src/spec/methods.cr:16:5 in '~procProc(Nil)'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # src/spec/methods.cr:16:5 in '~procProc(Nil)'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # spec/std/io/multi_writer_spec.cr:1:1 in '__crystal_main'
         # src/crystal/main.cr:11:3 in '_crystal_main'
         # src/crystal/main.cr:112:5 in 'main_user_code'
         # src/crystal/main.cr:101:7 in 'main'
         # src/crystal/main.cr:135:3 in 'main'
         # ???

     # spec/std/io/io_spec.cr:809

  2) IO encoding encode raises on incomplete byte sequence
     Failure/Error: expect_raises ArgumentError, "Incomplete multibyte sequence" do

       Expected ArgumentError with "Incomplete multibyte sequence", got #<ArgumentError: Invalid encoding: GB2312> with backtrace:
         # src/iconv.cr:0:11 in 'initialize'
         # src/iconv.cr:7:3 in 'new'
         # src/io/encoding.cr:20:16 in 'initialize'
         # src/io/encoding.cr:19:5 in 'new'
         # src/io.cr:494:7 in 'encoder'
         # src/io.cr:484:8 in 'write_utf8'
         # src/string.cr:4139:5 in 'to_s'
         # src/io.cr:192:5 in '<<'
         # src/io.cr:206:5 in 'print'
         # src/spec/methods.cr:255:3 in 'it'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # src/spec/methods.cr:16:5 in '~procProc(Nil)'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # src/spec/methods.cr:16:5 in '~procProc(Nil)'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # spec/std/io/multi_writer_spec.cr:1:1 in '__crystal_main'
         # src/crystal/main.cr:11:3 in '_crystal_main'
         # src/crystal/main.cr:112:5 in 'main_user_code'
         # src/crystal/main.cr:101:7 in 'main'
         # src/crystal/main.cr:135:3 in 'main'
         # ???

     # spec/std/io/io_spec.cr:824

  3) String encode raises if illegal byte sequence
     Failure/Error: expect_raises ArgumentError, "Invalid multibyte sequence" do

       Expected ArgumentError with "Invalid multibyte sequence", got #<ArgumentError: Invalid encoding: GB2312> with backtrace:
         # src/iconv.cr:0:11 in 'initialize'
         # src/iconv.cr:7:3 in 'new'
         # src/string.cr:36:13 in 'encode'
         # src/string.cr:1162:5 in 'encode'
         # src/kernel.cr:28:5 in '~procProc(Nil)'
         # src/spec/methods.cr:255:3 in 'it'
         # src/kernel.cr:28:5 in '~procProc(Nil)'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # src/spec/expectations.cr:210:7 in '~procProc(Nil)'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # spec/std/thread/condition_variable_spec.cr:1:1 in '__crystal_main'
         # src/crystal/main.cr:11:3 in '_crystal_main'
         # src/crystal/main.cr:112:5 in 'main_user_code'
         # src/crystal/main.cr:101:7 in 'main'
         # src/crystal/main.cr:135:3 in 'main'
         # ???

     # spec/std/string_spec.cr:2236

  4) String encode raises if incomplete byte sequence
     Failure/Error: expect_raises ArgumentError, "Incomplete multibyte sequence" do

       Expected ArgumentError with "Incomplete multibyte sequence", got #<ArgumentError: Invalid encoding: GB2312> with backtrace:
         # src/iconv.cr:0:11 in 'initialize'
         # src/iconv.cr:7:3 in 'new'
         # src/string.cr:36:13 in 'encode'
         # src/string.cr:1162:5 in 'encode'
         # src/kernel.cr:28:5 in '~procProc(Nil)'
         # src/spec/methods.cr:255:3 in 'it'
         # src/kernel.cr:28:5 in '~procProc(Nil)'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # src/spec/expectations.cr:210:7 in '~procProc(Nil)'
         # src/spec/context.cr:255:3 in 'describe'
         # src/spec/methods.cr:16:5 in 'describe'
         # spec/std/thread/condition_variable_spec.cr:1:1 in '__crystal_main'
         # src/crystal/main.cr:11:3 in '_crystal_main'
         # src/crystal/main.cr:112:5 in 'main_user_code'
         # src/crystal/main.cr:101:7 in 'main'
         # src/crystal/main.cr:135:3 in 'main'
         # ???

     # spec/std/string_spec.cr:2246

  5) Time changes timezone with ENV["TZ"]
     Failure/Error: offset1.should_not eq(offset2)

       Expected: actual_value != 0
            got: 0

     # spec/std/time/time_spec.cr:709

  6) IO::Buffered encoding decode gets big GB2312 string

       Invalid encoding: GB2312
       src/iconv.cr:0:11 in 'initialize'
       src/iconv.cr:7:3 in 'new'
       src/string.cr:36:13 in 'encode'
       src/string.cr:1162:5 in 'encode'
       src/spec/expectations.cr:210:7 in '~procProc(Nil)'
       src/spec/methods.cr:255:3 in 'it'
       src/spec/expectations.cr:210:7 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/expectations.cr:210:7 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/expectations.cr:210:7 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       spec/std/io/hexdump_spec.cr:1:1 in '__crystal_main'
       src/crystal/main.cr:11:3 in '_crystal_main'
       src/crystal/main.cr:112:5 in 'main_user_code'
       src/crystal/main.cr:101:7 in 'main'
       src/crystal/main.cr:135:3 in 'main'
       ???

  7) IO encoding decode gets big GB2312 string

       Invalid encoding: GB2312
       src/iconv.cr:0:11 in 'initialize'
       src/iconv.cr:7:3 in 'new'
       src/string.cr:36:13 in 'encode'
       src/string.cr:1162:5 in 'encode'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/methods.cr:255:3 in 'it'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       spec/std/io/multi_writer_spec.cr:1:1 in '__crystal_main'
       src/crystal/main.cr:11:3 in '_crystal_main'
       src/crystal/main.cr:112:5 in 'main_user_code'
       src/crystal/main.cr:101:7 in 'main'
       src/crystal/main.cr:135:3 in 'main'
       ???

  8) IO encoding decode reads utf8

       Invalid encoding: GB2312
       src/iconv.cr:0:11 in 'initialize'
       src/iconv.cr:7:3 in 'new'
       src/string.cr:36:13 in 'encode'
       src/string.cr:1162:5 in 'encode'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/methods.cr:255:3 in 'it'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       spec/std/io/multi_writer_spec.cr:1:1 in '__crystal_main'
       src/crystal/main.cr:11:3 in '_crystal_main'
       src/crystal/main.cr:112:5 in 'main_user_code'
       src/crystal/main.cr:101:7 in 'main'
       src/crystal/main.cr:135:3 in 'main'
       ???

  9) IO encoding decode skips invalid byte sequences

       Invalid encoding: GB2312
       src/iconv.cr:0:11 in 'initialize'
       src/iconv.cr:7:3 in 'new'
       src/string.cr:36:13 in 'encode'
       src/string.cr:1162:5 in 'encode'
       src/spec/expectations.cr:210:7 in '~procProc(Nil)'
       src/spec/methods.cr:255:3 in 'it'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       spec/std/io/multi_writer_spec.cr:1:1 in '__crystal_main'
       src/crystal/main.cr:11:3 in '_crystal_main'
       src/crystal/main.cr:112:5 in 'main_user_code'
       src/crystal/main.cr:101:7 in 'main'
       src/crystal/main.cr:135:3 in 'main'
       ???

 10) IO encoding encode skips on invalid byte sequence

       Invalid encoding: GB2312
       src/iconv.cr:0:11 in 'initialize'
       src/iconv.cr:7:3 in 'new'
       src/io/encoding.cr:20:16 in 'initialize'
       src/io/encoding.cr:19:5 in 'new'
       src/io.cr:494:7 in 'encoder'
       src/io.cr:484:8 in 'write_utf8'
       src/string.cr:4139:5 in 'to_s'
       src/io.cr:192:5 in '<<'
       src/io.cr:206:5 in 'print'
       src/spec/methods.cr:255:3 in 'it'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/methods.cr:16:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       spec/std/io/multi_writer_spec.cr:1:1 in '__crystal_main'
       src/crystal/main.cr:11:3 in '_crystal_main'
       src/crystal/main.cr:112:5 in 'main_user_code'
       src/crystal/main.cr:101:7 in 'main'
       src/crystal/main.cr:135:3 in 'main'
       ???

 11) String encode doesn't raise on invalid byte sequence

       Invalid encoding: GB2312
       src/iconv.cr:0:11 in 'initialize'
       src/iconv.cr:7:3 in 'new'
       src/string.cr:36:13 in 'encode'
       src/string.cr:1162:5 in 'encode'
       src/kernel.cr:28:5 in '~procProc(Nil)'
       src/spec/methods.cr:255:3 in 'it'
       src/kernel.cr:28:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/expectations.cr:210:7 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       spec/std/thread/condition_variable_spec.cr:1:1 in '__crystal_main'
       src/crystal/main.cr:11:3 in '_crystal_main'
       src/crystal/main.cr:112:5 in 'main_user_code'
       src/crystal/main.cr:101:7 in 'main'
       src/crystal/main.cr:135:3 in 'main'
       ???

 12) String encode doesn't raise if incomplete byte sequence

       Invalid encoding: GB2312
       src/iconv.cr:0:11 in 'initialize'
       src/iconv.cr:7:3 in 'new'
       src/string.cr:36:13 in 'encode'
       src/string.cr:1162:5 in 'encode'
       src/kernel.cr:28:5 in '~procProc(Nil)'
       src/spec/methods.cr:255:3 in 'it'
       src/kernel.cr:28:5 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       src/spec/expectations.cr:210:7 in '~procProc(Nil)'
       src/spec/context.cr:255:3 in 'describe'
       src/spec/methods.cr:16:5 in 'describe'
       spec/std/thread/condition_variable_spec.cr:1:1 in '__crystal_main'
       src/crystal/main.cr:11:3 in '_crystal_main'
       src/crystal/main.cr:112:5 in 'main_user_code'
       src/crystal/main.cr:101:7 in 'main'
       src/crystal/main.cr:135:3 in 'main'
       ???

Finished in 21.61 seconds
5555 examples, 5 failures, 7 errors, 5 pending

These failures are quite expected.

@jirutka jirutka changed the title WIP: Add support for target aarch64-linux-musl Add support for target aarch64-linux-musl Mar 25, 2018
@RX14
Copy link
Contributor

RX14 commented Mar 25, 2018

IIRC iconv is broken on x86 musl too, can you confirm? Is the time related spec due to a missing zonedb?

Please submit a patch to wrap the asm_spec.cr in {% if flag?(:i686) || flag?(:x86_64) %} and add # TODO: arm asm tests, and replace flag?(:x86_64) with flag?(:bits64) in sizeof_spec.cr.

@jirutka
Copy link
Contributor Author

jirutka commented Mar 25, 2018

IIRC iconv is broken on x86 musl too, can you confirm?

Yes, the same errors are on x86_64.

Is the time related spec due to a missing zonedb?

Yes, the error is gone after installing tzdata!

@ysbaddaden
Copy link
Contributor

Yeah, skip the GB2312 encoding issues. Musl's iconv doesn't support this encoding, and specs have always failed.

@RX14
Copy link
Contributor

RX14 commented Mar 25, 2018

@jirutka can you confirm that (with all the PRs and patches):

  • you can compile a native compiler using a cross-compiled compiler
  • you can make spec with the native compiler and only the GB2312 encoding specs fail

If so, this is 👍 to me.

@jirutka
Copy link
Contributor Author

jirutka commented Mar 25, 2018

@RX14, yes, I can confirm that. I can also confirm that both dynamically and statically linked crystal works on aarch64 machine.

Package for aarch64 crystal is already available in the testing repository (see here).

Copy link
Contributor

@RX14 RX14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fantastic, great to have another libc supported, and thank you so so much for your packaging efforts.

Copy link
Contributor

@ysbaddaden ysbaddaden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 thanks a lot for this!

@ysbaddaden ysbaddaden merged commit 9980a1f into crystal-lang:master Mar 26, 2018
@jirutka jirutka deleted the aarch64-linux-musl branch March 26, 2018 08:51
@RX14 RX14 added this to the Next milestone Mar 26, 2018
sdogruyol pushed a commit that referenced this pull request Apr 13, 2018
* Fix codegen/asm specs to run x86 ASM only on x86_64 and i686

See #5861 (comment)

* Fix codegen/sizeof specs for aarch64 (and other 64bit arches)

See #5861 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants