WIP: Add support for arm-linux-musleabihf #5862

jirutka · 2018-03-25T14:46:19Z

⚠️ There’s some problem, see comments…

Related to #5467

RX14 · 2018-03-25T15:14:14Z

src/compiler/crystal/semantic/flags.cr

@@ -36,9 +36,11 @@ class Crystal::Program
    set.add "x86_64" if set.any?(&.starts_with?("amd64"))
    set.add "i686" if set.any? { |flag| %w(i586 i486 i386).includes?(flag) }

+    set.add "musl" if set.includes?("musleabihf")


you should probably add "gnu" if it includes gnueabihf, just to make this symmetrical. It's not currently used but it may be in the future.

RX14 · 2018-03-25T15:23:17Z

Same as #5861: if you can cross-compile the compiler and run the spec suite sucesfully on the target machine this is good to go.

jirutka · 2018-03-25T15:25:30Z

if you can cross-compile the compiler and run the spec suite sucesfully on the target machine this is good to go.

I did not yet, it’s in progress. ;)

jirutka · 2018-03-25T16:16:18Z

It seems that I did something wrong, compiling crystal on armhf with cross-compiled crystal fails:

$ ./bin/crystal build --release --progress --threads 8 --static  --target armv6-alpine-linux-musleabihf  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Invalid memory access (signal 7) at address 0xab70227b
[0xab6cfbf0] __crystal_sigfault_handler +8865944
[0xaae5b388] __crystal_sigfault_handler +48
make: *** [Makefile:118: .build/crystal] Error 7

$ strace ./bin/crystal build --release --progress --threads 8 --static  --target armv6-alpine-linux-musleabihf  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
...
access("/home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/ecr/./lexer.cr", F_OK) = 0
open("/home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/ecr/lexer.cr", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 8
fcntl64(8, F_SETFD, FD_CLOEXEC)         = 0
fstat64(8, {st_mode=S_IFREG|0644, st_size=4016, ...}) = 0
read(8, "# :nodoc:\nclass ECR::Lexer\n  cla"..., 4096) = 4016
read(8, "", 4096)                       = 0
close(8)                                = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=746226671}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=746297972}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=749744730}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=749847751}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=796043327}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=796122868}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=803931975}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=804027436}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=805860037}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=805948298}) = 0
brk(0xabd2c000)                         = 0xabd2c000
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=838290639}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=838452660}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=113001805}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=113125246}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=115341491}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=115441232}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=119568498}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=119682340}) = 0
brk(0xabd2d000)                         = 0xabd2d000
brk(0xabd2f000)                         = 0xabd2f000
brk(0xabd30000)                         = 0xabd30000
brk(0xabd31000)                         = 0xabd31000
brk(0xabd32000)                         = 0xabd32000
--- SIGBUS {si_signo=SIGBUS, si_code=BUS_ADRALN, si_addr=0xab5ee27b} ---
fcntl64(0, F_GETFL)                     = 0x20402 (flags O_RDWR|O_APPEND|O_LARGEFILE)
fcntl64(0, F_SETFL, O_RDWR|O_APPEND|O_NONBLOCK|O_LARGEFILE) = 0
fcntl64(1, F_GETFL)                     = 0x20c02 (flags O_RDWR|O_APPEND|O_NONBLOCK|O_LARGEFILE)
fcntl64(2, F_GETFL)                     = 0x20c02 (flags O_RDWR|O_APPEND|O_NONBLOCK|O_LARGEFILE)
writev(2, [{iov_base="Invalid memory access (signal 7)"..., iov_len=55}, {iov_base=NULL, iov_len=0}], 2Invalid memory access (signal 7) at address 0xab5ee27b
) = 55
writev(2, [{iov_base="[0xab5bbbf0] __crystal_sigfault_"..., iov_len=49}, {iov_base=NULL, iov_len=0}], 2[0xab5bbbf0] __crystal_sigfault_handler +8865944
) = 49
writev(2, [{iov_base="[0xaad47388] __crystal_sigfault_"..., iov_len=44}, {iov_base=NULL, iov_len=0}], 2[0xaad47388] __crystal_sigfault_handler +48
) = 44
exit_group(7)                           = ?

codenoid

you need to fix

RX14 · 2018-03-25T17:06:23Z

@jirutka can you run in gdb and show a backtrace when it segfaults?

jirutka · 2018-03-25T19:07:45Z

(gdb) run build --release --progress --threads 8  --target armv6-alpine-linux-musleabihf  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Starting program: /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2-armv6-alpine-linux-musleabihf/bin/crystal build --release --progress --threads 8  --target armv6-alpine-linux-musleabihf  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
[2/13] Semantic (top level)
Program received signal SIGBUS, Bus error.
__crystal_personality () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/raise.cr:40
40	{% if flag?(:arm) %}

(gdb) backtrace
#0  __crystal_personality () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/raise.cr:40
#1  0xf538a3a4 in ?? () from /usr/lib/libgcc_s.so.1
#2  0xf538aec0 in _Unwind_RaiseException () from /usr/lib/libgcc_s.so.1
#3  0xaaab653c in __crystal_raise () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/raise.cr:167
#4  0xaac2f518 in raise () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/raise.cr:189
#5  0xaade2334 in raise () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/ast.cr:13
#6  raise:exception_type () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/ast.cr:12
#7  interpret_raise () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/methods.cr:169
#8  interpret_top_level_call? () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/methods.cr:32
#9  0xaadd8d1c in interpret_top_level_call () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/methods.cr:8
#10 visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/interpreter.cr:359
#11 accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#12 0xaadd9824 in visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/interpreter.cr:334
#13 accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#14 0xaaf0459c in visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/interpreter.cr:99
#15 accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#16 expand_macro () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/macros.cr:48
#17 expand_inline_macro () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/semantic_visitor.cr:386
#18 0xaaeab2c0 in expand_inline_macro () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/semantic_visitor.cr:372
#19 visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/semantic_visitor.cr:155
#20 accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#21 0xaaf01bc8 in visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/top_level_visitor.cr:619
#22 0xaaeaae04 in accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#23 0xaaea729c in top_level_semantic () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic.cr:62
#24 0xaaea4ca4 in semantic () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic.cr:22
#25 0xab2d5dec in compile () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/compiler.cr:144
#26 0xaaaee0dc in compile () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:265
#27 compile () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:263
#28 build () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:165
#29 run () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:70
#30 0xaaab5e84 in __crystal_main () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:49
#31 0xaaabbd38 in _crystal_main () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/crystal/main.cr:11
#32 main_user_code () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/crystal/main.cr:112
#33 main () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/crystal/main.cr:101
#34 main () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/crystal/main.cr:135

(I’m building crystal 0.24.2 with patch for armhf and patch for aarch64 support)

RX14 · 2018-03-25T21:41:24Z

Hmm, looks like raising/libunwind is different on musl arm. They don't seem to be different on /x86(_64)?/ musl.

To be symetrical with musl. It's not currently used but it may be in the future.

jirutka · 2018-03-25T22:09:19Z

I’ve tried to cross-compile for armhf with -fno-omit-frame-pointer (we use -fomit-frame-pointer on Alpine by default) and used it to successfully built crystal on armhf. However, I’ve also built it in non-release mode now, to avoid waiting so long, so I’m now not sure what actually helped.

jirutka · 2018-03-25T22:30:04Z

Oh crap, it worked thanks to non-release build, not -fno-omit-frame-pointer. 😿

Failed to raise an exception: FAILURE
[0x910e88] ???
[0x76898] ???
[0x77a34] ???
[0x91126c] ???
[0x9c600] ???
make: *** [Makefile:118: .build/crystal] Error 9

Hm, what --release actually do? It probably doesn’t just increase optimization level, does it?

RX14 · 2018-03-25T22:32:22Z

@jirutka yes, it's just compiler optimization. That's strange. Probably a compiler (or llvm) bug. We're still failing to raise/unwind anyway so it hasn't helped much...

ysbaddaden · 2018-03-25T23:26:52Z

Unwinding is specific on ARM EHABI architectures. I spent a lot of time to decipher it, and mostly based the implementation on how libgcc / libstdc++ handle this... But it may be different for musl.

See src/callstack/lib_unwind.cr, and there are closed ARM related issues in which I explained things as I found them.

There is also an opened issue with unwinding and static binaries on musl/alpine.

jirutka · 2018-03-25T23:32:16Z

@ysbaddaden Aha! The above error is from static binary (I’m building static crystal binaries to be used for bootstrapping normal builds), so I’ll try dynamic binary.

ysbaddaden · 2018-03-25T23:38:44Z

You can build glibc static binaries to help bootstrapping. There will be the usual warnings, but meh, the binary will be good enough for bootstrap a native build —this is how I initially added support for alpine x86.

jirutka · 2018-03-26T00:28:50Z

It seems that even dynamically linked does not work. :(

Building crystal with dynamically linked crystal built on armv6 machine:

$ ./bin/crystal build --release --progress --threads 8  --verbose --target armv6-alpine-linux-musleabihf  --stats -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Parse:                             00:00:00.000168842 (   0.25MB)
Semantic (top level):              00:00:01.578067935 (  50.77MB)
Semantic (new):                    00:00:00.010198093 (  50.77MB)
Semantic (type declarations):      00:00:00.205687239 (  58.77MB)
Semantic (abstract def check):     00:00:00.009524184 (  58.77MB)
Semantic (ivars initializers):     00:00:00.529359294 (  58.80MB)
Semantic (cvars initializers):     00:00:00.012355161 (  58.80MB)
Invalid memory access (signal 7) at address 0xab4db28b
[0xab4a8bf0] __crystal_sigfault_handler +8865944
[0xaac34388] __crystal_sigfault_handler +48
make: *** [Makefile:118: .build/crystal] Error 7

ysbaddaden · 2018-03-26T08:09:04Z

You may look for the :gnueabihf flag. When present we set the :armhf flag that will eventually enable a conservative default hardware floating point unit, otherwise LLVM defaults to a software floating point, which can crash with C bindings, because they expect fp registers but LLVM uses int registers (and vice versa).

If it still fails, you may investigate by cross building each spec and running it one by one on the ARM target —with an automated script— and find what's crashing.

Supporting ARM32 isn't an easy feat...

jirutka · 2018-03-28T15:52:46Z

I wanted to give you access to ARMv7 machine, so you can troubleshoot it, but… it seems that the same binary (built on another ARM machine) does not crash here! 😕 It’s still running, but already passed the time when it repeatedly crashed on the another ARM machine.

The machine I use is actually ARMv8, but it is capable running in 32 bit mode. The new one is VM on Scaleway, real ARMv7 machine.

jirutka · 2018-03-29T01:19:31Z

Okay, I’ve successfully built dynamically linked crystal (3) on musl ARMv7 using crystal (2) compiled on the same machine using crystal (1) cross-compiled from x86_64 musl. I also run specs with the following result:

Finished in 20:29 minutes
11444 examples, 24 failures, 5 errors, 11 pending

Failed examples:

crystal spec spec/compiler/codegen/and_spec.cr:32 # Code gen: and codegens and with primitive type other than bool with union
crystal spec spec/compiler/codegen/c_struct_spec.cr:10 # Code gen: struct codegens struct property setter
crystal spec spec/compiler/codegen/c_struct_spec.cr:14 # Code gen: struct codegens struct property setter via pointer
crystal spec spec/compiler/codegen/c_struct_spec.cr:18 # Code gen: struct codegens struct property setter via pointer
crystal spec spec/compiler/codegen/c_union_spec.cr:10 # Code gen: c union codegens union property default value 2
crystal spec spec/compiler/codegen/c_union_spec.cr:18 # Code gen: c union codegens union property setter 2
crystal spec spec/compiler/codegen/c_union_spec.cr:26 # Code gen: c union codegens union property setter 2 via pointer
crystal spec spec/compiler/codegen/class_spec.cr:48 # Code gen: class codegens method call of instance var
crystal spec spec/compiler/codegen/const_spec.cr:26 # Codegen: const finds nearest constant first
crystal spec spec/compiler/codegen/const_spec.cr:42 # Codegen: const allows constants with same name
crystal spec spec/compiler/codegen/if_spec.cr:32 # Code gen: if codegens if with union
crystal spec spec/compiler/codegen/or_spec.cr:32 # Code gen: or codegens or with primitive type other than bool with union
crystal spec spec/compiler/codegen/primitives_spec.cr:45 # Code gen: primitives codegens f32
crystal spec spec/compiler/codegen/primitives_spec.cr:49 # Code gen: primitives codegens f64
crystal spec spec/compiler/codegen/union_type_spec.cr:4 # Code gen: union type codegens union type when obj is union and no args
crystal spec spec/compiler/codegen/union_type_spec.cr:8 # Code gen: union type codegens union type when obj is union and arg is union
crystal spec spec/compiler/codegen/union_type_spec.cr:12 # Code gen: union type codegens union type when obj is not union but arg is
crystal spec spec/compiler/codegen/union_type_spec.cr:16 # Code gen: union type codegens union type when obj union but arg is not
crystal spec spec/compiler/codegen/union_type_spec.cr:20 # Code gen: union type codegens union type when no obj
crystal spec spec/compiler/codegen/union_type_spec.cr:24 # Code gen: union type codegens union type when no obj and restrictions
crystal spec spec/compiler/codegen/union_type_spec.cr:28 # Code gen: union type codegens union type as return value
crystal spec spec/compiler/codegen/union_type_spec.cr:32 # Code gen: union type codegens union type for instance var
crystal spec spec/std/crc32_spec.cr:10 # CRC32 should be able to calculate crc32 combined
crystal spec spec/std/socket_spec.cr:416 # TCPServer fails when port is in use
crystal spec spec/std/socket_spec.cr:451 # TCPSocket sends and receives messages
crystal spec spec/std/socket_spec.cr:530 # TCPSocket fails when connection is refused
crystal spec spec/std/socket_spec.cr:562 # UDPSocket reads and writes data to server
crystal spec spec/std/socket_spec.cr:582 # UDPSocket sends and receives messages over IPv4
crystal spec spec/std/socket_spec.cr:613 # UDPSocket sends and receives messages over IPv6

More details for two randomly selected tests reveals something interesting:

  1) Code gen: struct codegens struct property setter
     Failure/Error: run("#{CodeGenStructString}; bar = LibFoo::Bar.new; bar.y = 2.5_f32; bar.y").to_f32.should eq(2.5)

       Expected: 2.5
            got: 7.006492321624085e-45

     # spec/compiler/codegen/c_struct_spec.cr:11

  1) Code gen: c union codegens union property default value 2
     Failure/Error: run("#{CodeGenUnionString}; bar = Pointer(LibFoo::Bar).malloc(1_u64); bar.value.z").to_f32.should eq(0)

       Expected: 0
            got: 7.006492321624085e-45

     # spec/compiler/codegen/c_union_spec.cr:11

Does these tests pass on glibc armhf? I have an impression that this may not be related to musl bindings at all and it looks like these failing tests and SIGBUS on ARMv8 in 32bit mode may be two different problems.

And there’s log for the first failing socket test, it hangs on it:

HTTP::WebSocket
...
  close
    closes with message
    closes without message
  negotiates over HTTP correctlyUnhandled exception in spawn:
bind: Address not available (Errno)
  from src/socket/tcp_server.cr:71:15 in 'initialize'
  from src/socket/tcp_server.cr:32:3 in 'initialize:reuse_port'
  from TCPServer::new:reuse_port<String, Int32, Bool>:TCPServer
  from src/http/server.cr:154:5 in 'bind'
  from src/http/server.cr:153:3 in 'bind'
  from src/spec/expectations.cr:210:7 in '~procProc(Nil)'
  from src/fiber.cr:255:3 in 'run'
  from src/concurrent.cr:61:3 in '~proc2Proc(Fiber, (IO::FileDescriptor | Nil))'

jirutka · 2018-03-29T01:50:24Z

require "./src/compiler/crystal/**"
require "primitives"

include Crystal

code = <<-CODE
  lib LibFoo
    struct Bar
      x : Int32
      y : Float32
    end
  end

  bar = LibFoo::Bar.new
  bar.y = 2.5_f32
  bar.y
CODE

x = Program.new.run(code, debug: Crystal::Debug::None).to_f32
puts x == 2.5
puts x

$ crystal build foo.rs
$ DUMP=1 ./foo

; ModuleID = 'main_module'
source_filename = "main_module"
target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"

%"struct.LibFoo::Bar" = type { i32, float }

@ARGC_UNSAFE = internal global i32 0
@ARGV_UNSAFE = internal global i8** null

define float @__crystal_main(i32 %argc, i8** %argv) {
alloca:
  %bar = alloca %"struct.LibFoo::Bar"
  %0 = alloca %"struct.LibFoo::Bar"
  br label %entry

entry:                                            ; preds = %alloca
  store i32 %argc, i32* @ARGC_UNSAFE
  store i8** %argv, i8*** @ARGV_UNSAFE
  %1 = call %"struct.LibFoo::Bar" @"*struct.LibFoo::Bar::new:struct.LibFoo::Bar"()
  store %"struct.LibFoo::Bar" %1, %"struct.LibFoo::Bar"* %0
  %2 = load %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %0
  store %"struct.LibFoo::Bar" %2, %"struct.LibFoo::Bar"* %bar
  %3 = getelementptr inbounds %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %bar, i32 0, i32 1
  store float 2.500000e+00, float* %3
  %4 = getelementptr inbounds %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %bar, i32 0, i32 1
  %5 = load float, float* %4
  ret float %5
}

declare i32 @printf(i8*, ...)

; Function Attrs: uwtable
define internal %"struct.LibFoo::Bar" @"*struct.LibFoo::Bar::new:struct.LibFoo::Bar"() #0 {
alloca:
  %x = alloca %"struct.LibFoo::Bar"
  %0 = alloca %"struct.LibFoo::Bar"
  br label %entry

entry:                                            ; preds = %alloca
  %1 = bitcast %"struct.LibFoo::Bar"* %0 to i8*
  call void @llvm.memset.p0i8.i32(i8* %1, i8 0, i32 ptrtoint (%"struct.LibFoo::Bar"* getelementptr (%"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* null, i32 1) to i32), i32 4, i1 false)
  %2 = load %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %0
  store %"struct.LibFoo::Bar" %2, %"struct.LibFoo::Bar"* %x
  %3 = load %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %x
  ret %"struct.LibFoo::Bar" %3
}

; Function Attrs: argmemonly nounwind
declare void @llvm.memset.p0i8.i32(i8* nocapture writeonly, i8, i32, i32, i1) #1

attributes #0 = { uwtable }
attributes #1 = { argmemonly nounwind }
false
7.006492321624085e-45

ysbaddaden · 2018-03-29T07:54:38Z

Except for the Target DataLayout, the LLVM IR dump is identical on x86_64 (that was expected). It looks more and more related to the hardware, and possibly related to the FPU, maybe a software vs hardware expectations between libraries.

Is a hardware FPU enabled? Since +vfp2 is set by default for the (gnu|musl)eabihf targets, I assume it does;
Maybe tweak --mcpu and --mattr to closely match the CPU/FPU?

Can the program itself print the FP value correctly?

no => possible LLVM code generation issue (or FPU mismatch);
yes => issue in LLVM GenericValue when it tries to access the evaluated Float32 value in memory: it's not where it's expected or invalid; maybe LLVM JitCompiler used the wrong register to get the value from to store it in memory —which would explain why the value is the same across different computations.

jirutka · 2018-03-29T12:59:55Z

lib LibFoo
  struct Bar
    x : Int32
    y : Float32
  end
end

bar = LibFoo::Bar.new
bar.y = 2.5_f32
x = bar.y

puts x.to_f32 == 2.5
puts x
puts x / 2

$ crystal run bar.rs
true
2.5
1.25

jirutka · 2018-03-29T13:16:12Z

@ysbaddaden I’ve prepared you environment on baremetal ARMv7 machine to troubleshoot this issue: ysbaddaden@163.172.130.162 (login with your SSH key).

ysbaddaden · 2018-03-30T07:36:42Z

Thanks! I'll try to have a look over the weekend.

I confirm the issue: the LLVMGenericValue doesn't get the value returned by the Crystal module, but from some other place. I had this issue with Crystal expecting the value in a regular CPU registers when a library actually put it in a VFP register.

jirutka · 2018-04-02T12:06:39Z

I'll try to have a look over the weekend.

How it goes? :)

ysbaddaden · 2018-04-03T08:41:09Z

Sorry, I didn't have much computer time. I'm not sure I'll have until a while :(

Serkan-devel · 2019-02-28T07:37:17Z

Is there anything missing for this to be merged?

ysbaddaden · 2019-02-28T08:46:04Z

I read the thread again, and:

The ARMv8 board is likely a Cavium ThunderX. It's an impressive AArch64 CPU but the AArch32 mode has issues. The SIGBUS is kinda expected. You need a ARMv7 or ARMv6 board for running 32-bit ARM builds.
The hardware float (HF) issue: I tried to debug it months ago on a Scaleway ARMv7 server, I tried to reproduce the features clang passes to LLVM (+vfp3, +neon, ...), I tried to specify --hard-float to the linker, but I never succeeded to get hardware floating point enabled in the binary (or there is a mismatch) on this server. It seems to always be compiled or linked as "soft-float". See HTTP::Client.post crashes on arm when inside a Kemal route #6954

Anyway: those issues aren't related to the arm-linux-musleabihf target. They easily reproduce on arm-linux-gnueabihf too. There is nothing blocking this Pull Request from being merged, except for rebasing and checking if they're up to date.

j8r · 2019-03-25T17:02:21Z

@jirutka could you rebase the PR? I can check on my side with QEMU if this works then.

vladfaust · 2019-06-03T21:23:05Z

🏓

j8r · 2019-06-03T21:39:17Z

@jirutka might be busy on other topics.
I can create a PR from this one, rebased against master.
I can test on qemu arm, eventually real hardware, but I've no knowledge on this port. All credits go to him and @ysbaddaden.

j8r · 2020-02-01T22:09:39Z

Can be closed, future work goes to #7859.

jirutka force-pushed the arm-linux-musleabihf branch 2 times, most recently from f770931 to 794525b Compare March 25, 2018 15:03

RX14 reviewed Mar 25, 2018

View reviewed changes

codenoid suggested changes Mar 25, 2018

View reviewed changes

jirutka added 2 commits March 26, 2018 00:04

Add support for arm-linux-musleabihf

51313bd

Add compiler flag "gnu" when "gnueabihf" is present

3236720

To be symetrical with musl. It's not currently used but it may be in the future.

jirutka force-pushed the arm-linux-musleabihf branch from 01be976 to 3236720 Compare March 25, 2018 22:04

straight-shoota added kind:feature platform pr:needs-work A PR requires modifications by the author. labels Mar 22, 2019

j8r mentioned this pull request Jun 5, 2019

Add support for arm-linux-musleabihf #7859

Open

ysbaddaden mentioned this pull request Jul 25, 2019

Concurrent invocations of puts seem to interfere with each other #7978

Open

RX14 closed this Feb 7, 2020

astrojuanlu mentioned this pull request Jan 23, 2021

Request: Docker image based on Alpine (for ARM64) iv-org/invidious#361

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Add support for arm-linux-musleabihf #5862

WIP: Add support for arm-linux-musleabihf #5862

jirutka commented Mar 25, 2018 •

edited

RX14 Mar 25, 2018

RX14 commented Mar 25, 2018

jirutka commented Mar 25, 2018

jirutka commented Mar 25, 2018

codenoid left a comment

RX14 commented Mar 25, 2018

jirutka commented Mar 25, 2018 •

edited

RX14 commented Mar 25, 2018

jirutka commented Mar 25, 2018 •

edited

jirutka commented Mar 25, 2018

RX14 commented Mar 25, 2018 •

edited

ysbaddaden commented Mar 25, 2018

jirutka commented Mar 25, 2018

ysbaddaden commented Mar 25, 2018

jirutka commented Mar 26, 2018 •

edited

ysbaddaden commented Mar 26, 2018

jirutka commented Mar 28, 2018 •

edited

jirutka commented Mar 29, 2018 •

edited

jirutka commented Mar 29, 2018

ysbaddaden commented Mar 29, 2018 •

edited

jirutka commented Mar 29, 2018

jirutka commented Mar 29, 2018

ysbaddaden commented Mar 30, 2018

jirutka commented Apr 2, 2018

ysbaddaden commented Apr 3, 2018

Serkan-devel commented Feb 28, 2019

ysbaddaden commented Feb 28, 2019 •

edited

j8r commented Mar 25, 2019

vladfaust commented Jun 3, 2019

j8r commented Jun 3, 2019

j8r commented Feb 1, 2020

WIP: Add support for arm-linux-musleabihf #5862

WIP: Add support for arm-linux-musleabihf #5862

Conversation

jirutka commented Mar 25, 2018 • edited

RX14 Mar 25, 2018

Choose a reason for hiding this comment

RX14 commented Mar 25, 2018

jirutka commented Mar 25, 2018

jirutka commented Mar 25, 2018

codenoid left a comment

Choose a reason for hiding this comment

RX14 commented Mar 25, 2018

jirutka commented Mar 25, 2018 • edited

RX14 commented Mar 25, 2018

jirutka commented Mar 25, 2018 • edited

jirutka commented Mar 25, 2018

RX14 commented Mar 25, 2018 • edited

ysbaddaden commented Mar 25, 2018

jirutka commented Mar 25, 2018

ysbaddaden commented Mar 25, 2018

jirutka commented Mar 26, 2018 • edited

ysbaddaden commented Mar 26, 2018

jirutka commented Mar 28, 2018 • edited

jirutka commented Mar 29, 2018 • edited

jirutka commented Mar 29, 2018

ysbaddaden commented Mar 29, 2018 • edited

jirutka commented Mar 29, 2018

jirutka commented Mar 29, 2018

ysbaddaden commented Mar 30, 2018

jirutka commented Apr 2, 2018

ysbaddaden commented Apr 3, 2018

Serkan-devel commented Feb 28, 2019

ysbaddaden commented Feb 28, 2019 • edited

j8r commented Mar 25, 2019

vladfaust commented Jun 3, 2019

j8r commented Jun 3, 2019

j8r commented Feb 1, 2020

jirutka commented Mar 25, 2018 •

edited

jirutka commented Mar 25, 2018 •

edited

jirutka commented Mar 25, 2018 •

edited

RX14 commented Mar 25, 2018 •

edited

jirutka commented Mar 26, 2018 •

edited

jirutka commented Mar 28, 2018 •

edited

jirutka commented Mar 29, 2018 •

edited

ysbaddaden commented Mar 29, 2018 •

edited

ysbaddaden commented Feb 28, 2019 •

edited