Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ARMv6 / ARMv7 support #3424

Merged
merged 10 commits into from
Oct 28, 2016
Merged

Conversation

ysbaddaden
Copy link
Contributor

@ysbaddaden ysbaddaden commented Oct 15, 2016

The ABI is a rework from the work @ssvb did (port from Rust) to which I merely added support for the LLVM Array type.

The arm-unknown-linux-gnueabihf target is for the Raspberry Pi (Raspbian). I only tested in QEMU emulating the ARM1176 CPU, not on a real hardware.

The fiber switch context took quite some time (×2), but seems to be working (now).

I could cross-compile Crystal itself (non release mode), as well as my posix project, which was capable to regenerate the C bindings (in QEMU). I didn't try to build Crystal with Crystal (QEMU is very slow) but Crystal was capable to build simple programs.

The one thing NOT working in unwind. Trying to unwind to generate the stacktrace segfaults, raising an exception fails (phase1 fatal error). Maybe this is because of libgcc_s, which is always linked in, but misses some symbols: _Unwind_GetIP, _Unwind_GetGR and _Unwind_SetGR (despite LSB stating they must be available). Linking against libunwind passes, but maybe it's causing conflicts? See my comment below. Now it works!

@meew0
Copy link

meew0 commented Oct 16, 2016

I ran into some issues using this PR to cross-compile to a Raspberry Pi:

Firstly, something was missing that caused code using the SecureRandom module to not compile properly. The minimal example

require "secure_random"

puts SecureRandom.urlsafe_base64(18)

led to the following error: https://gist.github.com/meew0/daf2c57b0de3878ae85e8e38bc6b0282

Fortunately, this wasn't a large problem for me, as SecureRandom wasn't actually necessary in my particular code. After replacing it and building again, I ran into the following segfault (here with the compiler -s flag): https://gist.github.com/meew0/07a8a063a31837cfd7fe275d54fa8f56

Note that this only occurred with the --release flag; without it the compilation worked fine. Other than that, thanks a lot for this PR; it came just in time when I needed it!

@ysbaddaden
Copy link
Contributor Author

Thanks for testing! There must be some tweaks to do in the stdlib, like you found out for SecureRandom. And certainly some issues to be debugged, too (maybe the fiber context switch is a bit wrong).

How did it crash? You cross compiled the compiler, then tried to build your app? I got a segfault with the compiler built in release mode, too (at the same pass), but my linux has LLVM 3.8 when the QEMU image has LLVM 3.5. We should verify with the exact LLVM version.

You can try to cross compile your app directly, and verify if it passes when compiled with --release, and see how it goes.

@meew0
Copy link

meew0 commented Oct 16, 2016

Sorry for being unclear there; I compiled the compiler with ARM support on x86 (in release mode), then used that to cross compile my app to ARM (in release/debug mode). Should I try cross-compiling the compiler in debug mode and then using that compiler to compile my app directly on the RasPi?

@ysbaddaden
Copy link
Contributor Author

You mean the compiler crashed on x86 while cross-compiling your app, it didn't crash an the ARM board? that's really unexpected 😕

@ysbaddaden
Copy link
Contributor Author

ysbaddaden commented Oct 16, 2016

After some more investigating, the Unwind issue is two fold:

  • libgcc_s:
    • is capable to throw exceptions (__crystal_personality is called),
    • is missing symbols (_Unwind_GetIP, _Unwind_SetIP and _Unwind_GetGR), because they're not part of ARM EHABI;
  • libgcc_s, using _Unwind_VRS_{Get,Set}:
    • is capable to unwind the stack EDIT: actually calls __crystal_personality with UA_FORCE_UNWIND and segfaults,
    • calls __crystal_personality but segfaults in _Unwind_GetRegionStart or _Unwind_VRS_Get if stack was unwinded before and always segfaults in _Unwind_GetLanguageSpecificData anyway;
  • libunwind (deb package, v1.1):
    • has all symbols (stubs?),
    • is capable to unwind the stack,
    • is incapable to raise an exception because it can't find an exception handler (with --debug then _Unwind_RaiseException returns END_OF_STACK and __crystal_personality is never called, otherwise ``);
  • llvm/libunwind (master, against LLVM 3.5):
    • has all symbols (stubs),
    • always segfaults in uwn_proc_info.

This kinda looks like a dead end. We could use libunwind to unwind the stack, and libgcc_s to raie the exception ... but we need SetIP and GetGR in the exception handler so it won't work.

Maybe using LLVM's libunwind would help. Maybe using the latest GNU's libunwind could help, too.

EDIT: I found the solution to use libgcc_s: use _Unwind_VRS_Get and _Unwind_VRS_Set. See llvm-mirror/libunwind@2d2bf2f or ARM EHABI for details.

@ysbaddaden
Copy link
Contributor Author

ysbaddaden commented Oct 17, 2016

@waj I give up the unwind stuff. This is too complicated for me to fix 😭 Maybe I'll try on Alpine Linux armhf but I think I'll have the same result.

If someone is motivated enough to follow up, here is a patch for using either libunwind or GetGR/SetGR: https://gist.github.com/ysbaddaden/67a2a4ab9c076aa96aa886f5f3d9a06b

@ysbaddaden
Copy link
Contributor Author

ysbaddaden commented Oct 17, 2016

I understood why __crystal_personality caused segfaults (using libgcc_s). It's being called with UNW_UA_FORCE_UNWIND (_Unwind_Backtrace) or UNW_UA_END_OF_STACK (_Unwind_RaiseException), but calling _Unwind_GetLanguageSpecificData in those cases segfaults because context is invalid.

Prepending the following to __crystal_personality fixes the segfaults:

if actions.end_of_stack?
  return LibUnwind::ReasonCode::END_OF_STACK
end

We end up with the following:

CallStack.unwind()
<collects the backtrace>
__crystal_personality(8, (FORCE_UNWIND, END_OF_STACK), 612757598630632, Pointer(LibUnwind::Exception).null, Pointer(Void).null)

__crystal_raise(Pointer(LibUnwind::Exception)@0x96f30)
__crystal_personality(0, (END_OF_STACK), 612757598630680, Pointer(LibUnwind::Exception)@0xffffffff, Pointer(Void)@0x96f30)
Failed to raise an exception: FAILURE
[244224] *CallStack::print_backtrace:Int32 +-1090522800
[143892] __crystal_raise +-1090522800
[151176] *raise<Exception>:NoReturn +-1090522800
[150888] *raise<String>:NoReturn +-1090522800
[238912] *failure:NoReturn +-1090522800
__crystal_personality(8, (FORCE_UNWIND, END_OF_STACK), 612757598630632, Pointer(LibUnwind::Exception).null, Pointer(Void).null)
[142324] ???
[Inferior 1 (process 5620) exited with code 011]

Note that unwind_ex (at 0x963f0) eventually ends up being the 5th parameter to __crystal_personality when it's supposed to be the 4th ... i.e. there is something fishy.

@asterite
Copy link
Member

Hey @ysbaddaden, this is really awesome!!

Do you think we can still merge this anyway? In this way it doesn't get outdated and trying to make the exception-handling work is a bit easier (no need to work in a separate branch and constantly rebase against master). What do you think?

@ssvb
Copy link

ssvb commented Oct 19, 2016

@ysbaddaden

The ABI is a rework from the work @ssvb did (port from Rust) to which I merely added support for the LLVM Array type.

I had omitted the LLVM Array type support on purpose, because it did not seem to be used by Crystal in any way. I generally don't feel very comfortable about having untested code branches. Missing this particular functionality is perfectly fine because in the case if Crystal ever needs it, the error message would be very clear and explicit. The rest of the ARM ABI code paths are more or less validated. Some of them even used a minimalistic standalone test program for this as mentioned in #324 (comment)

Thanks a lot for taking care of the fiber context switch code. The ancient Crystal did not need it, so we could get away without this functionality.

Now the remaining bits are the support for raising exceptions (the libunwind stuff) and LLVM code generation problems (enabling/disabling optimizations was a bit fragile). I haven't tested your branch yet, but maybe will try to find some time for this.

@ssvb
Copy link

ssvb commented Oct 19, 2016

@asterite

Do you think we can still merge this anyway? In this way it doesn't get outdated and trying to make the exception-handling work is a bit easier (no need to work in a separate branch and constantly rebase against master). What do you think?

I fully support this! And I'm glad that you changed your mind. My impressions was that you had the "all or nothing" approach, so that the ARM ABI bits could not land before we have the exceptions handling working too and before the ARM port is fully passing the Crystal test suite.

@ssvb
Copy link

ssvb commented Oct 19, 2016

By the way, ARM development boards are very cheap nowadays. If the Raspberry Pi is too expensive, then I can recommend one of the Orange Pi boards. The shipping cost from China is very low. Also there are board variants with 2GB of RAM (this might be important for Crystal) and on-board WIFI / eMMC for a little bit higher price.

@ysbaddaden
Copy link
Contributor Author

@ssvb I needed the array type for my posix project, and the x86_64 ABI do references it, now.

I don't have a Raspberry yet, because I'm not sure which one to get. I wish there was a bunch of Zero with different processors (big or little endian, hard or soft float).

@ssvb
Copy link

ssvb commented Oct 19, 2016

@ysbaddaden OK, thanks. If the array type is used now, then it makes sense.

As for big/little endian and hard/soft float, these are purely software things and you can try all of them on the same ARM processor. But big endian is a very unpopular choice and pretty much immature at the moment. As far as I know, NetBSD tried the big endian mode for fun (but I haven't tested it myself). Nowadays it may be a good idea to get a 64-bit ARMv8 board though.

@ysbaddaden ysbaddaden force-pushed the arm-support branch 2 times, most recently from 9a89ba9 to 69e88d0 Compare October 21, 2016 13:11
@ysbaddaden
Copy link
Contributor Author

Unwind is kinda different on ARM EHABI. The exception struct is different, the personality signature is different, the software is required to unwind the stack frame, ... Finding, then reading the GNU GCC __gxx_personality_v0 and Unwind implementation really helped to understand what the hell was happening.

Anyway, exceptions are now working on ARM EHABI. At least a begin/raise/rescue did work.

@ysbaddaden
Copy link
Contributor Author

ysbaddaden commented Oct 22, 2016

Unwind is almost working. Catch all rescues do catch the raised exceptions, but trying to catch a specific exception class doesn't. For example the following snippet doesn't work. It prints "Catch-All: Bar" instead of "Gotcha: Bar":

class Bar < Exception
end

def foo
  raise Bar.new("oh no")
end

begin
  foo
rescue ex : Bar
  puts "Gotcha: #{ex.class}"
rescue ex
  puts "Catch-All: #{ex.class}"
end

Note that in the catch-all handler, the object_id and crystal_type_id are correct and ex.is_a?(Bar) returns true 😭

@ysbaddaden
Copy link
Contributor Author

Fixed! A stupid register error (I put the type in R2 when assembly expected R1).

Now, I have a segfault in spec/std/channel_spec.cr, more specifically in can be closed from different fiber. An exception is raised, but _Unwind_Backtrace segfaults when trying to collect the callstack. It's most likely caused by an issue in the fiber context switch.

@meew0
Copy link

meew0 commented Oct 23, 2016

After the updates to this PR I've revisited the cross compilation I was doing before (see my earlier comments), here are my results:

The SecureRandom thing now works perfectly. Great, thanks!

The x86 compiler crash still exists, and the backtrace I get doesn't seem to have changed. The problem doesn't affect me that much and it may be unrelated to this PR, but I'd still be happy to provide any information necessary in fixing it.

I've also now tried to do the actual linking step on the Raspberry Pi itself, with the command output by the compiler on x86; and after installing and updating some missing libraries, building libcrystal.a and editing the command to point to it, I get the following output:

/usr/bin/ld: error: elgyem uses VFP register arguments, elgyem.o does not
/usr/bin/ld: failed to merge target specific data of file elgyem.o
collect2: error: ld returned 1 exit status

(In case there is confusion, elgyem is the name of the project.) From some Google searches, this should be fixable by adding -mfloat-abi=softfp (or soft or hard, I've tried all three) to the output, but nothing I've tried worked.

Apologies if this is something obvious I'm missing (I'm not very experienced in cross compilation) or if this problem is unrelated to the PR, but I feel like if this problem is fixable by adding a compiler flag, then that compiler flag should probably be added to the output when building on x86.

@ysbaddaden
Copy link
Contributor Author

Thanks for testing. Please do!

I didn't tested on a real Pi. I only did on QEMU with the versatilepb machine, arm1176 CPU, 256M of RAM and the latest Raspbian image, which targets ARMv6 with hard float. That is a Pi A or Pi Zero (I'm waiting for thepihut to stock some zeros to test on real hardware).

That being said, I never got a soft/hard float issue. I cross compile with --cross-compile --target=arm-unknown-linux-gnueabihf to get an object file, then link it in Raspbian/QEMU. Maybe toying with the target triple, so LLVM knows what features to use could help?

@meew0
Copy link

meew0 commented Oct 23, 2016

Ah, it seems like using arm-unknown-linux-gnueabihf works; I've only been using arm-linux-gnueabihf previously. Now it compiles and links successfully, thanks!

However I'm now getting a segfault at runtime:

Invalid memory access (signal 11) at address 0x1c
[1519956] ???
[475236] __crystal_sigfault_handler +64

This trace alone probably isn't of much use to you; tell me if you need any more information. I've been able to trace it down to a minimal example that simply makes an HTTP request: https://gist.github.com/meew0/c0683f76f603f8c8da135f9264245ffc

(Note that this specific example produces a slightly different segfault, however I don't think that matters much.)

@ysbaddaden
Copy link
Contributor Author

@meew0 you may comment out LibExt.setup_sigfault_handler in src/signal.cr and compile with --debug then use gdb to get a backtrace. Yet, I have a feeling the fiber context switch ASM is the culprit.

@meew0
Copy link

meew0 commented Oct 23, 2016

This is what I got when running the minimal example I posted above, with my very limited gdb experience:

(gdb) run
Starting program: /home/pi/minimal2 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) backtrace
#0  0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

@ysbaddaden
Copy link
Contributor Author

ysbaddaden commented Oct 23, 2016

I found in the D runtime the answer to my unwind crash (I believe), and also notes on which registers should actually be saved (r4-r11, lr, d8-d15):
https://github.com/dlang/druntime/blob/master/src/core/threadasm.S#L438

I'm working on that, and need to get my head around registers, stacks, etc. Done!

@ysbaddaden
Copy link
Contributor Author

ysbaddaden commented Oct 24, 2016

@meew0 I just fixed the fiber context switch segfault, and it fixed your reduced test case!

I can now run the std spec suite without any crash. It merely hangs a bit when computing the BCrypt digests, which is unavoidable.

./std_spec
Finished in 1:07 minutes
3451 examples, 187 failures, 69 errors, 1 pending

@ysbaddaden
Copy link
Contributor Author

ysbaddaden commented Oct 24, 2016

After fixing the path at which std specs expect some files to be, I now have:

4466 examples, 194 failures, 5 errors, 3 pending

Of which there are:

  • 1 unwind error (failed to raise an XML::Error exception) —no clue, the callstack is popoulated corectly, but when raising, the personality routine is never called;
  • 8 socket errors because my QEMU machine lacks IPv6 support —we'll may want specs to check for IPv6 support;
  • (almost) everything else are Float errors: the generated code is failing on every FPU calculations —no clue, yet.

@meew0
Copy link

meew0 commented Oct 24, 2016

Hm, when trying to cross-compile, I now get the following error (on x86):

Codegen (crystal):                 00:00:00.0136101 (  35.99MB)
<inline asm>:4:9: error: instruction requires: VFP2
        vstmdb sp!, {d8-d15}
        ^
<inline asm>:7:9: error: instruction requires: VFP2
        vldmia sp!, {d8-d15}
        ^
LLVM ERROR: Error parsing inline asm

It happens with anything I'm trying to cross-compile, even an empty file. Is it something with the target triple again? It happens with both arm-unknown-linux-gnueabihf and arm-linux-gnueabihf.

@ysbaddaden
Copy link
Contributor Author

No, it's the fiber context switch. I now push/pop the FPU registers too. LLVM complained for me, too, but I added .fpu vfp to the ASM (see) and it stopped complaining. I compiled Crystal against LLVM 3.9 (with ARM codegen fixes) by-the-way. Or maybe we must specify another version, like .fpu vfpv2?

@ysbaddaden
Copy link
Contributor Author

@zeiv you must first compile the latest master of Crystal, then you'll be able to target ARM (or wait for the 0.20 to be released). Also, do not specify an empty prelude, std and core lib are fine, and they're required to compile Crystal.

You may either cross compile Crystal itself (again, latest master). Be sure to have the same LLVM version on both the host and the guest. You'll may also want to install/compile a recent LLVM version (eg: 3.8 or 3.9). Be sure to not pass an empty prelude (compilation with fail).

bin/crystal build src/compiler/crystal.cr \
  --cross-compile --target arm-unknown-linux-gnueabihf \
  --release -Dwithout_openssl -Dwithout_zlib

Or you may cross-compile your applications for ARM:

bin/crystal build myapp.cr \
  --cross-compile --target arm-unknown-linux-gnueabihf

Once an object file is created, copy it to the the ARM board (or VM), but before linking it by copy/pasting the printed cc command, you'll need a bunch of libraries to be installed on the ARM board. Please refer to Required Libraries for an exhausted list. You'll have to compile Boehm GC 7.6; clone crystal and run make deps to compile the src/llvm/ext/llvm_ext.o and src/ext/libcrystal.a objects.

Eventually tweak the copied cc command to refer correctly to llvm_ext.o and libcrystal.a, and you should have a working Crystal compiler or application running on ARM.

@zeiv
Copy link

zeiv commented Nov 2, 2016

@ysbaddaden Thanks for the reply. That's what I was missing, I was using bin/crystal from master without actually compiling the entire source. I'm trying to compile master on my mac and am now running into the following error:

$ make LLVM_CONFIG=/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 
Using /usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 [version=3.6.2]
c++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc `/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 --cxxflags`
cc -fPIC    -c -o src/ext/sigfault.o src/ext/sigfault.c
ar -rcs src/ext/libcrystal.a src/ext/sigfault.o
CRYSTAL_CONFIG_PATH=`pwd`/src ./bin/crystal build  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Undefined symbols for architecture x86_64:
  "_LLVMInitializeARMAsmParser", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMAsmPrinter", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTarget", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTargetInfo", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTargetMC", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: execution of command failed with code: 1: `cc -o "/Users/xavier/OneDrive/Documents/Dev/crystal/crystal/.build/crystal" "${@}"  -rdynamic  /Users/xavier/OneDrive/Documents/Dev/crystal/crystal/src/llvm/ext/llvm_ext.o `/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lgc -lpthread /Users/xavier/OneDrive/Documents/Dev/crystal/crystal/src/ext/libcrystal.a -levent -liconv -ldl -L/usr/lib -L/usr/local/lib`
make: *** [.build/crystal] Error 

@zeiv
Copy link

zeiv commented Nov 2, 2016

UPDATE
I was able to successfully cross-compile the compiler WITHOUT the --release option using Ubuntu.

I was compiling on a digital ocean droplet. When I first tried to compile with --release, this happened...

$ bin/crystal build src/compiler/crystal.cr --cross-compile --target arm-unknown-linux-gnueabihf --release -Dwithout_openssl -Dwithout_zlib
Using compiled compiler at .build/crystal

Invalid memory access (signal 11) at address 0x18
[9183605] *CallStack::print_backtrace:Int32 +117
[9062216] __crystal_sigfault_handler +56
[35486375] sigfault_handler +40
[140131403375584] ???
[26166049] _ZN4llvm12SelectionDAG7getNodeEjNS_5SDLocENS_3EVTENS_7SDValueE +65
[26947182] ???
[26949563] ???
[26603809] ???
[26604895] _ZN4llvm12SelectionDAG13LegalizeTypesEv +943
[26456599] _ZN4llvm16SelectionDAGISel17CodeGenAndEmitDAGEv +215
[26470802] _ZN4llvm16SelectionDAGISel20SelectAllBasicBlocksERKNS_8FunctionE +1058
[26476045] _ZN4llvm16SelectionDAGISel20runOnMachineFunctionERNS_15MachineFunctionE +669
[23793557] ???
[34592087] _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE +487
[34592747] _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE +43
[34591156] _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE +772
[33208958] ???
[33210344] LLVMTargetMachineEmitToFile +456
[21001963] *LLVM::TargetMachine#emit_to_file<LLVM::Module, String, LLVM::CodeGenFileType>:Bool +91
[21001851] *LLVM::TargetMachine#emit_obj_to_file<LLVM::Module, String>:Bool +11
[17811350] *Crystal::Compiler#cross_compile<Crystal::Program, Array(Crystal::Compiler::CompilationUnit), String, String>:Nil +326
[17810546] *Crystal::Compiler#codegen<Crystal::Program, Crystal::ASTNode+, Array(Crystal::Compiler::Source), String>:(Array(String) | Nil) +1746
[17800966] *Crystal::Compiler#compile<Array(Crystal::Compiler::Source), String>:Crystal::Compiler::Result +150
[17825657] *Crystal::Command::CompilerConfig#compile<String>:Crystal::Compiler::Result +57
[17825570] *Crystal::Command::CompilerConfig#compile:Crystal::Compiler::Result +34
[10002466] *Crystal::Command#build:Crystal::Compiler::Result +290
[9999371] *Crystal::Command#run:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +491
[9998603] *Crystal::Command::run<Array(String)>:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +27
[9998537] *Crystal::Command::run:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +25
[8987984] ???
[9061913] main +41
[140131383912496] __libc_start_main +240
[8984489] _start +41
[0] ???

I thought it might be a memory leak or something, so I resized my droplet to their super-duper 64GB / 16 core option and tried again, but it crashed again. (I was using screen and it terminated when it died, so I don't have a trace unfortunately).

@ysbaddaden
Copy link
Contributor Author

@zeiv homebrew is missing some targets by default. AArch64 isn't enabled for 3.9 and it seems ARM wasn't in 3.6 :-(

I'll try to reproduce the crash with --release. It's a crash in LLVM.

@meew0
Copy link

meew0 commented Nov 2, 2016

That segfault looks like the one I got; it was fixed by upgrading LLVM to 3.9.

@schovi
Copy link
Contributor

schovi commented Nov 3, 2016

Hi, I am super exited about running Crystal on Raspi. I am trying to do Docker playground.

FROM debian

##########
# Install utils
RUN apt-get update && \
    apt-get install -y \
      git \
      make \
      dh-autoreconf \
      apt-transport-https

##########
# Install LLVM
RUN echo "deb http://apt.llvm.org/trusty/ llvm-toolchain-trusty-3.9 main" >> /etc/apt/sources.list && \
    echo "deb-src http://apt.llvm.org/trusty/ llvm-toolchain-trusty-3.9 main" >> /etc/apt/sources.list

RUN apt-get update && \
    apt-get install -y --force-yes clang-3.9 lldb-3.9

##########
# Install other necessary libraries
RUN apt-get update && \
    apt-get install -y \
      libbsd-dev \
      libedit-dev \
      libevent-core-2.0-5 \
      libevent-dev \
      libevent-extra-2.0-5 \
      libevent-openssl-2.0-5 \
      libevent-pthreads-2.0-5 \
      libgmp-dev \
      libgmpxx4ldbl \
      libssl-dev \
      libxml2-dev \
      libyaml-dev \
      libreadline-dev

# Install BOEHM GC
ADD https://api.github.com/repos/ivmai/bdwgc/compare/master...HEAD /dev/null
RUN git clone https://github.com/ivmai/bdwgc.git /tmp/bdwgc

ADD https://api.github.com/repos/ivmai/libatomic_ops/compare/master...HEAD /dev/null
RUN git clone https://github.com/ivmai/libatomic_ops.git /tmp/bdwgc/libatomic_ops

RUN cd /tmp/bdwgc && autoreconf -vif
RUN cd /tmp/bdwgc && automake --add-missing
RUN cd /tmp/bdwgc && ./configure
RUN cd /tmp/bdwgc && make
RUN cd /tmp/bdwgc && make check
RUN cd /tmp/bdwgc && make install

##########
# Install crystal
RUN apt-key adv --keyserver keys.gnupg.net --recv-keys 09617FD37CC06B54 && \
    echo "deb https://dist.crystal-lang.org/apt crystal main" > /etc/apt/sources.list.d/crystal.list

RUN apt-get update && \
    apt-get install -y crystal

##########
# Add crystal repo
ADD https://api.github.com/repos/crystal-lang/crystal/compare/master...HEAD /dev/null
RUN git clone https://github.com/crystal-lang/crystal.git /tmp/crystal

##########
# Compile it
RUN cd /tmp/crystal && make libcrystal

RUN cd /tmp/crystal && \
    bin/crystal build \
      src/compiler/crystal.cr \
      --cross-compile \
      --target arm-unknown-linux-gnueabihf \
      --release -Dwithout_openssl -Dwithout_zlib

CMD ["sleep", "3600"]

Maybe it will help to somebody. When you have docker installed and running, just save this file as Dockerfile anywhere and then in same directory run docker build . -t crystal

Unfortunately it is not 100% yet and there is few problems. Like installing Boehm GC 7.6 throws error when calling: ./configure

...
checking sys/dg_sys_info.h presence... no
checking for sys/dg_sys_info.h... no
./configure: line 16943: syntax error near unexpected token `ATOMIC_OPS,'
./configure: line 16943: `   PKG_CHECK_MODULES(ATOMIC_OPS, atomic_ops, ,'
The command '/bin/sh -c cd /tmp/bdwgc && ./configure' returned a non-zero code: 2

@Sija
Copy link
Contributor

Sija commented Nov 3, 2016

@schovi Did you install autotools (as per ivmai/bdwgc#69 (comment))?

@ysbaddaden
Copy link
Contributor Author

@schovi just use gc 7.6.0 and libatomic_ops 4.4.0 tarballs, instead of cloning the Git master.

@rdp
Copy link
Contributor

rdp commented Nov 29, 2016

@schovi ran into that as well, seems the work around was to install and run 'pkg-config' once there was a note added to the README recently about it I think. GL!

@elorest
Copy link
Contributor

elorest commented Mar 16, 2017

When trying to use the version of crystal I compiled on my raspi I get the following error when trying to compile a simple hello world app. Error in hello.cr:1: while requiring "prelude": can't find file 'prelude' relative to '/home/pi/crystal' I've been using crystal for a couple years on OSX and Linux (x86) and I've never seen this before. What did I do wrong?

@lbguilherme
Copy link
Contributor

Is the prelude.cr file there? Everything in src/ must be available to the compiler at runtime.

@elorest
Copy link
Contributor

elorest commented Mar 16, 2017 via email

@elorest
Copy link
Contributor

elorest commented Mar 16, 2017

I moved my hello.cr into crystal/src which I verified did have prelude.cr. When I then ran pi@raspberrypi:~/crystal/src $ ../bin/crystal build hello.cr
but still got this error:
Error in hello.cr:1: while requiring "prelude": can't find file 'prelude' relative to '/home/pi/crystal/src'

@RX14
Copy link
Member

RX14 commented Mar 16, 2017

You need to set CRYSTAL_PATH=lib:/path/to/crystal/source/src. I've updated my stackoverflow answer here.

@elorest
Copy link
Contributor

elorest commented Mar 16, 2017

@ysbaddaden I finally got crystal working on my raspberry pi. 👍 When I try to compile with it though I get this error https://dl.dropboxusercontent.com/s/m5mvghbb8q043be/2017-03-16%20at%2012.19%20PM.png

@RX14 Advised me to change crystal build hello.cr to crystal build --target arm-unknown-linux-gnueabihf hello.cr which worked. He also said that it was a bug and I should report it here. Thanks.

@ysbaddaden
Copy link
Contributor Author

@elorest you copied the binary over to the pi, right? The compiler needs a bash script that sets CRYSTAL_PATH and eventually delegates to the binary —see any Linux or macOS distribution.

Default target issue: how did you install or compile LLVM for the rpi? Crystal asks LLVM what the default host target is. You can check with llvm-config --host-target; it should be arm-unknown-linux-gnueabihf but I guess it's not?

@RX14
Copy link
Member

RX14 commented Mar 17, 2017

@ysbaddaden I got @elorest to fix the crystal path issue by setting CRYSTAL_CONFIG_PATH.

@elorest
Copy link
Contributor

elorest commented Mar 18, 2017

I installed in from a debian repo that i needed to add since the newest stable version was 3.5. It installed it as llvm-config-3.9 which I specified when I compiled on my arm machine from cyrstal.o.
It returns armv7l-unknown-linux-gnueabihf from --host-target.

@ysbaddaden
Copy link
Contributor Author

The issue may be that Crystal doesn't deal correctly with the armv7l part, and searches for a lib_c/armv7l-linux-gnueabihf which I'm not really sure how to fix. Maybe force it to arm-....

@ysbaddaden
Copy link
Contributor Author

@elorest Does the fix in #4167 fixes the target issue? are Crystal specs passing on your RPI?

ysbaddaden added a commit to ysbaddaden/crystal that referenced this pull request Mar 23, 2017
Fixes the target triple used by Crystal to select the LibC bindings
when the triple specifies a specific ARM architecture, e.g. `armv6`,
`armv7l` or `armv8.1-a`.

refs crystal-lang#3424 (comment)
ysbaddaden added a commit that referenced this pull request Mar 23, 2017
Fixes the target triple used by Crystal to select the LibC bindings
when the triple specifies a specific ARM architecture, e.g. `armv6`,
`armv7l` or `armv8.1-a`.

refs #3424 (comment)
@elorest
Copy link
Contributor

elorest commented Mar 23, 2017

@ysbaddaden Yes they pass when I pass arm-unknown-linux-gnueabih as the target. I've since messed something up on the raspberry pi system I was using so can't try that latest fix yet. I'll let you know when I check that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet