-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ARMv6 / ARMv7 support #3424
Conversation
I ran into some issues using this PR to cross-compile to a Raspberry Pi: Firstly, something was missing that caused code using the require "secure_random"
puts SecureRandom.urlsafe_base64(18) led to the following error: https://gist.github.com/meew0/daf2c57b0de3878ae85e8e38bc6b0282 Fortunately, this wasn't a large problem for me, as Note that this only occurred with the |
Thanks for testing! There must be some tweaks to do in the stdlib, like you found out for SecureRandom. And certainly some issues to be debugged, too (maybe the fiber context switch is a bit wrong). How did it crash? You cross compiled the compiler, then tried to build your app? I got a segfault with the compiler built in release mode, too (at the same pass), but my linux has LLVM 3.8 when the QEMU image has LLVM 3.5. We should verify with the exact LLVM version. You can try to cross compile your app directly, and verify if it passes when compiled with --release, and see how it goes. |
Sorry for being unclear there; I compiled the compiler with ARM support on x86 (in release mode), then used that to cross compile my app to ARM (in release/debug mode). Should I try cross-compiling the compiler in debug mode and then using that compiler to compile my app directly on the RasPi? |
You mean the compiler crashed on x86 while cross-compiling your app, it didn't crash an the ARM board? that's really unexpected 😕 |
After some more investigating, the Unwind issue is two fold:
|
@waj I give up the unwind stuff. This is too complicated for me to fix 😭 Maybe I'll try on Alpine Linux armhf but I think I'll have the same result. If someone is motivated enough to follow up, here is a patch for using either |
I understood why Prepending the following to if actions.end_of_stack?
return LibUnwind::ReasonCode::END_OF_STACK
end We end up with the following:
Note that |
Hey @ysbaddaden, this is really awesome!! Do you think we can still merge this anyway? In this way it doesn't get outdated and trying to make the exception-handling work is a bit easier (no need to work in a separate branch and constantly rebase against master). What do you think? |
I had omitted the LLVM Array type support on purpose, because it did not seem to be used by Crystal in any way. I generally don't feel very comfortable about having untested code branches. Missing this particular functionality is perfectly fine because in the case if Crystal ever needs it, the error message would be very clear and explicit. The rest of the ARM ABI code paths are more or less validated. Some of them even used a minimalistic standalone test program for this as mentioned in #324 (comment) Thanks a lot for taking care of the fiber context switch code. The ancient Crystal did not need it, so we could get away without this functionality. Now the remaining bits are the support for raising exceptions (the libunwind stuff) and LLVM code generation problems (enabling/disabling optimizations was a bit fragile). I haven't tested your branch yet, but maybe will try to find some time for this. |
I fully support this! And I'm glad that you changed your mind. My impressions was that you had the "all or nothing" approach, so that the ARM ABI bits could not land before we have the exceptions handling working too and before the ARM port is fully passing the Crystal test suite. |
By the way, ARM development boards are very cheap nowadays. If the Raspberry Pi is too expensive, then I can recommend one of the Orange Pi boards. The shipping cost from China is very low. Also there are board variants with 2GB of RAM (this might be important for Crystal) and on-board WIFI / eMMC for a little bit higher price. |
@ssvb I needed the array type for my posix project, and the x86_64 ABI do references it, now. I don't have a Raspberry yet, because I'm not sure which one to get. I wish there was a bunch of Zero with different processors (big or little endian, hard or soft float). |
@ysbaddaden OK, thanks. If the array type is used now, then it makes sense. As for big/little endian and hard/soft float, these are purely software things and you can try all of them on the same ARM processor. But big endian is a very unpopular choice and pretty much immature at the moment. As far as I know, NetBSD tried the big endian mode for fun (but I haven't tested it myself). Nowadays it may be a good idea to get a 64-bit ARMv8 board though. |
9a89ba9
to
69e88d0
Compare
Unwind is kinda different on ARM EHABI. The exception struct is different, the personality signature is different, the software is required to unwind the stack frame, ... Finding, then reading the GNU GCC Anyway, exceptions are now working on ARM EHABI. At least a begin/raise/rescue did work. |
Unwind is almost working. Catch all rescues do catch the raised exceptions, but trying to catch a specific exception class doesn't. For example the following snippet doesn't work. It prints "Catch-All: Bar" instead of "Gotcha: Bar": class Bar < Exception
end
def foo
raise Bar.new("oh no")
end
begin
foo
rescue ex : Bar
puts "Gotcha: #{ex.class}"
rescue ex
puts "Catch-All: #{ex.class}"
end Note that in the catch-all handler, the |
8940f30
to
8a00092
Compare
Fixed! A stupid register error (I put the type in R2 when assembly expected R1). Now, I have a segfault in |
After the updates to this PR I've revisited the cross compilation I was doing before (see my earlier comments), here are my results: The The x86 compiler crash still exists, and the backtrace I get doesn't seem to have changed. The problem doesn't affect me that much and it may be unrelated to this PR, but I'd still be happy to provide any information necessary in fixing it. I've also now tried to do the actual linking step on the Raspberry Pi itself, with the command output by the compiler on x86; and after installing and updating some missing libraries, building
(In case there is confusion, Apologies if this is something obvious I'm missing (I'm not very experienced in cross compilation) or if this problem is unrelated to the PR, but I feel like if this problem is fixable by adding a compiler flag, then that compiler flag should probably be added to the output when building on x86. |
Thanks for testing. Please do! I didn't tested on a real Pi. I only did on QEMU with the versatilepb machine, arm1176 CPU, 256M of RAM and the latest Raspbian image, which targets ARMv6 with hard float. That is a Pi A or Pi Zero (I'm waiting for thepihut to stock some zeros to test on real hardware). That being said, I never got a soft/hard float issue. I cross compile with |
Ah, it seems like using However I'm now getting a segfault at runtime:
This trace alone probably isn't of much use to you; tell me if you need any more information. I've been able to trace it down to a minimal example that simply makes an HTTP request: https://gist.github.com/meew0/c0683f76f603f8c8da135f9264245ffc (Note that this specific example produces a slightly different segfault, however I don't think that matters much.) |
@meew0 you may comment out |
This is what I got when running the minimal example I posted above, with my very limited gdb experience:
|
I found in the D runtime the answer to my unwind crash (I believe), and also notes on which registers should actually be saved (r4-r11, lr, d8-d15):
|
8a00092
to
b4f60b9
Compare
@meew0 I just fixed the fiber context switch segfault, and it fixed your reduced test case! I can now run the std spec suite without any crash. It merely hangs a bit when computing the BCrypt digests, which is unavoidable.
|
After fixing the path at which std specs expect some files to be, I now have:
Of which there are:
|
Hm, when trying to cross-compile, I now get the following error (on x86):
It happens with anything I'm trying to cross-compile, even an empty file. Is it something with the target triple again? It happens with both |
No, it's the fiber context switch. I now push/pop the FPU registers too. LLVM complained for me, too, but I added |
@zeiv you must first compile the latest master of Crystal, then you'll be able to target ARM (or wait for the 0.20 to be released). Also, do not specify an empty prelude, std and core lib are fine, and they're required to compile Crystal. You may either cross compile Crystal itself (again, latest master). Be sure to have the same LLVM version on both the host and the guest. You'll may also want to install/compile a recent LLVM version (eg: 3.8 or 3.9). Be sure to not pass an empty prelude (compilation with fail).
Or you may cross-compile your applications for ARM:
Once an object file is created, copy it to the the ARM board (or VM), but before linking it by copy/pasting the printed Eventually tweak the copied |
@ysbaddaden Thanks for the reply. That's what I was missing, I was using
|
UPDATE I was compiling on a digital ocean droplet. When I first tried to compile with
I thought it might be a memory leak or something, so I resized my droplet to their super-duper 64GB / 16 core option and tried again, but it crashed again. (I was using screen and it terminated when it died, so I don't have a trace unfortunately). |
@zeiv homebrew is missing some targets by default. AArch64 isn't enabled for 3.9 and it seems ARM wasn't in 3.6 :-( I'll try to reproduce the crash with |
That segfault looks like the one I got; it was fixed by upgrading LLVM to 3.9. |
Hi, I am super exited about running Crystal on Raspi. I am trying to do Docker playground. FROM debian
##########
# Install utils
RUN apt-get update && \
apt-get install -y \
git \
make \
dh-autoreconf \
apt-transport-https
##########
# Install LLVM
RUN echo "deb http://apt.llvm.org/trusty/ llvm-toolchain-trusty-3.9 main" >> /etc/apt/sources.list && \
echo "deb-src http://apt.llvm.org/trusty/ llvm-toolchain-trusty-3.9 main" >> /etc/apt/sources.list
RUN apt-get update && \
apt-get install -y --force-yes clang-3.9 lldb-3.9
##########
# Install other necessary libraries
RUN apt-get update && \
apt-get install -y \
libbsd-dev \
libedit-dev \
libevent-core-2.0-5 \
libevent-dev \
libevent-extra-2.0-5 \
libevent-openssl-2.0-5 \
libevent-pthreads-2.0-5 \
libgmp-dev \
libgmpxx4ldbl \
libssl-dev \
libxml2-dev \
libyaml-dev \
libreadline-dev
# Install BOEHM GC
ADD https://api.github.com/repos/ivmai/bdwgc/compare/master...HEAD /dev/null
RUN git clone https://github.com/ivmai/bdwgc.git /tmp/bdwgc
ADD https://api.github.com/repos/ivmai/libatomic_ops/compare/master...HEAD /dev/null
RUN git clone https://github.com/ivmai/libatomic_ops.git /tmp/bdwgc/libatomic_ops
RUN cd /tmp/bdwgc && autoreconf -vif
RUN cd /tmp/bdwgc && automake --add-missing
RUN cd /tmp/bdwgc && ./configure
RUN cd /tmp/bdwgc && make
RUN cd /tmp/bdwgc && make check
RUN cd /tmp/bdwgc && make install
##########
# Install crystal
RUN apt-key adv --keyserver keys.gnupg.net --recv-keys 09617FD37CC06B54 && \
echo "deb https://dist.crystal-lang.org/apt crystal main" > /etc/apt/sources.list.d/crystal.list
RUN apt-get update && \
apt-get install -y crystal
##########
# Add crystal repo
ADD https://api.github.com/repos/crystal-lang/crystal/compare/master...HEAD /dev/null
RUN git clone https://github.com/crystal-lang/crystal.git /tmp/crystal
##########
# Compile it
RUN cd /tmp/crystal && make libcrystal
RUN cd /tmp/crystal && \
bin/crystal build \
src/compiler/crystal.cr \
--cross-compile \
--target arm-unknown-linux-gnueabihf \
--release -Dwithout_openssl -Dwithout_zlib
CMD ["sleep", "3600"] Maybe it will help to somebody. When you have docker installed and running, just save this file as Unfortunately it is not 100% yet and there is few problems. Like installing Boehm GC 7.6 throws error when calling:
|
@schovi Did you install |
@schovi just use gc 7.6.0 and libatomic_ops 4.4.0 tarballs, instead of cloning the Git master. |
@schovi ran into that as well, seems the work around was to install and run 'pkg-config' once there was a note added to the README recently about it I think. GL! |
When trying to use the version of crystal I compiled on my raspi I get the following error when trying to compile a simple hello world app. |
Is the prelude.cr file there? Everything in |
I just checked crystal out from master and didn't delete any files. Are you
saying that I need all crystal-lang source files in the folder with my
crystal binary or every one of my crystal projects? How would I build a stand alone binary crystal compiler like I have on my mac and linux x86 boxes?
|
I moved my hello.cr into crystal/src which I verified did have prelude.cr. When I then ran |
You need to set |
@ysbaddaden I finally got crystal working on my raspberry pi. 👍 When I try to compile with it though I get this error https://dl.dropboxusercontent.com/s/m5mvghbb8q043be/2017-03-16%20at%2012.19%20PM.png @RX14 Advised me to change |
@elorest you copied the binary over to the pi, right? The compiler needs a bash script that sets Default target issue: how did you install or compile LLVM for the rpi? Crystal asks LLVM what the default host target is. You can check with |
@ysbaddaden I got @elorest to fix the crystal path issue by setting CRYSTAL_CONFIG_PATH. |
I installed in from a debian repo that i needed to add since the newest stable version was 3.5. It installed it as llvm-config-3.9 which I specified when I compiled on my arm machine from cyrstal.o. |
The issue may be that Crystal doesn't deal correctly with the |
Fixes the target triple used by Crystal to select the LibC bindings when the triple specifies a specific ARM architecture, e.g. `armv6`, `armv7l` or `armv8.1-a`. refs crystal-lang#3424 (comment)
Fixes the target triple used by Crystal to select the LibC bindings when the triple specifies a specific ARM architecture, e.g. `armv6`, `armv7l` or `armv8.1-a`. refs #3424 (comment)
@ysbaddaden Yes they pass when I pass |
The ABI is a rework from the work @ssvb did (port from Rust) to which I merely added support for the LLVM Array type.
The
arm-unknown-linux-gnueabihf
target is for the Raspberry Pi (Raspbian). I only tested in QEMU emulating the ARM1176 CPU, not on a real hardware.The fiber switch context took quite some time (×2), but seems to be working (now).
I could cross-compile Crystal itself (non release mode), as well as my posix project, which was capable to regenerate the C bindings (in QEMU). I didn't try to build Crystal with Crystal (QEMU is very slow) but Crystal was capable to build simple programs.
The one thing NOT working inNow it works!unwind
. Trying to unwind to generate the stacktrace segfaults, raising an exception fails (phase1 fatal error). Maybe this is because oflibgcc_s
, which is always linked in, but misses some symbols:_Unwind_GetIP
,_Unwind_GetGR
and_Unwind_SetGR
(despite LSB stating they must be available). Linking againstlibunwind
passes, but maybe it's causing conflicts? See my comment below.