Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

haproxy: Cleaning up and enable LUA and PCRE #23901

Closed
wants to merge 3 commits into from

Conversation

fuzzy-id
Copy link
Contributor

Motivation for this change

This solves #23806.

By the way I also enabled PCRE. Quoting the haproxy README: "If your system supports PCRE (Perl Compatible Regular Expressions), then you really should build with libpcre which is between 2 and 10 times faster than other libc implementations."

Things done
  • Tested using sandboxing
    (nix.useSandbox on NixOS,
    or option build-use-sandbox in nix.conf
    on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • Linux
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

Thomas Bach added 3 commits March 14, 2017 21:39
`preConfigure` line became too long to edit comfortably. Split up
flags into ones necessary during both build and install phase and
install phase only.
Quoting the README: "If your system supports PCRE (Perl Compatible
Regular Expressions), then you really should build with libpcre which
is between 2 and 10 times faster than other libc implementations."
@mention-bot
Copy link

@fuzzy-id, thanks for your PR! By analyzing the history of the files in this pull request, we identified @rbvermaa, @pmahoney and @garbas to be potential reviewers.

@fuzzy-id fuzzy-id changed the title Cleaning up and enabpled LUA and PCRE Cleaning up and enabled LUA and PCRE Mar 14, 2017
@fuzzy-id fuzzy-id changed the title Cleaning up and enabled LUA and PCRE haproxy: Cleaning up and enabled LUA and PCRE Mar 15, 2017
@fuzzy-id fuzzy-id changed the title haproxy: Cleaning up and enabled LUA and PCRE haproxy: Cleaning up and enable LUA and PCRE Mar 15, 2017
@fuzzy-id
Copy link
Contributor Author

fuzzy-id commented Mar 15, 2017

Fails on OS X with:

make flags: SHELL=/nix/store/0z0ll7i4xxjhdl8gzpizppcyhyhpmak5-bash-4.4-p5/bin/bash PREFIX=${out} TARGET=generic  USE_LUA=yes LUA_LIB=/nix/store/d0qygi7yd6n2imf765h66ks9vd5z05sl-lua-5.3.0/lib LUA_INC=/nix/store/d0qygi7yd6n2imf765h66ks9vd5z05sl-lua-5.3.0/include USE_PCRE=yes USE_PCRE_JIT=yes USE_OPENSSL=yes USE_ZLIB=yes CC=cc USE_KQUEUE=1 
Makefile:610: *** unable to automatically detect the Lua library name, you can enforce its name with LUA_LIB_NAME=<name> (where <name> can be lua5.3, lua53, lua, ...).  Stop.

I have no clue why this is not working. The files in /nix/store/d0qygi7yd6n2imf765h66ks9vd5z05sl-lua-5.3.0/lib are liblua.5.3.0.dylib, liblua.5.3.dylib and liblua.dylib. The relevant section in the Makefile looks like this:

ifneq ($(USE_LUA),)
check_lua_lib = $(shell echo "int main(){}" | $(CC) -o /dev/null -x c - $(2) -l$(1) 2>/dev/null && echo $(1))

BUILD_OPTIONS   += $(call ignore_implicit,USE_LUA)
OPTIONS_CFLAGS  += -DUSE_LUA $(if $(LUA_INC),-I$(LUA_INC))
LUA_LD_FLAGS := -Wl,--export-dynamic $(if $(LUA_LIB),-L$(LUA_LIB))
ifeq ($(LUA_LIB_NAME),)
# Try to automatically detect the Lua library
LUA_LIB_NAME := $(firstword $(foreach lib,lua5.3 lua53 lua,$(call check_lua_lib,$(lib),$(LUA_LD_FLAGS))))
ifeq ($(LUA_LIB_NAME),)
$(error unable to automatically detect the Lua library name, you can enforce its name with LUA_LIB_NAME=<name> (where <name> can be lua5.3, lua53, lua, ...))
endif
endif

Darwin-support was added by @pmahoney. Can you look into this?

@c0bw3b
Copy link
Contributor

c0bw3b commented Mar 15, 2017

According to the Makefile there is a dedicated TARGET for Darwin which implies USE_KQUEUE.
And for the Lua lib, it seems it can't be found because it's named liblua.dylib and make is looking for liblua.so

This should do the trick but I'm not a Darwin user so I can't test it :

makeFlags = [
    "PREFIX=\${out}"
    "TARGET=${if stdenv.isSunOS then "solaris" else if stdenv.isLinux then "linux2628" else if stdenv.isDarwin then "osx" else "generic"}"
  ];
  buildFlags = [
    "USE_LUA=yes"
    "LUA_LIB=${pkgs.lua5_3}/lib"
    "LUA_INC=${pkgs.lua5_3}/include"
    "USE_PCRE=yes"
    "USE_PCRE_JIT=yes"
    "USE_OPENSSL=yes"
    "USE_ZLIB=yes"
    (stdenv.lib.optionalString stdenv.isDarwin "CC=cc LUA_LIB_NAME=liblua.dylib")
];

@LnL7 LnL7 added the 6.topic: darwin Running or building packages on Darwin label Mar 15, 2017
@LnL7
Copy link
Member

LnL7 commented Mar 16, 2017

@c0bw3b That patch still fails with this.

cc  -g -o haproxy src/haproxy.o src/base64.o src/protocol.o src/uri_auth.o src/standard.o src/buffer.o src/log.o src/task.o src/chunk.o src/channel.o src/listener.o src/lru.o src/xxhash.o src/time.o src/fd.o src/pipe.o src/regex.o src/cfgparse.o src/server.o src/checks.o src/queue.o src/frontend.o src/proxy.o src/peers.o src/arg.o src/stick_table.o src/proto_uxst.o src/connection.o src/proto_http.o src/raw_sock.o src/backend.o src/tcp_rules.o src/lb_chash.o src/lb_fwlc.o src/lb_fwrr.o src/lb_map.o src/lb_fas.o src/stream_interface.o src/stats.o src/proto_tcp.o src/applet.o src/session.o src/stream.o src/hdr_idx.o src/ev_select.o src/signal.o src/acl.o src/sample.o src/memory.o src/freq_ctr.o src/auth.o src/proto_udp.o src/compression.o src/payload.o src/hash.o src/pattern.o src/map.o src/namespace.o src/mailers.o src/dns.o src/vars.o src/filters.o src/flt_http_comp.o src/flt_trace.o src/flt_spoe.o src/cli.o src/ev_poll.o src/ev_kqueue.o src/ssl_sock.o src/shctx.o src/hlua.o src/hlua_fcn.o ebtree/ebtree.o ebtree/eb32tree.o ebtree/eb64tree.o ebtree/ebmbtree.o ebtree/ebsttree.o ebtree/ebimtree.o ebtree/ebistree.o    -lz  -lssl -lcrypto -Wl,--export-dynamic -L/nix/store/rmk5j5x5pbxmba10r3r8qgcqjmqsf89g-lua-5.3.0/lib -lliblua.dylib -lm -L/nix/store/2byhkql1kfcbrx53pyd99z0ad1idg38f-pcre-8.39/lib -lpcreposix -lpcre 
ld: unknown option: --export-dynamic
clang-3.7: error: linker command failed with exit code 1 (use -v to see invocation)

@@ -1,4 +1,4 @@
{ stdenv, pkgs, fetchurl, openssl, zlib }:
{ stdenv, pkgs, fetchurl, lua5_3, openssl, pcre, zlib }:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be nice to have useLua and usePcre options so this can be customized with an override.

@c0bw3b
Copy link
Contributor

c0bw3b commented Mar 16, 2017

Thanks for trying the build on Darwin @LnL7. I guess it means no Lua support for haproxy on Darwin :)

Options could be a way to tackle this. But PCRE (at least) should be there by default.

fuzzy-id pushed a commit to fuzzy-id/nixpkgs that referenced this pull request Mar 23, 2017
PCRE is enabled by default as it brings. LUA support does not compile on
Darwin (see NixOS#23901). Hence, it is disabled on this platform by default.
@fuzzy-id
Copy link
Contributor Author

Closing in favor of #24248.

@fuzzy-id fuzzy-id closed this Mar 23, 2017
fuzzy-id pushed a commit to fuzzy-id/nixpkgs that referenced this pull request Mar 29, 2017
Both are enabled by default. Except for LUA on Darwin where
compilation fails. (See NixOS#23901.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: darwin Running or building packages on Darwin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants