New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
foundationdb: add fdb 6.1 with cmake build #61009
Merged
thoughtpolice
merged 6 commits into
NixOS:master
from
thoughtpolice:nixpkgs/fdb-61-fixes
May 15, 2019
Merged
foundationdb: add fdb 6.1 with cmake build #61009
thoughtpolice
merged 6 commits into
NixOS:master
from
thoughtpolice:nixpkgs/fdb-61-fixes
May 15, 2019
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thoughtpolice
force-pushed
the
nixpkgs/fdb-61-fixes
branch
from
May 12, 2019 19:42
1f92595
to
a061933
Compare
ofborg
bot
added
6.topic: python
10.rebuild-darwin: 1-10
and removed
10.rebuild-darwin: 0
labels
May 12, 2019
thoughtpolice
force-pushed
the
nixpkgs/fdb-61-fixes
branch
4 times, most recently
from
May 14, 2019 19:15
0d8ee53
to
f868dc9
Compare
thoughtpolice
force-pushed
the
nixpkgs/fdb-61-fixes
branch
from
May 15, 2019 11:07
f868dc9
to
add91b1
Compare
thoughtpolice
changed the title
foundationdb: add fdb 6.1 with cmake build [WIP]
foundationdb: add fdb 6.1 with cmake build
May 15, 2019
thoughtpolice
force-pushed
the
nixpkgs/fdb-61-fixes
branch
from
May 15, 2019 11:29
add91b1
to
729c1db
Compare
@GrahamcOfBorg build foundationdb61 python3Packages.foundationdb61 |
Let's go ahead and merge this. The |
FoundationDB is currently in the process of migrating to CMake, and it will eventually be the only build system. In preparation for this, split off the current (somewhat nasty) builder into its own file, and allow default.nix to be more declarative -- containing only the main supported versions. Similarly, a cmake.nix file will be added later. There is no functional change here (NFC), only an organizational change (file moves, no hash changes). Signed-off-by: Austin Seipp <aseipp@pobox.com>
Signed-off-by: Austin Seipp <aseipp@pobox.com>
This adds a new build of FoundationDB 6.1, using the new, much improved with CMake build system with fewer patches and rough edges. Signed-off-by: Austin Seipp <aseipp@pobox.com>
This allows us to specify JSON trace logging, which is useful for tooling to injest/transform logs. Signed-off-by: Austin Seipp <aseipp@pobox.com>
Slight oversight: this allows members of the FoundationDB group to read logs. Signed-off-by: Austin Seipp <aseipp@pobox.com>
Signed-off-by: Austin Seipp <aseipp@pobox.com>
thoughtpolice
force-pushed
the
nixpkgs/fdb-61-fixes
branch
from
May 15, 2019 14:54
729c1db
to
2525b88
Compare
pull bot
pushed a commit
to avitex/nixpkgs
that referenced
this pull request
May 16, 2019
This was a testing oversight that came from NixOS#61009 -- I forgot to test the new traceFormat option with older server versions while I was working on FDB 6.1. Since trace_format is only available in 6.1+, emitting it unconditionally caused older versions of the database fail to start, reporting an error. We simply gate it behind a version check instead, and assert the format is always XML on older versions. This avoids the case where the user has an old version, changes traceFormat willingly, and then is confused by why it didn't work. As reported by @timothyklim in the comments on commit c55b923. See NixOS@c55b923#r33566132 Signed-off-by: Austin Seipp <aseipp@pobox.com>
@thoughtpolice Hi! Is there any chance that 6.1.8 (official release) will be released here soon? Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation for this change
FoundationDB 6.1 is going to ship somewhat soon(-ish) and has a new CMake build system. While it would have been possible to keep wobbling on with the old one (a strange beast), CMake has a number of advantages and the overall result for the 6.1+ builds is looking much cleaner in the future with minimal patching, no old boost, or GCC 4.9, etc.
Some notes:
clang
andlibcxx
as the stdenv ifuseClang = true
. This significantly shrinks (~45%) the resulting closure size because all foundationdb binaries are statically linked to their source code (effectively duplicating large amounts of the codebase in each binary). Enabled by default, tentatively.libcxxStdenv
needs investigation, it dramatically seems to reduce memory requirements but something weird happens andlibtls
doesn't get itsRPATH
appropriately set(????)fdb_install
custom command, so unfortunately we still need a mostly custom install phase. I want to try and fix this so normalninja install
or whatever works (maybe I'm missing something) but it'll take more coordination upstream.gcc8Stdenv
for now as well, since it gives some binary size improvements. This can probably go away if we move to GCC 9.1 or something instdenv
(soon, hopefully!).out
output. These are tiny compared to the binaries so it's not a big deal, but the improvement is that we can now run self-tests against any closures we build. (My intention is to eventually automate this and use it in some of the FoundationDB NixOS tests.)test-lists.txt
.This is a draft and a WIP, but the core shape looks OK I guess. This expression is also very viable and suitable for development on FDB itself using
nix-shell '<nixpkgs>' -A foundationdb61 --pure
, etc.Still TODO
fdb-java.jar
file again (should be easy, it gets built anyway)..pythonsrc
so the Python bindings still work.libtls
weirdness going on in the build system, it should just detect it frombuildInputs
I think...Find a way to get rid oftest-lists.txt
stdenv49
since nothing else needs it, and all the other old stuff! (The on-disk formats are still compatible, too!)Things done
sandbox
innix.conf
on non-NixOS)nix-shell -p nix-review --run "nix-review wip"
./result/bin/
)