Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mariadb-client: create symlinks for libmariadb #70879

Closed
wants to merge 1 commit into from
Closed

mariadb-client: create symlinks for libmariadb #70879

wants to merge 1 commit into from

Conversation

callahad
Copy link
Member

@callahad callahad commented Oct 9, 2019

Fixes #67144

⚠️ This won't build on macOS until #70841 gets merged. With that patch included, it passes nix-review:

nix-review output on darwin 
$ nix build --no-link --keep-going --max-jobs 8 --option build-use-sandbox true -f /Users/dan/.cache/nix-review/rev-4073dddb87c600b52e967b0f695a65aa2d445ded/build.nix
warning: ignoring the user-specified setting 'sandbox', because it is a restricted setting and you are not a trusted user
[7 built, 12 copied (627.1 MiB), 197.7 MiB DL]
8 package were build:
diesel-cli lua51Packages.luadbi-mysql lua52Packages.luadbi-mysql lua53Packages.luadbi-mysql luajitPackages.luadbi-mysql mariadb mysql-client shmig
Motivation for this change

I can't build things that link to mariadb's client libraries.

I ran into this issue trying to install Python's mysqlclient module. Other folks ran into issues installing OCaml bindings to mysql (#67144).

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @thoughtpolice @Izorkin @matthewbauer

@callahad
Copy link
Member Author

callahad commented Oct 9, 2019

This shell.nix shows the problem that this patch addresses:

with import <nixpkgs> {};
with python27Packages;

stdenv.mkDerivation {
  name = "testPythonEnv";
  src = null;
  buildInputs = [
    python27Full
    python27Packages.virtualenv
    python27Packages.pip

    mariadb-client.dev
  ];

  shellHook = ''
    SOURCE_DATE_EPOCH=$(date +%s)  # Required for building Python wheels
    cd $(mktemp -d)
    virtualenv venv
    export PATH=$PWD/venv/bin:$PATH
    pip install --no-cache-dir mysqlclient
  '';
}

Without this patch, pip install fails to build mysqlclient:

clang -bundle -undefined dynamic_lookup build/temp.macosx-10.12-x86_64-2.7/MySQLdb/_mysql.o -L/nix/store/x5k6mxv8sa5vwj3qk2anflszirdan7aa-mariadb-client-10.3.18-dev/lib/ -L/nix/store/dk69d9l6qnvzajl7fv9drsx6ihfkl8qr-python-2.7.16/lib -lmariadb -o build/lib.macosx-10.12-x86_64-2.7/MySQLdb/_mysql.so
ld: library not found for -lmariadb
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'clang' failed with exit status 1

With the patch, everything is great:

Collecting mysqlclient
  Downloading https://files.pythonhosted.org/packages/4d/38/c5f8bac9c50f3042c8f05615f84206f77f03db79781db841898fde1bb284/mysqlclient-1.4.4.tar.gz (86kB)
     |████████████████████████████████| 92kB 1.9MB/s 
Building wheels for collected packages: mysqlclient
  Building wheel for mysqlclient (setup.py) ... done
  Created wheel for mysqlclient: filename=mysqlclient-1.4.4-cp27-cp27mu-macosx_10_12_x86_64.whl size=54308 sha256=00b07238957214422abd7fe79505426261197a0b3e7fa9b6b1b0d60623f16d52
  Stored in directory: /private/var/folders/lv/0v7w3t1j1t70tlmtkvdnn2j00000gn/T/pip-ephem-wheel-cache-NB2NIA/wheels/a0/04/57/031b9b01df38999df7dc7f4ee998a98ecdbd2d781f73e3ffbf
Successfully built mysqlclient
Installing collected packages: mysqlclient
Successfully installed mysqlclient-1.4.4

@Izorkin
Copy link
Contributor

Izorkin commented Oct 9, 2019

@callahad not cheked with libmysql?

New python executable in /run/user/1737/tmp.1CMUCsROyX/venv/bin/python2.7
Also creating executable in /run/user/1737/tmp.1CMUCsROyX/venv/bin/python
Installing setuptools, pip, wheel...
done.
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting mysqlclient
  Downloading https://files.pythonhosted.org/packages/4d/38/c5f8bac9c50f3042c8f05615f84206f77f03db79781db841898fde1bb284/mysqlclient-1.4.4.tar.gz (86kB)
     |████████████████████████████████| 92kB 6.1MB/s
Building wheels for collected packages: mysqlclient
  Building wheel for mysqlclient (setup.py) ... done
  Stored in directory: /run/user/1737/pip-ephem-wheel-cache-p1gBtB/wheels/a0/04/57/031b9b01df38999df7dc7f4ee998a98ecdbd2d781f73e3ffbf
Successfully built mysqlclient
Installing collected packages: mysqlclient
Successfully installed mysqlclient-1.4.4
WARNING: You are using pip version 19.1.1, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

nix-info -m

 - system: `"x86_64-linux"`
 - host os: `Linux 5.3.4-hardened, NixOS, 20.03.git.b7a41e0 (Markhor)`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.3`
 - channels(user): `"nixpkgs-20.03pre196329.e904777dc9b"`
 - channels(root): `"nixos-19.09pre191485.3104cb28673"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

@callahad
Copy link
Member Author

callahad commented Oct 9, 2019 •
edited

@callahad not cheked with libmysql?

I don't really know my way around nixpkgs or mariadb, so this was a bit of a stab in the dark. :)

I thought libmysql was gone since c13c903, but trying this with mariadb-connector-c does seem to work, even without this patch.

...so maybe I'm holding it wrong when trying to depend on mariadb-client.dev?

@callahad
Copy link
Member Author

callahad commented Oct 9, 2019

In case it helps, it looks like 3a6e362 is when we started rm'ing libmariadb.so but only re-symlinking it as libmysqlclient.so and libmysqlclient_r.so.

@Izorkin
Copy link
Contributor

Izorkin commented Oct 9, 2019

#65222 - here changed to use a separate package - libmysql (mariadb-connector-c)

@Izorkin
Copy link
Contributor

Izorkin commented Oct 9, 2019

Probably to me need to double-check the build configuration

@Izorkin
Copy link
Contributor

Izorkin commented Oct 10, 2019

@callahad please check this patch

diff --git a/pkgs/servers/sql/mariadb/default.nix b/pkgs/servers/sql/mariadb/default.nix
index ebd9938f12f..fe0735a5452 100644
--- a/pkgs/servers/sql/mariadb/default.nix
+++ b/pkgs/servers/sql/mariadb/default.nix
@@ -140,10 +140,6 @@ client = stdenv.mkDerivation (common // {
   postInstall = common.postInstall + ''
     rm -r "$out"/share/doc
     rm "$out"/bin/{mysqltest,mytop,wsrep_sst_rsync_wan}
-    libmysqlclient_path=$(readlink -f $out/lib/libmysqlclient${libExt})
-    rm "$out"/lib/{libmariadb${libExt},libmysqlclient${libExt},libmysqlclient_r${libExt}}
-    mv "$libmysqlclient_path" "$out"/lib/libmysqlclient${libExt}
-    ln -sv libmysqlclient${libExt} "$out"/lib/libmysqlclient_r${libExt}
   '';
 });

@callahad
Copy link
Member Author

That patch appears to work and passes nix-review.

After building with that patch (and #70841), this is my lib/:

$ ls -lhF /nix/store/p1agfqxri38jaq38yijxgc6h4sk7y40a-mariadb-client-10.3.18/lib/                                                                                                          
total 528
-r-xr-xr-x  1 root  wheel   261K Jan  1  1970 libmariadb.3.dylib*
lrwxr-xr-x  1 root  wheel    18B Jan  1  1970 libmariadb.dylib@ -> libmariadb.3.dylib
lrwxr-xr-x  1 root  wheel    18B Jan  1  1970 libmysqlclient.dylib@ -> libmariadb.3.dylib
lrwxr-xr-x  1 root  wheel    18B Jan  1  1970 libmysqlclient_r.dylib@ -> libmariadb.3.dylib
dr-xr-xr-x  3 root  wheel    96B Jan  1  1970 mysql/

Compared to with my patch:

$ ls -lhF /nix/store/g33dklngpahpqchwj37scsshn3zql11k-mariadb-client-10.3.18/lib/                                                                                         
total 528
lrwxr-xr-x  1 root  wheel    20B Jan  1  1970 libmariadb.dylib@ -> libmysqlclient.dylib
-r-xr-xr-x  1 root  wheel   261K Jan  1  1970 libmysqlclient.dylib*
lrwxr-xr-x  1 root  wheel    20B Jan  1  1970 libmysqlclient_r.dylib@ -> libmysqlclient.dylib
dr-xr-xr-x  3 root  wheel    96B Jan  1  1970 mysql/

Let's go with yours. :)

@Izorkin
Copy link
Contributor

Izorkin commented Oct 10, 2019

It’s better to use libmysql

@callahad
Copy link
Member Author

Yep, everything also works fine with mariadb-connector-c.

@callahad
Copy link
Member Author

So I think where we're at is:

  1. mariadb-client.dev is unnecessarily broken by its postInstall script
  2. ...but mariadb-connector-c still works, and it's what most people should be using anyway.

Still feels like we should fix mariadb-connector-c (libmysql), right? (And if so, I think your patch is simpler and better than mine.)

@Izorkin
Copy link
Contributor

Izorkin commented Oct 10, 2019

theoretically, need to remove mysql-client.dev and use libmysql

@callahad
Copy link
Member Author

Maybe I'm not understanding you correctly. What exactly do you mean by "use libmysql?"

@callahad
Copy link
Member Author

callahad commented Oct 10, 2019 •
edited

Oh! Do you mean we should remove the .dev output from mariadb-client in mariadb/default.nix?

@Izorkin
Copy link
Contributor

Izorkin commented Oct 10, 2019

@callahad yes.

@callahad
Copy link
Member Author

Great. I'll close this and open a new PR later tonight to do that. Thank you.

@callahad callahad closed this Oct 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

package with attribute path nixpkgs.mysql-client.dev is incoherent on darwin
2 participants
@callahad @Izorkin