postgresql: add debug versions #103287

happysalada · 2020-11-10T09:40:01Z

Motivation for this change

Add postgres versions with debug symbols to get backtrace in case of a segfault

Not sure yet what the best approach is. This is my naive first approach.

closes #103024

@ocharles @thoughtpolice @danbst @globin @marsam

Interested in your thoughts

Things done

symphorien · 2020-11-10T10:33:37Z

Can you try making enable-debug true all the time and setting separateDebugInfo to true ? Then check that the closure size of the non-debug outputs is still the same. More info on separateDebugInfo in the manual https://nixos.org/manual/nixpkgs/stable/#ssec-fixup-phase and the wiki https://nixos.wiki/wiki/Debug_Symbols

happysalada · 2020-11-10T11:54:08Z

@symphorien Thanks for the tip, this way seems to make a lot more sense.

Here is what I did.

cd in the nixpkgs forked repository
then run nix-build ./ -A postgresql_13
run du -hs ./result/ and check the size. (about 20 mb for each version)

I tried not adding separateDebugInfo = true; and it didn't seem to make a difference.

I feel there is something about building packages locally that I don't understand.

I also tried inspecting the built derivation, but couldn't find the debugPath
nix show-derivation /nix/store/d7bhq0dnhdi69rsskiizbplh306ysahy-postgresql-13.0.drv | jq '.[]|.outputs.debug.path'
did not show anything

symphorien · 2020-11-10T21:02:45Z

You put separateDebugInfo on the postgresqlWithPackage derivation, not postgresql itself.

To compute the closure size, use nix path-info -S as recommented by the PR template instead of du.

happysalada · 2020-11-11T00:11:32Z

Thank you for your patience on this!

Enabling debug info with separate debug the size is 134966120
without enable debug, the size is 134965104

grahamcofborg-eval-check-meta seems to fail but on a text-cp437 package, It seems unrelated. Not sure what to do about it.

I tried checking for the debug output in the build derivation but couldn't find them.
Here is what I did
nix show-derivation /nix/store/457pkf3vzv171fbchmdyik25whqbvw5k-postgresql-13.0.drv

I got

{
  "/nix/store/457pkf3vzv171fbchmdyik25whqbvw5k-postgresql-13.0.drv": {
    "outputs": {
      "doc": {
        "path": "/nix/store/9mb8s0f337a8d206izv503igw219l5l2-postgresql-13.0-doc"
      },
      "lib": {
        "path": "/nix/store/fcqi755d09f6c50zszs1kdsvw47yrx6i-postgresql-13.0-lib"
      },
      "man": {
        "path": "/nix/store/1m16nhn9hgx0hh0yyidl9kggr79887x2-postgresql-13.0-man"
      },
      "out": {
        "path": "/nix/store/q8d78fpjlvypkvx041vawvgx7iaxcnfn-postgresql-13.0"
      }
    },
    "inputSrcs": [
      "/nix/store/75a44m59dwpwwgm7pcwwh9hmlx17gq63-specify_pkglibdir_at_runtime.patch",
      "/nix/store/7s83qirx82lfwhym8zc4dnpi4hfbrd22-less-is-more-96.patch",
      "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh",
      "/nix/store/jn9shys2nd8qjfsn4px8a9wky6askj8m-hardcode-pgxs-path-96.patch",
      "/nix/store/pwn3wii8whzpm3ffa6zxbwqm4rzha0hm-stabilize-timetz-dst.patch",
      "/nix/store/zpsqrnfpmmw840vr6xgsmynp6k4lk3ki-findstring.patch",
      "/nix/store/zv4waczq5k45138d2sdy7dyidas90lhh-disable-resolve_symlinks-94.patch"
    ],
    "inputDrvs": {
      "/nix/store/0lfv6bp3a81qrihjrvhkdrf0xk9s0vsg-postgresql-13.0.tar.bz2.drv": [
        "out"
      ],
      "/nix/store/37fzbqb4mh4vx1chandd4kndnm34x4a3-clang-wrapper-7.1.0.drv": [
        "out"
      ],
      "/nix/store/3dn8rqpb3g0kcnbhh8wgavkr6hx948q6-pkg-config-wrapper-0.29.2.drv": [
        "out"
      ],
      "/nix/store/3x4y5xjy65xnla0jgii6zj3h6bcq0nxs-hook.drv": [
        "out"
      ],
      "/nix/store/7dkd8lid6bc03szg05ba4x6qzrjf0sv5-tzdata-2020c.drv": [
        "out"
      ],
      "/nix/store/a21x6w6z0g219avypwg04kvfxg3cj03k-stdenv-darwin.drv": [
        "out"
      ],
      "/nix/store/b2m9j6gzwk5ady0irnqsnffq8r51p824-openssl-1.1.1h.drv": [
        "dev"
      ],
      "/nix/store/cjm5gliwkrmcwv4ar9wg33pb9a609vry-icu4c-67.1.drv": [
        "dev"
      ],
      "/nix/store/h84xz4538rywg84f8achwivrd6n13vw3-zlib-1.2.11.drv": [
        "dev"
      ],
      "/nix/store/jqk401vza112qy2fqnna1cnlc1ppmsx1-bash-4.4-p23.drv": [
        "out"
      ],
      "/nix/store/n73aas8i5ck59vp3d4j6mzwczqv3rs6l-readline-6.3p08.drv": [
        "dev"
      ],
      "/nix/store/rpzhcar4avfd84pxizbmhmbgiyf9h6dd-libxml2-2.9.10.drv": [
        "dev"
      ]
    },
    "platform": "x86_64-darwin",
    "builder": "/nix/store/7kzamk0abm530kkr5v106hnc3cfscwkc-bash-4.4-p23/bin/bash",
    "args": [
      "-e",
      "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"
    ],
    "env": {
      "LC_ALL": "C",
      "NIX_CFLAGS_COMPILE": "-I/nix/store/13csl9l1nzqfgnavx9hdilmxagp7nshg-libxml2-2.9.10-dev/include/libxml2",
      "__darwinAllowLocalNetworking": "",
      "__impureHostDeps": "/bin/sh /usr/lib/libSystem.B.dylib /usr/lib/system/libunc.dylib /dev/zero /dev/random /dev/urandom /bin/sh",
      "__propagatedImpureHostDeps": "",
      "__propagatedSandboxProfile": "",
      "__sandboxProfile": "",
      "buildFlags": "world",
      "buildInputs": "/nix/store/mgzn96nia7i7mxg396rzr3ll5blgyl2a-zlib-1.2.11-dev /nix/store/m8qcqm15dqxlgyj42dgpnybrk7z6accn-readline-6.3p08-dev /nix/store/2h6clxlyr03ak65m4zb2nfc9q2sviq27-openssl-1.1.1h-dev /nix/store/13csl9l1nzqfgnavx9hdilmxagp7nshg-libxml2-2.9.10-dev /nix/store/asj54pn1fqjbwl4fz96g9pxs6qg01133-hook /nix/store/48zib55dydpj6nlmlzg39zy3fzzknf26-icu4c-67.1-dev",
      "builder": "/nix/store/7kzamk0abm530kkr5v106hnc3cfscwkc-bash-4.4-p23/bin/bash",
      "checkTarget": "check",
      "configureFlags": "--with-openssl --with-libxml --sysconfdir=/etc --libdir=$(lib)/lib --with-system-tzdata=/nix/store/j3qmi5zgc41i9c6hcrnkr7zqzilpa1lq-tzdata-2020c/share/zoneinfo --enable-debug  --with-uuid=e2fs --with-icu",
      "depsBuildBuild": "",
      "depsBuildBuildPropagated": "",
      "depsBuildTarget": "",
      "depsBuildTargetPropagated": "",
      "depsHostHost": "",
      "depsHostHostPropagated": "",
      "depsTargetTarget": "",
      "depsTargetTargetPropagated": "",
      "disallowedReferences": "/nix/store/phsmyd540ip5hrn25wj303w8qir34hv0-clang-wrapper-7.1.0",
      "doCheck": "",
      "doInstallCheck": "",
      "doc": "/nix/store/9mb8s0f337a8d206izv503igw219l5l2-postgresql-13.0-doc",
      "enableParallelBuilding": "",
      "installTargets": "install-world",
      "lib": "/nix/store/fcqi755d09f6c50zszs1kdsvw47yrx6i-postgresql-13.0-lib",
      "man": "/nix/store/1m16nhn9hgx0hh0yyidl9kggr79887x2-postgresql-13.0-man",
      "name": "postgresql-13.0",
      "nativeBuildInputs": "/nix/store/rzkjabipvbk7wfqmfkahhlw3mplxyp0g-pkg-config-wrapper-0.29.2",
      "out": "/nix/store/q8d78fpjlvypkvx041vawvgx7iaxcnfn-postgresql-13.0",
      "outputs": "out lib doc man",
      "patches": "/nix/store/zv4waczq5k45138d2sdy7dyidas90lhh-disable-resolve_symlinks-94.patch /nix/store/7s83qirx82lfwhym8zc4dnpi4hfbrd22-less-is-more-96.patch /nix/store/jn9shys2nd8qjfsn4px8a9wky6askj8m-hardcode-pgxs-path-96.patch /nix/store/75a44m59dwpwwgm7pcwwh9hmlx17gq63-specify_pkglibdir_at_runtime.patch /nix/store/zpsqrnfpmmw840vr6xgsmynp6k4lk3ki-findstring.patch /nix/store/pwn3wii8whzpm3ffa6zxbwqm4rzha0hm-stabilize-timetz-dst.patch",
      "pname": "postgresql",
      "postConfigure": "# Hardcode the path to pgxs so pg_config returns the path in $out\nsubstituteInPlace \"src/common/config_info.c\" --replace HARDCODED_PGXS_PATH $out/lib\n",
      "postFixup": "",
      "postInstall": "moveToOutput \"lib/pgxs\" \"$out\" # looks strange, but not deleting it\nmoveToOutput \"lib/libpgcommon*.a\" \"$out\"\nmoveToOutput \"lib/libpgport*.a\" \"$out\"\nmoveToOutput \"lib/libecpg*\" \"$out\"\n\n# Prevent a retained dependency on gcc-wrapper.\nsubstituteInPlace \"$out/lib/pgxs/src/Makefile.global\" --replace /nix/store/phsmyd540ip5hrn25wj303w8qir34hv0-clang-wrapper-7.1.0/bin/ld ld\n\nif [ -z \"${dontDisableStatic:-}\" ]; then\n  # Remove static libraries in case dynamic are available.\n  for i in $out/lib/*.a $lib/lib/*.a; do\n    name=\"$(basename \"$i\")\"\n    ext=\".dylib\"\n    if [ -e \"$lib/lib/${name%.a}$ext\" ] || [ -e \"${i%.a}$ext\" ]; then\n      rm \"$i\"\n    fi\n  done\nfi\n",
      "preConfigure": "CC=cc",
      "propagatedBuildInputs": "",
      "propagatedNativeBuildInputs": "",
      "separateDebugInfo": "1",
      "setOutputFlags": "",
      "src": "/nix/store/sm2lbr1br9xnzrshzd89v55qyz6ankzz-postgresql-13.0.tar.bz2",
      "stdenv": "/nix/store/nssc5ppljv528hhihsbrj0gcjf4m0vgd-stdenv-darwin",
      "strictDeps": "",
      "system": "x86_64-darwin",
      "version": "13.0"
    }
  }
}

It shows "separateDebugInfo": "1" but I couldn't find the debug path like described in https://nixos.wiki/wiki/Debug_Symbols.
Perhaps I'm missing something?

symphorien · 2020-11-11T17:34:24Z

I can build the debug info with your pr:

$  nix-build . -A postgresql.debug
/nix/store/wxvhkdfl501cy380kfwzdwfid4rrd0vi-postgresql-11.9-debug

happysalada · 2020-11-12T00:53:18Z

Maybe I need to enable something else on my end to enable the debug to show?

I'm using nix version nix (Nix) 3.0pre20200829_f156513

I can see that the derivation is built with --enable-debug and separateDebugInfo but when I try to build .debug like you did, this is what I get

❯ nix-build . -A postgresql.debug
error: --- AttrPathNotFound -------------------------------------------------------------------------------------------------------------------------- nix-build
attribute 'debug' in selection path 'postgresql.debug' not found

symphorien · 2020-11-12T19:27:37Z

Huh ? If we have the same checkout, we should get the same evaluation... can you double check?

happysalada · 2020-11-13T00:16:57Z

I've attached more information below. I have checked my branch. I have tried building the normal postgres. It seems I still get the same error.

nixpkgs on  postgresql_add_debug
❯ nix-build . -A postgresql.debug
error: --- AttrPathNotFound -------------------------------------------------------------------------------------------------------------------------- nix-build
attribute 'debug' in selection path 'postgresql.debug' not found
nixpkgs on  postgresql_add_debug
❯ nix-build --version
nix-build (Nix) 3.0pre20200829_f156513
nixpkgs on  postgresql_add_debug
❯ nix-build . -A postgresql
/nix/store/cxrv523j6k4hr7ck5a5hagsk44a7n9yk-postgresql-11.9

It there is anything else that I might have forgotten, let me know. I can't think of anything though.

Perhaps it's something wrong with my environment only? I'm on macos, not sure if that should make a difference though.

happysalada · 2020-11-14T23:44:33Z

When you have a moment, we probably need another pair of eyes on this. I'm not sure how to move forward
@ocharles @thoughtpolice @danbst @globin @marsam
(no rush if you're busy)

happysalada · 2020-11-20T08:11:46Z

When you have a moment, we probably need another pair of eyes on this. I'm not sure how to move forward
@ocharles @thoughtpolice @danbst @globin @marsam
(no rush if you're busy)

happysalada · 2020-11-23T07:52:47Z

The debug symbols don't work on macOS, however I tested on nixOS and was able to build them.
I don't understand why the debug symbols don't work on macOS, but most probably they won't be used on that platform.
debug symbols would mostly be used to debug a prod issue in my opinion (That is also what I intend to use them for).

As far as I'm concerned this is ready to merge.
@ocharles @thoughtpolice @danbst @globin @marsam

Let me know if anything.

nixos-discourse · 2020-11-23T07:53:37Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/382

happysalada · 2020-11-27T11:24:48Z

@ocharles @thoughtpolice @danbst @globin @marsam
I'm ating everybody as I don't know who is the best person to ask.
When you have a moment, I think this is ready to merge.
Let me know if anything of course

symphorien · 2020-11-28T20:32:24Z

Maintainers are not responsive but I think this is low risk enough for me to merge it.

@GrahamcOfBorg test postgresql

happysalada · 2020-11-29T06:15:09Z

@symphorien Thank you!
I'll be testing this in staging next week

happysalada · 2020-11-30T06:12:30Z

I'm seeing some debug symbols on my staging instance.

I am currently having a coredump on production sometimes. I will switch my production and try to inspect the coredump. I'll create an issue if there are any unexpected problems.

Thanks again for the help!

postgresql: add debug versions

2d9c667

happysalada requested a review from thoughtpolice as a code owner November 10, 2020 09:40

ofborg bot added the 8.has: package (new) label Nov 10, 2020

ofborg bot requested review from globin, marsam, ocharles and danbst November 10, 2020 09:56

ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 501-1000 10.rebuild-linux: 501+ 10.rebuild-linux: 2501-5000 labels Nov 10, 2020

happysalada added 2 commits November 10, 2020 20:43

postgresql: use separateDebugInfo

4c5931e

postgresql: remove local name change

b433c79

postgresql: use separateDebugInfo correctly

ef08d4c

symphorien merged commit ebe455a into NixOS:master Nov 28, 2020

happysalada deleted the postgresql_add_debug branch April 28, 2023 19:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

postgresql: add debug versions #103287

postgresql: add debug versions #103287

happysalada commented Nov 10, 2020

symphorien commented Nov 10, 2020

happysalada commented Nov 10, 2020 •

edited

symphorien commented Nov 10, 2020

happysalada commented Nov 11, 2020 •

edited

symphorien commented Nov 11, 2020

happysalada commented Nov 12, 2020

symphorien commented Nov 12, 2020

happysalada commented Nov 13, 2020

happysalada commented Nov 14, 2020

happysalada commented Nov 20, 2020

happysalada commented Nov 23, 2020 •

edited

nixos-discourse commented Nov 23, 2020

happysalada commented Nov 27, 2020

symphorien commented Nov 28, 2020

happysalada commented Nov 29, 2020

happysalada commented Nov 30, 2020

postgresql: add debug versions #103287

postgresql: add debug versions #103287

Conversation

happysalada commented Nov 10, 2020

Motivation for this change

Things done

symphorien commented Nov 10, 2020

happysalada commented Nov 10, 2020 • edited

symphorien commented Nov 10, 2020

happysalada commented Nov 11, 2020 • edited

symphorien commented Nov 11, 2020

happysalada commented Nov 12, 2020

symphorien commented Nov 12, 2020

happysalada commented Nov 13, 2020

happysalada commented Nov 14, 2020

happysalada commented Nov 20, 2020

happysalada commented Nov 23, 2020 • edited

nixos-discourse commented Nov 23, 2020

happysalada commented Nov 27, 2020

symphorien commented Nov 28, 2020

happysalada commented Nov 29, 2020

happysalada commented Nov 30, 2020

happysalada commented Nov 10, 2020 •

edited

happysalada commented Nov 11, 2020 •

edited

happysalada commented Nov 23, 2020 •

edited