Since @jtojnar cherry-picked the other version of jsonrpc-glib and template-glib onto master today, I've rebased this PR dropping my commits that packaged those libraries. I had to make a minor fix to my gnome-builder packaging as a result but this version works just as well for me too.

I finally figured out that I just needed to patch src/libide/meson.build to replace pkglibdir with pkglibdir_abs instead of messing around with changes to the gobjectIntrospection package.

Sounds good.

This build works okay for me except that Builder seems to carefully clear the environment before running build steps, which is a behavior that I think needs to be patched somehow. I'm not sure how yet.

What do you mean?

In particular, I'm more fond of my approach of passing the Nix value python3.libPrefix (which currently evaluates to "python3.6") in as a Meson config option rather than substituting python3.sitePackages into a patch. One advantage is that I think my patch could be accepted upstream, possibly with whitespace changes. Also it doesn't run a Python command whose result it then ignores.

I am all for it if upstream accepts it.

#33270 also patched all the plugins to use hard-coded paths, which is a very invasive patch, and difficult to verify that it's correct. I see what problem that exists to solve, but I'm wondering if there's an easier way... If that's the best we can do, it's probably not hard to apply that patch to my branch too.

I like to make everything explicit in patches because they make it easier to track changes but I agree that there are downsides. Instead of patch, we could also use something like gappsWrapperArgs+=(--prefix PATH : "${stdenv.lib.makeBinPath [ meson ninja … ]}").

My suspicion is, though, that upstream Builder should do a better job of handling per-project environment variables, possibly by supporting tools like https://direnv.net/ (or nix-shell directly). Then if you want to use a particular plugin with a particular project, you'd include that plugin's dependencies in your project's environment. That should help keep the closure for Builder be not quite so huge...

We could make the binpath overridable and have a gnome-builderLight package that would set it to empty list.

I've just submitted some patches upstream toward this effort, and we'll see what the maintainers think:

https://gitlab.gnome.org/GNOME/gnome-builder/merge_requests/86

I've pushed an additional commit to this PR that I think addresses all your review comments, although I'm still not sure how to tell whether a dependency is only needed at build-time or also at run-time, so I might still have the buildInputs and nativeBuildInputs split incorrectly. Anyway, feel free to squash the branch if you want.

I'd disabled flatpak because it didn't build (and I'll point out that #33270 also disabled flatpak) but it turns out a minor patch fixes that, so it's enabled now. I've added that to my upstream merge request too.

The problem I observed with the environment getting cleared is that when I have a project open in Builder, clicking the "Build" button fails because it can't find programs like pkg-config. But opening a terminal inside Builder and running the same build command succeeds. So I think Builder sets up the environment differently for build commands that it runs. And in fact there's a clear-env property in the Builder implementation that among other things sets PATH to whatever was chosen as SAFE_PATH at configure time, which I believe is winding up as /bin:/usr/bin. But I haven't traced things through carefully enough to prove that's what's going on or figure out what to do about it.

I've been talking with Christian Hergert about some of the issues I've been having, and regarding the build environment problem, he suggests:

implement an IdeRuntimeProvider that lets us specify those "containers of tools" with some description file. Then you'd set your projects runtime to that instead of "Host".

I think that may be something I can tackle...

Hooray, my upstream merge request has been rebased and merged onto master! I think that was early enough to get in before the 3.30 release freeze, so hopefully those patches will be in the next stable release. But I don't actually understand the GNOME release cycle... 😅

The patches in this branch are now exactly the patches that are on master upstream, which I hope will make them easy to evaluate when upgrading.

I'm never quite sure when GitHub sends notifications, either. Did it tell you that I pushed my changes so far when you asked for them an hour ago, or should I have written a comment saying so...?

Hah, it looks like neither this package nor #33270 actually managed to get the gtk-doc output into $out, even when it was built; we both should have had outputDevdoc = 'out'. Fixed now...

Also the documentation totally didn't build correctly because it tried to get the Docbook DTDs from the internet since neither of us had pulled in docbook_xml_dtd_43 so it could use a local catalog instead.

And then once I fixed that, there was a locale problem in the Meson driver for gtkdoc.

Blech. But I can read the generated HTML documentation now, which is nice because the online copy is several major releases old.

I'm never quite sure when GitHub sends notifications, either. Did it tell you that I pushed my changes so far when you asked for them an hour ago, or should I have written a comment saying so...?

Simple build fixes are unaffected by the freezes: https://wiki.gnome.org/ReleasePlanning/Freezes

The patches in this branch are now exactly the patches that are on master upstream, which I hope will make them easy to evaluate when upgrading.

I would use fetchpatch then.

I'm never quite sure when GitHub sends notifications, either. Did it tell you that I pushed my changes so far when you asked for them an hour ago, or should I have written a comment saying so...?

I am not sure either, this time I received a notification but sometimes I do not. The difference seems to be that other projects use appstream-util validate command, whereas Builder uses appstream-util validate-relaxng. Ignoring it for now is probably the best choice of action.

Okay, as suggested I used fetchpatch and split out the separate devdoc output. As you said in the other pull request, I'm also "Not sure what else is missing". This is kind of usable as it stands now...

I tried running it with nix-shell -p gnome-builder autoconf clang clang-tools cmake ctags flatpak-builder gnome3.gjs gnumake indent meson packagekit pkgconfig python3 python3.pkgs.jedi python3.pkgs.lxml python3.pkgs.sphinx python3.pkgs.sphinx_rtd_theme rustup unzip which --run gnome-builder and it works, except for getting jedi not found, python auto-completion not possible.

Also, since we have decided to use it with nix-shell/direnv, comment explaining why it does not work from the get-go and redirecting users to one of these solutions should be added to the top of the file.

Do you think the comment explaining why it doesn't work without a nix-shell should go in meta.longDescription, perhaps?

Some people prefer it there, as it it visible on the package list on the website. Other people grep nixpkgs repo and comment on the top is more convenient for them. There is not any preferrence as fair as I know.

We could put the whole meta section at the top of the file and make both groups happy... 😁

Okay, I've pushed a longDescription. I'm going to bed now, but I'll check in tomorrow if there's anything else you'd like me to change. Thank you for your help getting this package into shape!

I find having meta at top makes the expressions less clean and also it is non-standard.

Do you mean having meta at the top of the file is non standard? Although I think there's actually a good argument for doing that, I wasn't seriously suggesting it. I left meta at the bottom as is normal elsewhere.

Yep, that was I meant. I have seen it in an expression or two but it is highly irregular and consistency is probably more important.

Also, do you have any idea how to fix the jedi thing?

Okay that took a lot of effort to track down, but the jedi thing is that nixpkgs has too new of a version of jedi for this version of gnome-builder. It should be fixed when gnome-builder 3.30 is released, as far as I can tell. Also, that version of gnome-builder will print the actual exception in addition to the "jedi not found" message, which would have made troubleshooting this a whole lot easier.

We could just disable building the jedi plugin with a comment that it can be re-enabled in 3.30, if you like?

(I also fixed the whitespace, thanks for pointing that out.)

Thanks so much for merging this, @jtojnar!

By the way, I just discovered a one-line kludge that allows me to actually compile things with Builder, so long as my environment was set up correctly before I launched Builder:

--- a/src/libide/buildsystem/ide-build-pipeline.c
+++ b/src/libide/buildsystem/ide-build-pipeline.c
@@ -2773,7 +2773,7 @@ ide_build_pipeline_create_launcher (IdeBuildPipeline  *self,
     {
       IdeEnvironment *env = ide_configuration_get_environment (self->configuration);

-      ide_subprocess_launcher_set_clear_env (ret, TRUE);
+      ide_subprocess_launcher_set_clear_env (ret, FALSE);
       ide_subprocess_launcher_overlay_environment (ret, env);
       /* Always ignore V=1 from configurations */
       ide_subprocess_launcher_setenv (ret, "V", "0", TRUE);

I'm pretty sure they'll never take this patch upstream. I'm not even sure it's a good idea in Nix. But at least it highlights precisely where I've been having trouble.

I'll still see if I can implement Christian Hergert's suggestion, of writing a plugin that allows a project to set its own environment, whether that's via nix-shell or direnv or whatever.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-install-gnome-sdk-dev-dependencies/5599/2