Reworked how dependencies are handled in pypandoc. glibcLocales doesn't seem to be necessary, so I removed it.

cc @bennofs @Twey

@GrahamcOfBorg build python2Packages.pypandoc python3Packages.pypandoc

I think there is a fundamental problem with packaging pypandoc (maybe even pandoc) for Nix:

First of all pandoc generally uses PATH quite heavily, which is generally fine, as it could theoretically use any program as filter, for example. This becomes problematic, when it comes to optional dependencies of pandoc, like pandoc-citeproc or pdflatex (xetex ...), because it generally assumes to find these programs in PATH, even when they are not explicitly given, like when pandoc is called with --bibliography.

It is possible to work around this issue, e. g. by giving --pdf-engine or --filter explicitly with a path to /nix/store/.... For applications that use pandoc another approach can be used: Setting up a PATH for just the applications containing the dependencies using makeWrapper, so the application doesn't need to be heavily patched.

A real issue comes up concerning pypandoc: It offers full control over pandoc and its options like --filter. This means we can't anticipate all possible usages of pypandoc and set static paths in those cases. Or maybe we could, but It'd mean heavy patching, from what I can tell. Additionally pypandoc has some problematic features for Nix, like the ability to download pandoc dynamically, although this is often not used, like in the case of uberwriter.

My last approach was to setup PATH inside of pypandoc to contain its most common dependencies, texlive and pandoc-citeproc. I agree that this approach isn't ideal. If we don't do that the following tests break, since the required dependencies are not found:

• test_conversion_with_citeproc_filter
• test_pdf_conversion

I think there are multiple solutions to this problem, that don't involve going back to the previous approach:

Patching pypandoc

I don't think this makes sense. It would be a lot of effort, as it'd mean to figure out, which dependencies are used from the given pandoc options and filters and replacing them with static paths in the pandoc call. I fear this wouldn't be possible to make 100% reliable and break some applications.

Don't care about dependencies in pypandoc

I guess this is our best bet. Instead of worrying about dependencies (except for pandoc itself which we can set the static path for quite easily) in pypandoc, we require all applications that use it to ensure pandoc will find its dependencies that are acually used, either by setting up the correct PATH using makeWrapper or similar, or by using static paths in the options to pandoc.

To fix the tests we could either disable those two or temporarily set up a PATH containing them just for the testPhase.

I have a mockup of this locally where I removed the addition concerning os.environ["PATH"] and set doCheck = false in pypandoc. It works fine.

Edit: Also it would be possible, although more effort, to patch uberwriter to explicitly use static paths whenever using pypandoc.

Add a pandocWithPackages derivation

Not sure if this is worth it, but essentially we'd add a pandoc derivations that can be called with certain dependencies that would be available to pandoc at runtime. Maybe a long term idea.

I'm not really sure how to proceed here, but I heavily favor option 2 at the moment, as it is the simplest and would make uberwriter work without a lot of effort. I'd love to hear what you think as well!

Implemented my idea for the second option on my branch now.

@GrahamcOfBorg build python2Packages.pypandoc python3Packages.pypandoc uberwriter

@GrahamcOfBorg build uberwriter

@sternenseemann would you like to become a maintainer of the pypandoc package as well? I'm not actively using it, and if you maintain uberwriter, you have a great way to check if it's actually working.

@bennofs yeah sure, already had a good look into it, so it probably makes sense for me to do it