Build QGIS and dependencies on darwin #36902

mpickering · 2018-03-13T10:04:15Z

Motivation for this change

QGIS didn't build on darwin but now it does. I have only tested these patches on OSX. I'm not sure the original QGIS derivation worked properly on nixos anyway, is anyone using it?

It would be good to test the following attributes using @GrahamcOfBorg

grass qgis

The rest should be dependencies of qgis.

cc @viric

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

lheckemann · 2018-03-13T10:06:33Z

pkgs/applications/gis/grass/noLn.patch

+ ifneq ($(findstring darwin,$(ARCH)),)
+ 	@# enable OSX Help Viewer
+-	@/bin/ln -sfh "$(INST_DIR)/docs/html" /Library/Documentation/Help/GRASS-$(GRASS_VERSION_MAJOR).$(GRASS_VERSION_MINOR)
+	#@/bin/ln -sfh "$(INST_DIR)/docs/html" /Library/Documentation/Help/GRASS-$(GRASS_VERSION_MAJOR).$(GRASS_VERSION_MINOR)


Is this commenting it out? If so, maybe prefer just removing it completely.

Yes it is commented out. I can remove it completely as well, I was just trying to make the least change possible.

I would just remove it, that makes it much easier to read the diff.

lheckemann · 2018-03-13T10:08:01Z

pkgs/applications/gis/qgis/default.nix

-    fcgi libspatialindex libspatialite postgresql qjson qca2 txt2tags ] ++
+    fcgi libspatialindex libspatialite postgresql qjson qca2 txt2tags pkgconfig
+    (stdenv.lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.IOKit)
+    (stdenv.lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.ApplicationServices)


Maybe append lib.optionals stdenv.isDarwin (with darwin.apple_sdk.frameworks; [IOKit ApplicationServices]) instead of having these individual entries

Extending the callPackage scope in all-packages.nix with frameworks is the most common approach.

lheckemann · 2018-03-13T10:09:14Z

pkgs/applications/gis/qgis/default.nix

+               ++ stdenv.lib.optional stdenv.isDarwin
+                    (["-DCMAKE_FIND_FRAMEWORK=never"]
+                      ++ ["-DQGIS_MACAPP_BUNDLE=0"]);
+#                     ++ ["-DCMAKE_BUILD_TYPE=RelWithDebInfo"];


Was this left in intentionally?

I was unsure whether to leave it in or not as it's very useful to see the terminal output when running the application. Like I said, the derivation is currently not perfect but much better than what is currently in the repo.

The release type affects the terminal output? 🤦‍♂️ ah well, maybe add a comment clarifying how it's useful then

Well it's a graphical application. Building with debug info makes the terminal output very verbose as you use the gui.

No, the actual thing that debug info refers to is the debugging symbols that map the machine code sections back to source code. Terminal output should not usually be affected, but it's up to the application developer whether that's actually the case.

lheckemann · 2018-03-13T10:10:43Z

pkgs/development/libraries/gdal/default.nix

@@ -68,7 +68,7 @@ stdenv.mkDerivation rec {
    wrapPythonPrograms
  '';

-  enableParallelBuilding = true;
+  enableParallelBuilding = false;


There were files missing in the installation before I set it to false.

This is worth documenting in a comment and/or the commit message.

lheckemann · 2018-03-13T10:16:43Z

pkgs/applications/gis/qgis/default.nix

+          wrapProgram $out/bin/qgis \
+            --prefix PYTHONPATH : $PYTHONPATH \
+            --prefix LD_LIBRARY_PATH : ${stdenv.lib.makeLibraryPath [ openssl ]}
+          '') ++


Currently evaluation fails as shown by our dear GrahamcOfBorg — strings are concatenated with +, not ++. Once this is fixed I can get ofborg to try building it :)

lheckemann · 2018-03-13T10:33:26Z

@GrahamcOfBorg build grass qgis

GrahamcOfBorg · 2018-03-13T11:21:13Z

Failure on x86_64-darwin (full log)

Attempted: grass, qgis

Partial log (click to expand)

1 warning generated.
[ 49%] Building CXX object src/providers/wms/CMakeFiles/wmsprovider_a.dir/qgswmtsdimensions.cpp.o
[ 49%] Building CXX object src/providers/wms/CMakeFiles/wmsprovider_a.dir/qgsxyzconnection.cpp.o
[ 49%] Building CXX object src/providers/wms/CMakeFiles/wmsprovider_a.dir/moc_qgswmscapabilities.cxx.o
[ 49%] Building CXX object src/providers/wms/CMakeFiles/wmsprovider_a.dir/moc_qgswmsprovider.cxx.o
[ 49%] Building CXX object src/providers/wms/CMakeFiles/wmsprovider_a.dir/moc_qgswmssourceselect.cxx.o
[ 49%] Building CXX object src/providers/wms/CMakeFiles/wmsprovider_a.dir/moc_qgswmsconnection.cxx.o
[ 49%] Building CXX object src/providers/wms/CMakeFiles/wmsprovider_a.dir/moc_qgswmsdataitems.cxx.o
building of '/nix/store/gm535ymqawka4hsxpf7n95zc4p2ljwc9-qgis-2.18.16.drv' timed out after 1800 seconds
error: build of '/nix/store/gm535ymqawka4hsxpf7n95zc4p2ljwc9-qgis-2.18.16.drv' failed

mpickering · 2018-03-13T11:23:57Z

The build takes over an hour locally on my machine so that time out seems expected.

GrahamcOfBorg · 2018-03-13T11:34:08Z

Failure on aarch64-linux (full log)

Attempted: grass, qgis

Partial log (click to expand)

                  ^~~~~
                  int
/build/hdf-4.2.12/hdf/src/tbbt.h:289:11: error: unknown type name 'VOID'
 HDFLIBAPI VOID        tbbtfree
           ^~~~
building of '/nix/store/6bmyb5w02zcghk85w67qb6v9wm1pambi-hdf-4.2.12.drv' timed out after 3600 seconds
cannot build derivation '/nix/store/4b809mcnfnhvilnm6rcm6zlw6bzcky20-gdal-2.2.3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/6qr0r6pfmsvn8xacnvkhi946r4cl6p4w-grass-7.2.2.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/gbji5bk9lrfmnrzn3349i17i0axxfinr-qgis-2.18.16.drv': 2 dependencies couldn't be built
�[31;1merror:�[0m build of '/nix/store/6qr0r6pfmsvn8xacnvkhi946r4cl6p4w-grass-7.2.2.drv', '/nix/store/gbji5bk9lrfmnrzn3349i17i0axxfinr-qgis-2.18.16.drv' failed

GrahamcOfBorg · 2018-03-13T12:22:45Z

Failure on x86_64-linux (full log)

Attempted: grass, qgis

Partial log (click to expand)

/tmp/nix-build-qgis-2.18.16.drv-0/qgis-2.18.16/src/gui/../core/qgis.h:507:32: warning: extra ';' [-Wpedantic]
   _Pragma("GCC diagnostic pop");
                                ^
/tmp/nix-build-qgis-2.18.16.drv-0/qgis-2.18.16/src/gui/../core/qgsexpression.h:1602:1: note: in expansion of macro 'Q_NOWARN_DEPRECATED_POP'
 Q_NOWARN_DEPRECATED_POP
 ^~~~~~~~~~~~~~~~~~~~~~~
[ 42%] Building CXX object src/gui/CMakeFiles/qgis_gui.dir/moc_qgsowssourceselect.cxx.o
[ 42%] Building CXX object src/gui/CMakeFiles/qgis_gui.dir/moc_qgssourceselectdialog.cxx.o
building of ‘/nix/store/3vbhhzwg370dcg9vgv11i3mlhribyncg-qgis-2.18.16.drv’ timed out after 3600 seconds
error: build of ‘/nix/store/3vbhhzwg370dcg9vgv11i3mlhribyncg-qgis-2.18.16.drv’ failed

mpickering · 2018-03-14T19:31:00Z

I rebased and applied the changes from the suggestions. I'm rebuilding now locally to verify that they work correctly.

FRidh · 2018-03-17T12:19:08Z

pkgs/applications/gis/qgis/default.nix

+  postInstall =
+    (stdenv.lib.optionalString stdenv.isLinux ''
+          wrapProgram $out/bin/qgis \
+            --prefix PYTHONPATH : $PYTHONPATH \


use of prefix allows leaking in PTYTHONPATH. Typically it is better to use set

It was like this before? Was it wrong before?

FRidh · 2018-03-17T12:20:10Z

pkgs/development/libraries/qscintilla/default.nix

  nativeBuildInputs = [ unzip ] ++ (if withQt5 then [ qmake ] else [ qmake4Hook ]);

+  # Fix Xcode 8 compilation problem
+  xcodePatch = fetchurl { url = "https://raw.githubusercontent.com/Homebrew/formula-patches/a651d71/qscintilla2/xcode-8.patch";


either add this directly in the patches list, or put it in a let in expression.

mpickering · 2018-03-17T16:07:56Z

I made the two changes @FRidh suggested.

There are still some problems with the withGrass option but the package is mostly functional.

mpickering · 2018-03-21T08:36:10Z

I rebased onto staging. Thanks @copumpkin

copumpkin · 2018-03-21T12:59:01Z

@grahamc I think borg got confused about the base branch switch somehow 😄or at least I hope this thing doesn't affect the stdenv!

@mpickering thanks!

mpickering · 2018-04-10T21:04:32Z

@copumpkin I am returning to this issue now looking at what is causing Syntax error at '__attribute__', have you got any ideas?

mpickering · 2018-04-10T21:12:20Z

The source seems to be when compiling in the lib/python/ctypes folder if anyone runs into this later. I'm stopping my chase now as I found my files elsewhere.

copumpkin · 2018-04-11T14:45:43Z

No real ideas, sorry!

copumpkin · 2018-06-14T02:33:12Z

@mpickering actually needed qgis on Darwin today and it worked flawlessly, thanks again for fixing this!

mpickering requested review from FRidh and ttuegel as code owners March 13, 2018 10:04

GrahamcOfBorg added the 6.topic: darwin Running or building packages on Darwin label Mar 13, 2018

lheckemann reviewed Mar 13, 2018

View reviewed changes

GrahamcOfBorg added 6.topic: python 6.topic: qt/kde labels Mar 13, 2018

lheckemann reviewed Mar 13, 2018

View reviewed changes

mpickering force-pushed the qgis-darwin branch from 829a0bf to ed5a700 Compare March 13, 2018 10:19

GrahamcOfBorg added 8.has: package (new) 10.rebuild-darwin: 11-100 10.rebuild-linux: 101-500 labels Mar 13, 2018

mpickering mentioned this pull request Mar 13, 2018

Building GRASS fails on darwin #36365

Closed

ttuegel removed their request for review March 14, 2018 12:29

mpickering force-pushed the qgis-darwin branch from ed5a700 to 3f40de2 Compare March 14, 2018 19:28

GrahamcOfBorg added 10.rebuild-linux: 11-100 and removed 6.topic: qt/kde 10.rebuild-linux: 101-500 labels Mar 14, 2018

mpickering force-pushed the qgis-darwin branch 2 times, most recently from 2beffc0 to 2f7dcdf Compare March 16, 2018 23:53

FRidh requested changes Mar 17, 2018

View reviewed changes

mpickering added 7 commits March 21, 2018 08:28

Fix qscintilla qt5 darwin build

3391006

Build qwt6 on darwin

ef4bce8

Build qscintilla on darwin

2c6cd5b

Add additional darwin dependency for qca2

4a4478b

Add darwin to qwt platforms attribute

92f76df

Package QGIS 2.18 for darwin

7842e5d

There are still some problems with the withGrass option but the package is mostly functional.

Remove darwin specific patch

9031300

mpickering force-pushed the qgis-darwin branch from 6696e6a to 9031300 Compare March 21, 2018 08:34

mpickering changed the base branch from master to staging March 21, 2018 08:36

copumpkin merged commit c7b3373 into NixOS:staging Mar 21, 2018

mpickering mentioned this pull request Apr 27, 2020

Attempting to add Darwin support for QGIS #71398

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build QGIS and dependencies on darwin #36902

Build QGIS and dependencies on darwin #36902

mpickering commented Mar 13, 2018

lheckemann Mar 13, 2018

mpickering Mar 13, 2018

LnL7 Mar 14, 2018

lheckemann Mar 13, 2018

LnL7 Mar 14, 2018

lheckemann Mar 13, 2018

mpickering Mar 13, 2018

lheckemann Mar 13, 2018

mpickering Mar 13, 2018

lheckemann Mar 13, 2018

lheckemann Mar 13, 2018

mpickering Mar 14, 2018

lheckemann Mar 15, 2018

lheckemann Mar 13, 2018

lheckemann commented Mar 13, 2018

GrahamcOfBorg commented Mar 13, 2018

mpickering commented Mar 13, 2018

GrahamcOfBorg commented Mar 13, 2018

GrahamcOfBorg commented Mar 13, 2018

mpickering commented Mar 14, 2018

FRidh Mar 17, 2018

mpickering Mar 17, 2018

FRidh Mar 17, 2018

mpickering Mar 17, 2018

mpickering commented Mar 17, 2018

mpickering commented Mar 21, 2018

copumpkin commented Mar 21, 2018

mpickering commented Apr 10, 2018

mpickering commented Apr 10, 2018

copumpkin commented Apr 11, 2018

copumpkin commented Jun 14, 2018

Build QGIS and dependencies on darwin #36902

Build QGIS and dependencies on darwin #36902

Conversation

mpickering commented Mar 13, 2018

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lheckemann commented Mar 13, 2018

GrahamcOfBorg commented Mar 13, 2018

mpickering commented Mar 13, 2018

GrahamcOfBorg commented Mar 13, 2018

GrahamcOfBorg commented Mar 13, 2018

mpickering commented Mar 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mpickering commented Mar 17, 2018

mpickering commented Mar 21, 2018

copumpkin commented Mar 21, 2018

mpickering commented Apr 10, 2018

mpickering commented Apr 10, 2018

copumpkin commented Apr 11, 2018

copumpkin commented Jun 14, 2018