[RDY] Vdirsyncer update #39195

teto · 2018-04-20T07:07:16Z

Motivation for this change

The new rust dependency kinda complicates things see #33050

@matthiasbeyer and @pbogdan made the patch, I just created the PR.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

GrahamcOfBorg · 2018-04-20T07:22:45Z

Success on x86_64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

  /nix/store/imk08z514fjvvnrpw0qnqfhym0ajyhsf-python3.6-hypothesis-3.45.2/lib/python3.6/site-packages/hypothesis/core.py:1001: HypothesisDeprecationWarning: In future derandomize will imply database=None, but your test is currently using examples from the database. To get the future behaviour, update your settings to include database=None.
    state.run()

tests/unit/utils/test_vobject.py::test_replace_uid[BEGIN:VCARD\nVERSION:3.0\nFN:Cyrus Daboo\nN:Daboo;Cyrus;;;\nADR;TYPE=POSTAL:;2822 Email HQ;Suite 2821;RFCVille;PA;15213;USA\nEMAIL;TYPE=PREF:cyrus@example.com\nNICKNAME:me\nNOTE:Example VCard.\nORG:Self Employed\nTEL;TYPE=VOICE:412 605 0499\nTEL;TYPE=FAX:412 605 0705\nURL;VALUE=URI:http://www.example.com\nX-SOMETHING:{r}\nUID:{uid}\nEND:VCARD]
  /nix/store/imk08z514fjvvnrpw0qnqfhym0ajyhsf-python3.6-hypothesis-3.45.2/lib/python3.6/site-packages/hypothesis/core.py:1001: HypothesisDeprecationWarning: In future derandomize will imply database=None, but your test is currently using examples from the database. To get the future behaviour, update your settings to include database=None.
    state.run()

-- Docs: http://doc.pytest.org/en/latest/warnings.html
============ 373 passed, 113 skipped, 6 warnings in 154.07 seconds =============
/nix/store/pg4xvns5gc0zjabb7haidbq86mp466r7-vdirsyncer-0.17.0a1

GrahamcOfBorg · 2018-04-20T07:50:26Z

Success on aarch64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

  /nix/store/9cj1v7gmlvxs17qnfnhz2cy5lawiqk1k-python3.6-hypothesis-3.45.2/lib/python3.6/site-packages/hypothesis/core.py:1001: HypothesisDeprecationWarning: In future derandomize will imply database=None, but your test is currently using examples from the database. To get the future behaviour, update your settings to include database=None.
    state.run()

tests/unit/utils/test_vobject.py::test_replace_uid[BEGIN:VCARD\nVERSION:3.0\nFN:Cyrus Daboo\nN:Daboo;Cyrus;;;\nADR;TYPE=POSTAL:;2822 Email HQ;Suite 2821;RFCVille;PA;15213;USA\nEMAIL;TYPE=PREF:cyrus@example.com\nNICKNAME:me\nNOTE:Example VCard.\nORG:Self Employed\nTEL;TYPE=VOICE:412 605 0499\nTEL;TYPE=FAX:412 605 0705\nURL;VALUE=URI:http://www.example.com\nX-SOMETHING:{r}\nUID:{uid}\nEND:VCARD]
  /nix/store/9cj1v7gmlvxs17qnfnhz2cy5lawiqk1k-python3.6-hypothesis-3.45.2/lib/python3.6/site-packages/hypothesis/core.py:1001: HypothesisDeprecationWarning: In future derandomize will imply database=None, but your test is currently using examples from the database. To get the future behaviour, update your settings to include database=None.
    state.run()

-- Docs: http://doc.pytest.org/en/latest/warnings.html
============ 373 passed, 113 skipped, 4 warnings in 131.40 seconds =============
/nix/store/pxzi66za8pkq701vgxyvv0b8x4znlp7c-vdirsyncer-0.17.0a1

dotlambda · 2018-04-20T09:36:10Z

pkgs/development/python-modules/milksnake/default.nix

+  pname = "milksnake";
+  version = "0.1.1";
+
+  src = fetchurl {


Please use fetchPypi

fetchPypi fails,maybe because it's a .zip and not a .tar.gz ? https://pypi.org/project/milksnake/#files

found a fix

You can pass extension = "zip" to fetchPypi.

dotlambda · 2018-04-20T09:37:45Z

pkgs/development/python-modules/milksnake/default.nix

+
+  meta = with lib; {
+    description = "A python library that extends setuptools for binary extensions.";
+    homepage = https://pypi.python.org/pypi/milksnake/;


I'd prefer https://github.com/getsentry/milksnake.

dotlambda · 2018-04-20T09:39:14Z

pkgs/development/python-modules/milksnake/default.nix

+  ];
+
+  meta = with lib; {
+    description = "A python library that extends setuptools for binary extensions.";


no period at the end

GrahamcOfBorg · 2018-04-20T09:39:31Z

Success on x86_64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

tests/storage/dav/test_carddav.py ssssssssssssssssssssssssssss           [ 91%]
tests/storage/dav/test_main.py .................................         [100%]

=========================== slowest 3 test durations ===========================
1.16s call     tests/storage/test_filesystem.py::TestFilesystemStorage::test_generic[VCARD]
0.99s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VEVENT]
0.98s call     tests/storage/test_singlefile.py::TestSingleFileStorage::test_generic[VEVENT]
=================== 250 passed, 137 skipped in 31.52 seconds ===================
bash codecov.sh -c -F storage
/nix/store/y5b57mx9g5ww86f190rgimirn9js9wld-vdirsyncer-0.17.0a2

GrahamcOfBorg · 2018-04-20T10:29:58Z

Success on aarch64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

tests/storage/dav/test_carddav.py ssssssssssssssssssssssssssss           [ 91%]
tests/storage/dav/test_main.py .................................         [100%]

=========================== slowest 3 test durations ===========================
0.11s call     tests/storage/test_http.py::test_list
0.08s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VCARD]
0.07s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VTODO]
=================== 250 passed, 137 skipped in 6.65 seconds ====================
bash codecov.sh -c -F storage
/nix/store/mkq42hlvv735675n431dqm3vfqja8xch-vdirsyncer-0.17.0a2

dotlambda · 2018-04-23T07:18:38Z

pkgs/tools/misc/vdirsyncer/default.nix

+    postInstall = ''
+      mkdir $out/include $out/lib
+      cp $out/bin/libvdirsyncer_rustext* $out/lib
+      rm -r $out/bin


dotlambda · 2018-04-23T07:18:42Z

pkgs/tools/misc/vdirsyncer/default.nix

  };
+in pythonPackages.buildPythonApplication rec {
+  inherit version name src;


Please specify pname instead of name.

dotlambda · 2018-04-23T07:19:02Z

pkgs/tools/misc/vdirsyncer/default.nix

  name = "vdirsyncer-${version}";
-
  src = fetchurl {


fetchPypi

dotlambda · 2018-04-23T07:20:39Z

@GrahamcOfBorg build python2.pkgs.milksnake python3.pkgs.milksnake

GrahamcOfBorg · 2018-04-23T07:21:10Z

Success on x86_64-linux (full log)

Attempted: python2.pkgs.milksnake, python3.pkgs.milksnake

Partial log (click to expand)

reading manifest file 'milksnake.egg-info/SOURCES.txt'
writing manifest file 'milksnake.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/3xnxm08y5r00s2x6nl3bkij9gvjr2v95-python2.7-milksnake-0.1.5
/nix/store/qjryznlnqmi0mb029d5br681s7crav51-python3.6-milksnake-0.1.5

GrahamcOfBorg · 2018-04-23T07:21:23Z

Success on aarch64-linux (full log)

Attempted: python2.pkgs.milksnake, python3.pkgs.milksnake

Partial log (click to expand)

reading manifest file 'milksnake.egg-info/SOURCES.txt'
writing manifest file 'milksnake.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/5kmnc7hagp8qhmsc3b1c6lbccs9a0f9p-python2.7-milksnake-0.1.5
/nix/store/lm8sq9ibf343hb7aly1qjx1cyicmn3wz-python3.6-milksnake-0.1.5

GrahamcOfBorg · 2018-04-23T07:26:20Z

Success on x86_64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

tests/storage/dav/test_carddav.py ssssssssssssssssssssssssssss           [ 91%]
tests/storage/dav/test_main.py .................................         [100%]

=========================== slowest 3 test durations ===========================
1.16s call     tests/storage/test_filesystem.py::TestFilesystemStorage::test_generic[VTODO]
1.03s call     tests/storage/test_singlefile.py::TestSingleFileStorage::test_generic[VCARD]
0.95s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VTODO]
=================== 250 passed, 137 skipped in 32.14 seconds ===================
bash codecov.sh -c -F storage
/nix/store/33cvli4igbh5jxmkgps17m47qppw61ld-vdirsyncer-0.17.0a2

dotlambda · 2018-04-23T07:29:05Z

Ran 0 tests in 0.000s

Please set doCheck = false for milksnake, as their test seems to use pip install: https://github.com/getsentry/milksnake/blob/master/tests/test_basic.py (include a comment on why tests are disabled).

GrahamcOfBorg · 2018-04-23T07:51:33Z

Success on x86_64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

tests/storage/dav/test_carddav.py ssssssssssssssssssssssssssss           [ 91%]
tests/storage/dav/test_main.py .................................         [100%]

=========================== slowest 3 test durations ===========================
1.18s call     tests/storage/test_filesystem.py::TestFilesystemStorage::test_generic[VCARD]
0.98s call     tests/storage/test_singlefile.py::TestSingleFileStorage::test_generic[VTODO]
0.95s call     tests/storage/test_filesystem.py::TestFilesystemStorage::test_update_others_stay_the_same[VCARD]
=================== 250 passed, 137 skipped in 30.93 seconds ===================
bash codecov.sh -c -F storage
/nix/store/q18mlwp6rhbp7dv51bvmw7785q6ajxwn-vdirsyncer-0.17.0a2

GrahamcOfBorg · 2018-04-23T07:57:37Z

Success on aarch64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

tests/storage/dav/test_carddav.py ssssssssssssssssssssssssssss           [ 91%]
tests/storage/dav/test_main.py .................................         [100%]

=========================== slowest 3 test durations ===========================
0.11s call     tests/storage/test_http.py::test_list
0.08s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VCARD]
0.07s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VTODO]
=================== 250 passed, 137 skipped in 6.73 seconds ====================
bash codecov.sh -c -F storage
/nix/store/nzd7i9baqry66k10qnmk10dqn38b0wsr-vdirsyncer-0.17.0a2

teto · 2018-04-23T07:58:10Z

I set doCheck to false and used fetchPypi to retrieve vdirsyncer. Binary seems to run fine.

dotlambda · 2018-04-23T07:58:14Z

pkgs/tools/misc/vdirsyncer/default.nix

  };
+  native = rustPlatform.buildRustPackage {
+    name = pname + "-native";


+ "-" + version

dotlambda · 2018-04-23T07:59:09Z

pkgs/tools/misc/vdirsyncer/default.nix

+  native = rustPlatform.buildRustPackage {
+    name = pname + "-native";
+    inherit src;
+    sourceRoot = pname + "-" + version + "/rust";


I'm not sure but I think there's something like src.name

dotlambda · 2018-04-23T08:00:30Z

pkgs/tools/misc/vdirsyncer/default.nix


  propagatedBuildInputs = with pythonPackages; [
    click click-log click-threading
    requests_toolbelt
    requests
    requests_oauthlib # required for google oauth sync
    atomicwrites
+    milksnake
  ];

  buildInputs = with pythonPackages; [hypothesis pytest pytest-localserver pytest-subtesthack setuptools_scm ] ++ [ glibcLocales ];


Everything but setuptools_scm belongs into checkInputs.

GrahamcOfBorg · 2018-04-23T08:00:52Z

Success on aarch64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

tests/storage/dav/test_carddav.py ssssssssssssssssssssssssssss           [ 91%]
tests/storage/dav/test_main.py .................................         [100%]

=========================== slowest 3 test durations ===========================
0.11s call     tests/storage/test_http.py::test_list
0.08s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VCARD]
0.08s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VTODO]
=================== 250 passed, 137 skipped in 6.74 seconds ====================
bash codecov.sh -c -F storage
/nix/store/y0q6l3j4dcq1ry4gjb7w79n4myla70bp-vdirsyncer-0.17.0a2

dotlambda · 2018-04-23T08:02:08Z

Sorry for the many comments, this should be the last batch. Did you try using rmdir?

teto · 2018-04-23T08:10:07Z

why rmdir ? no pb for the comments, on the contrary it's my fault, I kinda know the python guidelines but didn't check the initial package. Also I might be a bit hasty at merging it since I reported the pb last year.

dotlambda · 2018-04-23T08:13:48Z

why rmdir ?

Because if every file in $out/bin is called libvdirsyncer_rustext*, but a new file is added after an upgrade, we would not notice because of rm -r.

GrahamcOfBorg · 2018-04-23T08:39:42Z

Success on x86_64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

tests/storage/dav/test_carddav.py ssssssssssssssssssssssssssss           [ 91%]
tests/storage/dav/test_main.py .................................         [100%]

=========================== slowest 3 test durations ===========================
2.26s call     tests/storage/test_filesystem.py::TestFilesystemStorage::test_generic[VEVENT]
1.04s call     tests/storage/test_singlefile.py::TestSingleFileStorage::test_generic[VEVENT]
0.98s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VTODO]
=================== 250 passed, 137 skipped in 33.28 seconds ===================
bash codecov.sh -c -F storage
/nix/store/x0lxcgpl5zw8lli149zrp5ws6aq4mmzh-vdirsyncer-0.17.0a2

GrahamcOfBorg · 2018-04-23T08:40:33Z

Success on aarch64-linux (full log)

Attempted: vdirsyncer

Partial log (click to expand)

tests/storage/dav/test_carddav.py ssssssssssssssssssssssssssss           [ 91%]
tests/storage/dav/test_main.py .................................         [100%]

=========================== slowest 3 test durations ===========================
0.11s call     tests/storage/test_http.py::test_list
0.08s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VCARD]
0.07s call     tests/storage/test_http_with_singlefile.py::TestHttpStorage::test_generic[VTODO]
=================== 250 passed, 137 skipped in 6.68 seconds ====================
bash codecov.sh -c -F storage
/nix/store/r8d33m1wd7a19vvh4x170wkdgy69sg8x-vdirsyncer-0.17.0a2

teto · 2018-04-23T08:43:48Z

rmdir doesn't work because there seems to be a few subfolders (not a rust expert but rm -r works well enough). src.name adds the extension too so it's not usable (without parseDrvName but I preferred to create a let name = ...) . I hope this is ok

teto requested a review from FRidh as a code owner April 20, 2018 07:07

GrahamcOfBorg added 6.topic: python 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Apr 20, 2018

teto force-pushed the vdirsyncer branch from 8b79975 to c157692 Compare April 20, 2018 09:25

dotlambda requested changes Apr 20, 2018

View reviewed changes

teto force-pushed the vdirsyncer branch from c157692 to 8df68f1 Compare April 23, 2018 07:11

dotlambda reviewed Apr 23, 2018

View reviewed changes

pythonPackages.milksnake: init at 0.1.5

f396f09

teto force-pushed the vdirsyncer branch from 8df68f1 to 9e9ef7b Compare April 23, 2018 07:37

teto changed the title ~~Vdirsyncer update~~ [RDY] Vdirsyncer update Apr 23, 2018

dotlambda reviewed Apr 23, 2018

View reviewed changes

vdirsyncer: 0.16.4 -> 0.17.0a2

49f55fa

teto force-pushed the vdirsyncer branch from 9e9ef7b to 49f55fa Compare April 23, 2018 08:19

dotlambda merged commit 1aaab71 into NixOS:master Apr 23, 2018

teto deleted the vdirsyncer branch April 24, 2018 01:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RDY] Vdirsyncer update #39195

[RDY] Vdirsyncer update #39195

teto commented Apr 20, 2018 •

edited

GrahamcOfBorg commented Apr 20, 2018

GrahamcOfBorg commented Apr 20, 2018

dotlambda Apr 20, 2018

teto Apr 23, 2018

teto Apr 23, 2018

dotlambda Apr 23, 2018

dotlambda Apr 20, 2018

dotlambda Apr 20, 2018

GrahamcOfBorg commented Apr 20, 2018

GrahamcOfBorg commented Apr 20, 2018

dotlambda Apr 23, 2018

dotlambda Apr 23, 2018

dotlambda Apr 23, 2018

dotlambda commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

dotlambda commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

teto commented Apr 23, 2018

dotlambda Apr 23, 2018

dotlambda Apr 23, 2018

dotlambda Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

dotlambda commented Apr 23, 2018

teto commented Apr 23, 2018

dotlambda commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

teto commented Apr 23, 2018

[RDY] Vdirsyncer update #39195

[RDY] Vdirsyncer update #39195

Conversation

teto commented Apr 20, 2018 • edited

Motivation for this change

Things done

GrahamcOfBorg commented Apr 20, 2018

GrahamcOfBorg commented Apr 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GrahamcOfBorg commented Apr 20, 2018

GrahamcOfBorg commented Apr 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

dotlambda commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

teto commented Apr 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GrahamcOfBorg commented Apr 23, 2018

dotlambda commented Apr 23, 2018

teto commented Apr 23, 2018

dotlambda commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

GrahamcOfBorg commented Apr 23, 2018

teto commented Apr 23, 2018

teto commented Apr 20, 2018 •

edited