Python CAN support #34473

sorki · 2018-01-31T22:13:51Z

Motivation for this change

CAN/CANOpen python stack

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

grahamc · 2018-01-31T22:35:30Z

@GrahamcOfBorg eval

(a commit broke master, and it is now fixed.)

dotlambda

Please put the derivations into spererate files inside pkgs/development/python-modules and use callPackage.

dotlambda · 2018-01-31T23:12:42Z

pkgs/top-level/python-packages.nix

@@ -2012,6 +2012,83 @@ in {
    doCheck = false;
  });

+  can = buildPythonPackage rec {
+    rev = "2.0.0";
+    name = "can-${rev}";


No name. Specify pname and version please.

dotlambda · 2018-01-31T23:12:56Z

pkgs/top-level/python-packages.nix

+
+  canopen = buildPythonPackage rec {
+    rev = "v0.5.1";
+    name = "canopen-${rev}";


dotlambda · 2018-01-31T23:13:06Z

pkgs/top-level/python-packages.nix

+
+  canmatrix = buildPythonPackage rec {
+    rev = "0.6";
+    name = "canmatrix-${rev}";


dotlambda · 2018-01-31T23:15:07Z

pkgs/top-level/python-packages.nix

+    rev = "2.0.0";
+    name = "can-${rev}";
+
+    src = pkgs.fetchFromGitHub {


Please use fetchPypi if possible. This makes it easier to override the src.

FRidh · 2018-02-01T08:45:31Z

pkgs/top-level/python-packages.nix

@@ -2012,6 +2012,83 @@ in {
    doCheck = false;
  });

+  can = buildPythonPackage rec {


We are moving Python expressions out of pkgs/top-level/python-packages.nix into pkgs/development/python-modules/<module>/default.nix.

Please move the expression there, and call it from pkgs/top-level/python-packages.nix using callPackage ../development/python-modules/<package> { };.

sorki · 2018-02-02T10:03:57Z

Should be ok now.

dotlambda · 2018-02-02T10:10:13Z

pkgs/development/python-modules/canopen/default.nix

+  doCheck = false;
+
+  propagatedBuildInputs =
+    [ nose


checkInputs

dotlambda · 2018-02-02T10:10:38Z

pkgs/development/python-modules/can/default.nix

+
+  propagatedBuildInputs =
+    [ nose
+      mock


checkInputs

dotlambda · 2018-02-02T10:10:54Z

pkgs/development/python-modules/can/default.nix

+buildPythonPackage rec {
+  pname = "python-can";
+  version = "2.0.0";
+  name = "${pname}-${version}";


dotlambda · 2018-02-02T10:11:19Z

pkgs/development/python-modules/canmatrix/default.nix

+buildPythonPackage rec {
+  pname = "canmatrix";
+  version = "0.6";
+  name = "${pname}-${version}";


dotlambda · 2018-02-02T10:11:28Z

pkgs/development/python-modules/canopen/default.nix

+buildPythonPackage rec {
+  pname = "canopen";
+  version = "0.5.1";
+  name = "${pname}-${version}";


dotlambda · 2018-02-02T10:13:30Z

pkgs/development/python-modules/canopen/default.nix

+
+  # due to missing test/sample.eds
+  # https://github.com/christiansandberg/canopen/pull/57
+  doCheck = false;


Thanks a lot for opening the PR! As long as no new version is published on PyPI, you could also rely on fetchFromGitHub.

dotlambda · 2018-02-02T10:17:24Z

@GrahamcOfBorg build python2.pkgs.canopen python3.pkgs.canopen

GrahamcOfBorg · 2018-02-02T10:17:49Z

Success on x86_64-linux (full log)

Partial log (click to expand)

Successfully installed canopen-0.5.1
/tmp/nix-build-python3.6-canopen-0.5.1.drv-0/canopen-0.5.1
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/sg6mvlqi6g5j3z9kmlbrv1hrhyn6laq3-python3.6-canopen-0.5.1
strip is /nix/store/mdyy001q67hiks0g24ra53z7ckm4jfr4-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/sg6mvlqi6g5j3z9kmlbrv1hrhyn6laq3-python3.6-canopen-0.5.1/lib 
patching script interpreter paths in /nix/store/sg6mvlqi6g5j3z9kmlbrv1hrhyn6laq3-python3.6-canopen-0.5.1
checking for references to /tmp/nix-build-python3.6-canopen-0.5.1.drv-0 in /nix/store/sg6mvlqi6g5j3z9kmlbrv1hrhyn6laq3-python3.6-canopen-0.5.1...
/nix/store/vl0s2prrnqdnardxq3jx092k5nkmcmsi-python2.7-canopen-0.5.1
/nix/store/sg6mvlqi6g5j3z9kmlbrv1hrhyn6laq3-python3.6-canopen-0.5.1

GrahamcOfBorg · 2018-02-02T10:18:06Z

Success on aarch64-linux (full log)

Partial log (click to expand)

Successfully installed canopen-0.5.1
/build/canopen-0.5.1
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/4hcpabbng74g0qdd4aw8gm2d79sqsabb-python3.6-canopen-0.5.1
strip is /nix/store/jwz859pxqj7sl2dbwvmxkx68jp774izb-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/4hcpabbng74g0qdd4aw8gm2d79sqsabb-python3.6-canopen-0.5.1/lib
patching script interpreter paths in /nix/store/4hcpabbng74g0qdd4aw8gm2d79sqsabb-python3.6-canopen-0.5.1
checking for references to /build in /nix/store/4hcpabbng74g0qdd4aw8gm2d79sqsabb-python3.6-canopen-0.5.1...
/nix/store/7wxnh9nhq84icajdn25d5v0xgdb4a15v-python2.7-canopen-0.5.1
/nix/store/4hcpabbng74g0qdd4aw8gm2d79sqsabb-python3.6-canopen-0.5.1

dotlambda · 2018-02-02T10:57:30Z

The following enables canmatrix tests. Because PyPI doesn't ship the test/ directory, fetchFromGitHub is used.

diff --git a/pkgs/development/python-modules/canmatrix/default.nix b/pkgs/development/python-modules/canmatrix/default.nix
index c62c43dc2df..6409010aa3e 100644
--- a/pkgs/development/python-modules/canmatrix/default.nix
+++ b/pkgs/development/python-modules/canmatrix/default.nix
@@ -1,7 +1,7 @@
 { lib
 , stdenv
 , buildPythonPackage
-, fetchPypi
+, fetchFromGitHub
 , lxml
 , xlwt
 , xlrd
@@ -14,9 +14,11 @@ buildPythonPackage rec {
   version = "0.6";
   name = "${pname}-${version}";
 
-  src = fetchPypi {
-    inherit pname version;
-    sha256 = "1jv0ry3qrrkjc32a0lwznf7ix8wl2s5lccqsqcyxnr89qbkjapqb";
+  src = fetchFromGitHub {
+    owner = "ebroecker";
+    repo = pname;
+    rev = version;
+    sha256 = "1lb0krhchja2jqfsh5lsfgmqcchs1pd38akvc407jfmll96f4yqz";
   };
 
   propagatedBuildInputs =
@@ -28,6 +30,11 @@ buildPythonPackage rec {
       future
     ];
 
+  checkPhase = ''
+    cd test
+    python ./test.py
+  '';
+
   meta = with lib; {
     homepage = https://github.com/ebroecker/canmatrix;
     description = "Support and convert several CAN (Controller Area Network) database formats .arxml .dbc .dbf .kcd .sym fibex xls(x)";

However, I don't know what the expected outcome is. Their travis build more or less show the same output as this does.

sorki · 2018-02-03T06:34:10Z

Had to disable doCheck in canopen now, rest looks fine, I would let canmatrix run its test even if they only ouput diffs if they ever decide to return exit code according to that.

FRidh · 2018-02-03T07:50:43Z

@GrahamcOfBorg build python2.pkgs.can python2.pkgs.canmatrix python3.pkgs.can python3.pkgs.canmatrix

FRidh · 2018-02-03T07:49:28Z

pkgs/development/python-modules/canmatrix/default.nix

+
+  checkPhase = ''
+    cd test
+    python ./test.py


${python.interpreter. This is needed e.g. for pypy

GrahamcOfBorg · 2018-02-03T07:51:37Z

Success on x86_64-linux (full log)

Partial log (click to expand)

Variable type 'string' found and skipped
Variable type 'string' found and skipped
Variable type 'string' found and skipped
Variable type 'string' found and skipped
xls -> ['arxml', 'csv', 'dbc', 'dbf', 'json', 'kcd', 'sym', 'xlsx', 'xml', 'yaml']
xlsx -> ['arxml', 'csv', 'dbc', 'dbf', 'json', 'kcd', 'sym', 'xls', 'xml', 'yaml']
yaml -> ['arxml', 'csv', 'dbc', 'dbf', 'json', 'kcd', 'sym', 'xls', 'xlsx', 'xml']


    Testing completed: difference found

GrahamcOfBorg · 2018-02-03T07:57:22Z

Success on aarch64-linux (full log)

Partial log (click to expand)

Variable type 'string' found and skipped
Variable type 'string' found and skipped
Variable type 'string' found and skipped
Variable type 'string' found and skipped
xls -> ['arxml', 'csv', 'dbc', 'dbf', 'json', 'kcd', 'sym', 'xlsx', 'xml', 'yaml']
xlsx -> ['arxml', 'csv', 'dbc', 'dbf', 'json', 'kcd', 'sym', 'xls', 'xml', 'yaml']
yaml -> ['arxml', 'csv', 'dbc', 'dbf', 'json', 'kcd', 'sym', 'xls', 'xlsx', 'xml']


    Testing completed: difference found

sorki requested a review from FRidh as a code owner January 31, 2018 22:13

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Jan 31, 2018

dotlambda reviewed Jan 31, 2018

View reviewed changes

FRidh requested changes Feb 1, 2018

View reviewed changes

FRidh added the 6.topic: python label Feb 1, 2018

sorki force-pushed the python-can-support branch from cb4e2f6 to 90b6abe Compare February 1, 2018 23:40

dotlambda reviewed Feb 2, 2018

View reviewed changes

sorki force-pushed the python-can-support branch from 90b6abe to 7d5d39d Compare February 3, 2018 06:24

python-can: init at 2.0.0

096a07f

sorki force-pushed the python-can-support branch from 7d5d39d to f7545fc Compare February 3, 2018 06:32

FRidh requested changes Feb 3, 2018

View reviewed changes

sorki force-pushed the python-can-support branch from f7545fc to f2f6f57 Compare February 3, 2018 08:46

sorki added 2 commits February 3, 2018 09:55

python-canmatrix: init at 0.6

6c52f36

python-canopen: init at 0.5.1

305ed7e

sorki force-pushed the python-can-support branch from f2f6f57 to 305ed7e Compare February 3, 2018 08:55

FRidh merged commit 9ba4b16 into NixOS:master Feb 4, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python CAN support #34473

Python CAN support #34473

sorki commented Jan 31, 2018

grahamc commented Jan 31, 2018

dotlambda left a comment

dotlambda Jan 31, 2018

dotlambda Jan 31, 2018

dotlambda Jan 31, 2018

dotlambda Jan 31, 2018

FRidh Feb 1, 2018

sorki commented Feb 2, 2018

dotlambda Feb 2, 2018

dotlambda Feb 2, 2018

dotlambda Feb 2, 2018

dotlambda Feb 2, 2018

dotlambda Feb 2, 2018

dotlambda Feb 2, 2018

dotlambda commented Feb 2, 2018

GrahamcOfBorg commented Feb 2, 2018

GrahamcOfBorg commented Feb 2, 2018

dotlambda commented Feb 2, 2018 •

edited

sorki commented Feb 3, 2018

FRidh commented Feb 3, 2018

FRidh Feb 3, 2018

GrahamcOfBorg commented Feb 3, 2018

GrahamcOfBorg commented Feb 3, 2018

Python CAN support #34473

Python CAN support #34473

Conversation

sorki commented Jan 31, 2018

Motivation for this change

Things done

grahamc commented Jan 31, 2018

dotlambda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sorki commented Feb 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda commented Feb 2, 2018

GrahamcOfBorg commented Feb 2, 2018

GrahamcOfBorg commented Feb 2, 2018

dotlambda commented Feb 2, 2018 • edited

sorki commented Feb 3, 2018

FRidh commented Feb 3, 2018

Choose a reason for hiding this comment

GrahamcOfBorg commented Feb 3, 2018

GrahamcOfBorg commented Feb 3, 2018

dotlambda commented Feb 2, 2018 •

edited