javacard-devkit: init at 2.2.2 #44148

Ekleog · 2018-07-27T10:48:35Z

Oracle has already released 3.0.5, but versions after 2.2.2 appear to be Windows-only.

This derivation will override javac in order to give it the paths required for JavaCard programming.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

Ekleog · 2018-07-27T10:52:50Z

~~Note to self (got to go right now and was reminded to test all binary files by the checklist): @Ekleog, figure out why the cref wrapper is unhappy and make libjpcsclite.so non-executable.~~

~~Sorry for opening this PR before having checked everything!~~

I think it's fixed :) Thank you @cleverca22 !

GrahamcOfBorg · 2018-07-27T11:02:23Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-27T11:02:43Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-29T08:35:22Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-29T08:35:45Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

aszlig

I turned the suggestions here into a small patch file. However, I didn't test the results.

aszlig · 2018-07-29T11:22:02Z

pkgs/development/compilers/javacard-devkit/default.nix

+    # the very few information on the net assume a hand-downloaded sdk…? Maybe in the
+    # manual instead… but noone I know thinks of reading the manual upon installing
+    # a package? Anyway, just pick a place (including /dev/null) and I'll try to
+    # move these instructions to there :)


Maybe longDescription in meta?

aszlig · 2018-07-29T11:25:44Z

pkgs/development/compilers/javacard-devkit/default.nix

+
+  nativeBuildInputs = [ unzip makeWrapper ];
+
+  unpackPhase = "true";


Something like this with oraclejdk8 in nativeBuildInputs should unpack the nested Zip file:

{ # ... sourceRoot = "."; zipPrefix = "java_card_kit-${underscoreVersion}"; unpackCmd = '' unzip -p "$curSrc" "$zipPrefix/$zipPrefix-rr-bin-linux-do.zip" | jar x ''; # ... }

Setting sourceRoot to . prevents unpackPhase from bailing out due to multiple directories.

Thanks, didn't know about the sourceRoot = "." trick!

aszlig · 2018-07-29T11:27:02Z

pkgs/development/compilers/javacard-devkit/default.nix

+    mkdir "$out"
+    cp -R bin "$out"
+    cp -R lib "$out"
+    cp -R api_export_files "$out"


I'd suggest to put this in $out/share/javacard-devkit instead of directly in $out.

aszlig · 2018-07-29T11:27:14Z

pkgs/development/compilers/javacard-devkit/default.nix

+
+    mkdir "$out"
+    cp -R bin "$out"
+    cp -R lib "$out"


Maybe $out/lib/java?

The SDK appears to expect to find the .jar in ../lib/*.jar from the bin/ directory, so, with a choice between either patching all the scripts (and likely mis-patching cref) I guess it's easier to just put it there? :)

aszlig · 2018-07-29T11:28:27Z

pkgs/development/compilers/javacard-devkit/default.nix

+
+    chmod +x bin/{apdutool,capdump,capgen,converter,cref,exp2text,jcwde,scriptgen,verifycap,verifyexp,verifyrev}
+    chmod u+w bin/cref
+    patchelf --set-interpreter "$(cat "$NIX_CC/nix-support/dynamic-linker")" bin/cref


There is also libjpcsclite.so (not sure which component uses it however), which needs libpcsclite.so. It might be a good idea to use autoPatchelfHook here.

I don't know where that's used either…

Good catch for noticing it needs libpcsclite.so! it also appears to need libnsl.so.1 and libc.so.6. However, for some reason I can't understand, adding glibc to the buildInputs doesn't appear to make autoPatchelfHook auto-correct it. I've now tried quite a number of variations of it, so if you have any idea about what's going wrong…

(the fact that libnsl-1.1.0 appears to package libnsl.so.2.0.0 and the fact I'm not even sure it is used somewhere are probably just as important reasons for my decision to ragequit this issue after a few hours, though :°)

Adding glibc to buildInputs shouldn't be needed. What's the output from ldd?

There's no output from ldd (it's a shared library), but here is the output from readelf -d:

Dynamic section at offset 0x20a0 contains 23 entries: Tag Type Name/Value 0x0000001d (RUNPATH) Library runpath: [/nix/store/n962knaz7h3qjdjf01g6r0dwld9qswyc-pcsclite-1.8.23/lib] 0x00000001 (NEEDED) Shared library: [libpcsclite.so] 0x00000001 (NEEDED) Shared library: [libc.so.6] 0x00000001 (NEEDED) Shared library: [libnsl.so.1] 0x0000000c (INIT) 0x50c 0x0000000d (FINI) 0x684 0x00000004 (HASH) 0xb4 0x00000005 (STRTAB) 0x2188 0x00000006 (SYMTAB) 0x18c 0x0000000a (STRSZ) 240 (bytes) 0x0000000b (SYMENT) 16 (bytes) 0x00000003 (PLTGOT) 0x17a0 0x00000002 (PLTRELSZ) 16 (bytes) 0x00000014 (PLTREL) REL 0x00000017 (JMPREL) 0x4fc 0x00000011 (REL) 0x4d4 0x00000012 (RELSZ) 40 (bytes) 0x00000013 (RELENT) 8 (bytes) 0x6ffffffe (VERNEED) 0x4b4 0x6fffffff (VERNEEDNUM) 1 0x6ffffff0 (VERSYM) 0x46c 0x6ffffffa (RELCOUNT) 2 0x00000000 (NULL) 0x0

So from what I can tell it's not adding ${glibc}/lib (for libc.so.6) in the RUNPATH. (and it didn't add the libnsl.so.1 either but that's pretty normal) Hence my attempts to add it into buildInputs :)

After discussion over IRC, I was just being stupid and assuming that nix-shell --pure -p pkgsi686Linux.glibc actually was pure, while it actually gave the 64-bits ldd (due to the internals of -p).

So with the actual ldd, that does perform recursive resolution of the runpath, the problem is solved and both libc.so.6 and libnsl.so.1 are already provided by the runpath. So no longer any problem here :)

aszlig · 2018-07-29T11:31:31Z

pkgs/development/compilers/javacard-devkit/default.nix

+    # TODO (question for the reviewer): The resulting javac can then be used with:
+    #     javac -source 1.2 -target 1.2 myPackage/myApplet.java
+    # Should I include the -source/-target arguments in this wrapper?
+    # I'm thinking this may break unaware users just adding javacard-devkit to their env


If it's in the same environment as the JDK, it will collide with the javac from the JDK, but maybe it's a good idea to use a different name?

Hmm, the issue is all the documentation online mentions only using javac for compiling (well, when there is any kind of documentation, that is), so I fear if we call it javacardc people will be lost… but maybe that's the least bad option? Having javacardc that would have both the right CLASSPATH and the right -source 1.5 -target 1.5…

Then, all tutorials online tell you to set the CLASSPATH and -source 1.5 -target 1.5 by hand, so I wonder how far we should go from upstream for end-user convenience… I think for CLASSPATH given it's hidden somewhere in the store there's little room for argumentation, but for -source/-target I wonder.

Ekleog · 2018-07-30T10:02:04Z

Thanks for the review and hints! I've done what I could, things appear to more or less work (at least for the ones for which I know how they are supposed to be used).

Though, still remain a few concerns voiced in my answers to your remarks, unfortunately hidden by github as no longer relevant due to underlying code change.

GrahamcOfBorg · 2018-07-30T10:13:41Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-30T10:13:52Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

Ekleog · 2018-07-31T01:04:38Z

Thank you for your help, @aszlig!

So I think the last remaining question is how to handle the differences of javac between javacard-devkit and oraclejdk* / jdk*.

Basically, the differences between the two are that javacard-devkit needs for successful compilation:

Additions to the CLASSPATH that point into the store (in order to find the required dependencies)
Additional arguments, for instance -source 1.5 -target 1.5 to specify the Java version (it can go at least from 1.2 to 1.5, converter doesn't support anything more recent but I haven't tried anything older)

What I'm thinking is, reasonably we can't leave it up to the user to do 1, because that points into the store and would be a PITA. So this must be done by the javacard-devkit derivation.

For 2… it's more complex: maybe someone wants to compile an older Java Card applet that was designed for use with 1.2? Yet at the same time it's most likely to be unnecessary boilerplate and a cause for head-scratching from someone who wouldn't know these arguments are required. So the question looks like it's a bit “How far do we want to go from upstream for our users' sake?”. I'd lean towards leaving these arguments outside of javacard-devkit, mostly for the ancient-addon use case.

Now, if the idea is to keep only 1 included in javacard-devkit, remains the question of how to do this. I see again two options:

Have a wrapped javac in the javacard-devkit derivation
Have a javacardc in the javacard-devkit derivation, that wraps oraclejdk's javac.

I can see advantages and disadvantages for both options:

Pros: Closer to upstream recommended building system. Consistent to most documentation that could be found online.
Cons: Might cause conflicts if installed globally at the same time as oraclejdk or jdk. (but is this actually an issue? anyway adding something to the CLASSPATH should hopefully not break anything, so it'd be a matter of making sure that javacard-devkit's javac overrides oraclejdk's or jdk's)
Pros: No conflict. Appears to be the name of the Java Card 3 tool (not sure whether this is actually a bonus or a malus, given JC3 has no linux version…)
Cons: Farther from the (few) docs that can be found online. Requires specific documentation and especially requires that the user actually reads said documentation.

A coin throw told me option 2 was better, but that's only a coin throw's opinion.

What do you think about this? :)

Ekleog · 2018-08-06T01:06:55Z

Given there appeared to be no contradictory opinion, I've now implemented the coin throw's result, and think this is ready to be merged :)

GrahamcOfBorg · 2018-08-06T01:19:18Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-08-06T01:19:48Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

makefu · 2018-08-14T11:15:07Z

hey all! i'd love to see this merged.

timokau · 2018-08-14T11:25:44Z

pkgs/top-level/all-packages.nix

@@ -6787,6 +6787,8 @@ with pkgs;
  oraclejdk10distro = packageType: _:
      (callPackage ../development/compilers/oraclejdk/jdk10-linux.nix { inherit packageType; });

+  javacard-devkit = pkgsi686Linux.callPackage ../development/compilers/javacard-devkit { };


Why is it pkgsi686Linux?

The devkit is available only for 32-bits linux, and includes a pre-compiled executable (cref)

Why is pkgsi686Linux used in stead of meta.platforms? Does it have something to do with cross compiling?

The devkit does work on 64-bit linux, but requires 32-bit libraries, that are defined by pkgsi686Linux :) That said, you're right, I had mis-set meta.platforms! This should be fixed with latest push :)

timokau · 2018-08-14T11:26:36Z

pkgs/development/compilers/javacard-devkit/default.nix

+    longDescription = ''
+      This Java Card SDK is the official SDK made available by Oracle for programming for the Java Card platform.
+
+      Instructions for usage:


Is it common to put documentation in the longDescription? Do you think it is likely that people find it here?

I don't really know, only followed the advice from #44148 (comment) :)

Doesn't seem ideal to me and I slightly prefer wrapping javac, but I don't really have a better proposal.

Oracle has already released 3.0.5, but versions after 2.2.2 appear to be Windows-only. Thanks-To: aszlig <aszlig@nix.build>

GrahamcOfBorg · 2018-08-14T12:25:32Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-08-14T12:25:58Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: javacard-devkit

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg added 8.has: package (new) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Jul 27, 2018

Ekleog force-pushed the javacard-devkit branch from 547734c to f7c02c5 Compare July 29, 2018 08:23

aszlig reviewed Jul 29, 2018

View reviewed changes

Ekleog force-pushed the javacard-devkit branch from f7c02c5 to 22f861c Compare July 30, 2018 09:59

Ekleog force-pushed the javacard-devkit branch 2 times, most recently from 6458fcd to 1b35a0f Compare August 6, 2018 01:06

timokau reviewed Aug 14, 2018

View reviewed changes

javacard-devkit: init at 2.2.2

bcf59b9

Oracle has already released 3.0.5, but versions after 2.2.2 appear to be Windows-only. Thanks-To: aszlig <aszlig@nix.build>

Ekleog force-pushed the javacard-devkit branch from 1b35a0f to bcf59b9 Compare August 14, 2018 12:14

timokau merged commit ec14b03 into NixOS:master Aug 14, 2018


		nativeBuildInputs = [ unzip makeWrapper ];

		unpackPhase = "true";

javacard-devkit: init at 2.2.2 #44148

javacard-devkit: init at 2.2.2 #44148

Conversation

Ekleog commented Jul 27, 2018 • edited

Things done

Ekleog commented Jul 27, 2018 • edited

GrahamcOfBorg commented Jul 27, 2018

GrahamcOfBorg commented Jul 27, 2018

GrahamcOfBorg commented Jul 29, 2018

GrahamcOfBorg commented Jul 29, 2018

aszlig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ekleog commented Jul 30, 2018

GrahamcOfBorg commented Jul 30, 2018

GrahamcOfBorg commented Jul 30, 2018

Ekleog commented Jul 31, 2018

Ekleog commented Aug 6, 2018

GrahamcOfBorg commented Aug 6, 2018

GrahamcOfBorg commented Aug 6, 2018

makefu commented Aug 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GrahamcOfBorg commented Aug 14, 2018

GrahamcOfBorg commented Aug 14, 2018

Ekleog commented Jul 27, 2018 •

edited

Ekleog commented Jul 27, 2018 •

edited