Update android sdk, add license config option and selectable build tools #48848

nicknovitski · 2018-10-22T23:40:33Z

I followed #36112 and some other license-requiring packages.

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

Mic92 · 2018-10-23T12:56:25Z

Do you have an example project I could use to test this?

nicknovitski · 2018-10-24T03:59:08Z

That's a great idea, I'll try to find one or set one up.

It's deprecated, and no longer prints the urls of any xml files for the script to parse.

Manually corrected three problems: - fetch.sh: android cli cannot find java executable - addons.nix: google_apis_25 expression not added correctly - addons.nix: android_support_extra expression deleted (replaced in android 9 with jetpack/androidx)

nicknovitski · 2018-10-30T20:09:20Z

@Mic92 I made these changes to create expo/expo#2506; you could use that PR's branch as a test.

After checking it out, you can look at android/shell.nix, which uses the updated sdk, the modified ndk, and the new build-tools versions argument. The command nix-shell --run make-release.sh should run a successful gradle build (and then error because you don't have our app signing keys).

I've also been able to get owncloud/android building with this shell.nix:

let
  nixsrc = builtins.fetchTarball {
    url = "https://github.com/nicknovitski/nixpkgs/archive/android-sdk-26-licenses.tar.gz";
    sha256 = "1c52sksgla3fprb2xrw47992jx3lphcr5wqs51xnl607dimflxgm";
  };

  nixpkgs = import nixsrc {
    config = {
      allowUnfree = true;
      android_sdk.accept_license = true;
    };
  };

  sdk = nixpkgs.androidenv.androidsdk {
     platformVersions = [ "26" ];
     buildToolsVersions = [ "27.0.3" ];
     abiVersions = [ "x86" "x86_64"];
     useGoogleAPIs = true;
   };
   sdk_root = "${sdk}/libexec";

in

nixpkgs.mkShell {

  LANG="en_US.UTF-8";
  JAVA_HOME=nixpkgs.openjdk8;
  ANDROID_SDK_ROOT=sdk_root;

  nativeBuildInputs = with nixpkgs;[
    nodejs-8_x
    openjdk8
    sdk
  ];
}

...and the command nix-shell --pure --run "./gradlew assembleDebug && ./gradlew :testDebug"

Mic92 · 2018-10-30T21:04:01Z

I had to set ANDROID_HOME instead of ANDROID_SDK_ROOT, but it worked out in the end:

 with import <nixpkgs> {
    config = {
      allowUnfree = true;
      android_sdk.accept_license = true;
    };
  };

  let
    sdk = androidenv.androidsdk {
       platformVersions = [ "26" ];
       buildToolsVersions = [ "27.0.3" ];
       abiVersions = [ "x86" "x86_64"];
       useGoogleAPIs = true;
     };
  in mkShell {
    LANG="en_US.UTF-8";
    ANDROID_HOME="${sdk}/libexec";

    nativeBuildInputs = [
      glibcLocales
      nodejs-8_x
      openjdk8
      sdk
    ];
  }

I wonder if we should add this an example like this into nixpkgs or in the wiki?

myguidingstar · 2018-10-31T13:20:11Z

this seems to work but the command emulator fails in my machine:

/nix/store/2gngv0lf60ibbgzp9wj54fsyncbbb0y3-android-sdk-26.1.1/bin/emulator: line 5: /nix/store/2gngv0lf60ibbgzp9wj54fsyncbbb0y3-android-sdk-26.1.1/libexec/tools/.emulator-wrapped: No such file or directory

can you have a look at that? @nicknovitski

nicknovitski · 2018-10-31T23:42:58Z

@myguidingstar Good catch, thank you. #49564

nicknovitski · 2018-10-31T23:45:10Z

@Mic92 The android infrastructure could definitely benefit from a section in the manual.

Mic92 · 2018-10-31T23:48:01Z

@nicknovitski I invite you to do that. If docbook is not your thing there is also markdown supported. (see python.md)

myguidingstar · 2018-11-01T04:07:46Z

@nicknovitski @Mic92 this pull request also breaks androidenv.emulateApp because older versions use /libexec/tools/android create avd to create a new virtual device. Now it yells Error: Flag '-t' is not valid for 'create avd'.
The command also says The "android" command is deprecated. and suggests android studio, sdkmanager and avdmanager instead.

myguidingstar · 2018-11-01T04:20:18Z

if I run avdmanager to create a virtual device myself, I get these warning.

Info: Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/build-tools/26.0.3/package.xml. Probably the SDK is read-only
Info: Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/android/m2repository/package.xml. Probably the SDK is read-only
Info: Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/android/support/package.xml. Probably the SDK is read-only
Info: Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/google/google_play_services/package.xml. Probably the SDK is read-only
Info: Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/google/m2repository/package.xml. Probably the SDK is read-only
Info: Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/platform-tools/package.xml. Probably the SDK is read-only
Info: Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/platforms/android-26/package.xml. Probably the SDK is read-only
Info: Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/sources/android-26/package.xml. Probably the SDK is read-only
Info: Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/system-images/android-26/x86_64/package.xml. Probably the SDK is read-only
Warning: Observed package id 'system-images;android-26;default;x86_64' in inconsistent location '/nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/system-images/android-26/x86_64' (Expected '/nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/system-images/android-26/default/x86_64')
Parsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/build-tools/26.0.3Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/build-tools/26.0.3/package.xml. Probably the SDK is read-onlyParsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/android/m2repositoryException while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/android/m2repository/package.xml. Probably the SDK is read-onlyParsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/android/supportException while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/android/support/package.xml. Probably the SDK is read-onlyParsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/google/google_play_servicesException while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/google/google_play_services/package.xml. Probably the SDK is read-onlyParsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/google/m2repositoryException while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/extras/google/m2repository/package.xml. Probably the SDK is read-onlyParsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/platform-toolsException while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/platform-tools/package.xml. Probably the SDK is read-onlyParsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/platforms/android-26Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/platforms/android-26/package.xml. Probably the SDK is read-onlyParsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/sources/android-26Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/sources/android-26/package.xml. Probably the SDK is read-onlyParsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/supportParsing legacy package: /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/system-images/android-26/x86_64Exception while marshalling /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/system-images/android-26/x86_64/package.xml. Probably the SDK is read-onlyParsing /nix/store/dfv8mxsjfh4mghawqrpa31a28b5wwdns-android-sdk-26.1.1/libexec/tools/package.xmlAuto-selecting single ABI x86_64

It also says Copying files then null but no virtual device is created.

myguidingstar · 2018-11-01T10:28:31Z

I also got this error when trying to run emulator:

[140623058774912]:ERROR:android/android-emu/android/qt/qt_setup.cpp:28:Qt library not found at /nix/store/ks3zbnxm0kaq1js6cdzxi1ji3q52h6r7-android-sdk-26.1.1/emulator/lib64/qt/lib
Could not launch '/nix/store/ks3zbnxm0kaq1js6cdzxi1ji3q52h6r7-android-sdk-26.1.1/emulator/qemu/linux-x86_64/qemu-system-x86_64': No such file or directory

Mic92 · 2018-11-01T10:35:20Z

@myguidingstar this will be fixed with: #49515

myguidingstar · 2018-11-01T10:49:16Z

@Mic92 actually I've picked that patch, and the above warnings and errors are made with that branch

Mic92 · 2018-11-01T11:37:09Z

Ah right. Did not pay enough attention. autoPatchelfHook might be a better fix since it will automatically detect missing libraries at build time.

svanderburg · 2018-11-01T13:57:39Z

Hi,

I am working on a revised implementation of the Android SDK that fixes most of our current problems, and provides several kinds of new facilities that make maintenance a lot easier.

More information can be found here: https://discourse.nixos.org/t/my-revised-experimental-android-build-functions/923

My revised implementation lives in a separate repository, but I intend to soon work on a reintegration strategy.

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Oct 22, 2018

nicknovitski force-pushed the android-sdk-26-licenses branch 2 times, most recently from 7c86095 to 8447d64 Compare October 23, 2018 18:41

nicknovitski force-pushed the android-sdk-26-licenses branch from 0e70328 to fbf0d79 Compare October 24, 2018 22:02

nicknovitski mentioned this pull request Oct 25, 2018

Add android/{shell.nix,nix-global-sdk.sh,.envrc} expo/expo#2506

Merged

nicknovitski added 2 commits October 25, 2018 11:56

androidsdk: 25.2.5 -> 26.1.1, license agreement

950608b

androidenv.buildtools: 28.0.2 -> 28.0.3

ad501e4

nicknovitski force-pushed the android-sdk-26-licenses branch from 5da7ccb to c954ae0 Compare October 26, 2018 17:34

nicknovitski added 5 commits October 26, 2018 20:45

androidenv.buildTools: patch hard-coded /bin/ls paths

f5c2960

androidenv.androidndk: patch prebuilt 64bit binaries

ec8f320

androidenv: remove android command from fetch.sh

6e07691

It's deprecated, and no longer prints the urls of any xml files for the script to parse.

androidenv: run ./fetch.sh

a61f31c

Manually corrected three problems: - fetch.sh: android cli cannot find java executable - addons.nix: google_apis_25 expression not added correctly - addons.nix: android_support_extra expression deleted (replaced in android 9 with jetpack/androidx)

androidenv: add androidsdk_{8_1,9_0}{,_extras}

4a0ae5d

nicknovitski force-pushed the android-sdk-26-licenses branch from c954ae0 to 413733f Compare October 27, 2018 04:10

nicknovitski changed the title ~~Update android sdk and build tools, add license config option~~ Update android sdk, add license config option and selectable build tools Oct 27, 2018

androidenv: version buildTools and add buildToolsVersions sdk argument

d861da1

nicknovitski force-pushed the android-sdk-26-licenses branch from 413733f to d861da1 Compare October 27, 2018 05:10

matthewbauer approved these changes Oct 28, 2018

View reviewed changes

Mic92 merged commit 0c1ee96 into NixOS:master Oct 30, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update android sdk, add license config option and selectable build tools #48848

Update android sdk, add license config option and selectable build tools #48848

nicknovitski commented Oct 22, 2018

Mic92 commented Oct 23, 2018

nicknovitski commented Oct 24, 2018

nicknovitski commented Oct 30, 2018

Mic92 commented Oct 30, 2018 •

edited

myguidingstar commented Oct 31, 2018

nicknovitski commented Oct 31, 2018

nicknovitski commented Oct 31, 2018

Mic92 commented Oct 31, 2018

myguidingstar commented Nov 1, 2018

myguidingstar commented Nov 1, 2018

myguidingstar commented Nov 1, 2018

Mic92 commented Nov 1, 2018

myguidingstar commented Nov 1, 2018

Mic92 commented Nov 1, 2018

svanderburg commented Nov 1, 2018

Update android sdk, add license config option and selectable build tools #48848

Update android sdk, add license config option and selectable build tools #48848

Conversation

nicknovitski commented Oct 22, 2018

Mic92 commented Oct 23, 2018

nicknovitski commented Oct 24, 2018

nicknovitski commented Oct 30, 2018

Mic92 commented Oct 30, 2018 • edited

myguidingstar commented Oct 31, 2018

nicknovitski commented Oct 31, 2018

nicknovitski commented Oct 31, 2018

Mic92 commented Oct 31, 2018

myguidingstar commented Nov 1, 2018

myguidingstar commented Nov 1, 2018

myguidingstar commented Nov 1, 2018

Mic92 commented Nov 1, 2018

myguidingstar commented Nov 1, 2018

Mic92 commented Nov 1, 2018

svanderburg commented Nov 1, 2018

Mic92 commented Oct 30, 2018 •

edited