aria2: refactor to avoid python2 dependency #101978

braunse · 2020-10-28T22:12:02Z

Use python3 Sphinx package instead of python2 Sphinx package,
to avoid build-time dependency on python 2.

Motivation for this change

To remove the build dependency on python2, as per #101964

Things done

Use python3 Sphinx package instead of python2 Sphinx package, to avoid build-time dependency on python 2, as per NixOS#101964

SuperSandro2000 · 2020-10-28T22:18:51Z

Result of nixpkgs-review pr 101978 run on x86_64-linux 1

6 packages failed to build:

aria
persepolis
python37Packages.aria2p
python38Packages.aria2p
uget
uget-integrator

From aria.log:

Can't exec "/bin/sh": No such file or directory at /nix/store/38vxlxrvg3yji3jms44qn94lxdysbj5j-perl-5.32.0/lib/perl5/5.32.0/x86_64-linux-thread-multi/IO/File.pm line 186.

braunse · 2020-10-28T22:26:34Z

Hmmm.... Here's what happens for me, on Debian if that matters:

❯ nix-shell -p nixpkgs-review --run "nixpkgs-review pr 101978"
$ git -c fetch.prune=false fetch --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/101978/head:refs/nixpkgs-review/1
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 10 (delta 5), reused 7 (delta 5), pack-reused 0
Unpacking objects: 100% (10/10), 3.20 KiB | 252.00 KiB/s, done.
From https://github.com/NixOS/nixpkgs
   0dc7ad6f1fc..4a64cdd39ba  master                -> refs/nixpkgs-review/0
 * [new ref]                 refs/pull/101978/head -> refs/nixpkgs-review/1
$ git worktree add /home/seb/.cache/nixpkgs-review/pr-101978/nixpkgs 4a64cdd39ba5efc692205df8f15bc0924c5e0bf1
Preparing worktree (detached HEAD 4a64cdd39ba)
Updating files: 100% (23067/23067), done.
HEAD is now at 4a64cdd39ba Merge pull request #101952 from mweinelt/p10k
$ nix-env -f /home/seb/.cache/nixpkgs-review/pr-101978/nixpkgs -qaP --xml --out-path --show-trace
$ git merge --no-commit aa8525a8ec73d832b0f7b2f1a0f0fe0d670c2f0d
Automatic merge went well; stopped before committing as requested
$ nix-env -f /home/seb/.cache/nixpkgs-review/pr-101978/nixpkgs -qaP --xml --out-path --show-trace --meta
6 packages updated:
aria2 persepolis python3.7-aria2p python3.8-aria2p uget uget-integrator

$ nix build --no-link --keep-going --option build-use-sandbox relaxed -f /home/seb/.cache/nixpkgs-review/pr-101978/build.nix
[6 built, 482 copied (1316.6 MiB), 293.5 MiB DL]
https://github.com/NixOS/nixpkgs/pull/101978
6 packages built:
aria persepolis python37Packages.aria2p python38Packages.aria2p uget uget-integrator

zopieux · 2020-10-28T23:01:50Z

Tested working on x86_64 Archlinux starting with an empty /nix directory. Managed to download a magnet torrent with the built aria2c.

Logs

$ sudo mount -t tmpfs -o size=20G tmpfs /nix && curl -L https://nixos.org/nix/install | sh && source ~/.nix-profile/etc/profile.d/nix.sh
$ nix-shell -p nixpkgs-review
$ nixpkgs-review pr 101978
(snip)
[7 built, 490 copied (1341.5 MiB), 297.0 MiB DL]
https://github.com/NixOS/nixpkgs/pull/101978
1 package marked as broken and skipped:
kiwix

6 packages built:
aria persepolis python37Packages.aria2p python38Packages.aria2p uget uget-integrator

$ aria2c 'magnet:?xt=urn:btih:dd8255ecdc7ca55fb0bbf81323d87062db1f6d1c&dn=Big+Buck+Bunny&tr=udp%3A%2F%2Fexplodie.org%3A6969&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Ftracker.empire-js.us%3A1337&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337&tr=wss%3A%2F%2Ftracker.btorrent.xyz&tr=wss%3A%2F%2Ftracker.fastcast.nz&tr=wss%3A%2F%2Ftracker.openwebtorrent.com&ws=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2F&xs=https%3A%2F%2Fwebtorrent.io%2Ftorrents%2Fbig-buck-bunny.torrent'

10/28 23:54:45 [NOTICE] Downloading 1 item(s)
10/28 23:54:45 [NOTICE] IPv4 DHT: listening on UDP port 6960
10/28 23:54:45 [NOTICE] IPv4 BitTorrent: listening on TCP port 6909
10/28 23:54:45 [NOTICE] IPv6 BitTorrent: listening on TCP port 6909
[#e226df 20KiB/20KiB(100%) CN:20 SD:1]                                                                                                                                                                                                                                                                                      
10/28 23:54:55 [NOTICE] Download complete: [MEMORY][METADATA]Big+Buck+Bunny
[#a6cd3e SEED(0.0) CN:1 SD:0]                                                                                                                                                                                                                                                                                               
10/28 23:55:40 [NOTICE] Shutdown sequence commencing... Press Ctrl-C again for emergency shutdown.
10/28 23:55:40 [NOTICE] Download complete: Big Buck Bunny
10/28 23:55:40 [NOTICE] Your share ratio was 0.0, uploaded/downloaded=0B/263MiB

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
e226df|OK  |       0B/s|[MEMORY][METADATA]Big+Buck+Bunny
a6cd3e|OK  |    11MiB/s|Big Buck Bunny/Big Buck Bunny.en.srt (2more)

Status Legend:
(OK):download completed.

SuperSandro2000 · 2020-10-29T01:18:07Z

Could it be that my fail is related to an error in the perl package rather than aria2?

braunse · 2020-10-29T07:49:12Z

I tested it again on a freshly installed NixOS 20.09 VM, and could not reproduce @SuperSandro2000's problem.

Logs:

[seb@alfons-nixdev:~/dev/nixpkgs]$ nix-shell -p nixpkgs-review --run "nixpkgs-review pr 101978"
$ git -c fetch.prune=false fetch --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/101978/head:refs/nixpkgs-review/1
$ git worktree add /home/seb/.cache/nixpkgs-review/pr-101978-1/nixpkgs df6068e934ad2ea8db02609c335b23ac96c3d286
Preparing worktree (detached HEAD df6068e934a)
Updating files: 100% (23069/23069), done.
HEAD is now at df6068e934a Merge pull request #101985 from rnhmjoj/monero
$ git merge --no-commit aa8525a8ec73d832b0f7b2f1a0f0fe0d670c2f0d
Auto-merging pkgs/top-level/all-packages.nix
Automatic merge went well; stopped before committing as requested
$ nix build --no-link --keep-going --option build-use-sandbox relaxed -f /home/seb/.cache/nixpkgs-review/pr-101978-1/build.nix
[7 built, 558 copied (1807.6 MiB), 397.7 MiB DL]
https://github.com/NixOS/nixpkgs/pull/101978
1 package marked as broken and skipped:
kiwix

6 packages built:
aria persepolis python37Packages.aria2p python38Packages.aria2p uget uget-integrator

[0.0 MiB DL]
$ nix-shell /home/seb/.cache/nixpkgs-review/pr-101978-1/shell.nix
these paths will be fetched (0.51 MiB download, 2.47 MiB unpacked):
  /nix/store/82bsy6f0bz7avmy8y62j0z2dnxgi8pxw-bash-interactive-4.4-p23-doc
  /nix/store/b6mp5xw5pjr3bsmk3fphkyxw73vc806j-bash-interactive-4.4-p23-info
  /nix/store/hfvjs6abgzadrv9f387v1ip5mr206jh9-bash-interactive-4.4-p23-dev
  /nix/store/xpmrr7kyzlbny0540kzd1vlqsca69y13-bash-interactive-4.4-p23-man
copying path '/nix/store/82bsy6f0bz7avmy8y62j0z2dnxgi8pxw-bash-interactive-4.4-p23-doc' from 'https://cache.nixos.org'...
copying path '/nix/store/xpmrr7kyzlbny0540kzd1vlqsca69y13-bash-interactive-4.4-p23-man' from 'https://cache.nixos.org'...
copying path '/nix/store/hfvjs6abgzadrv9f387v1ip5mr206jh9-bash-interactive-4.4-p23-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/b6mp5xw5pjr3bsmk3fphkyxw73vc806j-bash-interactive-4.4-p23-info' from 'https://cache.nixos.org'...

[nix-shell:~/.cache/nixpkgs-review/pr-101978-1]$ aria2
aria2c  aria2p  

[nix-shell:~/.cache/nixpkgs-review/pr-101978-1]$ aria2c http://google.com

10/29 08:45:40 [NOTICE] Downloading 1 item(s)

10/29 08:45:40 [NOTICE] CUID#7 - Redirecting to http://www.google.com/

10/29 08:45:40 [NOTICE] Download complete: /home/seb/.cache/nixpkgs-review/pr-101978-1/index.html

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
18ab42|OK  |    71KiB/s|/home/seb/.cache/nixpkgs-review/pr-101978-1/index.html

Status Legend:
(OK):download completed.

[nix-shell:~/.cache/nixpkgs-review/pr-101978-1]$ exit
$ git worktree prune

SuperSandro2000 · 2020-10-29T09:13:45Z

Result of nixpkgs-review pr 101978 run on x86_64-linux 1

1 package marked as broken and skipped:

kiwix

2 packages failed to build:

uget
uget-integrator

4 packages built:

aria (aria2)
persepolis
python37Packages.aria2p
python38Packages.aria2p

configuring
configure flags: --disable-dependency-tracking --prefix=/nix/store/pjsfmk5gq3n7vr17znhjrzxmf4d08hra-uget-2.2.3-1
/nix/store/q1zjp9grl4w92qalkdqjs2bj5d0pf8ih-stdenv-linux/setup: ./configure: /bin/sh: bad interpreter: No such file or directory

Uget depends on Aria and would be broken with this PR https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/networking/uget/default.nix#L2. Please fix.

braunse · 2020-10-29T09:40:48Z

AFAICT, there should be nothing in this PR that relates to whether uget's configure script can find /bin/sh in any way, so I'm unsure how to go about fixing this. Any hints would be appreciated.

FWIW, uget builds fine for me with this PR, both on my Debian machine with Nix installed and on a NixOS 20.09 VM. @SuperSandro2000 can you check that you are able to build uget on the master branch without this PR?

Update: I have a fix for this in https://github.com/braunse/nixpkgs/tree/patch-configure-shebang-in-uget.
Since the fix (patchShebangs . in preConfigure) seems unrelated to aria2, I'm unsure whether I should add it to this PR, or open a fresh PR?

To fix build failure referenced in NixOS#101978 (comment)

SuperSandro2000 · 2020-10-29T10:59:13Z

Result of nixpkgs-review pr 101978 run on x86_64-linux 1

1 package marked as broken and skipped:

kiwix

1 package failed to build:

persepolis

5 packages built:

aria (aria2)
python37Packages.aria2p
python38Packages.aria2p
uget
uget-integrator

SuperSandro2000 · 2020-10-29T11:22:25Z

Uget gets pulled from cache for me so I can't check if it build or not but I suppose it should.

Edit: Now Uget builds after collecting nix store garbage. The persepolis build failure is unrelated to this PR.

error: can't copy 'man/persepolis.1.gz': doesn't exist or not a regular file

But this is fixed on master.

jonringer

LGTM

Result of nixpkgs-review pr 101978 1

1 package marked as broken and skipped:

kiwix

6 packages built:

aria (aria2)
persepolis
python37Packages.aria2p
python38Packages.aria2p
uget
uget-integrator

aria2: refactor to avoid python2 dependency

aa8525a

Use python3 Sphinx package instead of python2 Sphinx package, to avoid build-time dependency on python 2, as per NixOS#101964

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Oct 28, 2020

braunse added a commit to braunse/nixpkgs that referenced this pull request Oct 29, 2020

uget: Patch /bin/sh reference in configure

1a7fcb8

To fix build failure referenced in NixOS#101978 (comment)

SuperSandro2000 approved these changes Oct 29, 2020

View reviewed changes

jonringer approved these changes Oct 29, 2020

View reviewed changes

jonringer merged commit 8bdb520 into NixOS:master Oct 29, 2020

braunse deleted the update-aria2-python2-dep branch October 30, 2020 08:13

Janik-Haag added the 12. first-time contribution label Jun 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

aria2: refactor to avoid python2 dependency #101978

aria2: refactor to avoid python2 dependency #101978

braunse commented Oct 28, 2020 •

edited

SuperSandro2000 commented Oct 28, 2020 •

edited

braunse commented Oct 28, 2020

zopieux commented Oct 28, 2020

SuperSandro2000 commented Oct 29, 2020

braunse commented Oct 29, 2020

SuperSandro2000 commented Oct 29, 2020 •

edited

braunse commented Oct 29, 2020 •

edited

SuperSandro2000 commented Oct 29, 2020

SuperSandro2000 commented Oct 29, 2020 •

edited

jonringer left a comment

aria2: refactor to avoid python2 dependency #101978

aria2: refactor to avoid python2 dependency #101978

Conversation

braunse commented Oct 28, 2020 • edited

Motivation for this change

Things done

SuperSandro2000 commented Oct 28, 2020 • edited

braunse commented Oct 28, 2020

zopieux commented Oct 28, 2020

SuperSandro2000 commented Oct 29, 2020

braunse commented Oct 29, 2020

SuperSandro2000 commented Oct 29, 2020 • edited

braunse commented Oct 29, 2020 • edited

SuperSandro2000 commented Oct 29, 2020

SuperSandro2000 commented Oct 29, 2020 • edited

jonringer left a comment

Choose a reason for hiding this comment

braunse commented Oct 28, 2020 •

edited

SuperSandro2000 commented Oct 28, 2020 •

edited

SuperSandro2000 commented Oct 29, 2020 •

edited

braunse commented Oct 29, 2020 •

edited

SuperSandro2000 commented Oct 29, 2020 •

edited