Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python37: init at 3.7.0b1 #34231

Closed
wants to merge 1 commit into from
Closed

python37: init at 3.7.0b1 #34231

wants to merge 1 commit into from

Conversation

FRidh
Copy link
Member

@FRidh FRidh commented Jan 24, 2018

Beta version of Python 3.7.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

The current version builds and works. Even though, we should:

  • use hash-based bytecode PEP 552
  • check the UTF-8 changes

@Mic92
Copy link
Member

Mic92 commented Jan 24, 2018

Finally we can remove many encoding work-arounds in checkPhase: https://docs.python.org/3.7/whatsnew/3.7.html#pep-538-legacy-c-locale-coercion

Also great for my own projects and sanity.


postPatch = ''
# Determinism
substituteInPlace "Lib/py_compile.py" --replace "source_stats['mtime']" "(1 if 'DETERMINISTIC_BUILD' in os.environ else source_stats['mtime'])"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not go through pep-0552 intensively, but could not use the fact that it offers the possibility to have timestamp-less .pyc files and replace this hack?

something like:

substituteInPlace "Lib/py_compile.py" --replace "invalidation_mode=PycInvalidationMode.TIMESTAMP):" "invalidation_mode=PycInvalidationMode.CHECKED_HASH):"

and remove DETERMINISTIC_BUILD env var ? (this wold require some testing…)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, that's on the todo list. We will need to modify some of this patching.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

substituteInPlace "Lib/py_compile.py" --replace "invalidation_mode=PycInvalidationMode.TIMESTAMP):" "invalidation_mode=(PycInvalidationMode.CHECKED_HASH if 'DETERMINISTIC_BUILD' in os.environ else PycInvalidationMode.TIMESTAMP)):"

would probably be better

@FRidh FRidh changed the title python37: init at 3.7.0a4 python37: init at 3.7.0b1 Feb 3, 2018
@FRidh FRidh added this to the 18.03 milestone Feb 25, 2018
@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: python37

Partial log (click to expand)

strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/7yyxibzvdanji3hc2471c2fqj8z0wqfy-python3-3.7.0b4/lib  /nix/store/7yyxibzvdanji3hc2471c2fqj8z0wqfy-python3-3.7.0b4/bin
patching script interpreter paths in /nix/store/7yyxibzvdanji3hc2471c2fqj8z0wqfy-python3-3.7.0b4
/nix/store/7yyxibzvdanji3hc2471c2fqj8z0wqfy-python3-3.7.0b4/bin/python3.7m-config: interpreter directive changed from "/bin/sh" to "/nix/store/xn5gv3lpfy91yvfy9b0i7klfcxh9xskz-bash-4.4-p19/bin/sh"
/nix/store/7yyxibzvdanji3hc2471c2fqj8z0wqfy-python3-3.7.0b4/lib/python3.7/config-3.7m-x86_64-linux-gnu/makesetup: interpreter directive changed from " /bin/sh" to "/nix/store/xn5gv3lpfy91yvfy9b0i7klfcxh9xskz-bash-4.4-p19/bin/sh"
/nix/store/7yyxibzvdanji3hc2471c2fqj8z0wqfy-python3-3.7.0b4/lib/python3.7/config-3.7m-x86_64-linux-gnu/install-sh: interpreter directive changed from "/bin/sh" to "/nix/store/xn5gv3lpfy91yvfy9b0i7klfcxh9xskz-bash-4.4-p19/bin/sh"
/nix/store/7yyxibzvdanji3hc2471c2fqj8z0wqfy-python3-3.7.0b4/lib/python3.7/ctypes/macholib/fetch_macholib: interpreter directive changed from "/bin/sh" to "/nix/store/xn5gv3lpfy91yvfy9b0i7klfcxh9xskz-bash-4.4-p19/bin/sh"
checking for references to /build in /nix/store/7yyxibzvdanji3hc2471c2fqj8z0wqfy-python3-3.7.0b4...
wrong ELF type
/nix/store/7yyxibzvdanji3hc2471c2fqj8z0wqfy-python3-3.7.0b4

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: python37

Partial log (click to expand)

    pid = forkpty(&master_fd, NULL, NULL, NULL);
          ^
./Modules/posixmodule.c:6234:11: warning: this function declaration is not a prototype [-Wstrict-prototypes]
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration   -I. -I./Include -I/nix/store/byjz3zsj2397x4bllp4axqcnc517jdl7-zlib-1.2.11-dev/include -I/nix/store/qggzkch99ab8dnzyv1dflvl97mqxqcd2-bzip2-1.0.6.0.1-dev/include -I/nix/store/lszb9xprsaj0qbb61i2603yhags07x3m-expat-2.2.5-dev/include -I/nix/store/gls4179iw8xlrvm3qwr3cwzpgy5rdshf-xz-5.2.3-dev/include -I/nix/store/si0g5b52xhrpdd8xjd67xdpskfgfscid-libffi-3.2.1-dev/include -I/nix/store/33pspxxdp0jc0q1hd4k629mpxi39f4f4-gdbm-1.14.1/include -I/nix/store/6xmmwfkbb1g0g420ndjr6bjd5sxx9wgf-sqlite-3.22.0-dev/include -I/nix/store/nblby1iqln9grw4yb03fmprch4k0kv48-readline-6.3p08-dev/include -I/nix/store/926k3c6zc5i7xv6f2fk15p408cs5hqld-ncurses-6.1-dev/include -I/nix/store/pf16lx06p07zklkdra52p7b5akpd35qj-openssl-1.0.2o-dev/include -I/nix/store/fgbq3pqgw4xqifc75wmvsz6vrs0z2pk1-CF-osx-10.10.5/include -I/nix/store/2yipfqi0jkvfnk69rih2dzccf01ly1jw-configd-osx-10.8.5/include -I/nix/store/byjz3zsj2397x4bllp4axqcnc517jdl7-zlib-1.2.11-dev/include -I/nix/store/qggzkch99ab8dnzyv1dflvl97mqxqcd2-bzip2-1.0.6.0.1-dev/include -I/nix/store/lszb9xprsaj0qbb61i2603yhags07x3m-expat-2.2.5-dev/include -I/nix/store/gls4179iw8xlrvm3qwr3cwzpgy5rdshf-xz-5.2.3-dev/include -I/nix/store/si0g5b52xhrpdd8xjd67xdpskfgfscid-libffi-3.2.1-dev/include -I/nix/store/33pspxxdp0jc0q1hd4k629mpxi39f4f4-gdbm-1.14.1/include -I/nix/store/6xmmwfkbb1g0g420ndjr6bjd5sxx9wgf-sqlite-3.22.0-dev/include -I/nix/store/nblby1iqln9grw4yb03fmprch4k0kv48-readline-6.3p08-dev/include -I/nix/store/926k3c6zc5i7xv6f2fk15p408cs5hqld-ncurses-6.1-dev/include -I/nix/store/pf16lx06p07zklkdra52p7b5akpd35qj-openssl-1.0.2o-dev/include -I/nix/store/fgbq3pqgw4xqifc75wmvsz6vrs0z2pk1-CF-osx-10.10.5/include -I/nix/store/2yipfqi0jkvfnk69rih2dzccf01ly1jw-configd-osx-10.8.5/include   -c ./Modules/_sre.c -o Modules/_sre.o
2 warnings and 2 errors generated.
make: *** [Makefile:1793: Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
builder for '/nix/store/9jp2akl2w46lscky4d0g6vrj2fg2ijsz-python3-3.7.0b4.drv' failed with exit code 2
error: build of '/nix/store/9jp2akl2w46lscky4d0g6vrj2fg2ijsz-python3-3.7.0b4.drv' failed

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: python37

Partial log (click to expand)

strip is /nix/store/gp7fylxwn18b7pl2c18ks89hsiaxyfvf-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/zj5zn76a37p2mihs2r4yy7zccfrvvcml-python3-3.7.0b4/lib  /nix/store/zj5zn76a37p2mihs2r4yy7zccfrvvcml-python3-3.7.0b4/bin
patching script interpreter paths in /nix/store/zj5zn76a37p2mihs2r4yy7zccfrvvcml-python3-3.7.0b4
/nix/store/zj5zn76a37p2mihs2r4yy7zccfrvvcml-python3-3.7.0b4/lib/python3.7/config-3.7m-aarch64-linux-gnu/install-sh: interpreter directive changed from "/bin/sh" to "/nix/store/a4qslf0yfs44mhnbagarn232sjnckhy0-bash-4.4-p19/bin/sh"
/nix/store/zj5zn76a37p2mihs2r4yy7zccfrvvcml-python3-3.7.0b4/lib/python3.7/config-3.7m-aarch64-linux-gnu/makesetup: interpreter directive changed from " /bin/sh" to "/nix/store/a4qslf0yfs44mhnbagarn232sjnckhy0-bash-4.4-p19/bin/sh"
/nix/store/zj5zn76a37p2mihs2r4yy7zccfrvvcml-python3-3.7.0b4/lib/python3.7/ctypes/macholib/fetch_macholib: interpreter directive changed from "/bin/sh" to "/nix/store/a4qslf0yfs44mhnbagarn232sjnckhy0-bash-4.4-p19/bin/sh"
/nix/store/zj5zn76a37p2mihs2r4yy7zccfrvvcml-python3-3.7.0b4/bin/python3.7m-config: interpreter directive changed from "/bin/sh" to "/nix/store/a4qslf0yfs44mhnbagarn232sjnckhy0-bash-4.4-p19/bin/sh"
checking for references to /build in /nix/store/zj5zn76a37p2mihs2r4yy7zccfrvvcml-python3-3.7.0b4...
wrong ELF type
/nix/store/zj5zn76a37p2mihs2r4yy7zccfrvvcml-python3-3.7.0b4

@jtojnar
Copy link
Contributor

jtojnar commented Jun 2, 2018

Would not this still need C.UTF-8 locale? #20192

@FRidh
Copy link
Member Author

FRidh commented Jun 2, 2018

Would not this still need C.UTF-8 locale? #20192

https://docs.python.org/3.7/whatsnew/3.7.html#pep-540-forced-utf-8-runtime-mode

When in UTF-8 mode, CPython ignores the locale settings, and uses the UTF-8 encoding by default. The error handlers for sys.stdin and sys.stdout streams are set to surrogateescape.

The forced UTF-8 mode can be used to change the text handling behavior in an embedded Python interpreter without changing the locale settings of an embedding application.

The UTF-8 mode is enabled by default when the locale is “C”. See PEP 538: Legacy C Locale Coercion for details.

So, yes. While your workstation typically has a UTF locale configured, this is or may not be the case on say servers or inside services (using C).

Beta version of Python 3.7.
@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: python37

Partial log (click to expand)

    if (openpty(&master_fd, &slave_fd, NULL, NULL, NULL) != 0)
        ^
./Modules/posixmodule.c:6234:11: error: implicit declaration of function 'forkpty' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    pid = forkpty(&master_fd, NULL, NULL, NULL);
          ^
./Modules/posixmodule.c:6234:11: warning: this function declaration is not a prototype [-Wstrict-prototypes]
2 warnings and 2 errors generated.
make: *** [Makefile:1793: Modules/posixmodule.o] Error 1
builder for '/nix/store/7lmwg1sb2b0gvkg5xmam08kx2sjcx8s5-python3-3.7.0b4.drv' failed with exit code 2
error: build of '/nix/store/7lmwg1sb2b0gvkg5xmam08kx2sjcx8s5-python3-3.7.0b4.drv' failed

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: python37

Partial log (click to expand)

gzipping man pages under /nix/store/ivymxqqgw85l74xrx28lbnmmp7w0ww4w-python3-3.7.0b4/share/man/
strip is /nix/store/21ymadblbmsbb2bk4q7gl4kjasp8zmgd-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/ivymxqqgw85l74xrx28lbnmmp7w0ww4w-python3-3.7.0b4/lib  /nix/store/ivymxqqgw85l74xrx28lbnmmp7w0ww4w-python3-3.7.0b4/bin
patching script interpreter paths in /nix/store/ivymxqqgw85l74xrx28lbnmmp7w0ww4w-python3-3.7.0b4
/nix/store/ivymxqqgw85l74xrx28lbnmmp7w0ww4w-python3-3.7.0b4/bin/python3.7m-config: interpreter directive changed from "/bin/sh" to "/nix/store/qckzjk3406va7h6s40cy9s75z2w715rq-bash-4.4-p19/bin/sh"
/nix/store/ivymxqqgw85l74xrx28lbnmmp7w0ww4w-python3-3.7.0b4/lib/python3.7/ctypes/macholib/fetch_macholib: interpreter directive changed from "/bin/sh" to "/nix/store/qckzjk3406va7h6s40cy9s75z2w715rq-bash-4.4-p19/bin/sh"
/nix/store/ivymxqqgw85l74xrx28lbnmmp7w0ww4w-python3-3.7.0b4/lib/python3.7/config-3.7m-x86_64-linux-gnu/makesetup: interpreter directive changed from " /bin/sh" to "/nix/store/qckzjk3406va7h6s40cy9s75z2w715rq-bash-4.4-p19/bin/sh"
/nix/store/ivymxqqgw85l74xrx28lbnmmp7w0ww4w-python3-3.7.0b4/lib/python3.7/config-3.7m-x86_64-linux-gnu/install-sh: interpreter directive changed from "/bin/sh" to "/nix/store/qckzjk3406va7h6s40cy9s75z2w715rq-bash-4.4-p19/bin/sh"
checking for references to /build in /nix/store/ivymxqqgw85l74xrx28lbnmmp7w0ww4w-python3-3.7.0b4...
wrong ELF type

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: python37

Partial log (click to expand)

strip is /nix/store/qg2agrqkf240s656d207zqhipl0bc2id-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/2w9s10p7rn3dcp916khc08hj5lsgcf4x-python3-3.7.0b4/lib  /nix/store/2w9s10p7rn3dcp916khc08hj5lsgcf4x-python3-3.7.0b4/bin
patching script interpreter paths in /nix/store/2w9s10p7rn3dcp916khc08hj5lsgcf4x-python3-3.7.0b4
/nix/store/2w9s10p7rn3dcp916khc08hj5lsgcf4x-python3-3.7.0b4/lib/python3.7/config-3.7m-aarch64-linux-gnu/install-sh: interpreter directive changed from "/bin/sh" to "/nix/store/9rv2bzx7jz6wjbaxij5qq580qvzksj4i-bash-4.4-p19/bin/sh"
/nix/store/2w9s10p7rn3dcp916khc08hj5lsgcf4x-python3-3.7.0b4/lib/python3.7/config-3.7m-aarch64-linux-gnu/makesetup: interpreter directive changed from " /bin/sh" to "/nix/store/9rv2bzx7jz6wjbaxij5qq580qvzksj4i-bash-4.4-p19/bin/sh"
/nix/store/2w9s10p7rn3dcp916khc08hj5lsgcf4x-python3-3.7.0b4/lib/python3.7/ctypes/macholib/fetch_macholib: interpreter directive changed from "/bin/sh" to "/nix/store/9rv2bzx7jz6wjbaxij5qq580qvzksj4i-bash-4.4-p19/bin/sh"
/nix/store/2w9s10p7rn3dcp916khc08hj5lsgcf4x-python3-3.7.0b4/bin/python3.7m-config: interpreter directive changed from "/bin/sh" to "/nix/store/9rv2bzx7jz6wjbaxij5qq580qvzksj4i-bash-4.4-p19/bin/sh"
checking for references to /build in /nix/store/2w9s10p7rn3dcp916khc08hj5lsgcf4x-python3-3.7.0b4...
wrong ELF type
/nix/store/2w9s10p7rn3dcp916khc08hj5lsgcf4x-python3-3.7.0b4

@xeji
Copy link
Contributor

xeji commented Jun 29, 2018

@FRidh I believe this is obsolete since 7261a66

@FRidh
Copy link
Member Author

FRidh commented Jul 11, 2018

Yep, closing.

@FRidh FRidh closed this Jul 11, 2018
@FRidh FRidh deleted the python37 branch July 11, 2018 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants