add pythonPackages.outcome, pythonPackages.trio #40191

catern · 2018-05-08T22:39:19Z

This pull request doesn't yet build. I have a separate pull request
for the first two packages in this series, since they build fine.

The problem starts at the "outcome" package. It fails with:

installing
/build/outcome-0.1.0a0/dist /build/outcome-0.1.0a0
Processing ./outcome-0.1.0a0-py2.py3-none-any.whl
Collecting attrs (from outcome==0.1.0a0)
  Could not find a version that satisfies the requirement attrs (from outcome==0.1.0a0) (from versions: )
No matching distribution found for attrs (from outcome==0.1.0a0)
builder for '/nix/store/2k15pq05a7iv97fy6p2yxzqd5hzlk5h8-python3.6-outcome-0.1.0a0.drv' failed with exit code 1

Even though attrs is in its propagatedBuildInputs... any ideas?

dotlambda · 2018-05-10T17:29:38Z

pkgs/development/python-modules/outcome/default.nix

+    sha256 = "0cqwakzigw0602dxlb7c1882jwr8hn5nrxk1l8iwlmzc9whh48wn";
+  };
+
+  propogatedBuildInputs = [ attrs ];


propagatedBuildInputs

catern · 2018-05-10T20:19:21Z

OK, that's fixed now.

The next issue is that the "outcome" package has a test dependency on "trio", and "trio" has a normal dependency on "trio". Is there any way to resolve this without disabling tests? Maybe run test it in a joint environment? I also filed an issue upstream about it: python-trio/outcome#12

Furthermore, even if I disable tests for outcome, the trio tests also then fail with a cryptic message about pytest.config. (in next comment)

catern · 2018-05-10T20:20:25Z

running install tests
running test
running egg_info
writing trio.egg-info/PKG-INFO
writing dependency_links to trio.egg-info/dependency_links.txt
writing requirements to trio.egg-info/requires.txt
writing top-level names to trio.egg-info/top_level.txt
reading manifest file 'trio.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.pem' under directory 'trio/tests/test_ssl_certs'
no previously-included directories found matching 'docs/build'
writing manifest file 'trio.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "nix_run_setup", line 8, in <module>
    exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))
  File "setup.py", line 113, in <module>
    "Framework :: Trio",
  File "/nix/store/0082j66l4lm1klkpk09jf82iixb8dl18-python3.6-bootstrapped-pip-9.0.3/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/nix/store/0082j66l4lm1klkpk09jf82iixb8dl18-python3.6-bootstrapped-pip-9.0.3/lib/python3.6/site-packages/setuptools/command/test.py", line 226, in run
    self.run_tests()
  File "/nix/store/0082j66l4lm1klkpk09jf82iixb8dl18-python3.6-bootstrapped-pip-9.0.3/lib/python3.6/site-packages/setuptools/command/test.py", line 248, in run_tests
    exit=False,
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/unittest/main.py", line 118, in parseArgs
    self._do_discovery(argv[2:])
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/unittest/main.py", line 229, in _do_discovery
    self.test = loader.discover(self.start, self.pattern, self.top)
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/unittest/loader.py", line 341, in discover
    tests = list(self._find_tests(start_dir, pattern))
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/unittest/loader.py", line 398, in _find_tests
    full_path, pattern, namespace)
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/unittest/loader.py", line 475, in _find_test_path
    tests = self.loadTestsFromModule(package, pattern=pattern)
  File "/nix/store/0082j66l4lm1klkpk09jf82iixb8dl18-python3.6-bootstrapped-pip-9.0.3/lib/python3.6/site-packages/setuptools/command/test.py", line 52, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/unittest/loader.py", line 190, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "/nix/store/0082j66l4lm1klkpk09jf82iixb8dl18-python3.6-bootstrapped-pip-9.0.3/lib/python3.6/site-packages/setuptools/command/test.py", line 52, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/unittest/loader.py", line 153, in loadTestsFromName
    module = __import__(module_name)
  File "/build/trio-0.4.0/trio/tests/test_socket.py", line 8, in <module>
    from .._core.tests.tutil import need_ipv6
  File "/build/trio-0.4.0/trio/_core/tests/tutil.py", line 10, in <module>
    not pytest.config.getoption("--run-slow", True),
AttributeError: module 'pytest' has no attribute 'config'
builder for '/nix/store/vc6ih2pz2fppahba3yz36bdkgjm8fxxs-python3.6-trio-0.4.0.drv' failed with exit code 1
error: build of '/nix/store/vc6ih2pz2fppahba3yz36bdkgjm8fxxs-python3.6-trio-0.4.0.drv' failed

Other than that, it works:

sbaugh@sbaugh-sandbox ~/.local/src/nixpkgs $ nix run '(import ./. {}).python36.withPackages (ps: [ps.trio])' -c python -c 'import trio; trio.run(trio.sleep, 1); print("works!")'
works!

(don't merge yet though)

dotlambda · 2018-05-11T14:43:34Z

pkgs/development/python-modules/outcome/default.nix

+
+  checkInputs = [ pytest ];
+  propagatedBuildInputs = [ attrs ];
+  doCheck = false;


add a comment why

dotlambda · 2018-05-11T14:47:12Z

The error about pytest.config might be fixed if you set

checkPhase = ''
  py.test
'';

dotlambda · 2018-05-11T16:57:50Z

pkgs/development/python-modules/trio/default.nix

+    sha256 = "0ib1x47knlad9pljb64ywfiv6m3dfrqqjwka6j1b73hixmszb5h4";
+  };
+
+  checkInputs = [ pytest pyopenssl trustme ];


You're sure the last two are only needed for testing?

Yes, they're only listed in test-requirements.txt, not in setup.py; and I don't believe there's any place that the trio library would use them.

dotlambda · 2018-05-11T16:58:47Z

@GrahamcOfBorg build python3.pkgs.trio

GrahamcOfBorg · 2018-05-11T16:59:36Z

Failure on x86_64-linux (full log)

Attempted: python3.pkgs.trio

Partial log (click to expand)


            """
        try:
>           return Value(sync_fn(*args))
E           OSError: protocol not found

trio/_core/_result.py:33: OSError
=============== 3 failed, 419 passed, 5 skipped in 19.60 seconds ===============
builder for '/nix/store/s7r7xjgdwc1zzkpjlbd2zxpj5a08x6lj-python3.6-trio-0.4.0.drv' failed with exit code 1
error: build of '/nix/store/s7r7xjgdwc1zzkpjlbd2zxpj5a08x6lj-python3.6-trio-0.4.0.drv' failed

catern · 2018-05-11T17:00:30Z

OK, with some changes and importing another test dependency, the tests for trio are mostly running. Just a few failing tests.

Now the obstacle is some apparent sandbox issue. It looks like maybe /etc/services isn't present in the sandbox? Can we suggest a workaround upstream?

running install tests
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-3.5.0, py-1.5.3, pluggy-0.6.0
rootdir: /build/trio-0.4.0, inifile:
collected 427 items

trio/_core/tests/test_epoll.py .                                         [  0%]
trio/_core/tests/test_io.py ..................................           [  8%]
trio/_core/tests/test_ki.py ..........                                   [ 10%]
trio/_core/tests/test_local.py ...........                               [ 13%]
trio/_core/tests/test_multierror.py .........sss                         [ 15%]
trio/_core/tests/test_parking_lot.py ....                                [ 16%]
trio/_core/tests/test_result.py ......                                   [ 18%]
trio/_core/tests/test_run.py ........................................... [ 28%]
.................................                                        [ 36%]
trio/_core/tests/test_tutil.py .                                         [ 36%]
trio/_core/tests/test_unbounded_queue.py .....                           [ 37%]
trio/_core/tests/test_windows.py s                                       [ 37%]
trio/tests/test_abc.py .                                                 [ 37%]
trio/tests/test_deprecate.py ...........                                 [ 40%]
trio/tests/test_exports.py .                                             [ 40%]
trio/tests/test_file_io.py ................                              [ 44%]
trio/tests/test_highlevel_generic.py ..                                  [ 44%]
trio/tests/test_highlevel_open_tcp_listeners.py ........                 [ 46%]
trio/tests/test_highlevel_open_tcp_stream.py ...................         [ 51%]
trio/tests/test_highlevel_open_unix_stream.py ..                         [ 51%]
trio/tests/test_highlevel_serve_listeners.py ....                        [ 52%]
trio/tests/test_highlevel_socket.py ......                               [ 54%]
trio/tests/test_highlevel_ssl_helpers.py ..                              [ 54%]
trio/tests/test_path.py ............................                     [ 61%]
trio/tests/test_signals.py ...                                           [ 61%]
trio/tests/test_socket.py ...F...s......F......F....                     [ 67%]
trio/tests/test_ssl.py ............................                      [ 74%]
trio/tests/test_sync.py .................................                [ 82%]
trio/tests/test_testing.py ........................                      [ 87%]
trio/tests/test_threads.py .......................                       [ 93%]
trio/tests/test_timeouts.py ...                                          [ 93%]
trio/tests/test_util.py ..........................                       [100%]

=================================== FAILURES ===================================
_______________________________ test_getnameinfo _______________________________

    async def test_getnameinfo():
        # Trivial test:
        ni_numeric = stdlib_socket.NI_NUMERICHOST | stdlib_socket.NI_NUMERICSERV
        with assert_checkpoints():
            got = await tsocket.getnameinfo(("127.0.0.1", 1234), ni_numeric)
        assert got == ("127.0.0.1", "1234")

        # getnameinfo requires a numeric address as input:
        with assert_checkpoints():
            with pytest.raises(tsocket.gaierror):
                await tsocket.getnameinfo(("google.com", 80), 0)

        with assert_checkpoints():
            with pytest.raises(tsocket.gaierror):
                await tsocket.getnameinfo(("localhost", 80), 0)

        # Blocking call to get expected values:
>       host, service = stdlib_socket.getnameinfo(("127.0.0.1", 80), 0)
E       socket.gaierror: [Errno -3] Temporary failure in name resolution

trio/tests/test_socket.py:176: gaierror
___________________________ test_SocketType_resolve ____________________________

    async def test_SocketType_resolve():
        # For some reason the stdlib special-cases "" to pass NULL to getaddrinfo
        # They also error out on None, but whatever, None is much more consistent,
        # so we accept it too.
        for null in [None, ""]:
            sock4 = tsocket.socket(family=tsocket.AF_INET)
            got = await sock4._resolve_local_address((null, 80))
            assert got == ("0.0.0.0", 80)
            got = await sock4._resolve_remote_address((null, 80))
            assert got == ("127.0.0.1", 80)

            sock6 = tsocket.socket(family=tsocket.AF_INET6)
            got = await sock6._resolve_local_address((null, 80))
            assert got == ("::", 80, 0, 0)

            got = await sock6._resolve_remote_address((null, 80))
            assert got == ("::1", 80, 0, 0)

        # AI_PASSIVE only affects the wildcard address, so for everything else
        # _resolve_local_address and _resolve_remote_address should work the same:
        for res in ["_resolve_local_address", "_resolve_remote_address"]:

            async def s4res(*args):
                return await getattr(sock4, res)(*args)

            async def s6res(*args):
                return await getattr(sock6, res)(*args)

>           assert await s4res(("1.2.3.4", "http")) == ("1.2.3.4", 80)

trio/tests/test_socket.py:408:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
trio/tests/test_socket.py:403: in s4res
    return await getattr(sock4, res)(*args)
trio/_socket.py:556: in _resolve_local_address
    return await self._resolve_address(address, AI_PASSIVE)
trio/_socket.py:535: in _resolve_address
    self._sock.proto, flags
trio/_socket.py:248: in getaddrinfo
    cancellable=True
trio/_threads.py:399: in run_sync_in_worker_thread
    return await _core.wait_task_rescheduled(abort)
trio/_core/_traps.py:159: in wait_task_rescheduled
    return (await _async_yield(WaitTaskRescheduled(abort_func))).unwrap()
trio/_core/_result.py:119: in unwrap
    raise self.error
trio/_core/_result.py:33: in capture
    return Value(sync_fn(*args))
trio/_threads.py:361: in do_release_then_return_result
    return result.unwrap()
trio/_core/_result.py:119: in unwrap
    raise self.error
trio/_core/_result.py:33: in capture
    return Value(sync_fn(*args))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

host = b'1.2.3.4', port = 'http', family = <AddressFamily.AF_INET: 2>, type = 1
proto = 0, flags = <AddressInfo.AI_PASSIVE: 1>

    def getaddrinfo(host, port, family=0, type=0, proto=0, flags=0):
        """Resolve host and port into list of address info entries.

        Translate the host/port argument into a sequence of 5-tuples that contain
        all the necessary arguments for creating a socket connected to that service.
        host is a domain name, a string representation of an IPv4/v6 address or
        None. port is a string service name such as 'http', a numeric port number or
        None. By passing None as the value of host and port, you can pass NULL to
        the underlying C API.

        The family, type and proto arguments can be optionally specified in order to
        narrow the list of addresses returned. Passing zero as a value for each of
        these arguments selects the full range of results.
        """
        # We override this function since we want to translate the numeric family
        # and socket type values to enum constants.
        addrlist = []
>       for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
E       socket.gaierror: [Errno -8] Servname not supported for ai_socktype

/nix/store/96wn2gz3mwi71gwcrvpfg39bsymd7gqx-python3-3.6.5/lib/python3.6/socket.py:745: gaierror
_____________________________ test_getprotobyname ______________________________

    async def test_getprotobyname():
        # These are the constants used in IP header fields, so the numeric values
        # had *better* be stable across systems...
>       assert await tsocket.getprotobyname("udp") == 17

trio/tests/test_socket.py:733:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
trio/_socket.py:280: in getprotobyname
    _stdlib_socket.getprotobyname, name, cancellable=True
trio/_threads.py:399: in run_sync_in_worker_thread
    return await _core.wait_task_rescheduled(abort)
trio/_core/_traps.py:159: in wait_task_rescheduled
    return (await _async_yield(WaitTaskRescheduled(abort_func))).unwrap()
trio/_core/_result.py:119: in unwrap
    raise self.error
trio/_core/_result.py:33: in capture
    return Value(sync_fn(*args))
trio/_threads.py:361: in do_release_then_return_result
    return result.unwrap()
trio/_core/_result.py:119: in unwrap
    raise self.error
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

sync_fn = <built-in function getprotobyname>, args = ('udp',)

    @staticmethod
    def capture(sync_fn, *args):
        """Run ``sync_fn(*args)`` and capture the result.

            Returns:
              Either a :class:`Value` or :class:`Error` as appropriate.

            """
        try:
>           return Value(sync_fn(*args))
E           OSError: protocol not found

trio/_core/_result.py:33: OSError
=============== 3 failed, 419 passed, 5 skipped in 14.68 seconds ===============
builder for '/nix/store/s7r7xjgdwc1zzkpjlbd2zxpj5a08x6lj-python3.6-trio-0.4.0.drv' failed with exit code 1
error: build of '/nix/store/s7r7xjgdwc1zzkpjlbd2zxpj5a08x6lj-python3.6-trio-0.4.0.drv' failed

dotlambda · 2018-05-11T17:01:14Z

I'd say just disable the failing tests.

GrahamcOfBorg · 2018-05-11T17:14:49Z

Failure on aarch64-linux (full log)

Attempted: python3.pkgs.trio

Partial log (click to expand)


            """
        try:
>           return Value(sync_fn(*args))
E           OSError: protocol not found

trio/_core/_result.py:33: OSError
============== 3 failed, 419 passed, 5 skipped in 536.54 seconds ===============
builder for '/nix/store/3gs3rc9jz1p7s68f2wbpr0dc68a7fxh2-python3.6-trio-0.4.0.drv' failed with exit code 1
�[31;1merror:�[0m build of '/nix/store/3gs3rc9jz1p7s68f2wbpr0dc68a7fxh2-python3.6-trio-0.4.0.drv' failed

dotlambda · 2018-05-11T18:28:52Z

@GrahamcOfBorg build python3.pkgs.trio

GrahamcOfBorg · 2018-05-11T18:30:03Z

Success on x86_64-linux (full log)

Attempted: python3.pkgs.trio

Partial log (click to expand)

trio/tests/test_socket.py ......s................                        [ 67%]
trio/tests/test_ssl.py ............................                      [ 74%]
trio/tests/test_sync.py .................................                [ 82%]
trio/tests/test_testing.py ........................                      [ 87%]
trio/tests/test_threads.py .......................                       [ 93%]
trio/tests/test_timeouts.py ...                                          [ 93%]
trio/tests/test_util.py ..........................                       [100%]

============= 419 passed, 5 skipped, 3 deselected in 14.80 seconds =============
/nix/store/cqhghj42751vsraps7nl5789fbvn8inz-python3.6-trio-0.4.0

GrahamcOfBorg · 2018-05-11T18:30:25Z

Success on aarch64-linux (full log)

Attempted: python3.pkgs.trio

Partial log (click to expand)

trio/tests/test_socket.py ......s................                        [ 67%]
trio/tests/test_ssl.py ............................                      [ 74%]
trio/tests/test_sync.py .................................                [ 82%]
trio/tests/test_testing.py ........................                      [ 87%]
trio/tests/test_threads.py .......................                       [ 93%]
trio/tests/test_timeouts.py ...                                          [ 93%]
trio/tests/test_util.py ..........................                       [100%]

============= 419 passed, 5 skipped, 3 deselected in 45.77 seconds =============
/nix/store/8s26g51c4qyyfm229pwwg0wqvy4pid4n-python3.6-trio-0.4.0

catern requested a review from FRidh as a code owner May 8, 2018 22:39

GrahamcOfBorg added 6.topic: python 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels May 8, 2018

FRidh added the 2.status: work-in-progress label May 9, 2018

dotlambda reviewed May 10, 2018

View reviewed changes

catern force-pushed the trio branch 2 times, most recently from ec2f07a to ab62319 Compare May 10, 2018 20:02

dotlambda reviewed May 11, 2018

View reviewed changes

catern added 2 commits May 11, 2018 16:38

pythonPackages.outcome: init at 0.1.0a0

46fa7ab

pythonPackages.trustme: init at 0.4.0

c983bbb

catern force-pushed the trio branch from ab62319 to 23dcce9 Compare May 11, 2018 16:56

dotlambda reviewed May 11, 2018

View reviewed changes

pythonPackages.trio: init at 0.4.0

f78c252

catern force-pushed the trio branch from 23dcce9 to f78c252 Compare May 11, 2018 17:17

dotlambda merged commit 4e7e246 into NixOS:master May 11, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add pythonPackages.outcome, pythonPackages.trio #40191

add pythonPackages.outcome, pythonPackages.trio #40191

catern commented May 8, 2018

dotlambda May 10, 2018

catern commented May 10, 2018

catern commented May 10, 2018 •

edited

dotlambda May 11, 2018

dotlambda commented May 11, 2018

dotlambda May 11, 2018

catern May 11, 2018

dotlambda commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

catern commented May 11, 2018

dotlambda commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

dotlambda commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

add pythonPackages.outcome, pythonPackages.trio #40191

add pythonPackages.outcome, pythonPackages.trio #40191

Conversation

catern commented May 8, 2018

dotlambda May 10, 2018

Choose a reason for hiding this comment

catern commented May 10, 2018

catern commented May 10, 2018 • edited

dotlambda May 11, 2018

Choose a reason for hiding this comment

dotlambda commented May 11, 2018

dotlambda May 11, 2018

Choose a reason for hiding this comment

catern May 11, 2018

Choose a reason for hiding this comment

dotlambda commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

catern commented May 11, 2018

dotlambda commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

dotlambda commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

catern commented May 10, 2018 •

edited