Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

home-assistant: 2020.12.2 -> 2021.1.0 (+entourage) #108602

Merged
merged 14 commits into from Jan 7, 2021

Conversation

mweinelt
Copy link
Member

@mweinelt mweinelt commented Jan 6, 2021

Motivation for this change

https://www.home-assistant.io/blog/2021/01/06/release-20211/

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@mweinelt
Copy link
Member Author

mweinelt commented Jan 6, 2021

I've prepared these updates during the home-assistant beta release cycle and therefore missed a few independent library updates, like:

This also supersedes

@mweinelt
Copy link
Member Author

mweinelt commented Jan 6, 2021

There is a slighty newer version of uvicorn (0.13.3) out there, but it creates circular dependencies with httpx during the testPhase. I didn't want to deal with that in here.

@mweinelt

This comment has been minimized.

@mweinelt
Copy link
Member Author

mweinelt commented Jan 6, 2021

Whoops, this got somewhat out of hand. Anyway, the remaining issues are
on the target branch and I think we should move this through quickly.

Except maybe the zigpy-znp test issues. They're timing out after
latestzigpy update. I'm not yet sure why. CC @mvnetbiz

Result of nixpkgs-review pr 108602 run on x86_64-linux 1

7 packages marked as broken and skipped:
  • python39Packages.databases
  • python39Packages.fastapi
  • python39Packages.httpcore
  • python39Packages.orm
  • python39Packages.pyarlo
  • python39Packages.starlette
  • python39Packages.uvicorn
18 packages failed to build:
  • google-music-scripts
  • python37Packages.audio-metadata
  • python37Packages.google-music
  • python37Packages.google-music-proto
  • python37Packages.google-music-utils
  • python37Packages.labelbox
  • python37Packages.salmon-mail
  • python37Packages.zigpy-znp
  • python38Packages.audio-metadata
  • python38Packages.google-music
  • python38Packages.google-music-proto
  • python38Packages.google-music-utils
  • python38Packages.labelbox
  • python38Packages.orm
  • python38Packages.salmon-mail
  • python38Packages.zigpy-znp
  • python39Packages.zigpy-znp
  • tts
121 packages built:
  • apache-airflow (python37Packages.apache-airflow)
  • check_systemd
  • cura
  • datasette (python38Packages.datasette)
  • evdevremapkeys
  • gns3-gui
  • gns3-server
  • home-assistant
  • keepkey_agent (python38Packages.keepkey_agent)
  • ledger_agent (python38Packages.ledger_agent)
  • luigi
  • mirage-im
  • mitmproxy (python38Packages.mitmproxy)
  • mnemosyne
  • octoprint
  • pantalaimon
  • python37Packages.PyRMVtransport
  • python37Packages.aiosqlite
  • python37Packages.alerta-server
  • python37Packages.ansible-kernel
  • python37Packages.ansible-runner
  • python37Packages.bellows
  • python37Packages.clldutils
  • python37Packages.csvw
  • python37Packages.databases
  • python37Packages.datasette
  • python37Packages.entrance
  • python37Packages.entrance-with-router-features
  • python37Packages.fastapi
  • python37Packages.gtts
  • python37Packages.h11
  • python37Packages.httpcore
  • python37Packages.httpx
  • python37Packages.keepkey_agent
  • python37Packages.ledger_agent
  • python37Packages.libagent
  • python37Packages.matrix-nio
  • python37Packages.mitmproxy
  • python37Packages.nagiosplugin
  • python37Packages.ndjson
  • python37Packages.openwebifpy
  • python37Packages.orm
  • python37Packages.phonemizer
  • python37Packages.pyarlo
  • python37Packages.python-daemon
  • python37Packages.rfc3986
  • python37Packages.sanic
  • python37Packages.sanic-auth
  • python37Packages.segments
  • python37Packages.sentry-sdk
  • python37Packages.starlette
  • python37Packages.tbm-utils
  • python37Packages.toggl-cli
  • python37Packages.trezor_agent
  • python37Packages.twine
  • python37Packages.uvicorn
  • python37Packages.wsproto
  • python37Packages.zha-quirks
  • python37Packages.zigpy
  • python37Packages.zigpy-deconz
  • python38Packages.PyRMVtransport
  • python38Packages.aiosqlite
  • python38Packages.alerta-server
  • python38Packages.ansible-kernel
  • python38Packages.ansible-runner
  • python38Packages.bellows
  • python38Packages.clldutils
  • python38Packages.csvw
  • python38Packages.databases
  • python38Packages.entrance
  • python38Packages.entrance-with-router-features
  • python38Packages.fastapi
  • python38Packages.gtts
  • python38Packages.h11
  • python38Packages.httpcore
  • python38Packages.httpx
  • python38Packages.libagent
  • python38Packages.matrix-nio
  • python38Packages.nagiosplugin
  • python38Packages.ndjson
  • python38Packages.openwebifpy
  • python38Packages.phonemizer
  • python38Packages.pyarlo
  • python38Packages.python-daemon
  • python38Packages.rfc3986
  • python38Packages.sanic
  • python38Packages.sanic-auth
  • python38Packages.segments
  • python38Packages.sentry-sdk
  • python38Packages.starlette
  • python38Packages.tbm-utils
  • python38Packages.toggl-cli
  • trezor_agent (python38Packages.trezor_agent)
  • python38Packages.twine
  • python38Packages.uvicorn
  • python38Packages.wsproto
  • python38Packages.zha-quirks
  • python38Packages.zigpy
  • python38Packages.zigpy-cc
  • python38Packages.zigpy-deconz
  • python38Packages.zigpy-xbee
  • python38Packages.zigpy-zigate
  • python39Packages.aiosqlite
  • python39Packages.bellows
  • python39Packages.clldutils
  • python39Packages.csvw
  • python39Packages.h11
  • python39Packages.matrix-nio
  • python39Packages.mitmproxy
  • python39Packages.openwebifpy
  • python39Packages.phonemizer
  • python39Packages.rfc3986
  • python39Packages.segments
  • python39Packages.wsproto
  • python39Packages.zha-quirks
  • python39Packages.zigpy
  • python39Packages.zigpy-cc
  • python39Packages.zigpy-deconz
  • python39Packages.zigpy-xbee
  • python39Packages.zigpy-zigate
  • weechatScripts.weechat-matrix

fabaff and others added 6 commits January 7, 2021 13:33
The zigpy-znp package didn't sync up with changes to the zigpy 0.29 in
time for the home-assistant 2021.1.0 release, so let's disable this test
until the next release.

Co-Authored-By: Matt Votava <mvnetbiz@gmail.com>
Neither the home-assistant nor the frontend contain strippable binaries,
but the stripping process will still iterate over 6600+ files and notice
that they're not in a strippable format.

On my 6C/12T desktop CPU this takes slightly over two minutes.
@wd15
Copy link
Contributor

wd15 commented Jan 8, 2021

Result of nixpkgs-review pr 108602 1

3 package marked as broken and skipped:
  • python39Packages.httpcore
  • python39Packages.pyarlo
  • python39Packages.uvicorn
1 package failed to build:
  • google-music-scripts
81 package built:
  • appdaemon
  • datasette (python38Packages.datasette)
  • elasticsearch-curator
  • esphome
  • home-assistant
  • mirage-im
  • mitmproxy (python38Packages.mitmproxy)
  • pantalaimon
  • python37Packages.PyRMVtransport
  • python37Packages.async-upnp-client
  • python37Packages.bellows
  • python37Packages.datasette
  • python37Packages.entrance
  • python37Packages.entrance-with-router-features
  • python37Packages.google-music
  • python37Packages.h11
  • python37Packages.httpcore
  • python37Packages.httpx
  • python37Packages.matrix-nio
  • python37Packages.mitmproxy
  • python37Packages.nestedtext
  • python37Packages.openwebifpy
  • python37Packages.pyarlo
  • python37Packages.pyhs100
  • python37Packages.sanic
  • python37Packages.sanic-auth
  • python37Packages.uvicorn
  • python37Packages.voluptuous
  • python37Packages.voluptuous-serialize
  • python37Packages.wsproto
  • python37Packages.zha-quirks
  • python37Packages.zigpy
  • python38Packages.zigpy-cc
  • python38Packages.zigpy-deconz
  • python38Packages.zigpy-xbee
  • python38Packages.zigpy-zigate
  • python38Packages.zigpy-znp
  • python39Packages.async-upnp-client
  • python39Packages.bellows
  • python39Packages.h11
  • python39Packages.matrix-nio
  • python39Packages.mitmproxy
  • python39Packages.nestedtext
  • python39Packages.openwebifpy
  • python39Packages.pyhs100
  • python39Packages.voluptuous
  • python39Packages.voluptuous-serialize
  • python39Packages.wsproto
  • python39Packages.zha-quirks
  • python39Packages.zigpy
  • python39Packages.zigpy-cc
  • python39Packages.zigpy-deconz
  • python39Packages.zigpy-xbee
  • python39Packages.zigpy-zigate
  • python39Packages.zigpy-znp
  • weechatScripts.weechat-matrix

@mweinelt
Copy link
Member Author

Yes?

@pinpox
Copy link
Member

pinpox commented Jan 22, 2021

@mweinelt sorry, I accidentaly deleted my comment. home-manager is failing to build because of some tests still:

▸ nix-build ./krops.nix -A birne && ./result
/nix/store/1qaii67qh7r8hdkdmqgw2y762bx8f55b-deploy-birne
+ /nix/store/grm0kmhcm8zld7i6l2lfiw6c1s504qjp-populate.192.168.2.84.machine-config
sending incremental file list
./
tags
.git/FETCH_HEAD
.git/objects/
modules/home-assistant/
modules/home-assistant/default.nix

sent 65,352 bytes  received 431 bytes  131,566.00 bytes/sec
total size is 9,751,245  speedup is 148.23
+ /nix/store/a0zv3w9sjilgf73icpnfv4d8fwkn76i8-populate.192.168.2.84.secrets
sending incremental file list
./
borg-server/
borg/
cloudflare/
monit/
ssh/
ssh/root/
wireguard/

sent 468 bytes  received 47 bytes  343.33 bytes/sec
total size is 1,212  speedup is 2.35
warning: Git tree '/var/src/machine-config' is dirty
building the system configuration...
warning: Git tree '/var/src/machine-config' is dirty
building '/nix/store/dabjnd0hc56zc8aas8305bcsq7zl50m9-homeassistant-2021.1.3.drv'...
building '/nix/store/54g8lfs08z8nb1946jg22xm60m5jrdm6-configuration.json.drv'...
building '/nix/store/skyfjw629ydnc17qy105w08axwm26qfn-configuration.yaml.drv'...
building '/nix/store/zmd7ln1avpm0sj048zkgipjivfacnlaw-unit-script-home-assistant-pre-start.drv'...
error: --- Error --- nix
builder for '/nix/store/dabjnd0hc56zc8aas8305bcsq7zl50m9-homeassistant-2021.1.3.drv' failed with exit code 1; last 10 log lines:
      app[KEY_BANNED_IPS] = await async_load_ip_bans_config(

  tests/components/shell_command/test_init.py::test_do_no_run_forever[pyloop]
    /nix/store/yi9a1fafijn0nf5m8qkmnl955vi2mncm-python3-3.8.6/lib/python3.8/asyncio/unix_events.py:917: RuntimeWarning: A loop is being detached from a child watcher with pending handlers
      warnings.warn(

  -- Docs: https://docs.pytest.org/en/stable/warnings.html
  =========================== short test summary info ============================
  ERROR tests/components/ssdp/test_init.py::test_scan_match_st[pyloop] - Runtim...
  ====== 2501 passed, 1 skipped, 11 warnings, 1 error in 169.07s (0:02:49) =======
error: --- Error --- nix
1 dependencies of derivation '/nix/store/7mbz7wqckfa19ls5milsmdfmxxj7ff07-unit-home-assistant.service.drv' failed to build
error: --- Error --- nix
1 dependencies of derivation '/nix/store/0ivfl11fc93vnz4bka2alif5n53pwfnb-system-units.drv' failed to build
error: --- Error --- nix
1 dependencies of derivation '/nix/store/gnc9yspvc6crf8n7f9irdgbqcdcxv4m6-etc.drv' failed to build
error: --- Error --- nix
1 dependencies of derivation '/nix/store/0ss54wzr3m8ycmh0sl3bxsbibjanx3fd-nixos-system-birne-21.03.20210117.68398d2.drv' failed to build
warning: Git tree '/var/src/machine-config' is dirty
building the system configuration...
warning: Git tree '/var/src/machine-config' is dirty
building '/nix/store/dabjnd0hc56zc8aas8305bcsq7zl50m9-homeassistant-2021.1.3.drv'...
error: --- Error --- nix
builder for '/nix/store/dabjnd0hc56zc8aas8305bcsq7zl50m9-homeassistant-2021.1.3.drv' failed with exit code 1; last 10 log lines:
      app[KEY_BANNED_IPS] = await async_load_ip_bans_config(

  tests/components/shell_command/test_init.py::test_do_no_run_forever[pyloop]
    /nix/store/yi9a1fafijn0nf5m8qkmnl955vi2mncm-python3-3.8.6/lib/python3.8/asyncio/unix_events.py:917: RuntimeWarning: A loop is being detached from a child watcher with pending handlers
      warnings.warn(

  -- Docs: https://docs.pytest.org/en/stable/warnings.html
  =========================== short test summary info ============================
  ERROR tests/components/ssdp/test_init.py::test_scan_match_st[pyloop] - Runtim...
  ====== 2501 passed, 1 skipped, 11 warnings, 1 error in 168.88s (0:02:48) =======
error: --- Error --- nix
1 dependencies of derivation '/nix/store/7mbz7wqckfa19ls5milsmdfmxxj7ff07-unit-home-assistant.service.drv' failed to build
error: --- Error --- nix
1 dependencies of derivation '/nix/store/0ivfl11fc93vnz4bka2alif5n53pwfnb-system-units.drv' failed to build
error: --- Error --- nix
1 dependencies of derivation '/nix/store/gnc9yspvc6crf8n7f9irdgbqcdcxv4m6-etc.drv' failed to build
error: --- Error --- nix
1 dependencies of derivation '/nix/store/0ss54wzr3m8ycmh0sl3bxsbibjanx3fd-nixos-system-birne-21.03.20210117.68398d2.drv' failed to build

I'm tryin to use it with this minimal config: https://gist.github.com/pinpox/9a6b5bd1d0f7cee3827f1106e35eeebd

@mweinelt
Copy link
Member Author

Can you please post the complete test phase output, where things go wrong? I have a feeling what kind of issue this is, but I want to confirm. Generally some tests experience flakyness/racyness, sadly.

@pinpox
Copy link
Member

pinpox commented Jan 22, 2021

@mweinelt How can I get more output? The deployment command above is mostly just running nixos-rebuild switch -v --show-trace --flake ${targetPath}/machine-config on the server, not sure how to get the testPhase output.

EDIT: also, the build takes suspicously long ( ~2-3 min for the home-assistant derivation)

@mweinelt
Copy link
Member Author

Please ask the maintainers of your deployment tool (krops), I'm not familiar with it. Also please note that home-assistant is a pretty big package, so 2-3 minutes to build and test it are expected.

@pinpox
Copy link
Member

pinpox commented Jan 22, 2021

@mweinelt I'm not sure why, but re-running the rebuild just succeeded. I'm still getting a 404 on the page though, but that might be a different issue with my config. Would it be possible to disable the python tests for the package somehow?

@Mic92
Copy link
Member

Mic92 commented Jan 22, 2021

@pinbox Usually you can do nix log /nix/store/dabjnd0hc56zc8aas8305bcsq7zl50m9-homeassistant-2021.1.3.drv to get the full log after having a failed build. And nix-collect-garbage to remove a succeeding build.

@mweinelt
Copy link
Member Author

mweinelt commented Jan 22, 2021

You can disable the tests by overriding doCheck like so:

services.home-assistant.package = (pkgs.home-assistant.overrideAttrs (old: {
  doCheck = false;
});

@pinpox
Copy link
Member

pinpox commented Jan 22, 2021

@Mic92 I ran sudo nix-collect-garbage then build on the machine itself (without krops) again successfully:

pinpox@birne /var/src/machine-config main* 7s
▸ sudo nixos-rebuild -v --show-trace --flake ".#birne" switch
warning: Git tree '/var/src/machine-config' is dirty
building the system configuration...
warning: Git tree '/var/src/machine-config' is dirty
updating GRUB 2 menu...
activating the configuration...
setting up /etc...
reloading user units for pinpox...
setting up tmpfiles
the following new units were started: cfdyndns.service

I can see, that home-assistant is running:


pinpox@birne /var/src/machine-config main*
▸ netstat -tulpn
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:8123          0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      -
tcp        0      0 192.168.7.4:2812        0.0.0.0:*               LISTEN      -
tcp6       0      0 :::80                   :::*                    LISTEN      -
tcp6       0      0 :::22                   :::*                    LISTEN      -
tcp6       0      0 :::443                  :::*                    LISTEN      -
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -
udp        0      0 0.0.0.0:58460           0.0.0.0:*                           -
udp6       0      0 :::546                  :::*                                -
udp6       0      0 :::58460                :::*                                -

But I can't access it. I tried using the configured url or the localhost IP on the machine itself:

pinpox@birne /var/src/machine-config main*
▸ curl 127.0.0.1:8123
404: Not Found

Is there any configuration needed to enable the WebUI? The configuration I'm using is this: https://termbin.com/j5d3

@mweinelt
Copy link
Member Author

You are missing important modules, that are part of default_config (https://www.home-assistant.io/integrations/default_config/).

Enable it by setting

services.home-assistant.config.default_config = {};

@pinpox
Copy link
Member

pinpox commented Jan 22, 2021

@mweinelt I'm back to failing. Here is more output regarding the python tests:


▸ sudo nixos-rebuild -v --show-trace --flake ".#birne" switch
[sudo] password for pinpox:
warning: Git tree '/var/src/machine-config' is dirty
building the system configuration...
warning: Git tree '/var/src/machine-config' is dirty
building '/nix/store/ddifxrf1yhmm7q42pbajypppdlj5ajns-homeassistant-2021.1.3.drv'...
error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
builder for '/nix/store/ddifxrf1yhmm7q42pbajypppdlj5ajns-homeassistant-2021.1.3.drv' failed with exit code 1; last 10 log lines:
      app[KEY_BANNED_IPS] = await async_load_ip_bans_config(

  tests/components/shell_command/test_init.py::test_do_no_run_forever[pyloop]
    /nix/store/yi9a1fafijn0nf5m8qkmnl955vi2mncm-python3-3.8.6/lib/python3.8/asyncio/unix_events.py:917: RuntimeWarning: A loop is being detached from a child watcher with pending handlers
      warnings.warn(

  -- Docs: https://docs.pytest.org/en/stable/warnings.html
  =========================== short test summary info ============================
  ERROR tests/components/ssdp/test_init.py::test_scan_match_st[pyloop] - Runtim...
  ====== 2501 passed, 1 skipped, 11 warnings, 1 error in 169.19s (0:02:49) =======
error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
1 dependencies of derivation '/nix/store/fkvq3xckqkhvb2c9p5rphb93a26ychdj-unit-home-assistant.service.drv' failed to build
error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
1 dependencies of derivation '/nix/store/ablfc6156hwnjj2k0i5qs9ifp3l3g269-system-units.drv' failed to build
error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
1 dependencies of derivation '/nix/store/2rhzvz6wkd47rf0p91vcs9nk4m8vfxsn-etc.drv' failed to build
error: --- Error ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
1 dependencies of derivation '/nix/store/xmq5n6yixdzr7yxji7fagsphqdjhlv72-nixos-system-birne-21.03.20210117.68398d2.drv' failed to build

I'll probably disable the tests, but this seems like something that should be fixed anyway?

@mweinelt
Copy link
Member Author

As you can see the test is flaky, I appears, vanishes, reappears. I still want to see the exact test failure so I can make up my mind whether or not I'll report it upstream or just blacklist it downstream.

@pinpox
Copy link
Member

pinpox commented Jan 22, 2021

@mweinelt I'm running it with nixos-rebuild now as you can see (no krops), still not sure how to get more output to help

@mweinelt
Copy link
Member Author

Sorry, I have no idea what curbs your log output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants