Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turn hydra-notify into a daemon #668

Merged
merged 11 commits into from Aug 14, 2019
Merged

Turn hydra-notify into a daemon #668

merged 11 commits into from Aug 14, 2019

Conversation

edolstra
Copy link
Member

hydra-notify is now a daemon that receives events from hydra-queue-runner by listening to PostgreSQL notifications. This makes it a lot more efficient: not starting a hydra-notify process for each event avoids the ~1s startup time loading all the plugins. Also, it allows future plugins to be written in any language as long as they can listen to PostgreSQL notifications, and to run on different machines from the queue runner.

The PostgreSQL notifications are:

  • step_started
  • step_finished
  • build_started
  • build_finished
  • eval_started
  • eval_failed
  • eval_cached
  • eval_added

@AmineChikhaoui
Copy link
Member

Awesome !

@edolstra edolstra changed the base branch from flake to master August 13, 2019 16:17
It now receives notifications about started/finished builds/steps via
PostgreSQL. This gets rid of the (substantial) overhead of starting
hydra-notify for every event. It also allows other programs (even on
other machines) to listen to Hydra notifications.
* 'eval_started' has the format '<tmpId>\t<project>\t<jobset>'.

* 'eval_failed' has the format '<tmpId>'. (The cause of the error can
  be found in the database.)

* 'eval_added' has the format '<tmpId>:<evalId>'.
Plugins are now disabled at startup time unless there is some relevant
configuration in hydra.conf. This avoids hydra-notify having to do a
lot of redundant work (a lot of plugins did a lot of database queries
*before* deciding they were disabled).

Note: BitBucketStatus users will need to add 'enable_bitbucket_status
= 1' to hydra.conf.
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.
@ajs124
Copy link
Member

ajs124 commented Oct 1, 2019

This broke declarative jobsets in our setup. They work fine on 5e9439e, but don't do anything in f17cd94.

@ajs124
Copy link
Member

ajs124 commented Oct 5, 2019

I take everything back I said. We just weren't running the new daemon, because we still had an older version of the module.

@grahamc grahamc deleted the notifications branch March 4, 2020 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants