Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LB-155: Make Redis connection establishment more reliable #176

Merged
merged 1 commit into from May 22, 2017

Conversation

paramsingh
Copy link
Collaborator

Do a ping and if ConnectionErrors are thrown, sleep for two
seconds and try to establish a new connection.

Do a ping and if ConnectionErrors are thrown, sleep for two
seconds and try to establish a new connection.
@paramsingh paramsingh changed the title Make Redis connection establishment more reliable LB-155: Make Redis connection establishment more reliable May 20, 2017
@paramsingh
Copy link
Collaborator Author

Doing a ping because ConnectionErrors get thrown only when the a request is made to the redis server. See here: https://stackoverflow.com/questions/12857604/python-how-to-check-if-redis-server-is-available

Fixed the connection establishment for

  1. webserver
  2. alpha_importer
  3. influx_writer
  4. bigquery_writer

The last two had the loops already but didn't ping the server, so only needed to add a ping. :)

Copy link
Member

@mayhem mayhem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@mayhem mayhem merged commit bc8d2d8 into metabrainz:master May 22, 2017
@paramsingh paramsingh deleted the redis-listenstore/startup branch May 22, 2017 13:21
@alastair
Copy link
Collaborator

Does it make sense to quit after some number of connection attempts (say, 5?) instead of continually trying to connect and failing?

@mayhem
Copy link
Member

mayhem commented May 30, 2017

The whole point is to keep trying -- containers may come up and down at any point. Waiting nicely and retrying allows the system to self heal, without having to restart containers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants