New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: If a thread is blocking on UDP send, it can block the entire main loop #9001
Conversation
…n loop Under some rare configurations, UDP sends can block. Since we hold a mutex on some objects for this, it can cause blocking of the main loop which will eventually cause clients to drop.
To put IRC here too: If I read this right, this basically only fixes the cases where For a server, many of the incoming UDP requests cause an outgoing UDP packet, so there are many places that these Do I understand that correct? (I have nothing against this PR, to be clear :D Just want to set expectations :P) |
Yes a server will respond to many incoming UDP requests with another UDP packet back, and those responses are in fact done on the same thread receiving the UDP request packet. However those happen on the |
Another, possibly better, solution might be to protect the |
I can confirm this was induced by the containerised environment I was running, adding delays to domain name resolution with using IPv6 resolvers. This, however, triggered interesting behaviour from the game, showing that difficulties impacting communication with the master server had effects on all networking, in this case blocking responses to client traffic, leading to clients disconnection. This can be reproduced with TrafficControl with the use of the tc qdisc add dev <interface> root netem delay 2s This is very easy to experiment in a Docker environment with custom network (not the default Of course, way more interesting effects with surgical precision can be simulated with other queing disciplines, like HTB, for instance to simulate slow traffic from/to specific IP addresses I also can confirm #8878 was actually impacted by the slow DNS traffic too. |
Does this PR alleviate the client connection issues while this slow DNS request is occurring? |
Motivation / Problem
Under some rare configurations, UDP sends can block. Since we hold a mutex on some objects for this, it can cause blocking of the main loop which will eventually cause clients to drop.
Description
For example, the advertise to master server thread takes the UDP lock and tries to send on a socket.
OpenTTD/src/network/network_udp.cpp
Lines 555 to 565 in 39b7ef3
If this send blocks, it will cause the main loop to block in an unexpected way.
Limitations
This may cause some other UDP responses (prospective clients' info requests) to be delayed/lost, but probably preferable over lose active clients.
Checklist for review
Some things are not automated, and forgotten often. This list is a reminder for the reviewers.