I'm developing a network server application in C ++ and I came across the thundering herd problem, since I designed to have multiple threads accepting ( accept()
) client connections simultaneously. p>
By my searches, the most common solution is to do a queue of FIFO tasks, which consists of pointers to functions that are being executed by threads. In these functions the client connection socket FD is already accepted, so the accept()
becomes the only responsibility of the main process, without thundering herd here.
However, I find it somewhat complicated to implement this and I had another idea, which is as follows:
Thread:
- has an ID;
- monitors the server socket / listens for new client connections using kqueue, epoll, etc.
- When a new connection arrives, it requests the Threads Controller the ID of the thread that should accept this connection. If the ID received is the ID of the thread itself:
- accepts (
accept()
) new connection; - tells Threads Controller to change the thread ID that should accept the connection to the next thread ID;
- tells Threads Controller that is now busy;
- receives the client requests on the socket and processes the responses;
- tells Threads Controller that it is now free to service another connection;
Threads Controller:
- maintains the thread ID that should accept (
accept()
) the next connection; - maintains an atomic access vector (
std::atomic<std::vector>
) ofstructs
that contains the thread ID and if it is busy (in the end it is similar to astd::map
); - When Thread performs part 2 above, Threads Controller traverses the vector, gets the next thread that is not busy, and sets its ID as the thread that should accept (
accept()
) the next connection.
Do you consider this solution better than the task queue? In both there is a time that only 1 thread does something at a time (get the next task in the queue and accept the new connection), in both there is the use of std::atomic
, both seem to solve the problem ...
In my view there are no problems in my solution, but as I'm not very experienced programming network applications, multithreading, etc., there may be something I'm letting go of, something that may be a bottleneck, prevent others from continuing, I do not know.
If there are no problems, this may be a much easier solution to implement for anyone else who is facing thundering herd . There is no way for me to do a test application here and simulate a real environment, since my only computer is an old and weak single core.
What do they say? Thanks in advance for your attention.