Multi-threading required for client and server, and thoughts on a proposed setup

[kurasu1415]

So, I'm trying to get a basic client-server setup working. I have looked at the provided example, but the server simply waits until it gets a request, responds, and then it will close at the next bit of user input.

I want to build a real working example where the server constantly simulates physics, until it receives a packet, and the client should draw graphics at all times while listening for packets from the server.

I would assume that the only way to solve this is for both the client and server to have separate threads for listening for connections.

I have a simple setup in mind that I would like input on. I can see some issues with this, which I will bring up as well.

Server :
-Has two threads, one for Listening/Processing packets, and one for constantly simulating physics.
-If it receives input, it makes this change in a thread-safe buffer that is available to the other thread.
-If it receives any information requests, the listener responds with the requested info.
-Server updates all clients with updated physics states at a regular interval.

Client:
-Has three threads, one for listening to the server, one for drawing graphics/game logic, and one for audio.
-There is a physical state buffer that is always updated on a regular interval with information from the server. this stores information about the entities that should be watched as they aren't at rest.

I do have some questions about this setup though.

-On the client, it seems that it requires at LEAST 3 threads, but what happens if your processor only supports 2 threads?

-Even if the computer can still handle all the threads, is one thread for networking enough? What if I need to send a packet to the server, at the exact time that the server is trying to send me a packet? Are both of our packets lost?

I appreciate any input you guys can give me, as I'm new to real-time network programming.

[binary1248]

So, I'm trying to get a basic client-server setup working. I have looked at the provided example, but the server simply waits until it gets a request, responds, and then it will close at the next bit of user input.

Read this, all the way at the bottom.

I would assume that the only way to solve this is for both the client and server to have separate threads for listening for connections.

After you read the above, you should know better.

Server :
-Has two threads, one for Listening/Processing packets, and one for constantly simulating physics.
-If it receives input, it makes this change in a thread-safe buffer that is available to the other thread.
-If it receives any information requests, the listener responds with the requested info.
-Server updates all clients with updated physics states at a regular interval.

You will kill yourself with the synchronization you would need to perform for such a simple task. Unless you expect to handle a really big physics scenario or expect more than 20 clients to be connected at the same time, just use a single thread.

Client:
-Has three threads, one for listening to the server, one for drawing graphics/game logic, and one for audio.
-There is a physical state buffer that is always updated on a regular interval with information from the server. this stores information about the entities that should be watched as they aren't at rest.

Read this. You will find this remark:
Sounds (and musics) are played in a separate thread. This means that you can do what you want after calling play() (except destroying the sound or its data, of course), the sound will continue to play until it's finished or stopped explicitly.
Creating a new thread to play audio is kind of overkill.

-On the client, it seems that it requires at LEAST 3 threads, but what happens if your processor only supports 2 threads?

You need to read up a bit about what a thread really is and how it is realized. Even before multi-core processors became mainstream, operating systems already let you create as many threads as you wanted until you ran out of system resources.

-Even if the computer can still handle all the threads, is one thread for networking enough? What if I need to send a packet to the server, at the exact time that the server is trying to send me a packet? Are both of our packets lost?

If this were true, it would defeat the purpose of full-duplex ethernet and the internet would probably not work as it does nowadays. There was a time, in the last millennium, where such "collisions" could occur, but they were completely transparent to the application developer. You really don't need to worry about this. Sockets are able to do many things at once, UDP sockets are even able to send and receive from multiple hosts all at the same time.

[kurasu1415]

Thanks for all the info. I read the packet tutorial, and I still have one question.

Let's assume I have something like the following code. If I am in the middle of handling physics, but one of my connection tries to send me data, what happens to that data? Is it just sort of cached in my socket until I get back there to handle it? If this is the case, what happens if I received 4 messages from one socket in the time it took to get back? I would assume I should expect this to happen.

while(serverIsUp)
{
    if (selector.wait(sf::seconds(10)))
    {
        // for each socket...
        {
            if (selector.isReady(socket))
            {
                socket.receive(...);
            }
        }
    }

    handlePhysics();
}
 

Thanks again for all the help. I want to make sure I understand what is going on.

[binary1248]

If you use TCP sockets, data is not lost in normal operation (so in your case always). The operating system stores it in an internal buffer until the application decides to retrieve it. The same goes for sending data, in normal operation it is buffered and sent when possible without data loss even if the connection cannot accept any new data.

[kurasu1415]

OK, so that bring up another issue. You are speaking in relation to TCP. The way I understand it, you should never use TCP for real-time data in a game. I have read this from multiple sources. For example : http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/

Why don't you use UDP?

[binary1248]

https://github.com/SFML/SFML/wiki/FAQ#wiki-network-tcp-vs-udp

Just stick with TCP. You are obviously a beginner in network programming so you shouldn't bother writing your own layer on top of UDP yet. The person who wrote the article over-exaggerates the drawbacks of TCP. Yes, you will notice a latency spike IF packets are lost, but as he said earlier as well, data is rarely dropped nowadays. Not only that, he also did not mention newer mechanisms implemented in TCP to reduce the performance drops in the case of packet losses, such as fast retransmit and fast recovery.

UDP does have a lower average latency per packet than TCP. But people keep forgetting, it's about how you compute the average latency of the connection. TCP retransmits data that it perceives is lost hence EVERYTHING will eventually arrive although a bit later in the case of packet loss, UDP just silently forgets about the packet and doesn't even notify you about it. Obviously if you average out the latency of all packets THAT ARRIVED, UDP will have a lower latency because it's either it arrived really fast, or not at all. Remember, this is over a lossy connection. If we say that the connection has a 1% packet loss rate, it would mean that UDP will yield the same latency as if the connection has 0% packet loss whereas TCP will display a slight latency increase of 1% because of the retransmits. Given that hardware more or less processes IP packets in the same time regardless of whether they contain a TCP segment or UDP datagram, that will result in a 1% latency increase when using TCP over that lossy connection. At 10% loss rate, it will probably be around 10% latency increase. The typical latency of an internet connection nowadays from your home computer to a not too distant server is <100ms. This means at 10% loss TCP would be "10ms slower than UDP". 10% is already a very extreme loss rate, normal values are around 1-5%, so the difference between using TCP and UDP will be typically 1-5ms over a connection that has a base 100ms latency.

You have to ask yourself: is it really worth it to spend weeks if not months to gain a 5ms latency decrease by implementing your own reliability layer on top of UDP when you can skip right ahead to coding whatever you envisioned your game to be if you were to use TCP? For big AAA companies like Epic Games and EA it is. They have dedicated network programmers working 8 hours a day on things like this. Are you willing to spend that much time on 5ms? I don't think so.

There are too many people on the internet who preach one or the other no matter what. Both are viable options, it just depends on the circumstances given. Using TCP or UDP "just because <some famous game> uses it" must be the most stupid advice someone can give a beginner who has no idea of the implications of either. Just ignore those trolls, they don't know better.

In your case, since you want to synchronize a physical game state between multiple independent hosts each running their own simulation at regular intervals, you will need to employ some form of dead reckoning for the inter-update periods. This already leads to the premise that the state changes you intend to send must be sent reliably or you will screw up the simulation when trying to make up for the deviation at a state update. This leads to the colloquial "rubber-banding" you see in some games (UDP based ones too). As such don't bother with UDP to start out with. Later on, when your core game is complete and you really have nothing else to do, you can swap out the TCP network backend for a UDP one if you still feel like you will benefit from it.

[kurasu1415]

Assuming your numbers are correct, the UDP argument seems pretty silly at this point. I was led to believe that TCP was MUCH slower in the end. In that case, I'll move forward using TCP.

Also, I'm not running simulation on every computer. I'm planning on just sending user input to the server, and receiving back any changes based on that input.

[binary1248]

Also, I'm not running simulation on every computer. I'm planning on just sending user input to the server, and receiving back any changes based on that input.

You always need to have some kind of minimal physics system on every host if you want the simulation to appear smooth. Between updates, you need to keep predicting the next simulation state in order to update your graphics/audio. If you do this only every time an update arrives, you will end up with a very "choppy" simulation.

[kurasu1415]

So, how is this setup then?

The server sends actual positions every physics frame at 30 physics frames per second.

I set it up with a one-frame lag by default, and then just interpolate between the last and most recent physical frames. Then if I incur lag from the server, then the client uses the previous correct frame to extrapolate a temporary "Current" , which I replace as soon as I get it from the server.

Does that make sense?

[binary1248]

You can't afford to send the game state 30 times a second. If you have more than a handful of objects the bandwidth requirement will explode. You just need to send state changes that the clients can't predict, such as changes in acceleration, and let the client side simulation take care of integrating the rest. Due to floating point imprecision and other effects it is still recommended to synchronize the game state every now and then

[kurasu1415]

So, the client handles the physics unless something like user input, is involved? Collisions and Gravity are all handled by the client? I assume you want to have some way for the server to make sure there isn't any fowl play going on.

[binary1248]

Clients just send their player's commands to the server as well as changing their own state based on those commands immediately. Everybody (including the server) runs the same simulation. The server just needs to have a master copy of the game state. If any client rejects the authoritative state from the server when it sends an update then it goes out of sync and gets kicked or whatever.

[kurasu1415]

How do you check if you have a successful connection between the client and server? I was basing it off of the values returned from the connect() and accept() functions.

The client's connect function returns "Not Ready", but the server's accept returns "Done". I was lead to believe that if the server's accept returns "Done", then the connection was successful. If that is the case, then why is the client saying it wasn't?

[binary1248]

Don't use non-blocking sockets. They are more of a headache than they are worth for your purpose. Just use selectors and it should become much more simple.

[kurasu1415]

I was using them to ensure that my program didn't get stuck. Would you recommend I put a wait time on a selector? I just don't want to wait too long since you don't want a game to fall 1 second behind.