Nonblocking Tcp Socket Status is Never sf::Socket::Done

[Ant]

I'm running into problems with nonblocking Tcp sockets where the client is unable to receive the sf::Socket::Done status despite that the server accepting the connection. After the server accepts the connection, the client's socket status becomes NotReady.

Server minimal example:

#include <chrono>
#include <iostream>
#include <SFML/Network.hpp>
#include <thread>

int main()
{
        std::cout << "Starting server\n";

        unsigned long updateRate = 30;

        sf::TcpListener listen;
        listen.setBlocking(false);
        if (listen.listen(2011) != sf::Socket::Done)
        {
                std::cout << "Failed to listen at port 2011\n";
                return -1;
        }

        std::cout << "Listening on port 2011\n";

        sf::TcpSocket client;
        client.setBlocking(false);
        while (true)
        {
                if (listen.accept(client) == sf::Socket::Done)
                {
                        break;
                }

                //simulate work
                std::this_thread::sleep_for(std::chrono::milliseconds(updateRate));
        }

        std::cout << "Connected... waiting for packet\n";
        sf::Packet packet;
        while (true)
        {
                sf::Socket::Status status = client.receive(packet);
                if (status == sf::Socket::Done)
                {
                        int num;
                        packet >> num;
                        std::cout << "Received data. " << num << " \n";
                        break; //TODO: loop a couple times to check if receiving multiple data.
                }
               
                //Simulate work
                std::this_thread::sleep_for(std::chrono::milliseconds(updateRate));
        }

        std::cout << "Disconnecting server.\n";
        client.disconnect();
        return 0;
}

Server output:

Starting server
Listening on port 2011
Connected... waiting for packet
 

Client minimal example:

#include <chrono>
#include <iostream>
#include <SFML/Network.hpp>
#include <thread>

int main()
{
        std::cout << "Starting client\n";

        unsigned long updateRate = 30;
        sf::TcpSocket client;
        client.setBlocking(false);

        while (true)
        {
                sf::Socket::Status status = client.connect(sf::IpAddress("127.0.0.1"), 2011);
                if (status == sf::Socket::Done)
                {
                        break;
                }

                //Simulate work
                std::this_thread::sleep_for(std::chrono::milliseconds(updateRate));
        }

        std::cout << "Connected... sending packet\n";
        sf::Packet packet;
        while (true)
        {
                int num = 456;
                packet << num;

                sf::Socket::Status status = client.send(packet);
                if (status == sf::Socket::Done)
                {
                        std::cout << "Sent data. " << num << " \n";
                        break;
                }
               
                //Simulate work
                std::this_thread::sleep_for(std::chrono::milliseconds(updateRate));
        }

        std::cout << "Disconnecting client.\n";
        client.disconnect();
        return 0;
}

Client output:

Starting client

When I set the client's socket to blocking (server remains unblocking), then it proceeds as expected.
Server output

Starting server
Listening on port 2011
Connected... waiting for packet
Received data. 456
Disconnecting server.

Client output

Starting client
Connected... sending packet
Sent data. 456
Disconnecting client.

This is tested on Windows 10.
I'm using SFML version 2.5.1.

I will continue looking into this, and I'll post anything I identified in my investigation.

[Ant]

Update:
on the client, Windows is returning WSAEWOULDBLOCK, and SocketImpl::getErrorStatus maps to that error to Socket::NotReady.

According to https://docs.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2
WSAEWOULDBLOCK is:

Resource temporarily unavailable.
    This error is returned from operations on nonblocking sockets that cannot be completed immediately, for example recv when no data is queued to be read from the socket. It is a nonfatal error, and the operation should be retried later. It is normal for WSAEWOULDBLOCK to be reported as the result from calling connect on a nonblocking SOCK_STREAM socket, since some time must elapse for the connection to be established.

If I ignore this specific error message (using a break on that specific case), and if I start the server before the client, the applications proceed as expected. Starting the client before the server is ready would cause it to fail.

I closely followed what parameters are we passing into the win socket methods (in particular creation, blocking, and connect), and I haven't found anything suspicious when following along SFML's code and MS's documentation.

I found a few posts that may be related to win sockets, but I haven't quite comprehend it yet.
https://forums.codeguru.com/showthread.php?317429-Why-do-I-get-WSAEWOULDBLOCK-when-calling-recv()
https://stackoverflow.com/questions/14010285/wsaewouldblock-handling

I've also read that winapi returns WSAEWOULDBLOCK when there is nothing to read from the socket. However I tested against this by having the server populate a packet and sending it immediately after successfully accepting the connection.


Another thing that puzzles me is that I'm not seeing this bug in my actual application. I created this minimal case to investigate a different bug where I'm getting multiple callbacks from one packet. However this particular bug only occurs on the Hello World programs. The only thing that comes to mind is the complexity between the two applications. The real application essentially has a continuous loop that does a host of other things every frame. I was hoping to simulate this using thread sleep. I've attempted at higher intervals, too (up to 5 seconds).

This quote from one of the SO posts lead me to suspect that it's related to the infinite while loop.

It is a performance optimization. You can speculatively try to send synchronously (which is very fast). And only if it fails you are supposed to schedule an asynchronous IO

I'll take another look at the forum posts to see if something clicks in my head.

[Ant]

I experimented with this again, and I didn't find anything new. I decided to simply interpret NotReady as a successful return code primarily because:

SocketImpl::getErrorStatus maps Status::NotReady to WSAEWOULDBLOCK and WSAEALREADY.

According to windows-sockets-error-codes:

WSAEALREADY
A nonblocking connect call is in progress on the specified socket.

This is fine. I can work around this particular code by checking if I already sent a request or not.

WSAEWOULDBLOCK
It is normal for WSAEWOULDBLOCK to be reported as the result from calling connect on a nonblocking SOCK_STREAM socket, since some time must elapse for the connection to be established.

Seeing that it is normal for Windows to report that error code for nonblocking sockets, I think it would be fine to treat a normal code as a success especially since I never seen it return anything else in a successful case.

I'm going to take a break on this. When I look into this again, the next thing I'll look into is the FD_CONNECT notification. The document points me to use the select function:
https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-select

I recall that SFML already implements a SocketSelector. Admittedly I haven't researched the SocketSelector extensively. Perhaps I've been using nonblocking sockets incorrectly, and I was suppose to use the socket selector this whole time. It's something I'll look into.

[eXpl0it3r]

The FAQ has a good section discussing blocking vs non-blocking sockets and suggestion to use a socket selector.

WSAEWOULDBLOCK
It is normal for WSAEWOULDBLOCK to be reported as the result from calling connect on a nonblocking SOCK_STREAM socket, since some time must elapse for the connection to be established.

Seeing that it is normal for Windows to report that error code for nonblocking sockets, I think it would be fine to treat a normal code as a success especially since I never seen it return anything else in a successful case.

The documentation also states: "It is a nonfatal error, and the operation should be retried later."
You can't treat this error as success, because the socket is not yet ready, but eventually it should return successfully.

I can't directly see an issue with the code, but I'd probably have to try it first myself and debug a bit.