Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: IsSocketReady or similar stand alone function  (Read 5351 times)

0 Members and 1 Guest are viewing this topic.

Joshua Flynn

  • Full Member
  • ***
  • Posts: 133
    • View Profile
IsSocketReady or similar stand alone function
« on: May 27, 2012, 08:28:20 am »
Whilst I know the idea is a bit C-ish, after using sockets on SFML, I've had numerous instances where I wanted to poll whether or not the socket is ready, but without causing a block. As noted, there have been problems with connect, receive in non-blocking which I've discussed previously with Laurent - a call to select or similar is still required for connecting non-blocking sockets (according to MSDN), which is how you poll if a socket is ready or not.

Whilst there is the Selector class with it's wait function - the Selector class is really for multiple sockets, and incurs unnecessary overheads - Adding and Removing sockets, notably. Where-as IsSocketReady (or similar) could easily be a single, direct call to wait that returns a simple boolean. Any element of efficiency is an improvement, and I feel an 'IsSocketReady' (or similarly named function) could be useful inside SFML itself (I've noticed sf:Http's SendRequest might block indefinitely on receive - and polling the socket with a timeout would be a way to solve the problem - but I don't want to incur Selector's overhead).


The function would be simple to implement - it would be a modified version of Selector's 'wait' function designed to take the first argument as a single socket, and the second argument as a timeout value, which returns a boolean value as to whether or not it's ready (returning false if it times out).


Thoughts?

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: IsSocketReady or similar stand alone function
« Reply #1 on: May 27, 2012, 09:24:26 am »
I feel like this feature is useless. If you want to know whether there's data on the socket or not, then you most likely want to receive it. In other words, what would this feature add that non-blocking sockets don't allow?
Laurent Gomila - SFML developer

Zefz

  • Newbie
  • *
  • Posts: 21
    • View Profile
Re: IsSocketReady or similar stand alone function
« Reply #2 on: May 27, 2012, 07:22:08 pm »
I found this feature useful so I implemented this myself in my code.

I can see why a socket.isSocketReady would be useful if you do not want to use a SocketSelector.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: IsSocketReady or similar stand alone function
« Reply #3 on: May 27, 2012, 08:08:58 pm »
Quote
I found this feature useful
Can you explain your use case?
Laurent Gomila - SFML developer

Zefz

  • Newbie
  • *
  • Posts: 21
    • View Profile
Re: IsSocketReady or similar stand alone function
« Reply #4 on: May 27, 2012, 10:32:23 pm »
It's to check if the socket has new data ready without pulling the data at the same time. This is useful in my specific code for both debugging and readable code.

I also encountered a problem where SocketSelector was reporting new data when there wasn't really anything. I guess this is rather a bug in the socket selector. Could be related to the non-blocking sockets connect bug in windows (issue 194). In any case, my implementation solved the problems this was causing.

Joshua Flynn

  • Full Member
  • ***
  • Posts: 133
    • View Profile
Re: IsSocketReady or similar stand alone function
« Reply #5 on: May 28, 2012, 01:00:52 am »
I feel like this feature is useless. If you want to know whether there's data on the socket or not, then you most likely want to receive it. In other words, what would this feature add that non-blocking sockets don't allow?

How would having a non-blocking socket permit me to create a timeout function? Because I can't access the socket directly as it's private (unless I create a WinSock socket from scratch), and why would I want to even implement a looped iterator that would take more hassle than a selector with a blocking socket that uses wait?

Besides, erroneous assumption: " If you want to know whether there's data on the socket or not, then you most likely want to receive it." - So you're mistakenly assuming that data is there to be read and receive should immediately try to receive data that isn't there? So if the connection fails (server side) and no data will be sent - receive shouldn't ignorantly assume data will be there. It's got to check first. sf::Http has frozen up several times because it assumes data is going to be sent when trying to receive data (which makes the 100% perfect connection fallacy - it won't get any additional data if the server terminates, and you can't reset the connection because receive is blocking).

Joshua Flynn

  • Full Member
  • ***
  • Posts: 133
    • View Profile
Re: IsSocketReady or similar stand alone function
« Reply #6 on: May 28, 2012, 01:07:53 am »
I found this feature useful so I implemented this myself in my code.

I can see why a socket.isSocketReady would be useful if you do not want to use a SocketSelector.

I'm pretty much implementing similar stuff in my code (but it uses the pre-existing Selector class), but Selector incurs overhead (and sf::Http's SendRequest fix would be to use an 'IsSocketReady' to poll for data with timeout lest the server is being unkind and dropping a connection). I know, it shouldn't drop inbetween connection, sending and receiving - but because it's a file it's receiving, the server has time enough to drop midway through (don't ask me why - I just come back to a frozen program).

@Laurent
The IsSocketReady would also solve that nonblocking socket connect issue - because people could call it after connect (which would permit it to work afterwards).

Please? I'm sure it can be useful.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: IsSocketReady or similar stand alone function
« Reply #7 on: May 28, 2012, 10:13:54 am »
Quote
How would having a non-blocking socket permit me to create a timeout function? Because I can't access the socket directly as it's private (unless I create a WinSock socket from scratch), and why would I want to even implement a looped iterator that would take more hassle than a selector with a blocking socket that uses wait?
We're talking about a IsReady function, not a ReceiveWithTimeout, right? So what I said is simply that IsReady is similar to Receive in non-blocking mode.
if (socket.isReady())
{
    if (socket.receive(...) == sf::Socket::Done)
    {
        // process data
    }
}
else
{
    // do nothing
}
if (socket.receive(...) == sf::Socket::Done)
{
    // process data
}
else
{
    // do nothing
}

Quote
So you're mistakenly assuming that data is there to be read and receive should immediately try to receive data that isn't there? So if the connection fails (server side) and no data will be sent - receive shouldn't ignorantly assume data will be there. It's got to check first.
Why? If there's no data, receive will simply tell you that there were nothing and return immediately. In other words, receive already checks for you, there's no need to have an extra function.

Quote
sf::Http has frozen up several times because it assumes data is going to be sent when trying to receive data (which makes the 100% perfect connection fallacy - it won't get any additional data if the server terminates, and you can't reset the connection because receive is blocking).
receive returns an error status if the connection is broken. It can't block in this case.
If you noticed a strange behaviour, please make a bug report :)

Quote
but Selector incurs overhead
No that much. And the only way to implement it (unless I completely rewrite SFML sockets) is to use a selector anyway.

Quote
The IsSocketReady would also solve that nonblocking socket connect issue - because people could call it after connect (which would permit it to work afterwards).
There's nothing that isReady would do that I can't do internally in connect. So this function is not needed to make connect work.

Quote
Please? I'm sure it can be useful.
I'm not. Please provide a relevant use case ;)
Laurent Gomila - SFML developer

Joshua Flynn

  • Full Member
  • ***
  • Posts: 133
    • View Profile
Re: IsSocketReady or similar stand alone function
« Reply #8 on: May 30, 2012, 02:41:25 pm »
We're talking about a IsReady function, not a ReceiveWithTimeout, right?

Correct. I'm just saying IsReady would be used within the (Blocking)ReceiveWithTimeout (as nonblocking requires additional complexity just to implement and can be... sporadic - either I can use a 'IsReady' and a 'BlockingReceive', or I have to write an iterative loop with a non-blocking socket that has a tendency not to complete any work - you'd agree two simple functions is easier than a complicated iterative for/while loop that has to handle nonblocking issues).

Why? If there's no data, receive will simply tell you that there were nothing and return immediately. In other words, receive already checks for you, there's no need to have an extra function.

Receive doesn't return if there is no more data. It blocks (the second post there even suggests using select to poll)... Especially with HTTP.

I personally found receive froze up if called too many times (which is why I had to write a Receive with timeout function for my Telnet class - it uses the Selector Wait function to poll - and I'm now using a modified version of SFML 1.6 where the Http's SendRequest function's timeout actually applies to the receive by using... again, Selector with the Wait function). Since modifying both, the program thus far has never blocked once (previously it would block after several hours). It's been running continuously for at least 1-2 days now (previously I'd be lucky if it went 12 hours without a block).

receive returns an error status if the connection is broken. It can't block in this case.
If you noticed a strange behaviour, please make a bug report :)

Receive does not return if there is no additional data to be received (I can swap the DLLs around and wait until the program eventually blocks - of which the message will tell you what point it's reached - which can be narrowed down to SendRequest). It's not strange behaviour, as noted in the link above - HTTP will block if there is no additional data (but I find Receive blocks anyway if there is no additional data in other circumstances). If I get time, I'll draft an example piece of code (it's for 1.6 so I don't know if 2.0 solves this).

No that much. And the only way to implement it (unless I completely rewrite SFML sockets) is to use a selector anyway.

Selector incurs additional overhead by including an unnecessary (for this specific task) std::map - which I shouldn't need. In theory, you would only need a modified version of Wait - you shouldn't need to 'rewrite SFML sockets'. Besides - you can't implement Selector under a Socket - you'd likely get a circular dependency (A socket with a defined function trying to make a function call to a not-yet defined selector... or a defined selector making a function call that requires... a defined socket... eep!).

All I'm saying is this is copied and modified into 'IsReady()' for a socket.

unsigned int SelectorBase::Wait(float Timeout)
{
    // Setup the timeout structure
    timeval Time;
    Time.tv_sec  = static_cast<long>(Timeout);
    Time.tv_usec = (static_cast<long>(Timeout * 1000) % 1000) * 1000;

    // Prepare the set of sockets to return
    mySetReady = mySet;

    // Wait until one of the sockets is ready for reading, or timeout is reached
    int NbSockets = select(myMaxSocket + 1, &mySetReady, NULL, NULL, Timeout > 0 ? &Time : NULL);

    return NbSockets >= 0 ? static_cast<unsigned int>(NbSockets) : 0;
}
 

So the needed variables are self-contained, and only the socket itself is passed. It wouldn't 'defeat' selector's wait as selector is for multiple sockets (and intended to be more optimised for multiple sockets) - but an 'IsReady' would help with single sockets.

I'm not. Please provide a relevant use case ;)

At present, I have a program that connects to a website, it extracts the image link from the HTML, and then connects to the image link, downloads the image and writes it to file. The program then sleeps for approximately 4 minutes and 30 seconds (a new image is generated every 5 minutes, but this is to prevent it from 'missing' an image), before attempting to do this again.

Because the website does not archive it's data, and this information is being used in an effort to see if ionospheric disturbances can predict earthquakes, it is important that as little of the data is lost, and the program must operate without human intervention (I cannot watch over it all day), this means it must:

A) Not block.
B) Avoid overly complicated code. (As it could fail whilst I am not around, and every moment wasted debugging complicated code is another image lost - the site doesn't archive).
B2) This means no complicated 'threaded' applications (which introduce race conditions and other nightmares).
B3) And no non-blocking sockets (as they introduce their own set of quirks and issues - which is even more complicated to implement than a simple poll-receive on a blocking socket).

Now I could implement a selector:

SelectorTCP Test;
Test.Add(SomeSocket);
while(Test.Wait(0.5) != 0)
{
SomeSocket.Receive(); //Yes, this doesn't go anywhere but it's pseudo-code
}
Test.Remove(); //Maybe this is unnecessary - but I can't be sure. For all I know Selector might close any sockets it has stored.
 
That's three unneccessary commands with the overhead of a std::map. Why not:

while(SomeSocket.IsReady(0.5))
{
SomeSocket.Receive();
}

Which could also be used to solve the non-blocking socket issue we were discussing via email earlier (one where non-blocking sockets need select to complete the connection):

NonblockingSocket.Connect("fakeaddress.com",23);
while(!NonblockingSocket.IsReady(0));

Also, Zefz arrived at a similar conclusion I did in regards to an IsReady() type function (maybe not a standalone, as I'm still C in my thinking - perhaps a subset function of the individual socket itself), so perhaps such a solution has merit.
« Last Edit: May 30, 2012, 02:47:01 pm by Joshua Flynn »

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: IsSocketReady or similar stand alone function
« Reply #9 on: May 30, 2012, 03:07:55 pm »
Quote
Correct. I'm just saying IsReady would be used within the (Blocking)ReceiveWithTimeout
There's already a ReceiveWithTimeout function in the tutorials, which is already very simple. So this is not really a strong argument.

Quote
Receive doesn't return if there is no more data
I was talking about a non-blocking socket, of course.

Quote
I personally found receive froze up if called too many times
Bugs are something different, they are not relevant in this discussion (they can be solved); here we are talking about design decisions.
If you have problems with SFML please report them appropriately in separate discussions.

So, let's continue assuming that there's no bug.

Quote
Receive does not return if there is no additional data to be received
I said "if the connection is broken", not "if there's no more data". Don't mix everything.

Quote
Selector incurs additional overhead by including an unnecessary (for this specific task) std::map
It's gone in SFML 2, there's no more overhead.
It would greatly help if this discussion was based on SFML 2, SFML 1.6 is already quite outdated.

Thanks for the detailed use case. I'm still not sure it can't be implemented efficiently (and according to your requirements) with the current SFML features, but unfortunately I don't have the time to check.

But, in any case, since the IsReady function can be implemented with 3 lines of code (and with no overhead in SFML 2), you can easily write your own implementation. If in the future, many users have to implement and use this function, maybe I can consider adding it to SFML. This would be a much better base for taking such a decision, I don't like implementing new stuff based on a single feedback. Don't you agree with this strategy?
Laurent Gomila - SFML developer

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: IsSocketReady or similar stand alone function
« Reply #10 on: May 30, 2012, 03:28:36 pm »
On a side note, I'd like to add that I'm aware that network programming is much more complete (complex?) than what SFML provides. If you go with OS specific functions, you can easily implement asynchronous calls, polling, events, etc.

In this regard, SFML will probably be improved in the future so this kind of in-depth discussions, with concrete use cases, are very useful.
Laurent Gomila - SFML developer

Joshua Flynn

  • Full Member
  • ***
  • Posts: 133
    • View Profile
Re: IsSocketReady or similar stand alone function
« Reply #11 on: May 31, 2012, 05:34:43 pm »
There's already a ReceiveWithTimeout function in the tutorials, which is already very simple. So this is not really a strong argument.

You had asked for clarification on what function I was referring to - I was clarifying it and providing an example of usage. The pre-existence of a tutorial for receive does not negate IsReady's practical applications in completing a non-blocking connect.

I was talking about a non-blocking socket, of course.

I wasn't. And sf::Http uses blocking sockets - so non-blocking sockets aren't really relevant. Besides, they introduce additional complexities (I don't want to keep hopping between non-blocking and blocking every time a problem crops up - can't connect with non-blocking, blocking on receive, etc - alternating just introduces confusing code).

Bugs are something different, they are not relevant in this discussion (they can be solved); here we are talking about design decisions.
If you have problems with SFML please report them appropriately in separate discussions.

So, let's continue assuming that there's no bug.

It's no bug as it's a blocking receive (although the HTTP comment might be a bug, IsReady() is the feature I'm suggesting as the fix - I can't suggest a feature to fix a bug without the feature first).

I said "if the connection is broken", not "if there's no more data". Don't mix everything.

Being pedantic. If the connection is broken by the time it gets to receive, there won't be any data to receive anyway. Not sure what the difference would be (given connect isn't the issue as I'm not using non-blocking - although non-blocking connect could benefit).

It's gone in SFML 2, there's no more overhead.
It would greatly help if this discussion was based on SFML 2, SFML 1.6 is already quite outdated.

You'll need to update the online files as this is my reference, and I cannot find Selector's hpp file here so I can't tell what variables have and have not been defined for it.

But, in any case, since the IsReady function can be implemented with 3 lines of code (and with no overhead in SFML 2), you can easily write your own implementation. If in the future, many users have to implement and use this function, maybe I can consider adding it to SFML. This would be a much better base for taking such a decision, I don't like implementing new stuff based on a single feedback. Don't you agree with this strategy?

To retort, people have previously used SocketSelector to perform a timeout on receive with single sockets:
http://en.sfml-dev.org/forums/index.php?topic=5781.0

Clearly there is an implicit use case, with Selector being used as the work around.

Previously, other HTTP reliant functions in SFML have blocked on trying to receive from an (unresponsive) server:
http://en.sfml-dev.org/forums/index.php?topic=928.0

So perhaps modifying SendRequest so it timeouts on Receive (similarly to GetPublicAddress)? (It presently only times out on connect - in my experience the server oddly drops the connection during receive). I'm merely suggesting an 'IsReady()' function as the solution to the SendRequest problem (you call 'IsReady(Timeout)' on the socket just before '.Receive()' is called).

To summarise points:

    Suggesting IsReady() function for sockets to:
  • Avoid Selector class overhead (and to prevent Selector from becoming the 'timeout' class).
  • Fix sf::Http's SendRequest so it doesn't block on receive due to server problems.
  • Fix non-blocking connect not connecting (as select completes the process).
  • Permit the user to quickly create non-symmetrical send/replies on a single socket (useful in HTTP, SMTP and IMAP operations).

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: IsSocketReady or similar stand alone function
« Reply #12 on: May 31, 2012, 06:12:11 pm »
This would require in-depth investigation, and I don't have the time to do it now anyway. So if you feel like summarizing all we said in a ticket on the tracker, I suggest that you do so, so that this issue can collect more feedback and eventually be processed later.
Laurent Gomila - SFML developer