SFML community forums
Help => Network => Topic started by: Joshua Flynn on April 24, 2012, 10:51:21 pm
-
I would have assumed that the HTML side of SFML would have a fail-safe (I.E. return NULL string, error state, etc) for not being connected to the internet (given this could easily occur) or when a website is not available (which can also easily occur during downtime).
Given I am trying to run an automated program to automatically download data from a website that doesn't archive data (and both the internet connect and the website can fail)...
How do you check for the lack of an internet connection, and how do you check for whether or not the website is there, without SFML crashing the program? Surely this was a consideration?
-
I have no idea what you mean with 'the HTML side of SFML'. SFML does not have anything that relates to HTML.
So I guess you're trying to connect to a server and download something and the connection should get established by SFML?
In that matter I guess you're using sf::TcpSocket (http://www.sfml-dev.org/documentation/2.0/classsf_1_1TcpSocket.php). Further on I guess you're using the connect(...) function which in fact returns a enum Status with which you can check if the connection failed.
For the part where the 'HTML can fail' I can't guess there what you're doing...
Provide some code otherwise we can't help you and just make lucky guess into the blue.
-
I have no idea what you mean with 'the HTML side of SFML'. SFML does not have anything that relates to HTML.
I guess he's talking about sf::Http.
How do you check for the lack of an internet connection, and how do you check for whether or not the website is there, without SFML crashing the program?
Could you please start by describing your problem, showing your code, etc. before assuming that it is a bug in SFML? ;)
-
I have no idea what you mean with 'the HTML side of SFML'. SFML does not have anything that relates to HTML.
I guess he's talking about sf::Http.
How do you check for the lack of an internet connection, and how do you check for whether or not the website is there, without SFML crashing the program?
Could you please start by describing your problem, showing your code, etc. before assuming that it is a bug in SFML? ;)
Laurent is correct here. I meant HTTP but I wrote HTML... even though they are completely different things.
I took a look through and found I had a captain stupid moment (a string is not an array). I seem to making a lot of these. Thank you for the help. I thought it was weird (I didn't think SFML would be unstable).
-
I still want to ask, is there a way to test if there is a working internet connection in SFML?
-
I still want to ask, is there a way to test if there is a working internet connection in SFML?
What about launching a connection to the remote host and checking if it fails?
-
I took a look through and found I had a captain stupid moment (a string is not an array). I seem to making a lot of these.
Maybe they'll happen less frequently if you don't use arrays, but instead STL containers ;)
-
I still want to ask, is there a way to test if there is a working internet connection in SFML?
What about launching a connection to the remote host and checking if it fails?
How would I check if it fails? Bear in mind I'm dealing with a dual problem where either the internet connection can fail whilst I'm not there, or the website itself can go down (and give the faux-pas impression it's an internet connection failure). I'm trying to write a program where it can attempt to correct numerous issues itself as I might be away from it for hours.
On a similar note, I'm trying to access my router via HTTP - which seems valid enough, but I am unsure how to transmit the authorisation for it? Fiddler2 suggests it's in the header, but I tried duplicating Fiddler2's setup for the authorisation and I only get a 400 (Bad Request) instead of a 401 (Authorisation Error). Any hints?
-
How would I check if it fails? Bear in mind I'm dealing with a dual problem where either the internet connection can fail whilst I'm not there, or the website itself can go down (and give the faux-pas impression it's an internet connection failure). I'm trying to write a program where it can attempt to correct numerous issues itself as I might be away from it for hours.
Ok, then why not just resend the request as long as it returns a connection error?
On a similar note, I'm trying to access my router via HTTP - which seems valid enough, but I am unsure how to transmit the authorisation for it? Fiddler2 suggests it's in the header, but I tried duplicating Fiddler2's setup for the authorisation and I only get a 400 (Bad Request) instead of a 401 (Authorisation Error). Any hints?
I have no idea, I haven't looked at authorization support for SFML yet.
-
I took a look through and found I had a captain stupid moment (a string is not an array). I seem to making a lot of these.
Maybe they'll happen less frequently if you don't use arrays, but instead STL containers ;)
Not really, given this was merely a programming typo. I don't think STL containers can prevent programming typos - it's never as good as people claim it to be.
-
Could you please start by describing your problem, showing your code, etc. before assuming that it is a bug in SFML? ;)
I've never used sf::Http so I still could be wrong. ;D
If the website goes down or to be precise the webserver can not be reached anymore and you're performing a request, you'll receive a sf::Response (http://www.sfml-dev.org/documentation/2.0/classsf_1_1Http_1_1Response.php) which has a function getStatus() with which you can figure out what exactly went wrong (sf::Status::ConnectionFailed or sf::Status::NotFound).
I assume the problem now is that you can't tell if it's the connection of the client that is disconnected or the one from the server. In this case you could try to reach your server (or any server that you know of should always be online) with sf::TcpSocket (http://www.sfml-dev.org/documentation/2.0/classsf_1_1TcpSocket.php). If this fails you know it's the internet connection of the client. If this succeeds you can try your http requests and check the response.
For you authentification problem, you'll have to read a bit about http header authentification but it shouldn't be that hard.
Wikipedia article (https://en.wikipedia.org/wiki/Basic_access_authentication)...
-
Ok, then why not just resend the request as long as it returns a connection error?
In the case of an internet connection failure, I'd like it to attempt to fix the failure (which might be a result of changing IPs and failing mid-way through). In this case it'd have to call RenewIP.
In the rare case of the website blocking, it actually has to change it's own IP (and it appears to have to connect directly to the router to make it reboot long enough to get a new IP assigned). The reason is, I'm running a program that archives ionospheric data from a website every 5 minutes, but it seems to be flagging up anti-denial of service attacks (I don't think once every 4-5 minutes is that severe?), of which a new IP would bypass.
Whilst normally I wouldn't raise the merits, I am trying to gather data so I can analyse the ionospheric patterns to see if they preclude earthquake events. But that's nigh on impossible if the website blocks me for hours on end (they don't even archive their own data - so every piece lost is lost permanently).
Last time I tried to contact them I didn't get a reply, so I'm not holding my breath.
-
I think eXpl0it3r's solution might be the best to know whether it's you or the host which is broken. The socket has no way to know this, so the only way is to check another (valid) host.
-
I think eXpl0it3r's solution might be the best to know whether it's you or the host which is broken. The socket has no way to know this, so the only way is to check another (valid) host.
Problem is, I need a website I know won't be down, so it's a bit of a catch there.
Conversely, in my time away, I've discovered telnet, learned about it, wrote a class to communicate with it, can reboot my router at will (and issue whatever other commands I'd prefer, including running a script of commands), and can now send emails to my (or any other, depending) yahoo account without flagging up spam filters, now with the intention of forming an email communication system between me and my program, and perhaps an email alert system.
All I wanted to do was change IPs.
-
Problem is, I need a website I know won't be down, so it's a bit of a catch there.
Try http://www.google.com/ (http://www.google.com/) ;D
-
Problem is, I need a website I know won't be down, so it's a bit of a catch there.
Try http://www.google.com/ (http://www.google.com/) ;D
Google gives 302 (redirect) and on following said redirect, returns 404 (page not found).
I suppose I mean more of a way the computer can connect to itself via the internet (but not locally) so it can check - rather than relying on a website to check - as I know the computer will be running.
-
There's not such thing in SFML and it's pretty OS specific (if I've understood you right).
As for google, I didn't meant you should connect with the HTTP class but just try to establish a connection and if the server responses you know the internet works or better you ping a fixed IP since the DNS server could be non functional either. ;D
Btw: Can you please edit the title from 'HTML' to 'HTTP' or better 'sf:Http'?
-
Whilst on the issue of networking, Laurent, I'm getting some strange behaviour when trying to connect to the yahoo mailing server when using the TCP socket.
When the socket is set to blocking mode, the connection is established, and it works fine.
However, when I set the socket to non-blocking mode (I need to be able to do this as otherwise the socket would stop the program), and run it in a loop, it never connects.
You can pretty much put the code as this (just comment out the one you want to test):
sf::SocketTCP Test;
//Test.SetBlocking(false); while( !(Test.Connect(25,"smtp.mail.yahoo.com") == sf::Socket::Done) );
//Test.Connect(25,"smtp.mail.yahoo.com");
printf("Connected!\n");
And the non-blocking one won't ever connect. Why is that?
Don't be fooled by the equality operator - it's bracketed and negated.
-
There's not such thing in SFML and it's pretty OS specific (if I've understood you right).
As for google, I didn't meant you should connect with the HTTP class but just try to establish a connection and if the server responses you know the internet works or better you ping a fixed IP since the DNS server could be non functional either. ;D
I'll have to get the TCP non-blocking non-connection issue solved before I can do that (otherwise, if I try to use blocking and connect when the internet is down, the program will freeze). Ping is a system command and I don't think it'd give me much in the way of feedback in C++.
It should be possible to 'connect to yourself'. Assuming you set up a socket to connect to your external IP and an open port, you probably could do that?
-
The non-working non-blocking connect is a known issue ;)
-
The non-working non-blocking connect is a known issue ;)
Argh! Okay, I've got the source code with the build projects for code::blocks, does it build the .DLL for the network in the lib folder? If so, I'll see if I can write a fixed version of the TCPSocket and run that.
-
Actually, I am confused.
In the Connect function, regardless of whether or not the socket is in blocking mode (block until something happens), or non-blocking mode (return as soon as possible), you have a timeout value.
Surely, if a timeout value is specified, the socket, by default, is non-blocking (it doesn't wait for something to happen)? Because it will quit out after a given time? If select() is passed a NULL pointer for it's time-value, then it's treated as blocking. (http://msdn.microsoft.com/en-us/library/windows/desktop/ms740141(v=vs.85).aspx) So surely, you'd want to re-write the Connect function, so that the time-out value only applies to non-blocking sockets?
This would solve both issues.
I took a further look at the code and after trying it, I found the key reason that the non-blocking socket won't connect is because when the socket is already connected, it returns an error. You could easily solve this by including:
if(Status == Socket::Error)
{
if(WSAGetLastError() == WSAEISCONN)
{
return (Status = Socket::Done);
}
}
Because if the purpose of the function is to connect, then it should return true if there is already a connection. I found on the first pass, select didn't catch it (default timeout value), on the second pass a 10037 (operation already in progress), and finally an 10056 (connection already established), which SFML should return not as an error, but a success (as the functions role is to connect - if it's already connected, it's succeeded).
The above function should be placed between:
Socket::Status Status = SocketHelper::GetErrorStatus();
//Here
// If we were in non-blocking mode, return immediatly
if (!IsBlocking)
-
Thanks for helping :)
Feel free to submit a patch, or at least to add a link to your message in the corresponding tracker issue.
-
Thanks for helping :)
Feel free to submit a patch, or at least to add a link to your message in the corresponding tracker issue.
I will do that, although I don't know where/how to submit a patch.
-
You can fork on github and submit a pull request, you can clone the repository and convert your modifications to a patch, or even just rewrite the function and sending the code to me.
-
Is there a way to inquire with a server to check to see if there is data waiting to be received?h
-
Could you elaborate? What is the problem that you're trying to solve?
-
Well, you know when you do fgetc, it returns EOF when there's no additional data to be read from the file?
Is there anything similar for recv? The problem is, if the socket is blocking (and I can't use non-blocking), if recv goes 'one byte over', it blocks. So I want to know if there is another byte in the data stream to receive, before I attempt to get it. Is there a function to poll the data on the stream/buffer?
-
If you want to know what I'm intending, I need to read all the information given by an IMAP server from an email of an unknown length (which, with attachments, could easily run into many megabytes and thus make static buffers a bad idea), but I read it in such a way receive won't block (I have no idea what indicates the end of the received data).
-
There are different approaches (depending on how your program works):
- non-blocking sockets (why can't you use them?)
- stop to wait and consider the data complete after a given timeout
- finding how the protocol encodes the end of data -- I'm sure it does
Explicitely encoding the end of data (or the total size to receive) is the only reliable way to know when to stop receiving with TCP (if the server doesn't close the connection when finished, like with HTTP), so almost all protocols do it.
-
There are different approaches (depending on how your program works):
- non-blocking sockets (why can't you use them?)
- stop to wait and consider the data complete after a given timeout
- finding how the protocol encodes the end of data -- I'm sure it does
Explicitely encoding the end of data (or the total size to receive) is the only reliable way to know when to stop receiving with TCP (if the server doesn't close the connection when finished, like with HTTP), so almost all protocols do it.
- SFML doesn't work with them, remember? And even with the hacky fix I made, I couldn't get recv to work with it.
- There's no way to know if the data is complete. If, say, I query 256 worth of data, and assume if I get 256's worth back, there's yet more to be received, what happens when there is strictly 256 worth of data on the stream? As unlikely as it sounds, it's not worth risking it, as if it's precisely 256 worth, it'd block.
- This may be possible, except the CRLF 'ending' is present in all the data streams at points where it isn't explicitly the end.
-
- SFML doesn't work with them, remember?
No. For me they work perfectly, except connect() but you don't need a non-blocking connect here.
- There's no way to know if the data is complete. If, say, I query 256 worth of data, and assume if I get 256's worth back, there's yet more to be received, what happens when there is strictly 256 worth of data on the stream? As unlikely as it sounds, it's not worth risking it, as if it's precisely 256 worth, it'd block.
That's why I said: use a timeout -- which means: if it waits for too long then it probably means that there's no more data to receive.
- This may be possible, except the CRLF 'ending' is present in all the data streams at points where it isn't explicitly the end.
If CRLF is a valid sequence that can appear in the middle, then the end is probably a double CRLF. Anyway, have you read carefully the specification of the protocol?
-
No. For me they work perfectly, except connect() but you don't need a non-blocking connect here.
Inverse for me. Connect works, but recv doesn't.
That's why I said: use a timeout -- which means: if it waits for too long then it probably means that there's no more data to receive.
Timeout might be worth trying, but doesn't that force receive into non-blocking mode... (see above... hehe).
If CRLF is a valid sequence that can appear in the middle, then the end is probably a double CRLF. Anyway, have you read carefully the specification of the protocol?
IMAP's FETCH <ID> BODY[TEXT] returns the text verbatim. The last command when sending an email in sockets is the return or \r\n (CRLF) command. SMTP specifies a CRLF with a dot and another CRLF, but this gets removed from the body of the text. The only way for me to be able to tell the end presently is if it matches the server return response, but it's risky - because if the text body contains the same response, it would end prematurely, and if the server doesn't return the same response, it won't end at all.
I'll try a timeout variant but I'm not holding my hopes out.
-
Inverse for me. Connect works, but recv doesn't
So please report it in a new thread, with detailed explanation and a complete and minimal example that reproduces the problem :)
Timeout might be worth trying, but doesn't that force receive into non-blocking mode... (see above... hehe).
A "receive with timeout" function is explained in the "selector" tutorial.
-
So please report it in a new thread, with detailed explanation and a complete and minimal example that reproduces the problem :)
I don't think the problem is with SFML, it's just that either the IMAP or the SMTP servers don't respond in a timely manner. I would need to look into it further before I could say either way, but I just want to get the email program finished first.
A "receive with timeout" function is explained in the "selector" tutorial.
This seems like the most useful solution to the problem.
-
A "receive with timeout" function is explained in the "selector" tutorial.
Worked, thank you!
-
I wanted to ask, in 1.6's Http class's SendRequest, it has a timeout function.
Given it only returns a Http::Page type, how does it indicate timeout occurred? (This is assuming it's the same Http class that previously succeeded in using a SendRequest, and this time times out).
-
It will return sf::Http::ConnectionFailed.
-
Thank you.
-
For some reason, in 1.6, when using sf::Http class' 'SendRequest' function in my program, after a while of numerous calls to SendRequest (we're talking once every 4 minutes and 30 seconds for hours at a time), SendRequest, even with a timeout defined, still hangs.
It'd be impossible for me to supply a 'functional snippet' as I've built numerous low level classes to form a series of upper level classes, but I know this - as my program prints out statements, I know it hangs at either 'DownloadPage' (basically a call to Http.SendRequest), or .GetPage().GetStatus() (again, which is basically GetStatus for a Http::Response). Seeing as GetStatus shouldn't do anything but inquire with data in memory, the only thing I can determinate is SendRequest itself is hanging during either the send or receive phase - as it acts suspiciously like blocking. Setting a timeout value has no apparent effect.
-
SFML 1.6 is already dead (which means: I won't fix anything), you should test with SFML 2 instead.