ResourceStream class for custom resource loading

[Laurent]

Well, there is an InputStream already. Now what about an OutputStream?

It may be added in the future, yes. But let's first see if InputStream works well enough, and if people really need an OutputStream.

Also I think it would be worth adding overloads of operator >> and << to read/write data to/from streams. Those overloads can be easily implemented using the pure virtual Read/Write operations.

This is not the purpose of this class. It's meant as a bridge between custom data sources and SFML resource classes. You're not supposed to use it, you just need to implement it.
Standard library's streams are perfect for generic streaming with operators << and >>.

[gsaurus]

It may be added in the future, yes. But let's first see if InputStream works well enough, and if people really need an OutputStream.

well I need to write SFML resources into my own file format (with encryption, etc). Though I can keep using the GetPixelsPtr/GetSamples etc to stream it with my custom stream, and the LoadFromMemory instead of InputStream to read from my stream as I was doing before SFML 2..

Standard library's streams are perfect for generic streaming with operators << and >>.

Maybe InputStream should inherit from std::istream then (though I wanted to create my own derived stream some time ago, gave up and ended by creating my own class).
If SFML resources are stored within other information (headers, etc) it makes sense to use only one stream instead of somehow creating an InputStream over some other custom stream class at specific points of the streaming process.

On the network example, we have to create an sf::InputStream over an sf::Packet. So we stream normal information through the packet and stream SFML resources through that InputStream. It works but it's not that elegant/simple. We're using two streams to the same channel, makes some confusion to me.

[Laurent]

The goal was to have something simple to implement. Standard streams are too complicated: who knows how to override the std::streambuf virtual functions? who knows that this class even exists?

Moreover, using them in the public API of SFML would create inconsistencies in functions/class naming.

But I agree that in a perfect world, standard streams should be always used for this kind of stuff.

[gsaurus]

That's why I suggested the overloads on InputStream (instead of std), like Packet.
I understand that you don't want to discourage the use of the standard, though somehow it's already happening with Packet, no?

I keep thinking that Packet isn't in conformance with Input/OutputStream, since packet is a streaming specialization (and the name InputStream suggests genericity)

[Silvah]

Maybe InputStream should inherit from std::istream then (though I wanted to create my own derived stream some time ago, gave up and ended by creating my own class).

Or, better still, write a class derived from std::streambuf that forwards everything to an InputStream.

[Laurent]

That's why I suggested the overloads on InputStream (instead of std)

And who would use them?
From SFML side, I only need the Read/Size/Seek/Tell functions to implement the callbacks required by the resource loading libraries.
From user side, you only need to implement the Read/Size/Seek/Tell functions to create a SFML-compatible stream.

I understand that you don't want to discourage the use of the standard, though somehow it's already happening with Packet, no?

I don't see how sf::Packet could use standard streams. I need my own overloads of << and >> operators, I can't use those provided by std::i/ostream. And if I can't use std::i/ostream, std::streambuf is useless too.

[gsaurus]

I need my own overloads of << and >> operators

That's my point. Sorry if I have been having trouble expressing myself.
You provide overloads of << and >> for the portable types on sf::Packet because it is important over the network streams. The same could apply for streams in general.
I think << and >> for the portable types would be a good addiction in sf::InputStream/OutputStream, and thus making sf::Packet a specialization of those.

How would it be useful? If someone (like me) want to stream portable types through the network with encryption, we can't use the Packet overloads. We have to reimplement those overloads ourselves in a custom stream, stream the encrypted data into memory, and finally append it into a sf::Packet.
If it was defined at the InputStream and OutputStream level we just had to create a custom intermediate stream to encrypt our data, and stream it directly into a packet stream.

This is mostly my point, obviously such feature would also be great for many other custom streams (like file streams, etc), for when we want to stream not only textures and sound, but other portable information like headers.

[Laurent]

You provide overloads of << and >> for the portable types on sf::Packet because it is important over the network streams. The same could apply for streams in general.
I think << and >> for the portable types would be a good addiction in sf::InputStream/OutputStream, and thus making sf::Packet a specialization of those.

But how would it work? All these operator overloads would have to be virtual, which makes no sense.

How would it be useful? If someone (like me) want to stream portable types through the network with encryption, we can't use the Packet overloads.

sf::Packet has two virtual functions exactly for this purpose.

[gsaurus]

But how would it work? All these operator overloads would have to be virtual, which makes no sense.

Like sf::Packet, those overloads rely on calls to the virtual Read/Write operations which provide/take raw data.

to stream portable types through the network with encryption

sf::Packet has two virtual functions exactly for this purpose.

Oh, the private functions, I should have looked at it more carefully, thanks

[Laurent]

Like sf::Packet, those overloads rely on calls to the virtual Read/Write operations which provide/take raw data.

Sorry, you lost me. This might require a little more details

[gsaurus]

Something like this

Code: [Select]

// Possible Stream example
virtual ~Stream ();
virtual Int64 Read(char* data, Int64 size)=0;
virtual void Write(const char* data, Int64 size)=0;
virtual Int64 Seek(Int64 position)=0;
virtual Int64 Tell()=0;
virtual Int64 GetSize()=0;


Stream& operator <<(Uint32 data)
{
    Uint32 toWrite = htonl(data);
    Write(&toWrite, sizeof(toWrite));
    return *this;
}


Stream& operator >>(Uint32& data)
{
    if (Tell() + sizeof(value) <= GetSize() )
    {
        Uint32 value;
        Read(&value,sizeof(value));
        data = ntohl(value);
    }
    return *this;
}

...

[Laurent]

Ok I see. You suggest to create a generic binary/data stream.

This is interesting, but it is usually done with two classes: the front-end, with operators << and >>, which formats data and then read/write it to an abstract back-end (a "buffer", or "device"), which is represented by a base class with read/write/seek/... virtual functions.

Examples are std::i/ostream and std::streambuf, QIODevice and QDataStream, etc.

I'm not sure if I want to go this way for SFML, I don't mean to provide such generic mechanisms. I prefer to stick to the required features, and design a more generic/flexible abstraction later if needed.

[gsaurus]

Ok, I can still do it myself over InputStream. I hope OutputStream doesn't take too long to come up though. I still think that a generic stream could work better in conformance with Packet anyway.

The packet mechanism to process data isn't very dynamic, maybe functors could work better. Edit: err assigning them every time a packet is created wouldn't be good neither :?