SFML community forums
General => SFML projects => Topic started by: Disch on June 13, 2011, 05:07:49 am
-
I figured I would mention this now that I'm more or less done with my other SFML related project.
I'm starting work on an add-on library to SFML... SFSL (SFML File System Library).
As the name implies, one of the primary goals is to provide cross-platform way to interact with the file system.
It'll have the expected goodies:
- File I/O
- File/Folder creation, renaming, deletion
- Directory scanning (iterating over files/folders)
Plus some perks. The biggest one being .zip file reading/writing in a sane manner (most libs I've seen/used expect you to bend over backwards to use zips)
Benefits over stdio/iostream for normal files
- 64-bit offsets
- Ability to truncate/shorten files without having to completely wipe them
- Unicode filenames (windows' standard libs choke on this hard)
- Utility functions for endian-safe read/write of integral types
Benefits over boost::filesystem
- Seamlessly integrates with SFML
- Abstracted interface. Not strictly a singleton.
Use it to...
- Manage user profiles
- Manipulate easily distributable "packages" to allow for additional/customized levels
- ...and more!
Imagine if i/o with files in .zips were as simple as i/o with normal files. There's no reason it can't be!
I did something similar to this with another lib I wrote a long while back. Some of the code should be adaptable, but most will need to be rewritten to make it thread safe and exception safe.
-
Interesting, especially the ability to write/read from zip archives! Are you going to provide a stream interface similar to the standard library? Maybe you could even use std::ostream and std::istream, so that existing << and >> operators also work with your library...
By the way, don't you want to reuse portable code like Boost.FileSystem internally? I don't think it makes much sense to reinvent the wheel for every platform... You can still provide an interface that abstracts from the details ;)
-
Looks good - as Nexus said archive handling will be useful. Do you intend just to use zip, or might you use something like the LZMA SDK to support .7z files, etc as well?
Unicode filenames
Unicode friendliness! From you? Who'd have guessed ;) Seriously, though, it's a useful feature :)
I assume you won't be providing encrypted file handling, whether due to annoying laws on export or whatever...
-
Happiness. This is making me feel that. I don't need this right now, but if I do, it's already done.
-
Are you going to provide a stream interface similar to the standard library? Maybe you could even use std::ostream and std::istream, so that existing << and >> operators also work with your library...
Deriving from ostream/istream is definitely on my todo list. The problem is it's ridiculously complicated to do. I don't know why they made it so hard. I mean you'd think all you'd have to do is provide seek/tell/read functions...
It won't have that support immediately, but it is planned. Hopefully I can make it work.
By the way, don't you want to reuse portable code like Boost.FileSystem internally? I don't think it makes much sense to reinvent the wheel for every platform... You can still provide an interface that abstracts from the details
For systems like *nix where additional dependencies are trivial, yes. You're pretty much expected to have boost installed if you're on Linux so it's not really an issue.
For Windows users though, the extra dependency will be a hurdle, so I would rather stick to WinAPI. I already have plenty of dependencies with SFML and zlib.
Do you intend just to use zip, or might you use something like the LZMA SDK to support .7z files, etc as well?
.zip is just the most common, so I was going to start with that. I see no reason why future archives couldn't be supported, though. In fact I'm designing the framework to be easily expandable to allow for new formats (or at least, making it as easy as I can).
I assume you won't be providing encrypted file handling, whether due to annoying laws on export or whatever
I hadn't really thought about it. Would there be any point?
-
I hadn't really thought about it. Would there be any point?
Well when I was planning an archive manager, I considered putting it in just for the sake of it - after having waded into the largely undocumented LZMA SDK to compression, it would seem to make sense to go the whole way. Then I discovered all the export laws :/
Perhaps these reasons:
1) preventing users from easily cheating by editing game files (I guess it would also discourage modding...)
2) something like Valve's preloading whereby users could download the software in readiness for it's release, but it couldn't be run until then (this one is probably a fairly contrived reason though).
.zip is just the most common
But not as good compression ratio IIRC. Glad to hear it'll be extensible then :)
-
Sounds like a great project. Good luck.
-
I guess it will not depend on SFML, so why do you call it "SFML file system library"?
-
I guess it will not depend on SFML, so why do you call it "SFML file system library"?
I actually was planning on making it depend on SFML and act as sort of an add-on library.
But you're right, it doesn't have to be dependant.
As for the name, I'm open to better suggestions. I honestly didn't put a whole lot of thought into the name XD
EDIT:
Actually, I remember now...
I need SFML for threading stuff. Archives will need to be threadsafe and since this lib is designed to be used with SFML it makes more sense for me to use SFML threads.
So yeah, SFML is a dependency. The system package only though.
-
Did you consider PhysicsFS ? http://icculus.org/physfs/
Thanks a lot for your previous work on the io stuff.
-
I hadn't, but I will review it when I get some time. It does seem like it would help, but it also seems to be lacking some features I want.
But I didn't look in detail.
-
A little bit off-topic, but... a PhysFSInputStream for SFML could be a cool (and useful!) example. It would be so straight-forward that I could even include it as an example in the doc of the InputStream class.
-
You might be interested by this wrapper of std i/ostreams for PhysicsFS: https://svn6.assembla.com/svn/tbt/branches/test-l10n/src/physfs_stream.hh
-
Thanks, fekmon. That actually is very useful.
I'll definitely check this out in more detail when I get the time.
-
Don't know weather this may be useful or not, but somehow it's related, so I though on pointing it.
A few months ago I did a small framework for data codification using a pipeline architecture, called Codex. It's not a compression/cryptographic library, just a way to stream data through such kind of filters (but it comes with zlib and Rijndael as default compression and encryption filters).
http://data-codex.sourceforge.net/
I created my own thread classes, they are pretty much the same as SFML ones plus another synchronization mechanism.
I also though on deriving from ostream/istream but I hadn't the time to fully understand it so I moved on with my own classes. But one can still stream using the << and >> operators for the most common types (basic types, std::string, etc), and from instances of classes deriving from cx::Resource.
I didn't implement seek, it's mainly streaming really (what would seek do if you're streaming through the network instead of to/from a file? and there may be filters not supporting it), but using standard file io to seek resources in a file and then streaming them is still a reasonable alternative.
I don't know but maybe it can be of some use to you? I would combine Codex and your file system anyway if I need it someday.
-
How is this project coming along? Got any files coded yet ? :D
Currently looking for a on-the-fly zip-reader as you described, if you have any working code for that i would be so happy (A)
-
Work keeps me very busy so I don't have as much time to work on this as I'd like.
I'm still making progress but it's sparadic at best. I wouldn't expect anything for at least a month or two.
-
Work keeps me very busy so I don't have as much time to work on this as I'd like.
I'm still making progress but it's sparadic at best. I wouldn't expect anything for at least a month or two.
Allright, its cool now tho, i made my own packaging format with the functionality i need, still looking forward to this tho since zip would be awesome :)
-
In the words of Altered Beast... "Wiiise fwom your gwaaave!"
Work has been killing me, but lately I've been pushing to get more work on this done.
Made another breakthrough today!
I'll let the screenshot cover it:
(http://i54.tinypic.com/x1y7wz.png)
PS: I know I'm leaking memory in that snippit =P
-
Wow, this would be very useful! I cannot seem to find the download link... :roll:
-
That's because there isn't one. It's still quite a ways away from seeing a release.
I might release a "read-only" version of the zip file thing once I feel it's stable enough. But writing will take a bit more time. I don't get to work on it as much as I'd like.