SFML community forums
General => SFML projects => Topic started by: Haikarainen on August 08, 2011, 07:47:53 pm
-
(http://fredrik.haikarainen.se/logo.png)
Website/Documentation is found here (http://fredrik.haikarainen.se/libmy/)
This is a pretty early release since a lot of users have asked me when I release it, and I'm pretty busy, so PLEASE report any bugs you find ;)
I haven't been able to generate any CMake-files yet since I'm a noob at that area, but I promise that once I get the time to learn it, I will upload a release with it.
Features:
CRC32-Integrity checks for both files and strings
Filepackaging in a simple and fast way!
Ability to load SFML-resources directly from package(no need to extract)
Exceptionhandling
Opensource under zlib/sfml-like license!
Usage example: https://legacy.sfmluploads.org/code/107
Downloads: Full Package (https://legacy.sfmluploads.org/file/70) - Contains full source, binaries for Win64 and examples
Win64 binaries (https://legacy.sfmluploads.org/file/69) - Contains binaries for library and examples.
Minimal source (https://legacy.sfmluploads.org/file/68) - Contains include/ and src/ directories only.
Codeblocks project (https://legacy.sfmluploads.org/file/71) - Contains source and a codeblocksproject, see readme.txt.
[/list]
Dependencies: SFML2 System module
Boost system and filesystem modules
If you run into any problems/bugs, or if you have any ideas to donate, don't hesitate to reply!
-
With SFML (WIP):
my::Data::Package Test;
Test.Load("Test.pkg");
sf::Image Img;
Img.LoadFromMemory(Test.GetFileData("irimage.png"), Test.GetFileSize("irimage.png"));
Test.Close();
I dont know... sf::InputStream is created for avoid load resources from memory.
-
I dont know... sf::InputStream is created for avoid load resources from memory.
Hm, wasn't aware of this, thanks, will create a derived class :)
Just got done with the packageclass, as soon as i've created the derived class and fixed compression & encryption i'll release it.
Edit; Btw i will also implement exceptions, please tell me about any ideas you might have to make this project more flexible/functional. Since its for a game its currently under pretty heavy development, so its now or probably never :P
-
Btw i will also implement exceptions, please tell me about any ideas you might have to make this project more flexible/functional. Since its for a game its currently under pretty heavy development, so its now or probably never Razz
Sorry, at the moment i don't have a idea. But now i concern a thing about this one thing
Test.ExtractAll("./Extraction/");
I think that is better read or load assets from packages without extract the package content in folders. Again, sf::InputStream will help you at this task.
Another thing, really exceptions will help? I think that C error checking is ok.
-
Sorry, at the moment i don't have a idea. But now i concern a thing about this one thing
Test.ExtractAll("./Extraction/");
I think that is better read or load assets from packages without extract the package content in folders. Again, sf::InputStream will help you at this task.
Another thing, really exceptions will help? I think that C error checking is ok.
Well that functionality is already in progress of being developed, but why would that automatically delete the funcitonality of extracting an archive? This lib is not only for resourcemanagement, but could also be used for installers etc.
Exceptions would be really useful, so you have a global errorhandling instead of checking switching a my::Data::Status for every request you do, but then you still have that opportunity if you wanna check if something specifi happened allright.
-
Sorry, at the moment i don't have a idea. But now i concern a thing about this one thing
Test.ExtractAll("./Extraction/");
I think that is better read or load assets from packages without extract the package content in folders. Again, sf::InputStream will help you at this task.
Another thing, really exceptions will help? I think that C error checking is ok.
Well that functionality is already in progress of being developed, but why would that automatically delete the funcitonality of extracting an archive? This lib is not only for resourcemanagement, but could also be used for installers etc.
Exceptions would be really useful, so you have a global errorhandling instead of checking switching a my::Data::Status for every request you do, but then you still have that opportunity if you wanna check if something specifi happened allright.
Ok, ok. I don't know that this library is not only for resource management. And with exceptions, yes, is other way for handle errors. ¡Ah! Exceptions not only can be used for error handling.
-
Is this available for download yet?
-
Is this available for download yet?
Yes, sorry, i kinda have a lot of balls in the air atm, so ive kinda forgot about it. Very little work left tho, will upload it soon :)
-
Bump, view threadstart again
-
I'm wondering, does it support encryption? Would be nice to pack all resources in an encrypted archive and it has to be unlocked with the serial key or something like that.
-
I'm wondering, does it support encryption? Would be nice to pack all resources in an encrypted archive and it has to be unlocked with the serial key or something like that.
Not yet, but its planned, as well as compression. Those will probably be used something like my::Data::Package::Open("./data/package.pkg", my::Data::UseEncryption|my::Data::UseCompression, "COMPRESSIONKEYHERE");
Currently having some problems with static linkage of boost libraries for static libmy-libraries :S Might take a day or so to fix this. If anyone wanna help me this is my issue:
Links for Static Release of libmy:
-lboost_filesystem-mgw44-s-1_47
-lboost_system-mgw44-s-1_47
It compiles allright, but when i link the compiled libmy-s (wich links static boost) in a testapplication wich uses libmy, i get "undefined reference"'s to boost::filesystem :S Works allright for dynamic libmy-linkage, wich also links boost statically. Wich is weird.
-
It compiles allright, but when i link the compiled libmy-s (wich links static boost) in a testapplication wich uses libmy, i get "undefined reference"'s to boost::filesystem :S Works allright for dynamic libmy-linkage, wich also links boost statically. Wich is weird.
There's no link step when building a static library, it's basically just an archive of object files. That's why you must put your linker options when a link step actually occurs, ie. when compiling your final executable.
-
There's no link step when building a static library, it's basically just an archive of object files. That's why you must put your linker options when a link step actually occurs, ie. when compiling your final executable.
So to use my library people must link towards boost libs as well? Or did i get it all wrong? Probably got it wrong since sfml-static works without its depending libraries for the enduser :P
-
No, you're perfectly right. SFML uses a trick to include dependencies directly into its static libs ;)
-
No, you're perfectly right. SFML uses a trick to include dependencies directly into its static libs ;)
Wanna share? ;p
Edit; Nevermind, found the macro, will play around with this :) thanks tho!
-
Ohwell, The library is finally released after a bumpy ride of ma-thumb-up-ma-ass.
-
Currently having some problems with static linkage of boost libraries for static libmy-libraries :S
Is this still the case or did you resolve it?
Since you don't provide any build script and I've absolutly no idea how to use dll.a and dll.def files, I made my own VS2010 project, which compiled libMy to a static library. That worked fine but trying to use it in my test application throws out some linker errors with boost (debug and release).
So it sounds the same as your problem...
1>libMy.lib(Package.obj) : error LNK2001: Nicht aufgelöstes externes Symbol ""class boost::system::error_category const & __cdecl boost::system::system_category(void)" (?system_category@system@boost@@YAAEBVerror_category@12@XZ)".
1>libMy.lib(Package.obj) : error LNK2001: Nicht aufgelöstes externes Symbol ""class boost::system::error_category const & __cdecl boost::system::generic_category(void)" (?generic_category@system@boost@@YAAEBVerror_category@12@XZ)".
1>libMy.lib(Package.obj) : error LNK2001: Nicht aufgelöstes externes Symbol ""class boost::filesystem3::file_status __cdecl boost::filesystem3::detail::status(class boost::filesystem3::path const &,class boost::system::error_code *)" (?status@detail@filesystem3@boost@@YA?AVfile_status@23@AEBVpath@23@PEAVerror_code@system@3@@Z)".
1>libMy.lib(Package.obj) : error LNK2001: Nicht aufgelöstes externes Symbol ""bool __cdecl boost::filesystem3::detail::create_directories(class boost::filesystem3::path const &,class boost::system::error_code *)" (?create_directories@detail@filesystem3@boost@@YA_NAEBVpath@23@PEAVerror_code@system@3@@Z)".
1>libMy.lib(Package.obj) : error LNK2001: Nicht aufgelöstes externes Symbol ""private: static class std::codecvt<wchar_t,char,int> const * & __cdecl boost::filesystem3::path::wchar_t_codecvt_facet(void)" (?wchar_t_codecvt_facet@path@filesystem3@boost@@CAAEAPEBV?$codecvt@_WDH@std@@XZ)".
1>libMy.lib(Package.obj) : error LNK2001: Nicht aufgelöstes externes Symbol ""void __cdecl boost::filesystem3::path_traits::convert(char const *,char const *,class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> > &,class std::codecvt<wchar_t,char,int> const &)" (?convert@path_traits@filesystem3@boost@@YAXPEBD0AEAV?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@AEBV?$codecvt@_WDH@5@@Z)".
"Nicht aufgelöstes externes Symbol" = "unresolved (external) symbol"
I've compiled the boost libraries and set the paths to the boost and the lib folder. Added in debug mode: "libMy-d.lib;libboost_system-vc100-mt-gd-1_47.lib;libboost_filesystem-vc100-mt-gd-1_47.lib" and in release mode: "libMy.lib;libboost_system-vc100-mt-1_47.lib;libboost_filesystem-vc100-mt-1_47.lib" but it doesn't link...
Any idea?
I've compiled libMy as x64 but used the standard script for boost, so I don't know if boost got x64 libraries too and if that could matter...
Or can you give an example of how to use dll.a and dll.def files (in Visual Studio)?
-
I'm wondering how it's going with encryption? I'd love to use the library with my project but I'll need encryption as well :wink:
Though will download boost today and try your library out.
How do you think this look? :)
(https://legacy.sfmluploads.org/cache/pics/174_loading%20screen.png)
-
Defently encryption would be awesome! =)
Boost x64 is compiling... so haven't tested if it'll run with it, but I certaintly hope so. ;-)
@Groogy: hehe it needs a bit more white/lighter color around the black "lib", otherwise it difusses to much with the background.
-
(https://legacy.sfmluploads.org/cache/pics/175_loading%20screen.png)
Now then? Edit: Added smooth = true to the logo textures as the aliasing on them annoyed me. The picture above does not contain the change.
Anyway I won't compile x64 and I was kind of annoyed that there were not at least makefiles with the project but no biggie, Netbeans will take care of that for me.
-
Yep, one can now clearly see it. :)
@Edit: Aliasing is (nearly) never that nice...
Just wanted to try the x64, it worked now quiet good for SFML and Thor. Hopefully it will work for boost & libMy too.
Btw it says, it's possible to load the SFML resourcess directly, but is it only for those resources or can I also load a text/XML file directly, without extracting it seperatly?
@Groogy: you display "Parsing config data" was that loaded from the libMy package?[/b]
-
Nothing there is loaded from libMy as of yet but eventually all of the data there will be loaded from a package file in libMy. I want encryption so that a unique serial key is used to unlock the resources. But in developer mode of course there won't be any encryption(As it is now plugins/mods will be supported and the editor will compile everything into a package for a developer without a serial key) :P
-
Ah okay, so I'll have to find it out myself...
Btw: YES! I did it! :lol:
Managed to link libMy x64 staticly.
I'm really getting used to this linking stuff. But why can't the compiles give you a usefull error. All those linking errors are so cryptic...
What I had to do:
- Compile boost for x64 (bjam --toolset=msvc-10.0 address-model=64 --build-type=complete stage)
- Create & setup a project for libMy libraries
- Use Multithread (/MT) instead of Multithread DLL (/MD) for libMy and SFML project.
- Link with libboost_system-vc100-s-1_47.lib & libboost_filesystem-vc100-s-1_47.lib;libMy.lib
- Defining SFML_STATIC
If someone is interessted in the project file or the binaries, just let me know! =)
-
What exactly are its dependencies on boost?
Would it be possible to make it use zlib?
-
What exactly are its dependencies on boost?
Would it be possible to make it use zlib?
EDIT; Actually it depends on boost for filesystem(checking dirs, creating dirs etc) as well.
boost/crc.hpp
No probably not, since its my own custom format and it will support only that.
This thread seems to have exploded lately, I haven't forgot about the project but I'm kinda busy with another one. Encryption is the nr 1 priority for this lib now tho since I'm gonna use it in the other project myself.
Groogy; I'd appreciate it if you dont tamper with the logo for public releases, hope you understand. I can provide more logo's later.
-
Groogy; I'd appreciate it if you dont tamper with the logo for public releases, hope you understand. I can provide more logo's later.
No problem, I'll switch back. My project will be on a private repo so there's no rush for me on getting this part working and correct right now.
-
Hi,
Your project seems really interesting but cant you get ride of boost ? I'd like to build it for my platform (mac) but building with it is really annoying.
You can use this lib to encrypt : http://www.aescrypt.com/ it's really simple to use.
good luck.
-
This thread seems to have exploded lately, I haven't forgot about the project but I'm kinda busy with another one. Encryption is the nr 1 priority for this lib now tho since I'm gonna use it in the other project myself.
Hmmm... I think it would be better if you first 'unbug' the library. ;-)
I've found two issues probably both compiler dependend:
Package.cpp:85:
// At the moment
if(strcmp(firstcheck, "MPF") != 0){
// But should acutally be
if(strncmp(firstcheck, "MPF", 3) != 0){
// Since the read function of a stream doesn't (always) add a zero delimiter.
// Maybe it would work too with
firstcheck = new char[4];
// 3 chars + 1 delemiter
Package.cpp:95:
char idsplitter;
// My compiler returns an error in line 98 because idsplitter is not initialized
char idsplitter = ' ';
And I would suggest to implement a better file-/pathname handling. It took me a while to understand, that I've to use "./" to use the current path...
Also path and filename are kinda too close related to understand how one relates to the filename in the package and the other to the path of the file on the harddrive.
Additionally it would be great to get an build script or so... ;-)
And yep boost independens would be nice.
Now I even managed to load a text file out of the package!
I like it (after the hill I had to climb to get here)! :lol:
-
This thread seems to have exploded lately, I haven't forgot about the project but I'm kinda busy with another one. Encryption is the nr 1 priority for this lib now tho since I'm gonna use it in the other project myself.
Hmmm... I think it would be better if you first 'unbug' the library. ;-)
I've found two issues probably both compiler dependend:
Package.cpp:85:
// At the moment
if(strcmp(firstcheck, "MPF") != 0){
// But should acutally be
if(strncmp(firstcheck, "MPF", 3) != 0){
// Since the read function of a stream doesn't (always) add a zero delimiter.
// Maybe it would work too with
firstcheck = new char[4];
// 3 chars + 1 delemiter
Package.cpp:95:
char idsplitter;
// My compiler returns an error in line 98 because idsplitter is not initialized
char idsplitter = ' ';
And I would suggest to implement a better file-/pathname handling. It took me a while to understand, that I've to use "./" to use the current path...
Also path and filename are kinda too close related to understand how one relates to the filename in the package and the other to the path of the file on the harddrive.
Additionally it would be great to get an build script or so... ;-)
And yep boost independens would be nice.
Now I even managed to load a text file out of the package!
I like it (after the hill I had to climb to get here)! :lol:
Thank you! This is the kind of response I was looking for! I actually never really used this except for debugging and making sure everything worked, So this is a really good thing you're doing. Please continue if you don't mind, I really need testers for this right now ;) I'll fix the bugs and try to get a script going.
-
// At the moment
if(strcmp(firstcheck, "MPF") != 0){
// But should acutally be
if(strncmp(firstcheck, "MPF", 3) != 0){
Actually, it should be if(memcmp(firstcheck, "MPF", 3) != 0){ to emphasize it's really a binary comparison.
Can I nitpick some more? ;)
So, why on Earth earlier in the same function it's char* firstcheck;
firstcheck = new char[3];and not the vastly simplerchar firstcheck[3];?
Why there are so much unadorned new[] and delete[]s in the code? Familiarize yourself with RAII and classes like std::vector or boost::scoped_array. It'll make the code simpler, easier to read and write, and more correct. Are you sure there are not memory leaks? I'm not. Given that there are so many ways for things to go wrong, I wouldn't dare to ignore RAII.
Why all the loop counters are of type int, not container::size_type (in most cases the same as std::size_t)? You not only have to cast to silence a warning about comparison between signed (int) and unsigned (size_t), you also make the code not work for all sizes larger than INT_MAX. On some common platforms, size_t is in fact larger than int, so the code may crash in a funny way.
Why all the std::strings are passed to functions by value? Pass by a reference to a constant, you'll get rid of unnecessary copying. It's very common idiom, so everyone will understand what you mean.
Why you're returning after throwing an exception? These returns are unreachable, throw already changes the control flow so much you probably are in a completely different function now, so the returns will have no chance to get executed.
Why I'm nitpicking on such issues? On this forum it's Nexus's job to do that, not mine ;)
Anyway, it's a nice library, keep on working on it, even though it uses it's own format instead of something widely supported. Could you document the details about the package's binary structure?
-
On this forum it's Nexus's job to do that, not mine ;)
:lol:
I guess there are a few more bugs to fix... :wink:
I'll keep looking and reporting.
-
// At the moment
if(strcmp(firstcheck, "MPF") != 0){
// But should acutally be
if(strncmp(firstcheck, "MPF", 3) != 0){
Actually, it should be if(memcmp(firstcheck, "MPF", 3) != 0){ to emphasize it's really a binary comparison.
Well is this really neccessary? Do you know of any platform that these 2 different methods give different results in a scenario like this?
Can I nitpick some more? ;)
So, why on Earth earlier in the same function it's char* firstcheck;
firstcheck = new char[3];and not the vastly simplerchar firstcheck[3];?
Good point, The original code worked flawlessly for me so I really didnt think about initializing it whatsoever.
Why there are so much unadorned new[] and delete[]s in the code? Familiarize yourself with RAII and classes like std::vector or boost::scoped_array. It'll make the code simpler, easier to read and write, and more correct. Are you sure there are not memory leaks? I'm not. Given that there are so many ways for things to go wrong, I wouldn't dare to ignore RAII.
Unadorned? They are the reason the lib is NOT filled with memoryleaks(I have tested this extensively and I have not detected a single byte escaping. Also I'm 100% sure filedata does not escape). Stop taking idioms so overseriously.
Also, I know std::vector, I use it very often. But would you REALLY store filedata using an std::vector? Wouldn't that be both stupid and slow?
Why all the loop counters are of type int, not container::size_type (in most cases the same as std::size_t)? You not only have to cast to silence a warning about comparison between signed (int) and unsigned (size_t), you also make the code not work for all sizes larger than INT_MAX. On some common platforms, size_t is in fact larger than int, so the code may crash in a funny way.
I meant to write (signed) before *.size but then I remembered this is not how to iterate a vector properly. This is on my todolist (iterate vectors properly using std::vector<T>::iterator and not ints).
Why all the std::strings are passed to functions by value? Pass by a reference to a constant, you'll get rid of unnecessary copying. It's very common idiom, so everyone will understand what you mean.
Why you're returning after throwing an exception? These returns are unreachable, throw already changes the control flow so much you probably are in a completely different function now, so the returns will have no chance to get executed.
.. They are strings, not RenderWindows (for example). do you really have so little time to be THAT nitpicky? But I guess your right I could use const char* instead.
Why I'm nitpicking on such issues? On this forum it's Nexus's job to do that, not mine ;)
Anyway, it's a nice library, keep on working on it, even though it uses it's own format instead of something widely supported. Could you document the details about the package's binary structure?
Well thank you. Yes when it's actually done, as stated this is a too early release on request by the "public domain". I have also made clear why I use my own custom format. The fact that you're questioning it gives me suspicions that you don't even know what the goal use of this lib is. If you made a game would you want anyone to be able to open your datafiles using winzip, and maybe a bit of bruteforce?
-
I'll keep looking and reporting.
Thanks, appreciated :)
-
Well is this really neccessary? Do you know of any platform that these 2 different methods give different results in a scenario like this?
I'm talking about readability here. You want to compare the first three bytes, right? So use the function that makes it clear you're comparing raw bytes, not strings of characters.
They are the reason the lib is NOT filled with memoryleaks(I have tested this extensively and I have not detected a single byte escaping. Also I'm 100% sure filedata does not escape).
They are the reason you have to write more code and to be very careful. And to check it later. If you relied on RAII instead, a quick glimpse over the code would do.
But would you REALLY store filedata using an std::vector?
Sure, why not? File data is an array of bytes. std::vector is a dynamic array. I think it's perfectly good (in fact, quite natural) to store what's an array in an array.
Wouldn't that be [...] slow?
Any decent compiler would optimize it out. If the performance still concerns you, you can use boost::scoped_array instead of a vector.
They are strings, not RenderWindows (for example).
Apart from RenderWindow being noncopyable, so you couldn't pass it by value, it's just a good and widely recognized practice to pass a heavy-to-copy objects, including containers, which strings are, by a reference to a constant.
do you really have so little time to be THAT nitpicky?
Okay, this question puzzles me. Did you mean "do you really have so much time"? Well, yes, I do ;). And you don't have to be defensive, I'm only trying to be helpful :roll:
I have also made clear why I use my own custom format.
Apparently I haven't noticed it. So, could you give a link or something?
If you made a game would you want anyone to be able to open your datafiles using winzip, and maybe a bit of bruteforce?
Why not? If someone would want to make a mod or something like that, I'd actually like them to do that. On the other hand, if someone would want to steal it, you're protected by the law anyway. And if someone really would want to steal it, she could simply take a screenshot ;)
-
Well I want to provide support for mods as well but since I am exposing the engine trough javascript inside my editor it won't be a problem. Actually I see it as an advantage because I can use the exact same system to release expansions/DLC.
So having your own format has the pro that you can protect your own code and let modders protect their code. As long as one does not have the key of course.
It's not Bullet proof no. But I don't see any big con's for not having a custom format.
-
...
Sadly, there are so many issues with the way you think, I cant afford the time or care to answer them all.
-
Sadly, there are so many issues with the way you think, I cant afford the time or care to answer them all.
Please, what's wrong with you? I really don't mean to offend anyone, I'm only trying to point out what I think should be changed in the code.
So why you're resorting to what's basically a glorified name-calling? Seriously, if I'm doing something wrong, I'd be very glad to hear why it's so wrong, so I can work on it in order to not let it happen again in the future.
-
Sadly, there are so many issues with the way you think, I cant afford the time or care to answer them all.
Please, what's wrong with you? I really don't mean to offend anyone, I'm only trying to point out what I think should be changed in the code.
So why you're resorting to what's basically a glorified name-calling? Seriously, if I'm doing something wrong, I'd be very glad to hear why it's so wrong, so I can work on it in order to not let it happen again in the future.
I dont mean to offend you either, and you make some good points(memcmp etc), but there are a lot of points your making that I cant agree with at all, and I cant explain why without responding with way too much text.
-
there are a lot of points your making that I cant agree with at all, and I cant explain why without responding with way too much text.
So what should we do know? I stand by what I said, I won't give up easily, as I don't even know what are you disagreeing with, let alone why ;)
-
With SFML (WIP):
my::Data::Package Test;
Test.Load("Test.pkg");
sf::Image Img;
Img.LoadFromMemory(Test.GetFileData("irimage.png"), Test.GetFileSize("irimage.png"));
Test.Close();
I dont know... sf::InputStream is created for avoid load resources from memory.
Sorry for being a baddie and asking stupid questions, but what's the advantage of using sf::InputStream instead of LoadFromMemory?
-
Sorry for being a baddie and asking stupid questions, but what's the advantage of using sf::InputStream instead of LoadFromMemory?
The syntax is slightly nicer, and it adds another layer of abstraction, so if the public API of my::Data::Package changes, you only have one class in the code to modify.
my::Data::Package Test;
Test.Load("Test.pkg");
my::Data::InputStream stream(Test, "irimage.png");
sf::Image Img;
Img.LoadFromStream(stream);
Test.Close();
-
How to Build it oô ? I use msvc2010pro can anyone help me please :S
-
How to Build it oô ? I use msvc2010pro can anyone help me please :S
Unfortunatly Haikarainen doesn't provide a cross platform way to compile the library easily, but if you're using Win64 use the provided binaries.
It's quiet simple to build the library on your own, if you know what you're doing. Just create a new project, add the source files, go to the settings and add all the dependencies, switch to building a library rather than an application.
I can't give you a step by step explenation but I think you'd also learn more if you tried it on your own.
-
Hmm okay thats bad i need something like pkg who i can insert all my data crypted.
-
Regarding building the library I just remembered it also uses boost, so it won't be such a smooth and easy ride, except if you love bjam...
where i can insert all my data encrypted.
Then libMy is anyway nothing for you since it doesn't support encryption at all or not yet. :wink:
-
How did I miss this thread? Could have saved me some time.. or maybe not, since it doesn't have encryption yet, right?
I'm working on my own packaging system, it already pack/unpack stuff with compression and encryption. I use zlib and Rijndael cipher (a.k.a. AES). Probably going to use SHA2 to hash keys for user data packages.
But my lib is rather simple, once packed, resources are accessed through their index in the package (because I personally don't need a better mapping, at least not yet).
I'm about to do a class derived from sf::InputStream to stream stuff (I need it for music mostly) but I'm not sure how to deal with seek and getSize. For the latter I guess I'll have to store the size of the original files.
-
How did I miss this thread? Could have saved me some time.. or maybe not, since it doesn't have encryption yet, right?
I'm working on my own packaging system, it already pack/unpack stuff with compression and encryption. I use zlib and Rijndael cipher (a.k.a. AES). Probably going to use SHA2 to hash keys for user data packages.
But my lib is rather simple, once packed, resources are accessed through their index in the package (because I personally don't need a better mapping, at least not yet).
I'm about to do a class derived from sf::InputStream to stream stuff (I need it for music mostly) but I'm not sure how to deal with seek and getSize. For the latter I guess I'll have to store the size of the original files.
Feel free to "steal code" from me, would be appreciated with some creds for it though if you do :)
-
Well the challenge on implementing seek is that it cannot simply use seek of *put compression lib name here* (eg. gzip seek provided by zlib), because first it's encrypted. So the process of knowing where to jump in the file, decrypt a chunk and then decompress the same chunk is a bit tricky. I would have to create my own resource header and deal with zlib random access capabilities.
I'll skip streaming for now and use loadFromMemory, and proceed with my engine. Eventually I'll add this functionality sometime later.
-
Why no encrypt each object separately? That way you can seek in the compressed file, then decrypt the single object.
The objects can still be extracted by any gzip reader but the extracted files are encrypted.
-
You suggest to encrypting first, then compress. That gives terrible results. Remember that an encrypted file is all "random" data, there are no data repetitions to be compressed.
Currently I compress and encrypt each object individually. An index table is kept at the start of the pack with the offset of each object, for random access the objects.
-
Does anyone have a download of the full package that they could zip up and host at mediafire or something? sfmluploads seems to have been down for a while now :/
-
Oh I even found a copy of it on my drive, but I've tempered with it a bit, so it's not 100% the original pack. ;)
There you go: http://pitload.org/3110 (http://pitload.org/3110)
-
Oh I even found a copy of it on my drive, but I've tempered with it a bit, so it's not 100% the original pack. ;)
There you go: http://pitload.org/3110 (http://pitload.org/3110)
Thank you very much!