Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: [Released] libMy - Datapackaging library  (Read 31844 times)

0 Members and 1 Guest are viewing this topic.

gsaurus

  • Sr. Member
  • ****
  • Posts: 262
    • View Profile
    • Evolution Engine
Re: [Released] libMy - Datapackaging library
« Reply #45 on: April 15, 2012, 07:35:52 pm »
How did I miss this thread? Could have saved me some time.. or maybe not, since it doesn't have encryption yet, right?

I'm working on my own packaging system, it already pack/unpack stuff with compression and encryption. I use zlib and Rijndael cipher (a.k.a. AES). Probably going to use SHA2 to hash keys for user data packages.

But my lib is rather simple, once packed, resources are accessed through their index in the package (because I personally don't need a better mapping, at least not yet).

I'm about to do a class derived from sf::InputStream to stream stuff (I need it for music mostly) but I'm not sure how to deal with seek and getSize. For the latter I guess I'll have to store the size of the original files.
« Last Edit: April 15, 2012, 07:45:31 pm by gsaurus »
Pluma - Plug-in Management Framework

Haikarainen

  • Guest
Re: [Released] libMy - Datapackaging library
« Reply #46 on: April 17, 2012, 01:10:45 pm »
How did I miss this thread? Could have saved me some time.. or maybe not, since it doesn't have encryption yet, right?

I'm working on my own packaging system, it already pack/unpack stuff with compression and encryption. I use zlib and Rijndael cipher (a.k.a. AES). Probably going to use SHA2 to hash keys for user data packages.

But my lib is rather simple, once packed, resources are accessed through their index in the package (because I personally don't need a better mapping, at least not yet).

I'm about to do a class derived from sf::InputStream to stream stuff (I need it for music mostly) but I'm not sure how to deal with seek and getSize. For the latter I guess I'll have to store the size of the original files.

Feel free to "steal code" from me, would be appreciated with some creds for it though if you do :)

gsaurus

  • Sr. Member
  • ****
  • Posts: 262
    • View Profile
    • Evolution Engine
Re: [Released] libMy - Datapackaging library
« Reply #47 on: April 17, 2012, 05:44:38 pm »
Well the challenge on implementing seek is that it cannot simply use seek of *put compression lib name here* (eg. gzip seek provided by zlib), because first it's encrypted. So the process of knowing where to jump in the file, decrypt a chunk and then decompress the same chunk is a bit tricky. I would have to create my own resource header and deal with zlib random access capabilities.

I'll skip streaming for now and use loadFromMemory, and proceed with my engine. Eventually I'll add this functionality sometime later.
Pluma - Plug-in Management Framework

Ceylo

  • Hero Member
  • *****
  • Posts: 2325
    • View Profile
    • http://sfemovie.yalir.org/
    • Email
Re: [Released] libMy - Datapackaging library
« Reply #48 on: April 17, 2012, 09:11:00 pm »
Why no encrypt each object separately? That way you can seek in the compressed file, then decrypt the single object.

The objects can still be extracted by any gzip reader but the extracted files are encrypted.
« Last Edit: April 17, 2012, 09:14:58 pm by Ceylo »
Want to play movies in your SFML application? Check out sfeMovie!

gsaurus

  • Sr. Member
  • ****
  • Posts: 262
    • View Profile
    • Evolution Engine
Re: [Released] libMy - Datapackaging library
« Reply #49 on: April 17, 2012, 09:25:36 pm »
You suggest to encrypting first, then compress. That gives terrible results. Remember that an encrypted file is all "random" data, there are no data repetitions to be compressed.

Currently I compress and encrypt each object individually. An index table is kept at the start of the pack with the offset of each object, for random access the objects.
Pluma - Plug-in Management Framework

aBallofWin

  • Jr. Member
  • **
  • Posts: 99
    • View Profile
Re: [Released] libMy - Datapackaging library
« Reply #50 on: July 04, 2012, 12:26:52 am »
Does anyone have a download of the full package that they could zip up and host at mediafire or something? sfmluploads seems to have been down for a while now :/

eXpl0it3r

  • SFML Team
  • Hero Member
  • *****
  • Posts: 10783
    • View Profile
    • development blog
    • Email
Re: [Released] libMy - Datapackaging library
« Reply #51 on: July 04, 2012, 12:34:02 am »
Oh I even found a copy of it on my drive, but I've tempered with it a bit, so it's not 100% the original pack. ;)
There you go: http://pitload.org/3110
Official FAQ: https://www.sfml-dev.org/faq.php
Official Discord Server: https://discord.gg/nr4X7Fh
——————————————————————
Dev Blog: https://duerrenberger.dev/blog/

aBallofWin

  • Jr. Member
  • **
  • Posts: 99
    • View Profile
Re: [Released] libMy - Datapackaging library
« Reply #52 on: July 04, 2012, 11:02:36 am »
Oh I even found a copy of it on my drive, but I've tempered with it a bit, so it's not 100% the original pack. ;)
There you go: http://pitload.org/3110

Thank you very much!