Bad memory allocation?

[Kebrian]

I have big problem with loading Tiled maps. Everything works fine for smaller maps even 2000x2000 but when i try to load 8000x4000 map only thing that debugger says is:

Code: [Select]

Building to ensure sources are up-to-date
Selecting target:
Debug
Adding source dir: D:\2Dwarf\
Adding source dir: D:\2Dwarf\
Adding file: D:\2Dwarf\bin\2Dwarf-debug.exe
Changing directory to: D:/2Dwarf/bin
Set variable: PATH=.;D:\2Dwarf\dependencies\lib;C:\CodeBlocks\MinGW\bin;C:\CodeBlocks\MinGW;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Windows\System32;C:\Windows;C:\Windows\System32\wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\CMake\bin
Starting debugger: C:\CodeBlocks\MINGW\bin\gdb.exe -nx -fullname  -quiet  -args D:/2Dwarf/bin/2Dwarf-debug.exe
done
Registered new type: wxString
Registered new type: STL String
Registered new type: STL Vector
Setting breakpoints
Debugger name and version: GNU gdb (GDB) 7.5
Child process PID: 4692
In __cxa_throw () ()
Code of files causing error:

Code: [Select]

#ifndef TMXMAP_H
#define TMXMAP_H

#include <SFML/Graphics.hpp>
#include <SFML/System.hpp>
#include <string>
#include <vector>

namespace keb
{

class TmxLayer;

class TmxTile //: sf::Drawable
{
    friend TmxLayer;
    public:
        TmxTile();
        virtual ~TmxTile();
    protected:
        unsigned int m_gid;
        unsigned int m_x;
        unsigned int m_y;
        sf::IntRect m_rect;
        sf::Texture* m_texture;
        //sf::Sprite m_sprite;

        // Draw this tile
        //virtual void draw(sf::RenderTarget& target, sf::RenderStates states) const;
};

class TmxTileset
{
    friend TmxLayer;
    public:
        TmxTileset(std::string name, int firstGid, int tileWidth, int tileHeight, std::string filename);
        virtual ~TmxTileset();
        sf::Texture* getTexture();
        sf::IntRect getTileRect(unsigned int tile);
    protected:
        sf::Texture m_texture;
        unsigned int m_firstGid;
        int m_tileWidth;
        int m_tileHeight;
        int m_width;
        int m_height;
        std::string m_name;
};

class TmxMap;

class TmxLayer : sf::Drawable
{
    friend TmxMap;
    public:
        TmxLayer(int width, int height, int tileWidth, int tileHeight, std::vector<TmxTileset*>* tilesets);
        virtual ~TmxLayer();

        void setTile(int &x, int &y, int &gid);
        void setDrawRect(int x, int y, int width, int height);
    protected:
        int m_width;
        int m_height;
        int m_tileWidth;
        int m_tileHeight;
        sf::IntRect m_drawRect;

        std::vector<std::vector< TmxTile > > m_tiles;
        std::vector<TmxTileset*>* m_tilesets;

        // Draw this layer
        virtual void draw(sf::RenderTarget& target, sf::RenderStates states) const;
};

class TmxMap : public sf::Drawable
{
    public:
        TmxMap();
        virtual ~TmxMap();
        void loadFromFile(const char* dir, const char* filename);
        bool isReady();
        void setDrawRect(int x, int y, int width, int height);
    protected:
        bool m_ready;
        sf::Mutex m_readyMutex;
        sf::Mutex m_dataMutex;
        sf::Thread* m_thread;
        void loadingThread();

        // Map properties
        std::string m_dir;
        std::string m_filename;
        std::string m_encoding;
        std::string m_compression;
        int m_width;
        int m_height;
        int m_tileWidth;
        int m_tileHeight;
        sf::IntRect m_drawRect;

        // Map data
        std::vector<TmxTileset*> m_tilesets;
        std::vector<TmxLayer*> m_layers;

        // Draw entrie map
        virtual void draw(sf::RenderTarget& target, sf::RenderStates states) const;

        // Decompress function
        std::vector<unsigned char> m_Decompress(const char* source, int inSize, int expectedSize);
    private:
};

}; // namespace keb

// Base64 decoding function
std::string base64_decode(std::string const& encoded_string);

#endif // TMXMAP_H


Code: [Select]

#include <iostream>
#include <sstream>
#include <tinyxml2.h>
#include <zlib.h>
#include <cstring>
#include "TmxMap.h"

namespace keb
{

TmxTile::TmxTile()
{
    m_gid = 0;
    m_rect.top = m_rect.left = m_rect.width = m_rect.height = 0;
    m_texture = NULL;
}

TmxTile::~TmxTile()
{

}

/*void TmxTile::draw(sf::RenderTarget& target, sf::RenderStates states) const
{
    target.draw(m_sprite);
}*/

TmxTileset::TmxTileset(std::string name, int firstGid, int tileWidth, int tileHeight, std::string filename)
{
    m_firstGid = firstGid;
    m_tileWidth = tileWidth;
    m_tileHeight = tileHeight;
    m_name = name;
    m_texture.loadFromFile(filename.c_str());
    m_width = m_texture.getSize().x/m_tileWidth;
    m_height = m_texture.getSize().y/m_tileHeight;
}

TmxTileset::~TmxTileset()
{

}

sf::Texture* TmxTileset::getTexture()
{
    return &m_texture;
}

sf::IntRect TmxTileset::getTileRect(unsigned int tile)
{
    sf::IntRect rect;
    rect.left = (tile%m_width)*m_tileWidth;
    rect.top = (int)(tile/m_width)*m_tileHeight;
    rect.width = m_tileWidth;
    rect.height = m_tileHeight;
    return rect;
}

TmxLayer::TmxLayer(int width, int height, int tileWidth, int tileHeight, std::vector<TmxTileset*>* tilesets)
{
    m_width = width;
    m_height = height;
    m_tileWidth = tileWidth;
    m_tileHeight = tileHeight;
    m_tilesets = tilesets;

    TmxTile emptyTile;
    // Fill layer with zeros
    for(int x=0; x<m_width; x++)
    {
        std::vector< TmxTile > column;
        for(int y=0; y<m_height; y++)
        {
            column.push_back(emptyTile);
        }
        m_tiles.push_back(column);
    }
}

TmxLayer::~TmxLayer()
{
    //dtor
}

void TmxLayer::setTile(int &x, int &y, int &gid)
{
    m_tiles[x][y].m_gid = gid;
    TmxTileset* tileset;

    if(gid!=0)
    {
        std::vector<TmxTileset*>::reverse_iterator it = m_tilesets->rbegin();
        while( it != m_tilesets->rend() )
        {
            if(gid >= (*it)->m_firstGid)
            {
                tileset = (*it);
                break;
            }
            ++it;
        }

        m_tiles[x][y].m_texture = &tileset->m_texture;
        //m_tiles[x][y].m_sprite.setTexture(tileset->m_texture);
        gid -= tileset->m_firstGid;
        //m_tiles[x][y].m_sprite.setTextureRect( tileset->getTileRect(gid) );
        //m_tiles[x][y].m_sprite.setPosition(x*m_tileWidth,y*m_tileHeight);
        m_tiles[x][y].m_x = x*m_tileWidth;
        m_tiles[x][y].m_y = y*m_tileHeight;
        m_tiles[x][y].m_rect = tileset->getTileRect(gid);
    }
}

void TmxLayer::setDrawRect(int x, int y, int width, int height)
{
    m_drawRect.left = (x/m_tileWidth) - 1;
    m_drawRect.top = (y / m_tileHeight ) - 1;
    m_drawRect.width = m_drawRect.left + ( width / m_tileWidth ) + 2;
    m_drawRect.height = m_drawRect.top + ( height / m_tileHeight ) + 2;
    if (m_drawRect.left < 0) m_drawRect.left = 0;
    if (m_drawRect.top < 0) m_drawRect.top = 0;
    if ( m_drawRect.left >= m_width ) m_drawRect.left = m_width;
    if ( m_drawRect.top >= m_height ) m_drawRect.top = m_height;
    if ( m_drawRect.width >= m_width ) m_drawRect.width = m_width;
    if ( m_drawRect.height >= m_height ) m_drawRect.height = m_height;
    //std::cout << m_drawRect.left << " " << m_drawRect.top << " " << m_drawRect.width << " " << m_drawRect.height << std::endl;
}

void TmxLayer::draw(sf::RenderTarget& target, sf::RenderStates states) const
{
    sf::Sprite spr;
    for(int x = m_drawRect.left; x<m_drawRect.width; x++)
    {
        for(int y = m_drawRect.top; y<m_drawRect.height; y++)
        {
            if(m_tiles[x][y].m_gid!=0)
            {
                spr.setTexture(*m_tiles[x][y].m_texture);
                spr.setTextureRect(m_tiles[x][y].m_rect);
                spr.setPosition(m_tiles[x][y].m_x,m_tiles[x][y].m_y);
                target.draw(spr);
            }
        }
    }
}

TmxMap::TmxMap()
{
    //ctor
    m_ready = false;
}

TmxMap::~TmxMap()
{
    //dtor
}

void TmxMap::loadFromFile(const char* dir, const char* filename)
{
    {
        sf::Lock dataLock(m_dataMutex);
        m_dir = dir;
        m_filename = filename;
    }
    //sf::Thread thread(&TmxMap::loadingThread,this);
    //thread.launch();
    //m_thread = sf::Thread(&TmxMap::loadingThread,this);
    //m_thread.launch();
    m_thread = new sf::Thread(&TmxMap::loadingThread,this);
    m_thread->launch();
}

bool TmxMap::isReady()
{
    sf::Lock lock(m_readyMutex);
    return m_ready;
}

void TmxMap::setDrawRect(int x, int y, int width, int height)
{
    std::vector<TmxLayer*>::iterator it = m_layers.begin();
    while ( it != m_layers.end() )
    {
        (*it)->setDrawRect(x,y,width,height);
        ++it;
    }
}

void TmxMap::loadingThread()
{
    sf::Lock dataLock(m_dataMutex);

    sf::Clock loadTime;

    // Load xml document
    tinyxml2::XMLDocument* mapFile = new tinyxml2::XMLDocument(true,tinyxml2::COLLAPSE_WHITESPACE);
    std::string fullPath = m_dir + m_filename;
    mapFile->LoadFile(fullPath.c_str());
    tinyxml2::XMLElement* root =  mapFile->FirstChildElement("map");

    // Read map propetries
    m_width = root->IntAttribute("width");
    m_height = root->IntAttribute("height");
    m_tileWidth = root->IntAttribute("tilewidth");
    m_tileHeight = root->IntAttribute("tileheight");

    // Read map elements
    tinyxml2::XMLElement* element = root->FirstChildElement();
    while(element!=NULL)
    {
        if(!strcmp(element->Name(),"tileset"))
        {
            std::string name = element->Attribute("name");
            int firstGid = element->IntAttribute("firstgid");
            int tileWidth = element->IntAttribute("tilewidth");
            int tileHeight = element->IntAttribute("tileheight");
            std::string filename = m_dir + element->FirstChildElement("image")->Attribute("source");

            TmxTileset* tileset = new TmxTileset(name,firstGid,tileWidth,tileHeight,filename);
            m_tilesets.push_back(tileset);
        }

        if(!strcmp(element->Name(),"layer"))
        {
            tinyxml2::XMLElement* dataElement = element->FirstChildElement("data");
            m_encoding = dataElement->Attribute("encoding");
            m_compression = dataElement->Attribute("compression");
            std::string data = dataElement->GetText();

            if(m_encoding == "base64")
            {
                int expectedSize = m_width * m_height * 4; //number of tiles * 4 bytes = 32bits / tile
                data = base64_decode(data);

                std::vector<unsigned char>byteArray;
                if(m_compression == "zlib" || m_compression == "gzip")
                {
                    //std::cout << "Found " << m_compression << " compressed data" << std::endl;
                    int dataSize = data.length() * sizeof(unsigned char);
                    byteArray =  m_Decompress(data.c_str(), dataSize, expectedSize);
                }
                else //already uncompressed
                {
                    for(unsigned int i = 0; i < data.length(); i++)
                    {
                        byteArray.push_back(data.c_str()[i]);
                    }
                }

                int actualSize = byteArray.size() * sizeof(unsigned char);

if(actualSize != expectedSize)
{
std::cout << "Unexpected decompression size" << std::endl;
std::cout << "Actual size: " << actualSize << std::endl;
std::cout << "Expected size: " << expectedSize << std::endl;
return;
}

                int x, y;
                x = y = 0;
                TmxLayer* layer = new TmxLayer(m_width,m_height,m_tileWidth,m_tileHeight,&m_tilesets);
for(int i = 0; i < expectedSize - 3; i +=4)
{
int tileGID = byteArray[i] | byteArray[i + 1] << 8 | byteArray[i + 2] << 16 | byteArray[i + 3] << 24;
//m_SetTile(subRects, layer, x, y, tileGID);
//std::cout << "[" << x << "," << y << "] - " << tileGID << std::endl;
//std::cout << m_width << std::endl;
layer->setTile(x,y,tileGID);

x++;
if(x == m_width)
{
x = 0;
y++;
}
}
m_layers.push_back(layer);
std::cout << "Layer fully readed." << std::endl;
            }
        }

        element = element->NextSiblingElement();
    }

    // Set map to ready
    sf::Lock lock(m_readyMutex);
    m_ready = true;

    std::cout << "Map loaded (" << loadTime.getElapsedTime().asSeconds() << "s)\n";
}

void TmxMap::draw(sf::RenderTarget& target, sf::RenderStates states) const
{
    //std::cout << "Trololo" << std::endl;

    std::vector<TmxLayer*>::const_iterator it = m_layers.begin();
    while( it != m_layers.end())
    {
        target.draw(*(*it),states);
        ++it;
    }
}

std::vector<unsigned char> TmxMap::m_Decompress(const char* source, int inSize, int expectedSize)
{
std::vector<unsigned char> retVal;
int currentSize = expectedSize;
unsigned char* byteArray = new unsigned char[expectedSize / sizeof(unsigned char)];
z_stream stream;
stream.zalloc = Z_NULL;
stream.zfree = Z_NULL;
stream.opaque = Z_NULL;
stream.next_in = (Bytef*)source;
stream.avail_in = inSize;
stream.next_out = (Bytef*)byteArray;
stream.avail_out = expectedSize;

if(inflateInit2(&stream, 15 + 32) != Z_OK)
{
std::cout << "inflate 2 failed" << std::endl;
return retVal;
}

int result;
do
{
result = inflate(&stream, Z_SYNC_FLUSH);

switch(result)
{
case Z_NEED_DICT:
case Z_STREAM_ERROR:
result = Z_DATA_ERROR;
case Z_DATA_ERROR:
case Z_MEM_ERROR:
inflateEnd(&stream);
std::cout << result << std::endl;
return retVal;
}

if(result != Z_STREAM_END)
{
int oldSize = currentSize;
currentSize *= 2;
unsigned char* newArray = new unsigned char[currentSize / sizeof(unsigned char)];
std::memcpy(newArray, byteArray, currentSize / 2);
delete[] byteArray;
byteArray = newArray;

stream.next_out = (Bytef*)(byteArray + oldSize);
stream.avail_out = oldSize;

}
}
while(result != Z_STREAM_END);

if(stream.avail_in != 0)
{
std::cout << "stream.avail_in is 0" << std::endl;
return retVal;
}

const int outSize = currentSize - stream.avail_out;
inflateEnd(&stream);

unsigned char* newArray = new unsigned char[outSize / sizeof(unsigned char)];
std::memcpy(newArray, byteArray, outSize);
delete[] byteArray;
byteArray = newArray;

//copy bytes to vector
int length = currentSize / sizeof(unsigned char);
for(int i = 0; i < length; i++)
retVal.push_back(byteArray[i]);
delete[] byteArray;
return retVal;
}

}; // namespace keb

//base64 decode function taken from:
/*
   base64.cpp and base64.h

   Copyright (C) 2004-2008 René Nyffenegger

   This source code is provided 'as-is', without any express or implied
   warranty. In no event will the author be held liable for any damages
   arising from the use of this software.

   Permission is granted to anyone to use this software for any purpose,
   including commercial applications, and to alter it and redistribute it
   freely, subject to the following restrictions:

   1. The origin of this source code must not be misrepresented; you must not
      claim that you wrote the original source code. If you use this source code
      in a product, an acknowledgment in the product documentation would be
      appreciated but is not required.

   2. Altered source versions must be plainly marked as such, and must not be
      misrepresented as being the original source code.

   3. This notice may not be removed or altered from any source distribution.

   René Nyffenegger rene.nyffenegger@adp-gmbh.ch

*/

static const std::string base64_chars =
             "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
             "abcdefghijklmnopqrstuvwxyz"
             "0123456789+/";


static inline bool is_base64(unsigned char c)
{
return (isalnum(c) || (c == '+') || (c == '/'));
}

std::string base64_decode(std::string const& encoded_string) {
  int in_len = encoded_string.length();
  int i = 0;
  int j = 0;
  int in_ = 0;
  unsigned char char_array_4[4], char_array_3[3];
  std::string ret;

  while (in_len-- && ( encoded_string[in_] != '=') && is_base64(encoded_string[in_])) {
char_array_4[i++] = encoded_string[in_]; in_++;
if (i ==4) {
for (i = 0; i <4; i++)
char_array_4[i] = base64_chars.find(char_array_4[i]);

char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);
char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);
char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];

for (i = 0; (i < 3); i++)
ret += char_array_3[i];
i = 0;
}
  }

  if (i) {
    for (j = i; j <4; j++)
      char_array_4[j] = 0;

    for (j = 0; j <4; j++)
      char_array_4[j] = base64_chars.find(char_array_4[j]);

    char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);
    char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);
    char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];

    for (j = 0; (j < i - 1); j++) ret += char_array_3[j];
  }
  //std::cout << ret.length() << std::endl;
  return ret;
}

Error is throw on the point of loading this map :/

[Laurent]

You should check the return value of loadFromFile. There's a maximum texture size, so I guess that it fails with big sizes and since you don't handle this error case, you let the program continue in an undefined state and eventually crash somewhere.

[Kebrian]

I know but problem is not big textures. I find out that its crashing when trying to decompress second layer from zip.

[Laurent]

You can't just paste your entire code on a forum and say " it crashes, please find my error". Nobody wants to read so much code and try to find an error just by looking at it. You have a debugger which should be able to give you relevant information (error message, callstack, context, ...), you can also play with the code, remove stuff, even try to reproduce the problem in a minimal app that focuses on the crash.

[Kebrian]

That was the problem that debugger says nothing (as i pasted in first post).

Problem solved and anyone is intrested it was problem with decompress function from user fallahn in this thread - http://en.sfml-dev.org/forums/index.php?topic=3023.msg62106#msg62106

It seems that it's always crash on bigger maps. Changing decompress code to that presented here - https://code.google.com/p/tmx-parser/ solved problem.

[Laurent]

That was the problem that debugger says nothing (as i pasted in first post).

It should. If it doesn't, it means that either your compile flags or your debugger is not configured properly, and you will soon fall into the same problems when the next crash occurs. So even if you solved your problem without it this time, it's probably worth making the debugger work

[Kebrian]

Debugger works perfectly I can't imagine work without it xD

[Laurent]

So, why not in this case?

[Kebrian]

I think i linked zlib release in debug build.