SFML community forums

Help => System => Topic started by: MarsCoreTeam on October 10, 2010, 11:52:53 am

Title: [fixed] SFML2: sf::Utf8::ToUtf32 conversion problem
Post by: MarsCoreTeam on October 10, 2010, 11:52:53 am

Hello there!

I am trying to load an utf8-encoded text file into a std::vector<sf::String>. When I use this litte, but great library (http://utfcpp.sourceforge.net/) everything works as supposed to. After doing

Code: [Select]


void load(std::vector<sf::String>& lines) {
    // Open the test file (contains UTF-8 encoded text)
    std::ifstream fileStream("unitext.txt");
    std::string line;
    while (std::getline(fileStream, line)) {
        std::basic_string<sf::Uint32> utf32line;
        // this line converts utf8 to utf32 with the help of the metioned library
        utf8::utf8to32(line.begin(), line.end(), back_inserter(utf32line));
        lines.push_back(utf32line);
    }
}


the vector "lines" contains utf-32 encoded sf::Strings.
But since SFML supports utf conversions as well, I tried to do the same, only with SFML. Therefore I changed

Code: [Select]

utf8::utf8to32( ... );
to

Code: [Select]

sf::Utf8::ToUtf32( ... );
which, as I thought, should result in exactly the same behaviour. But it doesn't: When there is a non-ascii character, all trailing characters are dropped.

Am I doing something wrong :? ? Or is it a bug :shock: ?

Greetings, Simon.

Title: [fixed] SFML2: sf::Utf8::ToUtf32 conversion problem
Post by: Laurent on October 10, 2010, 10:22:36 pm

Can you show me a complete source code (with a sample text file similar to the one you load), so that I can try it myself?

Title: [fixed] SFML2: sf::Utf8::ToUtf32 conversion problem
Post by: MarsCoreTeam on October 11, 2010, 10:14:57 am

Thanks for your reply, Laurent!

There we go... the following example loads an utf8-encoded text file. I used  this one  (http://mars-game.sourceforge.net/wordpress/wp-content/uploads/unitext.txt) (right Click... save as). Since I'm on Linux, it's line endings are simple line feeds. Therefore it will look like one single line on Windows, but this shouldn't be a problem.
The lines of this file are displayed on screen with sf::Text's. The following code does not work for me. Nevertheless, when I include "utf8.h" (from the library mentioned in my first post) and use it's conversion function, it works...

Code: [Select]


# include <SFML/Graphics.hpp>
# include <fstream>
# include <sstream>
# include <iostream>
# include <vector>

//# include "utf8.h"

void load(std::vector<sf::String>& lines) {
    // load text file
    std::ifstream fileStream("unitext.txt");
    std::string line;
    // read line by line
    while (std::getline(fileStream, line)) {
        std::basic_string<sf::Uint32> utf32line;

        //utf8::utf8to32(line.begin(), line.end(), back_inserter(utf32line));
        sf::Utf8::ToUtf32(line.begin(), line.end(), back_inserter(utf32line));

        lines.push_back(utf32line);
    }
}

int main() {
    std::vector<sf::String> strings;
    load(strings);

    sf::RenderWindow App(sf::VideoMode(400, 300), "Unicode Test");
    App.SetFramerateLimit(60);

    // create a sf::Text for each sf::String
    std::vector<sf::Text> lines;
    int y(10);
    for (std::vector<sf::String>::iterator it = strings.begin(); it != strings.end(); ++it, y += 40) {
        sf::Text line(*it, sf::Font::GetDefaultFont(), 30.f);
        line.SetPosition(10.f, y);
        lines.push_back(line);
    }

    while (App.IsOpened())
    {
        sf::Event Event;
        while (App.GetEvent(Event))
        {
            if (Event.Type == sf::Event::Closed)
                App.Close();
        }
        App.Clear();

        // draw them all
        for (std::vector<sf::Text>::iterator it = lines.begin(); it != lines.end(); ++it)
            App.Draw(*it);

        App.Display();
    }
    return EXIT_SUCCESS;
}


Title: [fixed] SFML2: sf::Utf8::ToUtf32 conversion problem
Post by: Laurent on October 11, 2010, 10:25:36 am

Thank you very much, I'll test it as soon as possible :)

Title: [fixed] SFML2: sf::Utf8::ToUtf32 conversion problem
Post by: Laurent on October 11, 2010, 10:39:02 am

It was a bug in SFML, the conversion code was missing a cast to unsigned when the input type was signed.

Thanks for your help :)

Title: [fixed] SFML2: sf::Utf8::ToUtf32 conversion problem
Post by: MarsCoreTeam on October 11, 2010, 11:46:19 am

Hey, cool, Thanks :D ! And you've already committed it to SVN! Awesome!

Greetings, Simon.