[Linux] using sf::String makes the accents disapear

[santiaboy]

Hey guys, I've made a game in windows and was trying to port it to Linux. The game uses accents, so in windows I use std::wstrings. However, in linux, you don't have to use wstrings, but std::strings.

However, I have a problem: In windows works perfectly, but in linux it doesn't.

Example code

#include <iostream>
#include <fstream>
#include <SFML/System.hpp>

int main()
{
    std::fstream fin("folder/file");
    std::string aux = "";
        sf::String sfmlAux;
    while(getline(fin, aux)){
        std::cout << aux << std::endl;
                sfmlAux = aux;
                aux = sfmlAux;
                std::cout << aux << std::endl;
    }

    return 0;
}

 

the file contains

Code: [Select]

así
in UTF-8

The output of the program:

Code: [Select]

así
as

Thing is, using wstrings in windows works like a charm. I just change wstrings for strings (linux) and it doesn't work. Any ideas?

[Laurent]

Did you try std::wstring on Linux?

[FRex]

Try these functions:

sf::String sf32FromStd8(const std::string& str)
        {
                sf::String ret;
                sf::Uint32 c;
                auto it=str.begin();
                while(it!=str.end())
                {
                        it=sf::Utf8::decode(it,str.end(),c,0u);
                        if(c!=0u) ret+=c;
                }
                return ret;
        }
         std::string std8FromSf32(const sf::String& str)
        {
                std::string ret;
                char buf[5];

                for(auto it=str.begin();it!=str.end();++it)
                {
                        auto end=sf::Utf8::encode(*it,buf);
                        *end='\0';
                        ret+=buf;
                }
                return ret;
        }

Edit: Sorry, from sf to std was wrong the first time, I just fixed it and it works for me, I never tested converting that way before.

[santiaboy]

Did you try std::wstring on Linux?

Yes I did. Weird thing is, the purpose of the std::string in linux are the std::wstring in windows.

@FRex, thanks! I'm going to try it and hopefully it will work

[FRex]

Weird thing is, the purpose of the std::string in linux are the std::wstring in windows.

That sentence didn't make any sense.

@FRex, thanks! I'm going to try it and hopefully it will work

Try it and get back with results, I'd like to know if it doesn't work on some Linux too, since I'm using it for myself and it's supposedly 100% crossplatform.

[santiaboy]

Weird thing is, the purpose of the std::string in linux are the std::wstring in windows.

That sentence didn't make any sense.

I meant that can you use string in linux for the accents (á, é, etc) and you CAN'T use wstring.
In windows, is the other way around, string doesn't cut it, wstring does.

@FRex, thanks! I'm going to try it and hopefully it will work

Try it and get back with results, I'd like to know if it doesn't work on some Linux too, since I'm using it for myself and it's supposedly 100% crossplatform.

Couldn't  try it yet, I will post results!

[Laurent]

I meant that can you use string in linux for the accents (á, é, etc) and you CAN'T use wstring.

It still doesn't make sense.

Do you know what's behind these string classes, their encoding and their character type? Do you know how they interact with your file and console stream? Do you know how sf::String works? Do you know how the current locale messes up with all this stuff?

That would be the first step towards a robust implementation

[santiaboy]

Upon further research, I did not know what happens on the string classes. I thought that the characters that didn't appear on std::cout << str just weren't there. My bad.

@FRex, is that code c++11? I mean because of the "auto". I couldn't even compile it on Windows 32 bits. (Haven't tried it in Linux) In order for it to work, I did this:

sf::String sf32FromStd8(const std::string& str)
    {
        sf::String ret;
        sf::Uint32 c;
        std::string::const_iterator it=str.begin();
        while(it!=str.end())
        {
            it=sf::Utf8::decode(it,str.end(),c,0u);
            if(c!=0u) ret+=c;
        }
        return ret;
    }

 

It worked on a small example, and it is almost working on my project (I don't know why, there's a line of text that's giving me trouble.) I will try it on linux as soon as I can, and will post results. Thanks both of you!

[Nexus]

@FRex, is that code c++11? I mean because of the "auto". I couldn't even compile it on Windows 32 bits.

Yes, it's C++11. What do you mean with "Windows", which compiler do you use? I'm asking because already Visual Studio 2010, which was released 3.5 years ago, is able to compile this code...

If you use g++, you'll have to specify the flag -std=c++0x or -std=c++11.

[santiaboy]

I'm sorry, as you can see I'm still a beginner.

I am using code::blocks 12.11 mingw, which includes the GCC compiler and GDB debugger from TDM-GCC (version 4.7.1, 32 bit). Is this enough info?

I never specified either -std=c++0x or -std=c++11. I am asuming code::blocks uses c++0x as a default, because the "auto" gave me some errors and warnings. I can post them if you like.

[Nexus]

I am asuming code::blocks uses c++0x as a default, because the "auto" gave me some errors and warnings.

I don't get your reasoning. The errors are rather an indication that Code::Blocks does not use C++11 as default.

Just specify -std=c++0x and you're fine.

[santiaboy]

Ah, thanks! Is there a reason to use auto and not use for example std::string::const_iterator? I mean, I know it's that type, why would I use auto and not the explicit type? Is it to make it "prettier" to the person programming?

If there's a chance of not being portable, I prefer to use the explicit type.

I am asuming code::blocks uses c++0x as a default, because the "auto" gave me some errors and warnings.

I don't get your reasoning. The errors are rather an indication that Code::Blocks does not use C++11 as default.

Just specify -std=c++0x and you're fine.

I messed up there..again. I meant I am not using c++11 as a default.

[Nexus]

Is there a reason to use auto and not use for example std::string::const_iterator?

Yes, of course: Abstraction and code clarity.

You are not interested in the exact syntax of the iterator type. You know it's an iterator, which is already visible in the variable name, the rest is irrelevant and can be left to the compiler. Similar to typedef, just that you don't need to define the type beforehand. As a result, code becomes much clearer to read.

If there's a chance of not being portable, I prefer to use the explicit type.

You have to decide which C++ version you use. But you should be aware that C++11 has been officially released two years ago, and compilers have started to implemented it even earlier. Unless you have a good reason to stick to the past, it is a good idea to benefit of the innovations C++11 brought. Some C++11 features are still not portably implemented, but the support for type inference is quite good.

[santiaboy]

 @Nexus, thanks, I'll look into c++11 when I have the chance!

@Frex, I tried it in Ubuntu 12.04 LTs (Linux 64 bits). I used the g++ compiler, using the console (build-essentials one). Thanks again!

[Laurent]

The latest revision adds a sf::String::fromUtf8 function.

By the way, assuming an UTF-8 encoding on Linux is not 100% safe; although it is the default, I guess it can be changed.