Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: [Linux] using sf::String makes the accents disapear  (Read 8399 times)

0 Members and 1 Guest are viewing this topic.

santiaboy

  • Full Member
  • ***
  • Posts: 118
    • View Profile
[Linux] using sf::String makes the accents disapear
« on: August 19, 2013, 02:52:10 am »
Hey guys, I've made a game in windows and was trying to port it to Linux. The game uses accents, so in windows I use std::wstrings. However, in linux, you don't have to use wstrings, but std::strings.

However, I have a problem: In windows works perfectly, but in linux it doesn't.

Example code


#include <iostream>
#include <fstream>
#include <SFML/System.hpp>

int main()
{
    std::fstream fin("folder/file");
    std::string aux = "";
        sf::String sfmlAux;
    while(getline(fin, aux)){
        std::cout << aux << std::endl;
                sfmlAux = aux;
                aux = sfmlAux;
                std::cout << aux << std::endl;
    }

    return 0;
}

 

the file contains
Code: [Select]
así
in UTF-8

The output of the program:
Code: [Select]
así
as

Thing is, using wstrings in windows works like a charm. I just change wstrings for strings (linux) and it doesn't work. Any ideas?

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: [Linux] using sf::String makes the accents disapear
« Reply #1 on: August 19, 2013, 07:50:24 am »
Did you try std::wstring on Linux?
Laurent Gomila - SFML developer

FRex

  • Hero Member
  • *****
  • Posts: 1845
  • Back to C++ gamedev with SFML in May 2023
    • View Profile
    • Email
Re: [Linux] using sf::String makes the accents disapear
« Reply #2 on: August 19, 2013, 11:48:29 am »
Try these functions:
sf::String sf32FromStd8(const std::string& str)
        {
                sf::String ret;
                sf::Uint32 c;
                auto it=str.begin();
                while(it!=str.end())
                {
                        it=sf::Utf8::decode(it,str.end(),c,0u);
                        if(c!=0u) ret+=c;
                }
                return ret;
        }
         std::string std8FromSf32(const sf::String& str)
        {
                std::string ret;
                char buf[5];

                for(auto it=str.begin();it!=str.end();++it)
                {
                        auto end=sf::Utf8::encode(*it,buf);
                        *end='\0';
                        ret+=buf;
                }
                return ret;
        }
Edit: Sorry, from sf to std was wrong the first time, I just fixed it and it works for me, I never tested converting that way before. ;D
« Last Edit: August 19, 2013, 12:21:26 pm by FRex »
Back to C++ gamedev with SFML in May 2023

santiaboy

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: [Linux] using sf::String makes the accents disapear
« Reply #3 on: August 19, 2013, 08:05:58 pm »
Did you try std::wstring on Linux?

Yes I did. Weird thing is, the purpose of the std::string in linux are the std::wstring in windows.

@FRex, thanks! I'm going to try it and hopefully it will work :)

FRex

  • Hero Member
  • *****
  • Posts: 1845
  • Back to C++ gamedev with SFML in May 2023
    • View Profile
    • Email
Re: [Linux] using sf::String makes the accents disapear
« Reply #4 on: August 19, 2013, 11:32:12 pm »
Quote
Weird thing is, the purpose of the std::string in linux are the std::wstring in windows.
That sentence didn't make any sense.

Quote
@FRex, thanks! I'm going to try it and hopefully it will work :)
Try it and get back with results, I'd like to know if it doesn't work on some Linux too, since I'm using it for myself and it's supposedly 100% crossplatform. ;D
Back to C++ gamedev with SFML in May 2023

santiaboy

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: [Linux] using sf::String makes the accents disapear
« Reply #5 on: August 20, 2013, 03:20:29 am »
Quote
Quote
Weird thing is, the purpose of the std::string in linux are the std::wstring in windows.
That sentence didn't make any sense.
I meant that can you use string in linux for the accents (á, é, etc) and you CAN'T use wstring.
In windows, is the other way around, string doesn't cut it, wstring does.

Quote
Quote
@FRex, thanks! I'm going to try it and hopefully it will work :)
Try it and get back with results, I'd like to know if it doesn't work on some Linux too, since I'm using it for myself and it's supposedly 100% crossplatform. ;D

Couldn't  try it yet, I will post results!

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: [Linux] using sf::String makes the accents disapear
« Reply #6 on: August 20, 2013, 07:43:50 am »
Quote
I meant that can you use string in linux for the accents (á, é, etc) and you CAN'T use wstring.
It still doesn't make sense.

Do you know what's behind these string classes, their encoding and their character type? Do you know how they interact with your file and console stream? Do you know how sf::String works? Do you know how the current locale messes up with all this stuff?

That would be the first step towards a robust implementation ;)
Laurent Gomila - SFML developer

santiaboy

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: [Linux] using sf::String makes the accents disapear
« Reply #7 on: August 21, 2013, 05:35:05 am »
Upon further research, I did not know what happens on the string classes. I thought that the characters that didn't appear on std::cout << str just weren't there. My bad.

@FRex, is that code c++11? I mean because of the "auto". I couldn't even compile it on Windows 32 bits. (Haven't tried it in Linux) In order for it to work, I did this:


sf::String sf32FromStd8(const std::string& str)
    {
        sf::String ret;
        sf::Uint32 c;
        std::string::const_iterator it=str.begin();
        while(it!=str.end())
        {
            it=sf::Utf8::decode(it,str.end(),c,0u);
            if(c!=0u) ret+=c;
        }
        return ret;
    }

 

It worked on a small example, and it is almost working on my project (I don't know why, there's a line of text that's giving me trouble.) I will try it on linux as soon as I can, and will post results. Thanks both of you! :)

Nexus

  • SFML Team
  • Hero Member
  • *****
  • Posts: 6286
  • Thor Developer
    • View Profile
    • Bromeon
Re: [Linux] using sf::String makes the accents disapear
« Reply #8 on: August 21, 2013, 10:25:57 am »
@FRex, is that code c++11? I mean because of the "auto". I couldn't even compile it on Windows 32 bits.
Yes, it's C++11. What do you mean with "Windows", which compiler do you use? I'm asking because already Visual Studio 2010, which was released 3.5 years ago, is able to compile this code...

If you use g++, you'll have to specify the flag -std=c++0x or -std=c++11.
Zloxx II: action platformer
Thor Library: particle systems, animations, dot products, ...
SFML Game Development:

santiaboy

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: [Linux] using sf::String makes the accents disapear
« Reply #9 on: August 21, 2013, 05:25:38 pm »
I'm sorry, as you can see I'm still a beginner.

I am using code::blocks 12.11 mingw, which includes the GCC compiler and GDB debugger from TDM-GCC (version 4.7.1, 32 bit). Is this enough info?

I never specified either -std=c++0x or -std=c++11. I am asuming code::blocks uses c++0x as a default, because the "auto" gave me some errors and warnings. I can post them if you like.

Nexus

  • SFML Team
  • Hero Member
  • *****
  • Posts: 6286
  • Thor Developer
    • View Profile
    • Bromeon
Re: [Linux] using sf::String makes the accents disapear
« Reply #10 on: August 21, 2013, 05:29:49 pm »
I am asuming code::blocks uses c++0x as a default, because the "auto" gave me some errors and warnings.
I don't get your reasoning. The errors are rather an indication that Code::Blocks does not use C++11 as default.

Just specify -std=c++0x and you're fine.
Zloxx II: action platformer
Thor Library: particle systems, animations, dot products, ...
SFML Game Development:

santiaboy

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: [Linux] using sf::String makes the accents disapear
« Reply #11 on: August 21, 2013, 05:54:55 pm »
Ah, thanks! Is there a reason to use auto and not use for example std::string::const_iterator? I mean, I know it's that type, why would I use auto and not the explicit type? Is it to make it "prettier" to the person programming?

If there's a chance of not being portable, I prefer to use the explicit type.

I am asuming code::blocks uses c++0x as a default, because the "auto" gave me some errors and warnings.
I don't get your reasoning. The errors are rather an indication that Code::Blocks does not use C++11 as default.

Just specify -std=c++0x and you're fine.

I messed up there..again. I meant I am not using c++11 as a default.

Nexus

  • SFML Team
  • Hero Member
  • *****
  • Posts: 6286
  • Thor Developer
    • View Profile
    • Bromeon
Re: [Linux] using sf::String makes the accents disapear
« Reply #12 on: August 21, 2013, 06:28:51 pm »
Is there a reason to use auto and not use for example std::string::const_iterator?
Yes, of course: Abstraction and code clarity.

You are not interested in the exact syntax of the iterator type. You know it's an iterator, which is already visible in the variable name, the rest is irrelevant and can be left to the compiler. Similar to typedef, just that you don't need to define the type beforehand. As a result, code becomes much clearer to read.

If there's a chance of not being portable, I prefer to use the explicit type.
You have to decide which C++ version you use. But you should be aware that C++11 has been officially released two years ago, and compilers have started to implemented it even earlier. Unless you have a good reason to stick to the past, it is a good idea to benefit of the innovations C++11 brought. Some C++11 features are still not portably implemented, but the support for type inference is quite good.
Zloxx II: action platformer
Thor Library: particle systems, animations, dot products, ...
SFML Game Development:

santiaboy

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: [Linux] using sf::String makes the accents disapear
« Reply #13 on: August 21, 2013, 06:47:47 pm »
 @Nexus, thanks, I'll look into c++11 when I have the chance!

@Frex, I tried it in Ubuntu 12.04 LTs (Linux 64 bits). I used the g++ compiler, using the console (build-essentials one). Thanks again!

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: [Linux] using sf::String makes the accents disapear
« Reply #14 on: August 24, 2013, 05:16:16 pm »
The latest revision adds a sf::String::fromUtf8 function.

By the way, assuming an UTF-8 encoding on Linux is not 100% safe; although it is the default, I guess it can be changed.
Laurent Gomila - SFML developer

 

anything