UTF-8 and stuff

[Danetta]

Hello.

EDIT: It seems like I resolved almost all the issues.

EDIT2: How much bytes I possibly need to convert from sf::String to std::string or std::wstring?
In case when I need to resize empty strings so the converted one would fit. Probably an easy question, but can't google it.

[eXpl0it3r]

Just make use of the convert functions the SFML provides, see the documentation.

[Danetta]

Almost all of them require iterators to begin of original string | end of original string | begin of output string.
Output string is empty by default and most conversion function from there would give me

string iterator + offset out of range

if I do not set the size of output string manually like that:

std::string ANSI_to_UTF8(const sf::String& original)
{
        std::string ansi;                                                              
        ansi.resize(original.getSize() * 4);

        std::string::iterator last = sf::Utf<8>::fromAnsi(original.begin(), original.end(), ansi.begin(), std::locale("Russian"));
        ansi.resize(last - ansi.begin());
        return ansi;
}

[eXpl0it3r]

Maybe describe your problem in detail, because I don't really understand the issue. Or maybe you just misunderstood the API?

If you have a sf::String that is UTF and want to convert to an ANSI string, all you have to do is call toAnsiString() (and toWideString() for wstring).

If you want to do it manually, you could also take a look how SFML does it internally for toAnsi.

[dabbertorres]

string iterator + offset out of range

if I do not set the size of output string manually like that

You want std::back_inserter

[Danetta]

Maybe describe your problem in detail, because I don't really understand the issue. Or maybe you just misunderstood the API?

If you have a sf::String that is UTF and want to convert to an ANSI string, all you have to do is call toAnsiString() (and toWideString() for wstring).

If you want to do it manually, you could also take a look how SFML does it internally for toAnsi.

What if my sf::String is widestring and I want to convert it to UTF-8?

string iterator + offset out of range

if I do not set the size of output string manually like that

You want std::back_inserter

Seems like true, will try to figure out.

[Laurent]

What if my sf::String is widestring and I want to convert it to UTF-8?

sf::String::toUtf8(). And you don't have to care about what is stored internally in sf::String (it's UTF-32, by the way).

You still have said nothing about your real problem. This conversation can last for days if you don't tell us what you really want to do.

[Danetta]

I find sf::String::toUtf8 too hard to use because output is std::basic_string while sf::Utf<8> allows me to ger result almost within in a single string. I tried it and got what I need, but with some problems like having to resize declared strings manually so converted ones would fit into them.

Answering your question about what I am trying to do:
Client terminal displaying strings in cp_1251, client gui (label and editboxes) using widestring (seems like?), server processing requests to Database using UTF-8 strings as arguments.

So, when I receive widestring (or UTF-32? I am still not sure, but widestring works well when I use it as parameter; I don't want to, however, but TGUI documentation only said it's "sf::String"; also, UTF-8 doesn't display well with TGUI unless I convert to widestring.. ugh..) from gui, I need to handle all the encodings and decodings from client to server and then back.


Edited.

[Laurent]

I find sf::String::toUtf8 too hard to use because output is std::basic_string

?
std::string is just a typedef to std::basic_string<char>; so what you get with this function is basically std::string with something else than char inside.

I tried it and got what I need, but with some problems like having to resize declared strings manually so converted ones would fit into them.

As already said, std::back_inserter is the solution. Look at sf::Utf source code for examples.

[Danetta]

std::string String::toAnsiString(const std::locale& locale) const
{
    // Prepare the output string
    std::string output;
    output.reserve(m_string.length() + 1);

    // Convert
    Utf32::toAnsi(m_string.begin(), m_string.end(), std::back_inserter(output), 0, locale);

    return output;
}

Why would you use string::reserve in this case? Is not it unneeded when you use std::back_inserter?

[eXpl0it3r]

I'm not certain that this is the primary reason, but by reserving the needed memory space, you can do one memory allocation opposed to the string incrementally allocating more and more space.

[Laurent]

Yes, it's just an optimization to avoid many small memory re-allocations. You can remove this line, the function will still work as expected.