Author Topic: Unicode utility function to create UTF32 strings (Read 10075 times)

Tank · « **on:** February 17, 2009, 04:36:58 pm »

I'm currently changing all strings in my application to Unicode::UTF32String. The only way to properly create a new UTF32String is by using Unicode::Text, but it's not really a suitable solution, since you need to construct a temporary object to get an UTF32String. Example, using a constant:

Code: [Select]

sf::Unicode::UTF32String str( sf::Unicode::Text( L"Hello World!" ) );

Since Unicode::UTF32String is actually a basic_string<Uint32>, I don't really know how to make that beast more graceful. But I dislike the idea that I can't immediately construct an UTF32String with a value.

So my opinion is that there should be an utility function that builds an UTF32String, at least from well-known types like const char*, const wchar_t*, const std::string& etc.

If there's another possibility I didn't see yet, I'd be happy to hear from that. In this case, declare this request as a request for help.

Laurent · « **Reply #1 on:** February 17, 2009, 04:58:25 pm »

Yeah, Unicode strings will be improved in SFML 2.0. But didn't I already answer that in your previous post?

Tank · « **Reply #2 on:** February 17, 2009, 05:30:16 pm »

Nope, I was more complaining about the default "ourDefaultCharset" and displaying invalid glyphs (patch provided). My last post in http://www.sfml-dev.org/forum/viewtopic.php?p=6573#6573 wasn't answered, so I just wanted to make sure that my point gets understood correctly as a request.

I don't like the fact that it's supposed for SFML2, because it's basic and belongs to SFML since it has been changed to support Unicode. I can't construct UTF32 strings besides the workaround with sf::Unicode::Text. This request asks for enhancing the *current* state of SFML, to be able to work better with.

Another problem is the time that this fundamental thing needs until implemented. I've got a rather huge project going on which needs exhaustive support in the Unicode area, and I can't really wait 3, 4 or 5 months to get that simple feature added.

Laurent · « **Reply #3 on:** February 17, 2009, 11:02:03 pm »

Yeah, I know it's quite frustrating

But there are so many changes, that I'd really like to make them all at once.

However it doesn't mean you won't be able to see the results before many months, it will probably be available on SVN "soon". Right now I'm rewritting the OpenGL context management, but I think Unicode is a good candidate for the next task

The most important is that you can make what you want with a little workaround, so this issue isn't really blocking you.

Tank · « **Reply #4 on:** February 18, 2009, 01:47:13 am »

That's good news, really.

Thanks for answering all that stuff. Do you think it'd be possible to update the roadmap more frequently? I know, it's a lot of work, but that keeps me from asking stupid questions.

dabo · « **Reply #5 on:** February 18, 2009, 03:17:32 am »

Quote from: "Laurent"

Right now I'm rewritting the OpenGL context management

So why did you stop the render-to-image implementation at 80%?

Laurent · « **Reply #6 on:** February 18, 2009, 07:54:46 am »

Quote

Do you think it'd be possible to update the roadmap more frequently?

This Unicode stuff wasn't initially planned for SFML 2.0, but after your requests it's obviously something that has to be done. I indeed haven't updated the roadmap yet, I'll do it as soon as possible.

Quote

So why did you stop the render-to-image implementation at 80%?

I had something working, but it required some rewritting and cleaning of image and context stuff, that's why I stopped and postponed it to SFML 2.0.

Tank · « **Reply #7 on:** February 18, 2009, 12:03:14 pm »

Thank you very much for your effort.

Tank · « **Reply #8 on:** September 03, 2009, 04:05:53 pm »

Just wanted to ask if there's any progress in the Unicode area. ;-) I'm again messing around with sf::Unicode::Text, which definitely lacks some useful methods (accessing the string in a non-const way, or provide methods for manipulating it, which wouldn't be my preferred method since we already have a basic_string up there).

At the moment I'm stumbling around in the SFML sources by myself but would love to have it improved from the official side.

Laurent · « **Reply #9 on:** September 03, 2009, 04:32:07 pm »

I already thought about a complete rewrite in SFML 2.0, which will most likely be the case, but I still haven't found a clean and good public interface.

Tank · « **Reply #10 on:** September 04, 2009, 06:27:28 pm »

A complete rewrite sounds good. Do you want to keep using basic_string?

Laurent · « **Reply #11 on:** September 04, 2009, 06:33:32 pm »

I still don't know.

Tank · « **Reply #12 on:** September 04, 2009, 06:42:41 pm »

If you have some random thoughts it would be nice to hear them. I'm also thinking about how Unicode could be properly handled in SFML (not to implement it, but to create an appropiate feature reqeust

).

One possibility could be extending sf::Unicode::Text so that it has the most needed features like string concatenation. The bad thing is that this would lead to re-implementation, since basic_string already has all that stuff implemented.

My current workaround is providing a "GetString()" method in sf::Unicode::Text that gives full access to its basic_string member (don't know the name of it, right now -- but since there's only one, you'll know which I mean). And when you think of it, it's not really thaaat bad. sf::Unicode::Text could be just a wrapper around a basic_string providing some extra functionality (in this case: conversions between several string types, like it's currently done in sf::Unicode::Text).

Another way to implement Unicode would be to create a complete own string class. But I don't like that idea, since that class wouldn't work with the rest of the standard library (especially streams) without doing shitloads of helper functions.

Laurent · « **Reply #13 on:** September 04, 2009, 06:47:54 pm »

That's more or less where I stopped the last time I thought about it

Rewriting a string class would solve some of the issues, but it's bad I agree.

I also thought about a sf::Char class that would handle the Unicode stuff (like QChar in Qt), and just provide a basic_string<sf::Char>. But it was still not a robust solution -- I don't exactly remember why, it was several months ago.

Tank · « **Reply #14 on:** September 04, 2009, 07:26:34 pm »

Quote from: "Laurent"

I also thought about a sf::Char class that would handle the Unicode stuff (like QChar in Qt), and just provide a basic_string<sf::Char>. But it was still not a robust solution -- I don't exactly remember why, it was several months ago.

I don't know the Qt implementation, but wouldn't be an sf::Char basically an Uint32? As far as I know, SFML uses UTF32 internally (which is absolutely fine). So sf::Unicode::Text could be a typedef to std::basic_string<sf::Uint32> and sf::Unicode could contain the known methods for converting between encodings.

This is just an idea, I haven't thought about it deeply, but it seems very easy and wouldn't need much changes in the code. Discussion/whatever welcome.