Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: New unicode / string / text handling in SFML 2  (Read 18705 times)

0 Members and 1 Guest are viewing this topic.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
New unicode / string / text handling in SFML 2
« on: November 26, 2009, 09:16:05 am »
Hi

I've done many changes regarding strings and text:

- The low level Unicode handling now happens the in sf::Utf<X> classes (Utf<8>, Utf<16>, Utf<32> -- typedef'd to Utf8, Utf16, Utf32), with static functions. There are more functions, and the API is more consistent

- The sf::Unicode::Text class was replaced with sf::String. It's still implemented with UTF-32, and still handles automatically the conversions from/to ANSI and wide standard strings. It now contains more operators and functions, so that it is directly usable without having to cast it to another string type. However, I'm not 100% fan of this design and it might change in the future. If you have interesting ideas regarding this class, feel free to share them with me :)

- The sf::String graphics class was renamed to sf::Text. GetText and SetText were renamed to resp. GetString and SetString

Enjoy :)
Laurent Gomila - SFML developer

Dominator

  • Newbie
  • *
  • Posts: 37
    • View Profile
New unicode / string / text handling in SFML 2
« Reply #1 on: November 26, 2009, 12:53:48 pm »
I'm using VS2008 and trying to recompile SFML2-SVN, but the linker  complains about unresolved external symbols regarding sf::String in csfml-network.

The other projects compiled fine though.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
New unicode / string / text handling in SFML 2
« Reply #2 on: November 26, 2009, 01:57:54 pm »
I fixed this bug this morning, but I think I forgot to commit the changes.
Laurent Gomila - SFML developer

Tank

  • SFML Team
  • Hero Member
  • *****
  • Posts: 1486
    • View Profile
    • Blog
    • Email
New unicode / string / text handling in SFML 2
« Reply #3 on: November 26, 2009, 08:01:31 pm »
I was really looking forward for this feature. :) I'll check it out and drop you some comments then.

Tank

  • SFML Team
  • Hero Member
  • *****
  • Posts: 1486
    • View Profile
    • Blog
    • Email
New unicode / string / text handling in SFML 2
« Reply #4 on: December 02, 2009, 01:18:32 am »
Okay, I took a quick look into it. Looks good so far, I think. I'm not so happy that it couldn't be implemented by keeping at the STL, but it's nearly impossible because of the other bindings, I guess. This way it's a lot more portable.

However some methods are missing, like Insert(), SubString() etc. Also, Erase() should accept an Iterator, too (overloaded method).

I will try to use the new sf::String in one of my bigger projects that's using SFML2. But before I can actually do that, I need at least Insert(). ;)

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
New unicode / string / text handling in SFML 2
« Reply #5 on: December 02, 2009, 09:05:48 am »
Quote
However some methods are missing, like Insert(), SubString() etc. Also, Erase() should accept an Iterator, too (overloaded method).

Absolutely ;)
Like I said, this solution is kind of temporary, I will implement all the missing functions only when I'm 100% sure that I keep it.
Of course I'll implement everything that is in std::string, although I don't like this solution. I guess it would have been much easier if string functions in the STL were generic algorithms operating on a pair of iterators.

Quote
but it's nearly impossible because of the other bindings, I guess

sf::String is not implemented in any binding, actually. Each language already has its own string class supporting unicode natively.
The reason is that I simply couldn't find a clean and easy way to mix the features I needed (mainly automatic conversions between types/encodings) and std::basic_string. Every solution I found involved using a more verbose syntax for every string manipulation.
Laurent Gomila - SFML developer

panithadrum

  • Sr. Member
  • ****
  • Posts: 304
    • View Profile
    • Skyrpex@Github
    • Email
New unicode / string / text handling in SFML 2
« Reply #6 on: December 02, 2009, 12:37:53 pm »
So what wil be the final product? Will sf::String stand or will you finally come with a std::string solution.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
New unicode / string / text handling in SFML 2
« Reply #7 on: December 02, 2009, 12:43:21 pm »
sf::String is the current final product, it is the best solution I've found so far. But I'm not really satisfied of it, so if I find a better solution it might change again.

Don't forget that you can still use std::string and std::wstring instead of sf::String for string manipulations. Or even a better external string class. You can also write your own, with the sf::Utf low-level functions.
Laurent Gomila - SFML developer

Tank

  • SFML Team
  • Hero Member
  • *****
  • Posts: 1486
    • View Profile
    • Blog
    • Email
New unicode / string / text handling in SFML 2
« Reply #8 on: December 02, 2009, 02:08:21 pm »
Quote
I guess it would have been much easier if string functions in the STL were generic algorithms operating on a pair of iterators.

Yep, that's absolutely true.

I remember you once talked about a templated version of a character, like QChar in Qt. This way you keep the STL algorithms but can still do conversions for specified templates.

However, I think you've already thought about that. The current sf::String is fine and will work as supposed. Indeed directly using the STL would be great, but if that complicates things a lot, then just keep the current design.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
New unicode / string / text handling in SFML 2
« Reply #9 on: December 02, 2009, 02:31:26 pm »
Quote
I remember you once talked about a templated version of a character, like QChar in Qt. This way you keep the STL algorithms but can still do conversions for specified templates.

There are two problems with a generic Char class:
1. It cannot conveniently manipulate UTF-8 and UTF-16, where a single character can be represented by multiple elements (what would Char::ToUtf8() return?).
2. It doesn't help regarding the "automatic conversion" feature, I still need a class on top of standard strings for that (I don't want to use non-member functions in order to keep conversions implicit; I don't want something like text.SetString(sf::Utf32("blah"))
Laurent Gomila - SFML developer

Tank

  • SFML Team
  • Hero Member
  • *****
  • Posts: 1486
    • View Profile
    • Blog
    • Email
New unicode / string / text handling in SFML 2
« Reply #10 on: December 02, 2009, 02:36:57 pm »
Okay I see, those are good reasons. Well, then the current implementation seems to be alright. :)

Tank

  • SFML Team
  • Hero Member
  • *****
  • Posts: 1486
    • View Profile
    • Blog
    • Email
New unicode / string / text handling in SFML 2
« Reply #11 on: February 24, 2010, 07:10:13 pm »
Are there any new thoughts on this? Would be very helpful when the sf::String class would be more feature-complete. ;)

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
New unicode / string / text handling in SFML 2
« Reply #12 on: February 24, 2010, 07:16:01 pm »
No, sorry. I'm working on something totally different now.

But if you have new ideas (or at least requests), don't hesitate to tell me ;)
Laurent Gomila - SFML developer

Tank

  • SFML Team
  • Hero Member
  • *****
  • Posts: 1486
    • View Profile
    • Blog
    • Email
New unicode / string / text handling in SFML 2
« Reply #13 on: February 24, 2010, 09:59:20 pm »
Not really new ideas, still the old ones.

I think the current design is fine (the discussion happened already). What it lacks are the missing methods. Important should be: Insert, Find, SubString, At...uhm, probably you better take a look at the std::string documentation. ;)

Nexus

  • SFML Team
  • Hero Member
  • *****
  • Posts: 6287
  • Thor Developer
    • View Profile
    • Bromeon
New unicode / string / text handling in SFML 2
« Reply #14 on: February 24, 2010, 10:25:28 pm »
Quote from: "Tank"
At...uhm, probably you better take a look at the std::string documentation. ;)
Probably it's better not to repeat the design mistake of the standard libary and write 103 member functions. Especially the 24 versions of find() should be given a thought, I don't think a  find_last_not_of() is necessary. Be aware that a lot of functionality can be achieved by using STL algorithms on the iterator range.

Here is an interesting article concerning std::string's design on Guru of the Week (GotW).
Zloxx II: action platformer
Thor Library: particle systems, animations, dot products, ...
SFML Game Development:

 

anything