Better keyboard handling

[Laurent]

Hi

I'm currently working on improving the keyboard support, and after having a look at how other libraries handle it, I think I have what I need to finally provide good support.

However a few design choices have to be made, so I'd like to collect as many ideas/feedback/use cases as possible to make the right decisions.

There are three problems so far:

1) Many keys have 2 or 3 symbols on them. Which one should be reported when the key is pressed? The obvious answer would be "the main one", i.e. the one that is produced when no modifier key is pressed. However the default symbol for a key may not be the most "standard" one; for example, on my french keyboard, pressing '1' yields a '&', and to get a '1' I have to press shift. SDL has a special handling for these keys, so that the numbers are always reported, even if they are not the default symbol. Same for 'ù' and '%', etc.

2) Should modifier keys (shift, alt, control) be interpreted or not? In other words, does the reported symbol changes when we press a modifier key, or should there be only one symbol reported for each key?

3) Should the scancode be reported too? Reminder: the scancode is a hardware-dependent code that represents the physical location of the key on the keyboard. According to SDL sources, although scancodes are hardware dependent, there seems to be standards. The scancode is typically useful when a program provides the ability to choose key bindings; in this case you don't care which key is pressed, you just want to save its code and be able to test it later. Scancodes are better than key codes here because there is always a unique code for each key (within a keyboard), whereas all key that are not part of the sf::Keyboard::Key enum will all have the same code, sf::Keyboard::Unknown.

What other libraries do:

- SDL is pretty accurate, only 2 or 3 keys are not reported correctly on my keyboard. Modifier keys are not interpreted. It provides scancodes.

- GLFW is bad, on Windows it simply returns what the OS gives, which is wrong for almost every non-trivial key on my keyboard. Modifier keys are not interpreted as far as I remember. It provides scancodes.

- Qt is the best, it always reports the right symbol (even ù, é, ç, £, §, ...), its key codes enum is impressive. Modifier keys are interpreted. It provides both scancodes and the native OS key code.

I haven't tested Allegro, which is usually in my reference libraries too.

[eXpl0it3r]

I'm glad you've found the time to look into this!

Well I guess my conclusion would be: Use Qt's code as much as possible.

Which translates into:

• Interprete the keys as if one would be writing something in a text editor, it's what people essentially expect (unless every game does it the other way around). Although your number keys thing on the French keyboard seems a bit odd... Could you elaborate a bit more on the issue?
• Interpret modifier keys.
• And scan codes seem to be kind of a must, since if everything else breaks at the programmers could at least just do everything with scan codes, rather than having to move on from SFML.

[Veltas]

EDIT 2:
Looking at the current KeyboardEvent, I'd keep that the way it is and then add a new event CharacterInputEvent (or something similarly named) that informs of a new character being inputted.

I say separate event because we think about the keyboard keys pressed and the characters inputted separately, and multiple keys can be combined to form one character, for example.

There are troubles, though. International input is quite complicated and sometimes it's not possible to "just input a character", in far eastern locales for example.

I imagine it's worth aiming for whatever a generic text box would receive, if possible.

If you think Qt's key codes enum is impressive perhaps expand sf::Keyboard::Key?

EDIT3:
What does the TextEvent do? Is that what I'm talking about? If so I'm not sure what I'd change other than expanding sf::Keyboard::Key.

EDIT 4:
Perhaps there should be a way SFML provides to get a string that represents the key as well? This would make it easier to refer to keys with localization. On my keyboard the '6' key is a 6, and ^ when combined with a shift, so should be called "6" by this functionality because as a user I would recognize it as such. This would make it easier to tell the user to press a key, etc.

[zsbzsb]

In my humble opinion,

1) Report the interpreted symbol, if no modifier key is pressed then this is "the main one". Or if a modifier key is pressed SFML should report the symbol that the modifier key maps to. On my keyboard pressing Shift+1 should be reported as '!'. As far as I know the physical keys (with different symbols that can be reported depending on the modifier key) vary from keyboard to keyboard. So reporting all possible symbols doesn't seem feasible.

2) As in #1, the symbol according to the modifier key should be returned.

3) Returning scan codes would be useful in certain cases, in the end it probably wouldn't matter in most cases if the scan codes are reported or not. I would say return them just because some people would find them useful and at the most it would add one more member to the key event structure.

[Veltas]

3) Returning scan codes would be useful in certain cases, in the end it probably wouldn't matter if the scan codes are reported or not. I would say return them just because some people would find them useful and at the most it would add one more member to the key event structure.

I disagree, it would matter if the scan codes were not reported.

When writing specialised controls you'll want to know which keys on the keyboard are being pressed, not the character inputted.

EDIT: And also you could not return them together, as multiple scan codes will correspond to certain characters (i.e. multiple keys pressed to get one input character).

EDIT2: Looking at the current KeyboardEvent, I'd keep that the way it is and then add a new event CharacterInputEvent (or something similarly named) that informs of a new character being inputted. There are troubles, though. International input is quite complicated and sometimes it's not possible to "just input a character", in far eastern locales for example.

Disregard this, look at my original post.

[wintertime]

For a realtime game one would typically want to know which key is pressed with a bitfield of modifier keys (or sometimes may want modifier keys reported on their own). If suddenly a weird symbol gets returned when a player uses a modifier key at same time it would only be an annoyance.
Even the numpad keys often are tried to make always work with and without num-lock by testing both the Num key and the other functions, but they are mostly standardised. If one player presses shift+3 and gets § send to the game and another ? thats no fun. Always annoying is if games read the < and > keys in the wrong way (as text events) and then on my keyboard those are at a single key near left shift instead of two keys at 3 and 2 positions left of the right shift and you then need to use shift always. Using the key left of backspace is also pretty annoying when on some keyboards its a dead key with ´ and not + or =.

For a more relaxed game one would be also interested in the keys, but would want to always get a Q, A, Z or Y when the player presses whatever key is labeled like that, for example to mark menu shortcuts. There I think the modifiers would be needed separately in a bitfield, too, as the game could be testing for ctrl+alt+Q or alt-gr+Q and not for @.

For typing text into a edit field or text editor you would still want always the right character, not key, from a separate text input event. It would be annoying to manually put together dead keys like ^ or ´ and a following key, especially because there are so many different layouts.

[Oldie]

• Interprete the keys as if one would be writing something in a text editor, it's what people essentially expect (unless every game does it the other way around). Although your number keys thing on the French keyboard seems a bit odd... Could you elaborate a bit more on the issue?
• Interpret modifier keys.
• And scan codes seem to be kind of a must, since if everything else breaks at the programmers could at least just do everything with scan codes, rather than having to move on from SFML.

Essentially agree with that.
In my current work in progress, players can enter custom player names. So far I have been able to type in special characters, using modifier keys or not, like: &, #, ü, € or ç. So the first two points are needed.
Scan codes sound like an interesting add, which could enable programmers to bind to any type of keyboard if they wished. This sounds like the Linux kernel providing numerous drivers for graphics cards.

If everything mentioned was to be implemented, would this mean that key events be bound to scan codes and not to sf::Keyboard::Key anymore? Or to both, like in Qt?

Also, exploiter, are you just asking how digits are displayed on a French keyboard? Well, instead of being the main characters on the 10-key top row, they are the Shift characters at the same places. So, you have to press Shift + & to get 1, Shift + é to get 2, etc. The joy of Azerty keyboards!

[eXpl0it3r]

Also, exploiter, are you just asking how digits are displayed on a French keyboard? Well, instead of being the main characters on the 10-key top row, they are the Shift characters at the same places. So, you have to press Shift + & to get 1, Shift + é to get 2, etc. The joy of Azerty keyboards!

That sounds horrible!

Thinking about it and discussing it on IRC, I wonder whether it would be enough to provide:

• Scan codes (physical location)
• Fully interpreted keys, similar to what the Text Event does.
• No printable keys

That way one has the full abstraction of the keyboard (scan codes) and their localized  input, while being able to find what other keys are pressed.

[Laurent]

Important
We don't care at all about text input here. It's the job of the TextEntered event, which already works perfectly. So let's not talk about text boxes or whatever, providing the typed character is not the goal of this modification. We are only talking about finding the most useful way to identify keys for key events.

Modifier keys are already provided separately in key events, so there's no need to discuss them. What must be decided is whether the returned key code is the default one, or the combined one. And I repeat that we don't care about text input

I think we all agree about the usefulness of scancodes.

About providing interpreted keys or "standard" ones, it's complicated. What if someone maps actions or inventory or weapon selection to number keys? On a french keyboard, these keys would be unreachable if we just report the default symbols.

Fully interpreted keys, similar to what the Text Event does.

Why should we provide something similar to the TextEntered event?

[Nexus]

Maybe we should not only look how other libraries, but also how games handle keys. If common libraries provide a certain way, but games work around it for the sake of usability, then we can directly look at the latter.

A good example is squad/group assignment in strategy games. Usually, the keys 0-9 are used to refer to a squad, and Ctrl+0-9 to assign it. How does this work on French keyboards, do you have to additionally press Shift?

Laurent also mentioned inventory or weapons, can we gather some ways how games handle them on different keyboards?

[wintertime]

The strategy game example shows you really want to know the non-modified key(+modifiers the game can interpret itself). The label the localized key would correspond to with current modiers is useless in that case.
Especially when you consider shift+number is often also mapped in strategy games to "add this group to current selection" which could collide with "select only this group".
Some even map alt+number, shift+alt+number, ctrl+alt+number and/or shift+ctrl+alt+number.

Do there exist even more contrived mappings where the numbers are mapped to some completely other keys? Then some games would want those, but others would want the row where they are commonly found.

[Laurent]

The strategy game example shows you really want to know the non-modified key

So how do you handle french keyboards, where the non-modified key is not the number?

[wintertime]

I think if a strategy game maps unmodified key to a hotkey it always wants the same id the number key on non-french keyboards would give. Then you could still use that hotkey only and the game switches to only select the group and if you press shift it would add to selection. That avoids having the mappings collide.

[eXpl0it3r]

Why should we provide something similar to the TextEntered event?

My point kind of was, that it's more or less useless/"ignorant" to actually define keys like we do now. Why do we assume that there exists a key called "A" or "Z" or ...? How do you define the key A on the following layout?

So since it's kind of useless trying to define keys after a certain or a few certain layouts, my idea of using scan codes to define the key, returning the actual "character" that gets generated when pressing the key and defining codes for special keys (break, scroll lock, etc), could work with any keyboard layout.
Than again as you said, scancodes are not very consistent either...

It's just an idea and I might be over looking the most obvious fact why one does not do it that way, but it just seems odd to me to define keys based on the latin letters only. Sure I guess the most common layouts are some variants of the American layout, but should one just ignore all the other layouts?

[Laurent]

I think if a strategy game maps unmodified key to a hotkey it always wants the same id the number key on non-french keyboards would give. Then you could still use that hotkey only and the game switches to only select the group and if you press shift it would add to selection.

I'm reading this again and again, but, although it looks ok, I fail to understand it