A proposal of how to improve keyboard event handling

[SoleSoul]

Hi.
I am working on a project which uses SFML and I use the keyboard as kind of piano keys. There is a problem with keyboard event handling. For example, when you press shift + number  the produced event carries the key code 0 instead of the proper one.
Here is a bug report for reference: https://github.com/SFML/SFML/issues/187

Interestingly, this behavior in SFML is inconsistent. Here are two ways of checking the key code while pressing Shift+Num1, one works and one doesn't. I work on Linux.
In the first example the key code is '0' therefore nothing would be printed:

Code: [Select]

while(window.isOpen)
{
    window.waitEvent(event);

    if(event.type == sf::Event::KeyPressed)
    {
        if(event.key.code == sf::Keyboard::Num1)
            cout<<"1 is pressed"<<endl;
    }
}

In the second example the check correctly sees that Num1 is pressed therefore "1 is pressed" would be printed:

Code: [Select]

while(window.isOpen)
{
    window.waitEvent(event);

    if(event.type == sf::Event::KeyPressed)
    {
        if(sf::Keyboard::isKeyPressed(sf::Keyboard::Num1))
            cout<<"1 is pressed"<<endl;
    }
}

The reason for the inconsistency lies in the implementation.
The first example uses the code from the file InputImpl.cpp line 63. The function gets an SFML key value, converts it to an X11 keysym (layout dependent) and then to an X11 keycode. Then the keycode is checked with XQueryKeymap. This is good because the keysym is used only internally to get the keycode (layout independent).

The second example uses the code from the file WindowImplX11 line 660. The keysym - which is layout dependent and changed by pressing shift which is undesired - is being requested from X11 with the function (line 666)

Code: [Select]

XLookupString(&windowEvent.xkey, buffer, sizeof(buffer), &symbol, &keyboard); and then converted to an SFML key code with

Code: [Select]

event.key.code    = keysymToSF(symbol);The problem with this approach is that the keysym we get from X11 is layout and shift dependent. This is good if we want the inserted character, but if we want the key, we have to request the X11 keycode which is available from XKeyEvent.keycode.

Since the problem seems to linger from the beginning of time (2008?) I assume you tried to fix it but there were problems. May I ask if you could share your opinion on the subject and the proposed change?

Thank you for the great library by the way.

Edit: Here is the output of xev (X events) when pressing Num1 and then Shift+Num1. I copied only the relevant parts:

Code: [Select]

KeyPress event, serial 42, synthetic NO, window 0x2600001,
    root 0xb6, subw 0x0, time 19059638, (309,-46), root:(1119,686),
    state 0x0, keycode 10 (keysym 0x31, 1), same_screen YES,
    XLookupString gives 1 bytes: (31) "1"
    XmbLookupString gives 1 bytes: (31) "1"
    XFilterEvent returns: False

KeyRelease event, serial 45, synthetic NO, window 0x2600001,
    root 0xb6, subw 0x0, time 19059710, (309,-46), root:(1119,686),
    state 0x0, keycode 10 (keysym 0x31, 1), same_screen YES,
    XLookupString gives 1 bytes: (31) "1"
    XFilterEvent returns: False

KeyPress event, serial 45, synthetic NO, window 0x2600001,
    root 0xb6, subw 0x0, time 19062213, (309,-46), root:(1119,686),
    state 0x0, keycode 50 (keysym 0xffe1, Shift_L), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

KeyPress event, serial 45, synthetic NO, window 0x2600001,
    root 0xb6, subw 0x0, time 19063011, (309,-46), root:(1119,686),
    state 0x1, keycode 10 (keysym 0x21, exclam), same_screen YES,
    XLookupString gives 1 bytes: (21) "!"
    XmbLookupString gives 1 bytes: (21) "!"
    XFilterEvent returns: False

KeyRelease event, serial 45, synthetic NO, window 0x2600001,
    root 0xb6, subw 0x0, time 19063083, (309,-46), root:(1119,686),
    state 0x1, keycode 10 (keysym 0x21, exclam), same_screen YES,
    XLookupString gives 1 bytes: (21) "!"
    XFilterEvent returns: False

KeyRelease event, serial 45, synthetic NO, window 0x2600001,
    root 0xb6, subw 0x0, time 19063583, (309,-46), root:(1119,686),
    state 0x1, keycode 50 (keysym 0xffe1, Shift_L), same_screen YES,
    XLookupString gives 0 bytes:
    XFilterEvent returns: False
The order is: 1 press, 1 release, shift press, 1 press, 1 release, shift release.
You can see that when pressing 1 with or without shift the keycode doesn't change while the keysym does change.

[Laurent]

Since the problem seems to linger from the beginning of time (2008?) I assume you tried to fix it but there were problems. May I ask if you could share your opinion on the subject and the proposed change?

I haven't tried to fix this problem yet, it's something that I kept for after SFML 2.0.

The main issue is not how to implement the solution, it's to decide which one is the best. It's not clear to me how keys should be handled, especially what to return when modifier keys are used. I have to investigate, probably have a look at other libraries, before deciding what to do.

[SoleSoul]

Let's have an open discussion with the users of SFML. It can cause no harm, I think
Here is my opinion, I hope it would be somewhat helpful.

There are two scenarios when one needs to know which key has been pressed. These are the two distinct uses of the keyboard, as a writer, and as a controller.

When using the keyboard as a writer, what the developer really wants is the character which the user expects to produce with the key press. For this purpose a library should not interfere with the standard OS way of dealing with layouts and modifier keys because a library cannot possibly deal with all the possibilities. I didn't use this functionality of SFML but as I've read the implementation is good and works as expected.

When using the keyboard as a controller, a use which is very common in games programming, the developer usually does not care about the character which would have normally been produced by the key press. The greatest concern in this scenario is to catch key presses in a unified way regardless of the current keyboard layout or language and even modifier keys.

For the two distinctive uses there exist two values (for this example, in X11) to represent a key, keysym and keycode. The former is the character which is expected by the user to be printed following the press, layout, modifier keys, languages, all are dealt with by the OS. The latter is the physical key pressed regardless of any custom setup which is exactly what we need for key events.

An opinion of someone who knows other library well is important but this way looks good for me from a developer point of view.
The current way SFML handles things is actually quite close to ideal. Character input works fine by using keysym (am I wrong?), checking of current state of a key works by using keycode (isKeyPressed), the only part which needs some modifications is the key code in the event so it would use keycode instead of keysym, that's all.

What do you think?
Thaks.

[Ceylo]

I like your idea. This way everything is well defined and the whole information is always available, at almost no cost for the user (but maybe something more explicit than key/keysym would be welcome).

[Hiura]

(From memory) SDL does something like that. i.e. it provides "scancode" (aka keysym) and the actual character.

Now there might be a pitfall (this need to be verified though) : a given scancode on a keyboard A and bound to some key position (e.g. the third key from the left on the second row) could not be bound to the same key position on a keyboard B.

It depends on the layouts. Maybe it should also be provided by SFML ?

[SoleSoul]

Just to clarify things, in X11, the keysym is the actual character (language dependent) so I guess the scancode is equivalent to keycode? and when you say layout, I guess you mean physical layout, rather than logical layouts (input languages) which can be cycled through with alt+shift.

[Hiura]

Yep, I'm speaking about physical layout.

Now you said it, I'm not sure if SDL's scancode are bound to physical key or "logical" key. This should also be verified.

[Laurent]

Now you said it, I'm not sure if SDL's scancode are bound to physical key or "logical" key. This should also be verified.

Scan codes represent physical keys. That's the reason why I'm wondering how these code can be used (ie. how do you refer to them?).

[SoleSoul]

Scan codes represent physical keys. That's the reason why I'm wondering how these code can be used (ie. how do you refer to them?).

I think I don't understand the question, or problem. Can you please clarify a bit more?
As said above, we have 3 ways of getting key presses, two of them need the physical key and one needs the logical key.

When using the keyboard as a controller we would be using two functions, IsKeyPressed and event.key.code, to check the physical state of the key. For example, in your game Shift is used for "jump" and '1' is used for "change weapon". You want "change weapon" to work regardless of whether you are in a middle of a jump or not therefore you are interested in the state of the physical key. Currently IsKeyPressed is good and works like that by using the keycode (scancode). event.key.code however, is being initialized with the logical key and this is the reason many keys produce 0 instead of the actual code.

When using the keyboard as a writer we would be using the event TextEntered to get the character the user is trying to enter taking into account everything, shift state, and logical layouts therefore we need the logical key, the keysym. As I said before, I did not check SFML's code of this function but as people said, it works, right?

[Laurent]

I think I don't understand the question, or problem. Can you please clarify a bit more?

I mean, a scancode is a raw number, which may not relate to the symbol of the corresponding key in your preferred layout. So how do you manipulate scancodes? How do you know where scancode number 20 will be located on your keyboard?

After looking at this page from the SDL 2.0 wiki, I can see that they have names for scancodes, based on the USB standard. So it kind of answers my question. But you still don't know where the key is located; is SDL_SCANCODE_A the 'A' from a QWERTY keyboard, or an AZERTY one?

In fact I have no idea what the typical use case for scancodes is.

[SoleSoul]

Unfortunately I never saw a keyboard other than qwerty so I can't comment about the scancodes which are produced by such keyboards. I think that what I said is still true about qwerty keyboards so it is worth considering. Maybe someone here who has a different keyboard can enlighten us about how he plays games which use [wasd] as direction keys or how he creates such a game in SFML.

About the use case, as I said in the previous post in the example with the "jump" and "change weapon",  the majority of uses of a keyboard in games is as a controller where you don't want your keyboard language or modifier keys to affect the way the game works. For all these cases a scan code is the way to go in my opinion.
Actually, the need of the keysym (character, language dependent) is less common in my experience.

Don't you see that as a use case? why?

[Laurent]

Maybe someone here who has a different keyboard can enlighten us about how he plays games which use [wasd] as direction keys or how he creates such a game in SFML.

As an AZERTY user, I can tell you: I've never played a game where the physical keys were used, it was always WASD -- absolutely unusable on AZERTY.

About the use case, as I said in the previous post in the example with the "jump" and "change weapon",  the majority of uses of a keyboard in games is as a controller where you don't want your keyboard language or modifier keys to affect the way the game works. For all these cases a scan code is the way to go in my opinion.

I understand and it is indeed a good use case. My problem is a different one: mapping raw scan codes to actual keys (I'm not talking about modifier keys, just the "0x20" -> "A" mapping, for example).

I've always thought that scancodes where the physical location of the key on the keyboard, like "row 1, column 2", which would result in a different scancode for the same key on different keyboard layouts. But I'm starting to think that I'm wrong, and that scancodes are bound to particular keys, ie. "A", "F1", "Space", etc. which would result in the same scancode for a key whether it is on a QWERTY keyboard or an AZERTY one.

I probably need to read more documentation on the subject before talking

[SoleSoul]

We are getting closer.

As an AZERTY user, I can tell you: I've never played a game where the physical keys were used, it was always WASD -- absolutely unusable on AZERTY.

I think it is safe enough to assume they use scan codes so language and modifier keys won't interfere, but this doesn't help AZERTY users because the physical keys are located elsewhere. If a developer wants to support different types of physical layouts he needs to allow the users to change the key mapping or to choose a layout.

I understand and it is indeed a good use case. My problem is a different one: mapping raw scan codes to actual keys (I'm not talking about modifier keys, just the "0x20" -> "A" mapping, for example).

If I understood you correctly I think it is pretty simple. You get the key code, in case of 'a' it is 38, and feed it to XKeycodeToKeysym in order to get the associated standard character which is case of 'a' is 97.
Here is a working converter

Code: [Select]

#include <X11/Xlib.h>
#include <iostream>
using namespace std;

int main()
{
    Display * display = XOpenDisplay(NULL);
    int keycode;
    while(true)
    {
        cout<<"Keycode: ";
        cin>>keycode;
        cout<<"Keysym: "<<XKeycodeToKeysym(display, keycode, 0)<<endl;
    }
}
which can be compiled on Linux with

Code: [Select]

g++ `pkg-config --cflags --libs x11` convert.cpp -o convertThe useful fact about this method is that XKeycodeToKeysym produces 97 regardless of the current layout or modifier keys.
This specific function is deprecated for some reason in favor of XkbKeycodeToKeysym but you get the idea.
Does this look ok?

[Laurent]

I'm not concerned about technical details yet (finding the function to call is straight-forward), I just want to clarify how it works and how it's supposed to be used.

The relevant part of my previous message was this one:

I've always thought that scancodes where the physical location of the key on the keyboard, like "row 1, column 2", which would result in a different scancode for the same key on different keyboard layouts. But I'm starting to think that I'm wrong, and that scancodes are bound to particular keys, ie. "A", "F1", "Space", etc. which would result in the same scancode for a key whether it is on a QWERTY keyboard or an AZERTY one.

In short: what is a scan code?

I'd like to get a clear answer to this question before talking about XKeycodeToKeysym

[SoleSoul]

The relevant part of my previous message was this one:

I've always thought that scancodes where the physical location of the key on the keyboard, like "row 1, column 2", which would result in a different scancode for the same key on different keyboard layouts. But I'm starting to think that I'm wrong, and that scancodes are bound to particular keys, ie. "A", "F1", "Space", etc. which would result in the same scancode for a key whether it is on a QWERTY keyboard or an AZERTY one.

In short: what is a scan code?

I tried to answer this question in the first part of my message by saying that scan codes are associated with specific keys rather than with specific locations. It seems like I wasn't clear enough.
Anyway, as I wrote, it is indeed just what I could deduce from the facts we have. If you want to perform a scientific test it can be easily done
On my box, with the QWERTY keyboard, the scan codes which I assume are the same as "keycodes" in X11 are as follows:
The keys

Code: [Select]

1234567890-=
backspace
tab
qwertyuiop[]produce the codes 10-35 (1 is 10, 2 is 11, ... , [ is 34, ] is 35).
Now, if your keyboard produces the same codes we know that codes are associated with keys. If they differ, we would have to find the logic behind it.
What do you say?

I used "xev" to get the keycodes from X.