KeyEvent with IME

[Passer By]

If you tried to read sf::Event::KeyEvent::code for sf::Event::EventType::KeyPressed on Windows with IME (input method editor), you get sf::Keyboard::Key::Unknown.

As far as I know, this is useless information, and if its possible, I'd think it should instead be the physical key on the keyboard that is being pressed. That said, I have no idea how all the OS handles IME, so maybe its impossible?

At least reading messages directly works on Windows

#include<Windows.h>
#include<iostream>

LRESULT CALLBACK echoer(int nCode, WPARAM wParam, LPARAM lParam)
{
    switch (wParam)
    {
    case WM_KEYDOWN:
        [[fallthrough]];
    case WM_SYSKEYDOWN:
        unsigned char c = ((PKBDLLHOOKSTRUCT)lParam)->vkCode;
        std::cerr << c << '\n';  // correctly reads even with IME
        break;
    }
    return CallNextHookEx(nullptr, nCode, wParam, lParam);
}

int main()
{
    auto hook = SetWindowsHookEx(WH_KEYBOARD_LL, echoer, nullptr, 0);

    MSG msg;
    while (!GetMessage(&msg, nullptr, 0, 0))
    {
        TranslateMessage(&msg);
        DispatchMessage(&msg);
    }

    UnhookWindowsHookEx(hook);
}
 

[eXpl0it3r]

For character output there's the TextEntered event.

We can't map out the who-knows-how-many possible "keys" when using IME.

[FRex]

TextEntered correctly reports the inputted unicode.
The field you try to access is a DWORD, not a char.
I've added that code and it causes me to segfault when pressing alt, win key, etc. I have no idea why.

[Passer By]

For character output there's the TextEntered event.

We can't map out the who-knows-how-many possible "keys" when using IME.

I realize there is TextEntered for reading the character inputted.

I'm asking if its possible to read the physical key pressed when using IME, which is useful when it is used not as a text input, say for a tetris game where Keyboard::isKeyPressed is unsuitable. Otherwise, it'll suffer from the annoying trait of most old games: you'd have to manually switch off IME before playing. If you don't know, its really really annoying.

Granted, I don't know much about how OS handles IME, but apparently they do expose such information on some level.

TextEntered correctly reports the inputted unicode.
The field you try to access is a DWORD, not a char.
I've added that code and it causes me to segfault when pressing alt, win key, etc. I have no idea why.

That was my bad, switch to unsigned char, vkCode is supposed to only be in the range [1, 254]. I'm not sure if there is other reasons for it to segfault.

[eXpl0it3r]

Some when you use IME, your physical keys on your keyboard that SFML has an entry in the sf::Keyboard::Key enum don't work anymore?
Oe is it random virtual key presses that SFML can't possibly map out, that you want to trigger?

[Passer By]

Some when you use IME, your physical keys on your keyboard that SFML has an entry in the sf::Keyboard::Key enum don't work anymore?
Oe is it random virtual key presses that SFML can't possibly map out, that you want to trigger?

The physical keys stop working, not the virtual keys.

A concrete example:

sf::Window window;
// ...
for(sf::Event event; window.PollEvent(event); )
{
    if(event.type == sf::Event::KeyPressed)
        std::cerr << event.key.code;  // prints -1 for Unkown with IME, works without IME

    if(event.type == sf::Event::KeyReleased)
        std::cerr << event.key.code;  // always works, prints physical key released
}
 

I'm asking if sf::Event::KeyPressed can possibly play nice with IME

[FRex]

I'm using a Japanese IME myself but not play with it on. You can press win + space to go between your input methods so you can just switch to English. Some IME (Japanese) also have more then one mode themselves, including a mode where you type just latin letters and that works no problem too, that's also toggled with a shortcut. I don't see whats annoying about pressing one shortcut before launching the game. If you really want you can use auto hot key to make it easier or change the shortcuts.

You shouldn't be playing games with IME enabled anyway, it makes no sense, Minecraft and few other games I tested similarly don't react to any key presses (and that's good or I'd be doing random things when I'm just writing using IME and don't want to do anything) but read the final unicode text correctly.

With what you propose we'd be duplicating events, the game might get a bunch of nonsense presses (n, e, k, o, space, enter) but it was actually the user using IME to type out something (猫) in it. In Japanese especially you press space, enter a lot and the four arrows occasionally when writing stuff via IME. I can't imagine a game reading these all presses as real input, it'd be a mess.

There's also this pop up over the game graphics, even in fullscreen, when an IME like Japanese is active and you're typing into it, can't play with that on the screen all the time and filling with garbage either:

[Passer By]

I'm using a Japanese IME myself but not play with it on. You can press win + space to go between your input methods so you can just switch to English. Some IME (Japanese) also have more then one mode themselves, including a mode where you type just latin letters and that works no problem too, that's also toggled with a shortcut. I don't see whats annoying about pressing one shortcut before launching the game. If you really want you can use auto hot key to make it easier or change the shortcuts.

You shouldn't be playing games with IME enabled anyway, it makes no sense, Minecraft and few other games I tested similarly don't react to any key presses (and that's good or I'd be doing random things when I'm just writing using IME and don't want to do anything) but read the final unicode text correctly.

With what you propose we'd be duplicating events, the game might get a bunch of nonsense presses (n, e, k, o, space, enter) but it was actually the user using IME to type out something (猫) in it. In Japanese especially you press space, enter a lot and the four arrows occasionally when writing stuff via IME. I can't imagine a game reading these all presses as real input, it'd be a mess.

There's also this pop up over the game graphics, even in fullscreen, when an IME like Japanese is active and you're typing into it, can't play with that on the screen all the time and filling with garbage either:

I disagree about the toggling being a minor annoyance. Consider when you have a chatting capability, you start to have to switch back and forth between the input methods. If your game have some element of reaction in it, you find yourself unable to chat at all.

If you are still unconvinced about the toggling, think about this: why make users deal with our problem?

As for the semantics, switching to reading the physical keys will not make any difference*. The thing is, you already have duplicate events now, a KeyPressed is generated on key press regardless, why not make it useful?

If you cared about the actual text inputted, you ignore KeyPressed as usual and read TextEntered. If you cared about the key being pressed, you ignore TextEntered and read either Event::KeyPressed or Keyboard::isKeyPressed, which by the way currently ignores IME.

Responding to some games not working with IME, I call that a bug, and I hate that. Unless there is ambiguity about what the user intends, which in the case of Minecraft, there isn't.

* The only way to break with the switch is if someone used KeyPressed for text input and only switch to TextEntered if the key is Unknown, but that sounds kinda iffy.

PS While writing this, I realized being able to disable IME from sfml also fixes the issue.

PPS I like the "cat" as your word of choice

[FRex]

If you really want you can bind the same (or some other) key as the chat key via AHK to switch to your IME and then bind some cancel key to going back to English.

In your 'way' there still would be a pop up all the time from Windows itself, over the game, in some IMEs that will contain random garbage and hang in the top left corner if the game handled it like you want. Minecraft is a globally renowned huge game and localized into many languages, including CJK ones, if it doesn't bother and it's seemingly okay (in multiplayer you sometimes need to switch between chat and game quickly) then why should SFML invent solutions for a problem that doesn't exist?

This is a very unreasonable request IMO, are there even games that do that exactly as you want, don't break on CJK, don't show the pop up when it's not needed, etc?

[Passer By]

If you really want you can bind the same (or some other) key as the chat key via AHK to switch to your IME and then bind some cancel key to going back to English.

In your 'way' there still would be a pop up all the time from Windows itself, over the game, in some IMEs that will contain random garbage and hang in the top left corner if the game handled it like you want. Minecraft is a globally renowned huge game and localized into many languages, including CJK ones, if it doesn't bother and it's seemingly okay (in multiplayer you sometimes need to switch between chat and game quickly) then why should SFML invent solutions for a problem that doesn't exist?

This is a very unreasonable request IMO, are there even games that do that exactly as you want, don't break on CJK, don't show the pop up when it's not needed, etc?

Yes there is a lot. Doing it nicely. Which is why I asked.

Drawing from personal experience:

• League of Legends
• Osu!
• Overwatch
• Path of Exile

They all don't have white boxes on the corner of your screen, accepts proper input given the context (chat vs play) and some is cross-platform. Although, I've come to think the game might just disable IME when it needs to, I'm not sure. Binding AHK to switch IME is too much of a hack and shouldn't be the canonical solution I don't think, for one thing, the user may switch it manually and break their input.

Vanilla Minecraft (at some point in the past) even fails to take Unicode input in chat, even though it's written in Java with UTF-16 strings as default. You might not want to hold it as the golden standard when it comes to this.

Please stop treating this as an unfortunate circumstance and treat it as what it is, a bug. It might not be the end of the world, but it shouldn't be considered correct.

[FRex]

Osu! disables the IME which you could confirm easily if you played it in a window because your IME icon is replaced by a grey X icon in the system tray (if you'd enable showing the IME/language icon there) with the tooltip 'IME Disabled'.

Disabling IME is easy in WinAPI and even possible in SFML on Windows already using the Window handle:
https://support.microsoft.com/en-us/help/171154/how-to-disable-ime-for-a-particular-window
https://gist.github.com/FRex/7a2b6e2bfc845050d59868c771cec03f

Your proposal for always filling KeyPresses even when IME is on is still absolutely wrong and unreasonable and would not be suitable for CJK-ish IMEs that have the white pop up.

[FRex]

Anyone else has opinions on this for Windows? Laurent?

On Linux I honestly have no idea what's going on other than it being a bit tedious to set IME up to always work. Trying to read up on it doesn't help: https://unix.stackexchange.com/a/262220

[Laurent]

Anyone else has opinions on this for Windows? Laurent?

I never use IME so I really have no opinion on the subject.