Improve keyboard input with Input<ScanCode> #2052

TheRawMeatball · 2021-04-29T18:13:29Z

What problem does this solve or what need does it fill?

Right now, getting the status of raw keys is difficult. An Input<ScanCode> would make this simpler.

What solution would you like?

Add a Input<ScanCode> resource by default in bevy_input, and change the ScanCode type to be an enum where the keys are represented as the key they correspond to on a standard QWERTY keyboard.

What alternative(s) have you considered?

Do nothing, or do this partially.

Additional context

Started discussion in discord

The text was updated successfully, but these errors were encountered:

inodentry · 2021-04-29T18:28:53Z

Key Codes are layout-dependent. Scan Codes are layout-independent. I think we should improve our keyboard APIs to encourage good practices:

Scan Codes should be used for game input.
Key Codes should be used for text input (and maybe display).

So I would suggest the following:

Create a new ScanCode enum, that is similar to the current KeyCode enum.
- It should contain enum variants for every key commonly found on typical keyboards.
- Alphanumeric keys could be named based on QWERTY (as that is what most people are familiar with).
- It should be trivially convertable to/from an integer.
- An Other variant could be included for extra keys on weird keyboards, when not covered by any enum variant.
Change the KeyCode enum to be better suited for text input and more indicative of its layout-centric nature.
- Keep the variants for for non-alphanumeric keys like Esc, Delete, Return, ...
- Replace all alphanumeric keys with a single Char(char) variant, containing the unicode character value, as per the keyboard layout of the user.
  - This should make it trivial to implement text input.
  - It would make it easy for users to support non-latin scripts for text input.
  - It would also encourage people to use ScanCode for game inputs, instead of KeyCode.
Both Input<KeyCode> and Input<ScanCode> resources should be available.
Replace the scan_code field in the KeyboardInput event type with a ScanCode (it is currently a bare u32).

These changes should result in a much cleaner keyboard input API that encourages good practices and is easy to use correctly.

cart · 2021-04-29T20:04:38Z

To me this isn't necessarily clear cut. Game input generally cares about the "shape" of the keyboard (ex: WASD movement or the position of the arrow keys). ScanCode is closer to the ground truth of the "shape". But it also isnt the ground truth because we live in a world with different physical keyboard layouts.

By adopting ScanCode as our standard for game inputs, we are essentially saying that non-qwerty-scancode-aligned keyboard owners need to remap all of the defaults a game provides (or have the game provide alternative mappings for each layout they want to support). This feels pretty english-centric to me.

Using ScanCode for input will likely result in better outcomes, but it isn't a complete solution:

A keyboard input model rust-windowing/winit#753
https://hacks.mozilla.org/2017/03/internationalize-your-keyboard-controls/
- The "reference keyboard" / KeyboardEvent.code is the W3C solution to this problem. It forces everyone to code to a QWERTY layout, but at least its consistent.

Semi-related Bevy conversation: #782

mrec · 2021-04-29T21:37:11Z

As part of this, would Bevy need a portable mechanism to translate a ScanCode to a KeyCode for the user's keyboard and locale? Otherwise I don't know how a game's instructions are supposed to tell the user what keys to press.

EDIT: hmm, this is the same use case as #1995 which is now closed, but as far as I can see #805 only added a ReceivedCharacter event for text input. Maybe I'm missing something, but I don't see how this helps describe a key which the user isn't pressing.

inodentry · 2021-04-29T22:17:01Z

By adopting ScanCode as our standard for game inputs, we are essentially saying that non-qwerty-scancode-aligned keyboard owners need to remap all of the defaults a game provides (or have the game provide alternative mappings for each layout they want to support). This feels pretty english-centric to me.

Hmm ... what? This seems backwards to me? What is a "non-qwerty-scancode-aligned keyboard"? I thought the scan codes represent the physical keys and are the same on every keyboard?

If you are referring to my suggestion about naming the enum variants after QWERTY, I can see the point, but those are just internal enum values. I suggest using the matching KeyCode for text display to users / as part of UI.

Game inputs should use scan codes, precisely so that they are not "english-centric". That way they will work regardless of whether i switch my keyboard to QWERTY, Dvorak, or Russian. The current KeyCode-based API is english-centric, because it makes it inconvenient to use non-qwerty (or worse, non-latin) layouts.

But it also isnt the ground truth because we live in a world with different physical keyboard layouts.

Of course some people have special nonstandard keyboards (like the various ergonomic keyboards with weird physical arrangement of keys), but those people are going to have to customize/remap their game inputs anyway.

cart · 2021-04-30T00:47:20Z

Hmm ... what? This seems backwards to me? What is a "non-qwerty-scancode-aligned keyboard"? I thought the scan codes represent the physical keys and are the same on every keyboard?

This is what I was contesting. I understand that scan code is the raw event sent from the keyboard and doesn't directly correlate to a letter (this is the OS's job, which is what produces KeyCode and is virtual-keyboard-layout dependent). I have only ever used "qwerty" keyboards, but I am currently under the impression that international keyboards exist (such as this azerty keyboard) with scan codes that don't line up with the physical positions of qwerty keyboard scancodes. This is confounded by the fact that other azerty keyboards apparently do use qwerty scan codes (and rely on the OS to configure an azerty layout).

I was under the impression that the W3C KeyboardEvent.code spec was designed to combat this by providing abstract non-scancode values. But maybe they also assume scancode positions are constant and they just 1:1 map scancode to new names.

Of course some people have special nonstandard keyboards (like the various ergonomic keyboards with weird physical arrangement of keys), but those people are going to have to customize/remap their game inputs anyway.

If these are truly "non-standard" in terms of "an overwhelming majority of the world uses the same scan code positions", then I'm totally cool with using scancodes because we can't support every fringe thing that exists. I just want to make sure we aren't alienating some significant portion of the world's population.

I'm definitely not an expert here, but I'd like some way to quantify the actual situation. Something like "windows / mac / linux all assume physical scancode positions are stable as defined by some standard X and only 0.1% of the population uses standard Y".

edit: adjusted qwerty scancode link to something that uses hex values for parity with the other example

Tsudico · 2021-05-02T20:47:52Z

The USB HID spec for keyboards is based off the physical layout of a QWERTY keyboard so I think any enumeration should use the spec's defined values where applicable. [See Keyboard / Keypad page (0x07) on pdf page 82]:
https://usb.org/document-library/hid-usage-tables-122

mockersf · 2021-05-02T20:49:47Z

Fun news! on macOS, the scan codes provided by winit don't seem to be the real scan codes, but the virtual key codes macOS is using... https://developer.apple.com/documentation/coregraphics/cgkeycode

my "q" key have a scan code of... 0

inodentry · 2021-05-02T20:55:27Z

Ugh, why does macOS always ... uhm ... "think different"

mockersf · 2021-05-02T21:58:56Z

I tried running Bevy with rust-windowing/winit#1890 (macOS impl for rust-windowing/winit#753) and events are much better for keyboard 👍

example event:

KeyEvent {
    physical_key: KeyA,
    logical_key: Character(
        "q",
    ),
    text: Some(
        "q",
    ),
    location: Standard,
    state: Pressed,
    repeat: false,
    platform_specific: KeyEventExtra {
        text_with_all_modifiers: Some(
            "q",
        ),
        key_without_modifiers: Character(
            "q",
        ),
    },
}

It seems done for Windows (rust-windowing/winit#1788), in review for macOS (rust-windowing/winit#1890), draft for web (rust-windowing/winit#1888) and under work for Linux.

Unless we are ready to implement something temporary, I think we should wait for this to be done on winit side.

cart · 2021-05-03T19:31:16Z

Wow that api is so much better. And yeah the scancodes for mac (even in the new impl) are completely out of sync with what I expect them to be. Sounds like the "physical_key" api is the way to go.

Relevant bit from the api redesign issue:

Which is basically what the W3C decided to roll with for their new keyboard api for the web. Unfortunately this is the sort of thing that could drag on for a long time (iirc winit's eventloop 2.0 started midway through 2018 and winit 20.0 dropped at the start of 2020). We should try to help them if possible.

Bevy contributors motivated to make input better (faster) should consider reaching out in this issue: rust-windowing/winit#1806

maroider · 2021-05-05T10:45:31Z

Unfortunately this is the sort of thing that could drag on for a long time (iirc winit's eventloop 2.0 started midway through 2018 and winit 20.0 dropped at the start of 2020).

Heh, rust-windowing/winit#753 has been open for well over two years now, so I hope we don't end up spending too much time getting this across the finish line.

Any help on this matter would be appreciated. The Linux impl should have a PR up later this week (hopefully), so what would be most helpful is help with code review, preferably from someone familiar with X11 and/or Wayland, as both these platforms (effectively) lack maintainers. I'm sure @ArturKovacs would also appreciate another pair of eyes (and more testing) on the macOS impl.

mockersf · 2021-05-07T16:00:23Z

and now rust-windowing/winit#1932 is up for anyone on Linux to test 🎉

janhohenheim · 2022-04-10T13:56:46Z

Are there any updates on this?

mlp1802 · 2022-04-10T20:10:02Z

I'm not sure I understand this discussion fully, I too also need a way to show the user which keys he has selected, in a controls panel. I did this simple thing. Is this bad?

pub fn create_keycode_map() -> HashMap<KeyCode, String> {
    let mut map = HashMap::default();
    map.insert(KeyCode::A, "A");
    map.insert(KeyCode::B, "B");
    map.insert(KeyCode::C, "C");
    map.insert(KeyCode::D, "D");
    map.insert(KeyCode::E, "E");
    map.insert(KeyCode::F, "F");
    map.insert(KeyCode::G, "G");
    map.insert(KeyCode::H, "H");
    map.insert(KeyCode::I, "I");
    map.insert(KeyCode::J, "J");
    map.insert(KeyCode::K, "K");
    map.insert(KeyCode::L, "L");
    map.insert(KeyCode::M, "M");
    map.insert(KeyCode::N, "N");
    map.insert(KeyCode::O, "O");
    map.insert(KeyCode::P, "P");
    map.insert(KeyCode::Q, "Q");
    map.insert(KeyCode::R, "R");
    map.insert(KeyCode::S, "S");
    map.insert(KeyCode::T, "T");
    map.insert(KeyCode::U, "Y");
    map.insert(KeyCode::V, "V");
    map.insert(KeyCode::X, "X");
    map.insert(KeyCode::Y, "Y");
    map.insert(KeyCode::Z, "X");
    map.insert(KeyCode::Key1, "1");
    map.insert(KeyCode::Key2, "2");
    map.insert(KeyCode::Key3, "3");
    map.insert(KeyCode::Key4, "4");
    map.insert(KeyCode::Key5, "5");
    map.insert(KeyCode::Key6, "6");
    map.insert(KeyCode::Key7, "7");
    map.insert(KeyCode::Key8, "8");
    map.insert(KeyCode::Key9, "9");
    map.insert(KeyCode::Numpad0, "Numpad0");
    map.insert(KeyCode::Numpad1, "Numpad1");
    map.insert(KeyCode::Numpad2, "Numpad2");
    map.insert(KeyCode::Numpad3, "Numpad3");
    map.insert(KeyCode::Numpad4, "Numpad4");
    map.insert(KeyCode::Numpad5, "Numpad5");
    map.insert(KeyCode::Numpad6, "Numpad6");
    map.insert(KeyCode::Numpad7, "Numpad7");
    map.insert(KeyCode::Numpad8, "Numpad8");
    map.insert(KeyCode::Numpad9, "Numpad9");
    map.insert(KeyCode::Home, "Home");
    map.insert(KeyCode::Delete, "Delete");
    map.insert(KeyCode::End, "End");
    map.insert(KeyCode::PageDown, "PgDown");
    map.insert(KeyCode::PageUp, "PgUp");
    map
}

inodentry · 2022-04-14T20:56:06Z

This issue is not about that. This issue is about being able to handle keyboard input in a way that does not break for users with multiple keyboard layouts set in the OS.

For instance, I have a Workman (previously Dvorak), QWERTY (because i need it to test stuff that assumes everyone is using qwerty, like Bevy currently), and a Cyrillic layout (for Bulgarian/Ukrainian/Russian). French users use AZERTY. Etc...

Currently, Bevy does not work properly, because key codes come from the currently active layout. If a game uses WASD for movement, and I play the game, I have to switch to qwerty, or the keys will be in the wrong places (physically). If I have a keybinding (like Alt-Shift or something) for switching layouts, there is a risk I might accidentally press it while in game, causing the keyboard input to get messed up in the middle of gameplay.

The solution to this are "scan codes". Scan codes identify the physical key on the keyboard, regardless of the software layout set in the OS.

In Bevy, scan codes are only available when using keyboard events, not when using the Input abstraction, which makes them annoying to use. This could be fixed by providing a Input<ScanCode> type.

Further, scan codes are only represented as opaque integers, not a friendly enum like KeyCode. Due to the platform differences in the exact values of the scan codes, there isn't much Bevy can do about this, without the assistance of winit.

Ryder17z · 2022-07-17T03:48:29Z

For reference, on Scandinavian keyboards, the keys painted black here are not detected currently:

# Objective - I wanted to have controls independent from keyboard layout and found that bevy doesn't have a proper implementation for that ## Solution - I created a `ScanCode` enum with two hundreds scan codes and updated `keyboard_input_system` to include and update `ResMut<Input<ScanCode>>` - closes both #2052 and #862 Co-authored-by: Bleb1k <[email protected]>

# Objective - I wanted to have controls independent from keyboard layout and found that bevy doesn't have a proper implementation for that ## Solution - I created a `ScanCode` enum with two hundreds scan codes and updated `keyboard_input_system` to include and update `ResMut<Input<ScanCode>>` - closes both bevyengine#2052 and bevyengine#862 Co-authored-by: Bleb1k <[email protected]>

james7132 · 2022-12-04T03:05:16Z

This was added in #5495. Closing.

# Objective - I wanted to have controls independent from keyboard layout and found that bevy doesn't have a proper implementation for that ## Solution - I created a `ScanCode` enum with two hundreds scan codes and updated `keyboard_input_system` to include and update `ResMut<Input<ScanCode>>` - closes both bevyengine#2052 and bevyengine#862 Co-authored-by: Bleb1k <[email protected]>

TheRawMeatball added C-Feature A new feature, making something new possible D-Trivial Nice and easy! A great choice to get started with Bevy A-Input Player input via keyboard, mouse, gamepad, and more labels Apr 29, 2021

cart mentioned this issue May 2, 2021

[Merged by Bors] - Alien cake addict: Allow holding movement keys #2072

Closed

mockersf removed the D-Trivial Nice and easy! A great choice to get started with Bevy label May 2, 2021

mockersf mentioned this issue Jan 11, 2022

Keys not being detected while keyboard layout is in not english #3641

Closed

alice-i-cecile mentioned this issue Jul 15, 2022

Support ScanCode input bindings Leafwing-Studios/leafwing-input-manager#179

Closed

Bleb1k mentioned this issue Jul 31, 2022

[Merged by Bors] - Added keyboard scan input event #5495

Closed

james7132 closed this as completed Dec 4, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve keyboard input with Input<ScanCode> #2052

Improve keyboard input with Input<ScanCode> #2052

TheRawMeatball commented Apr 29, 2021

inodentry commented Apr 29, 2021 •

edited

Loading

cart commented Apr 29, 2021

mrec commented Apr 29, 2021

inodentry commented Apr 29, 2021 •

edited

Loading

cart commented Apr 30, 2021 •

edited

Loading

Tsudico commented May 2, 2021

mockersf commented May 2, 2021 •

edited

Loading

inodentry commented May 2, 2021

mockersf commented May 2, 2021 •

edited

Loading

cart commented May 3, 2021

maroider commented May 5, 2021

mockersf commented May 7, 2021 •

edited

Loading

janhohenheim commented Apr 10, 2022

mlp1802 commented Apr 10, 2022 •

edited

Loading

inodentry commented Apr 14, 2022

Ryder17z commented Jul 17, 2022

james7132 commented Dec 4, 2022 •

edited

Loading

Improve keyboard input with Input<ScanCode> #2052

Improve keyboard input with Input<ScanCode> #2052

Comments

TheRawMeatball commented Apr 29, 2021

What problem does this solve or what need does it fill?

What solution would you like?

What alternative(s) have you considered?

Additional context

inodentry commented Apr 29, 2021 • edited Loading

cart commented Apr 29, 2021

mrec commented Apr 29, 2021

inodentry commented Apr 29, 2021 • edited Loading

cart commented Apr 30, 2021 • edited Loading

Tsudico commented May 2, 2021

mockersf commented May 2, 2021 • edited Loading

inodentry commented May 2, 2021

mockersf commented May 2, 2021 • edited Loading

cart commented May 3, 2021

maroider commented May 5, 2021

mockersf commented May 7, 2021 • edited Loading

janhohenheim commented Apr 10, 2022

mlp1802 commented Apr 10, 2022 • edited Loading

inodentry commented Apr 14, 2022

Ryder17z commented Jul 17, 2022

james7132 commented Dec 4, 2022 • edited Loading

inodentry commented Apr 29, 2021 •

edited

Loading

inodentry commented Apr 29, 2021 •

edited

Loading

cart commented Apr 30, 2021 •

edited

Loading

mockersf commented May 2, 2021 •

edited

Loading

mockersf commented May 2, 2021 •

edited

Loading

mockersf commented May 7, 2021 •

edited

Loading

mlp1802 commented Apr 10, 2022 •

edited

Loading

james7132 commented Dec 4, 2022 •

edited

Loading