RFC: a new solution to Input Method + Keyboard Layout

This is not related to KDE itself, but I’d like to hear some opinion from keyboard layout users, especially from those who use more than one keyboard layout.

Right now I’m designing a new feature for fcitx (for people who doesn’t know it, it’s an input method framework under Linux), currently called “input method group”. The goal of this feature is to solve the conflict between keyboard layout and input method (mostly conceptually) . It can also solve some other problem, but the original goal is about keyboard layout.

Before I introduce the feature, I’d like to talk about the problem itself first.

People need to type their own language with their keyboard. For most latin character based languages, keyboard layout is a good solution to them, because the number of different characters in their language is small enough to be handled by the keyboard (26 * (1..2)). But in CJK world (Chinese, Japanese, Korean), number of characters are so large (especially for Chinese, number of characters in Chinese is over thousand) that each character can’t be simply mapped to a key on the keyboard. So people invent input method, which basically maps a sequence of key to a limited number of characters and people can select from it.

But if we re-think about the keyboard layout itself, it can actually be considered as a very simple kind of input methods. The only difference is that the mappings from “keys” to “characters” in keyboard layout is much simpler than those in CJK input method. And that’s why keyboard layouts are handled as input method in modern OS, like windows and mac OSX, and ibus/fcitx nowadays.

It looks so far so good, until you remember that keyboard are not only used to type text. Suppose you want to type French and German, and you add them as your input methods. But hey, French layout is azerty and German is qwertz. Your favorite “Ctrl+Z” for undo is at the different position on the keyboard! If you add Chinese pinyin into consideration, it will be even more messed up. Not to mention if you want to use Dovrak layout with Pinyin and but pinyin is forced you to US layout on Windows.

Though keyboard layout can be considered as an input method, it’s actually a mapping from physical keys to virtual keys (and to characters). And regular CJK input method is a mapping from virtual keys to characters. So they should be able to composed arbitrary to serve user’s own need (e.g. Pinyin + Dovrak).

And we previously had a user provided one of his scenerio:

* Kain(fake name, you know) has two keyboards. One is laptop built-in with US layout, which he will use at  home. Another is a usb keyboard, but it’s in fr layout (key printing).
* He usually use Pinyin and Mozc to type Chinese and Japanese.

He want to automatically switch layout when he’s using different keyboard. While X/wayland doesn’t provide “per physical keyboard” layout (I don’t think “real” per physical keyboard layout is useful though, since you’ll still only use one physical keyboard at a time.), I’d like to support such use case and that’s why I invent the concept of “input method group”.

Basically a input method group contains a list of input methods, and has a layout. This layout will be the actual “system” layout. It affects all the hotkey bindings for your desktop. And for Kain’s use case

His configuration will look like this:
* Default group (us layout):
* Keyboard (default), Pinyin, Mozc
* Alternative group (fr layout):
* Input methods are the same as default.

And we can listen to udev event, once the keyboard is plugged in, automatically switch to group fr and he can happily type with fr layout.

For regular CJK user, they can stick to one group. For regular keyboard layout users who want old fashion keyboard layout like setup, they can use different groups with single “keyboard” input method.

And I’d also like to support multiple layout based input method in same group. With help with xkbcommon, this can be actually done. So you can have “Keyboard (US English)” and “Keyboard (Arabic)” in same group, but your system layout will always be US layout even you’re using “Keyboard (Arabic)” to type.

And I think it can also help users who only use some specific language at work and another language at home. And I think is fit quite well with KDE’s activities, e.g. bind different activities with specific input method group. After all, if you want 3 different input methods, but only two of them at a time, it would be easier to switch input method between 2 instead of 3.

What do you think of it? I’d like to make this feature supports the needs of both keyboard layout users and input method users. Do you have any other use case related that you want to share?

This entry was posted in KDE, Linux and tagged . Bookmark the permalink.

11 Responses to RFC: a new solution to Input Method + Keyboard Layout

  1. Eike Hein says:
    Google Chrome 51.0.2704.81 Android 6.0.1

    I’ve repeatedly mentioned in Plasma discussions that we need to evolve the Keyboard Layout KCM from managing layouts to managing input languages 🙂

  2. csslayer says:
    Firefox 47.0 Windows 10 x64 Edition

    @Eike Hein yeah, but “how”? I’d like to hear more detailed opinion about this solution especially from you :).

  3. Wolfgang says:
    Google Chrome 51.0.2704.84 GNU/Linux x64

    Thank for working in this field. I must say, I like your idea. Let me shortly describe my setup and then highlight the issue I have with the current implementation of keyboard layout switching in kde.

    In my setup, I have a notebook with naturally comes with its own keyboard (mac). Most of the time, I attach an also Mac usb keyboard to it, but sometimes I also use a more standard conform keyboard from Dell – whatever I have available at the moment. Apart from the fact that the Mac keyboard defaults to its special keys for F1 to F10, both keyboards behave reasonably consistent. My issue with the current layout switching in KDE is somewhere else, and if I got you correctly, you would also tackle this issue, which is.

    I use Neo as my keyboard layout. Neo is an “ergonomic” alternative to the usual German layout. However, shortcuts in KDE applications stop working once the primary keyboard layout used is not a standard layout (which is I think in the best case US, but DE also seems to work). Therefor, I have DE configured as my primary keyboard layout only to make shortcuts work. Moreover, if Neo is the active layout, shortcuts
    are not consistently mapped to the Neo layout. Thus, Strg+C ist still Strg+C, as printed on the keyboard which reflects the DE layout. Since you also mention shortcuts, I thought it might be worth bringing the issue to your mind. It would be great if this could be sorted out along the way. Two randomly picked bug reports on the matter are [1,2].

    [1] https://bugs.kde.org/show_bug.cgi?id=320423
    [2] https://bugs.kde.org/show_bug.cgi?id=209699

  4. Johan Ouwerkerk says:
    Google Chrome 50.0.2661.94 GNU/Linux x64

    @Eike Hein

    Do note that language is not necessarily the right abstraction:
    * Swedish programmers have on more than one occasion cursed the Swedish keyboard layout. Just about every common code syntax element is placed most inconveniently.
    * Very few people in the Netherlands use the rare “Dutch” keyboard layout, US International with Euro sign on 5 is “the normal one” — but there are dead keys (and AltGr) to consider. Euro layout with AltGr dead keys kind of works.
    * People might switch between typing languages often enough — probably quite normal in multilingual companies/governments/organisations.
    * Many people aren’t touch typists, plenty actually look at their keys from time to time. So they might not enjoy a mismatch between chosen keyboard layout/language and the actual printing on the keycaps.

    So a direct configuration that links keyboard layout (input) and language definitely seems like asking for trouble/regressions/bug reports/unhappy users.

  5. csslayer says:
    Firefox 47.0 GNU/Linux x64

    @Johan Ouwerkerk Actually, using “US International with Euro sign ” to type Dutch doesn’t mean that it is the wrong abstraction. The layout itself doesn’t need to be linked to the language in its property (The language property defined in evdev.xml from xkeyboard-config). You know there’s even a “Chinese layout” (I don’t know if there’s any difference between it and us layout…), but no one using Chinese cares about it.

    The basic problem is that people are using keyboard layout for different reasons. There are two different purpose for a keyboard layout:
    1. map physical key to virtual key
    2. type text.

    Layout like Arabic are nearly only used for type text. But layout like “US International” may also be preferred to be used as the virtual key mapping.

    What I tried to provide with this functionality is that, providing an both composable and orthogonal configuration between keyboard layout and input method.

    There are different groups, each group has its assigned layout for key mapping purpose (purpose 1). You can add “regular” input or “layout-based” input method to each group for text typing (purpose 2).

  6. csslayer says:
    Firefox 47.0 GNU/Linux x64

    @Wolfgang Yeah I did notice such behavior. The actual reason is probably rooted in toolkit itself. I notice that Qt seems to prefer the “first layout” if you set a list of layout with setxkbmap, while other program uses the “current” one.

    That’s why changing system layout for “text typing” could be harmful. You’ll simply confuse the applications. Back to the time I introduce layout integration in fcitx, the xkbcommon library isn’t there yet. For layout based input method, I have to switch system layout for that. But now I can easily write a layout based input method that doesn’t use the same layout as system layout, which might be a good way to solve this problem.

  7. Stefan says:
    Firefox 46.0 Windows 7 x64 Edition

    Hey, thanks for taking this opportunity to poll the community on this. I predominantly use the Colemak layout, but have re-trained myself so I can type on qwerty when I need to. For my general typing and shortcut use I find myself well enough supported by current systems, but there are a couple of things that I think would be valuable for you to consider.

    First is a real pain point. Games. Most games seem to assume a qwerty layout, some not even providing an option to configure the bindings. Sometimes I can’t even get a game to work by changing my layout – it still uses my default layout (Colemak) even when the rest of my system is using qwerty. I’ve given up on games entirely because I was unable to play them. Not all games are problematic—I’d swear I’ve even encountered a game or two that allowed me to use my keyboard as though it were in qwerty mode even though it wasn’t, and still managed to read my text input (typing in chat) using Colemak.

    Second, is the compose key. I really, really, really love the compose key. As an English speaker I don’t find myself frequently going for accents, but it’s nice to be able to type them when I have need. Also more esoteric punctuation (like em-dashes for instance) are available as well. The advantage I see in the compose key is that it’s mnemonic, meaning I rarely have to consult a reference, and I most definitely don’t need to label the keys. I think the compose key could use more love – it almost seems forgotten these days.

    Hmm.. also possibly on topic are macro buttons. My keyboard has a variety of additional keys on it that are supported by proprietary software on Windows. I haven’t gone to the effort of getting them working on Linux because they don’t seem to send keystrokes that are recognized by default. I poked around a bit, and managed to read the events off the usb device, but didn’t really know what to do with them at that point.

    Anyhow, thanks for taking the time to review feedback, I hope you’re able to make some meaningful software that makes interaction better. Thanks for your work.

  8. souch says:
    Firefox 47.0 Ubuntu x64

    Yes, something like this is really needed. I’ve designed my own keyboard layout by remapping layout file in x11/xkb (not for english layout) but found that you cannot use “.” and “,” on regular letter keys, because many applications didn’t see shortcuts bound on that relative key. So pressing Ctrl+V isn’t registered by application when my custom layout chosen because there is actually “Ctrl+.” on that place.
    And another use case might be Emacs, by default it can’t register any shortcuts typed in wrong layout, you have to change layout to use shortcuts, or use internal switcher Ctrl+\
    And of course dvorak with qwerty shortcuts, yeah good luck with that.
    We are living in 21st. century and still can’t use our computers in the way we really want it, come on guys!!

  9. Laurent says:
    Safari 9.1.1 Mac OS X  10.11.5

    I do think “real” per physical keyboard layout is useful, but looks like I’m out of luck with that. I worked in several companies where we were doing a lot of pair programming, and developers from different countries, so we had like qwerty and azerty keyboards plugged in, and remembering to hit a special combination of keys before starting typing was a pain. I wrote a post about that a long time ago ( https://blog.desgrange.net/2008/10/20/keyboards-and-layouts.html ) and according to this post ( http://www.barreverte.fr/binomer-avec-un-clavier-dvorak-et-un-clavier-azerty/ sorry it’s in French) it seems that setxkbmap with the –device option can set layout per device.

  10. csslayer says:
    Firefox 47.0 GNU/Linux x64

    @Laurent Ah, I didn’t aware of that. So it’s indeed supported. I looked into it’s technical details, looks quite silly, X server will send a keyboard mapping event if keyboard changes.. OK…

    So indeed we can support that, if the underlying system support that.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.