Input Challenges

Posted On: 2019-04-15

By Mark

Previously, I wrote about the design choices that are involved when creating an input system for a game. This week, I'd like to talk a bit about some of the technical challenges associated with trying to implement a flexible input system. I will focus on several specific goals that a game developer might intend from the design, and describe what obstacles must be overcome in order to achieve those goals. Along the way, I will reference a couple Unity assets that have tried to solve some of these problems - these are by no means exhaustive, they merely represent the assets that I have looked at while trying to solve these for my own work.

Keyboard or Mouse in Menus

As a designer, one of the goals is to make as intuitive an experience as possible. From the standpoint of navigating menus, this typically means that the primary form of input used in the game (such as keyboard or controller) should be usable, but it also means that the primary way of interacting with menus on that device should also be usable (such as touch on mobile or mouse on the computer.) While some games have the same primary input form as their device (for example, mouse-based PC games) many have some kind of limitation in that regard, either due to game design choices or multi-platform constraints (as an example: consoles typically use controllers for all activities, while a PC port would expect mouse support in menu navigation.) From a design standpoint, this seems like a straight-forward problem to solve: describe a control scheme for each kind of input and allow users to freely switch between different kinds of input.

Unity provides a Navigation System which might, at first glance, appear to meet the needs described above: every element is connected to other elements, and a user with keyboard/controller can use those connections to change which element is currently focused. Additionally, a player using mouse/touch can interact with any of those elements, as they would with any normal PC/mobile application. Unfortunately, there are issues that emerge from trying to support a player being able to seamlessly switch between these:

On the Unity side of things, a mouse hovering over a button triggers a "hover" state, while navigating to an element triggers a "select" state. Since these are different states they do not override each other, which means one element may be hovered while a different element is selected. This can be a source of confusion especially since the built-in controls use the same UI change (color-tint/animation) for these two states.
On the more technical side of things, precise forms of input (such as mice and analog sticks) may detect changes in input that are not initiated by the user (such as a loose analog stick wiggling or hair reflecting the mouse laser.) Developers have some tools available to workaround these issues (such as deadzone settings for analog sticks) but in general, these issues must be taken into consideration when designing systems that can be controlled by multiple input devices. Specifically, if one tries to detect the intent to use a mouse, then noise from the mouse signal could potentially cause false-positives - which can range from benign (briefly showing the mouse cursor) to disruptive (disabling controller input) depending on how the game responds to having simultaneous input from multiple competing input sources.

Controller support

Controller support for PC or mobile gaming can provide a lot of value for players that are accustomed to using one. Many game types require quick reflexes, and using a familiar input device can help players leverage their existing skills. Unfortunately, there are many obstacles to providing quality controller support - some of which may make the experience worse than having no support whatsoever.

By default, Unity's input system will automatically map controller inputs into standardized input categories, such as "button 1" or "Axis 0". Unfortunately, these mappings are controller and operating-system specific (for example, the A button on the Xbox360 controller is button 0 on Windows but button 16 on Mac .) Fortunately, developers have made assets that work around these limitations (two examples are Rewired and inControl). Any workaround, however, will be limited to a subset of supported controllers - due to the nature of this problem (lack of standardization) it is impossible to create something that works for every possible situation.
Note: Unity's upcoming new input system includes providing its own implementation for these workarounds - including some non-gaming controllers such as artists' tablets.
Controllers disconnecting mid-use is (as I understand it) a significant technical challenge. Incorrect implementation of controller support can cause such an event to lead to a hardware-related crash (such as the "Blue Screen of Death".) Even if the disconnect is handled correctly, a reconnect is typically impossible to distinguish from a new controller being connected (from what I've read on the subject: controllers do not have any unique identifier, so the computer cannot tell which controller connected.) As before, some assets are advertised as solving this, but I personally have not looked much beyond merely researching the issue.
Note: As a developer, the risk of a Blue Screen of Death is the main reason I use a commercial engine: I want to make sure my software won't cause a BSoD, so I rely on the commercial engine to handle low-level implementation details like reading input and writing to the graphics buffer.
Pairing controllers (when one controller is associated with one player) is fairly common (especially on gaming hardware with wireless controllers) but it is made difficult by some of the same factors that affect controller disconnects (for example, that controllers cannot be uniquely identified.) Frankly, of the three controller-related topics, this is the one I've researched least (since my game will be single-player) but I wanted to mention it as it is a very important consideration for those that are affected by it.

Remappable Buttons

Being able to support changing controller mappings is extremely important: both from an accessibility standpoint and from a general device support perspective. The accessibility standpoint is important for reasons that should be reasonably self-evident (differences in motor capability should not impact a player's ability to enjoy the game.) The general device support is important because, as mentioned previously, it is impossible to support all possible controllers, and players that happen to use an unsupported controller should be afforded at least some recourse (creating the mapping for themselves.) Unfortunately, while this seems like an obvious choice, there are, as always, pitfalls that make it more difficult.

Unity's input system allows players to remap their controls in the launcher. While this does make sure that the player can map their controls, even if they can't navigate in-game menus, trying to map controls without having played the game is often inadequate (for example, if "Dash" and "Jump" actions are available, players would not be able to tell if they will ever need to dash and jump at the same time.) To make things worse, there is no way to hide input mappings from the launcher, so assets that aim to standardize input often have the side effect of flooding the menu (Rewired for example, adds over 500 entries to the list.)
Including an in-game menu to remap buttons solves many of the issues with having it in the launcher, but it runs into the risk of causing players to become stuck. Specifically, if the input that is used to navigate menus is changed, then the player may not be able to navigate menus anymore. Conversely, if that input cannot be changed, then players may never be able to comfortably navigate in-game menus. Furthermore, anything that is allowed to be remapped in-game will not have its remapping reflected in Unity's launcher, but hiding the launcher may make it impossible for some players to reach the menu responsible for remapping input.
Buttons must be represented in a recognizable way for players. This can be as a string (such as "B Button") or as an image. Using images have the advantage of being more recognizable and usually taking up less space, but have the extra cost of creating the images. Regardless of whether a string or image is used, the representation of that particular button is unique for each piece of hardware (and operating system.) Furthermore, if the mapping between buttons is not one-to-one (for example, the X button being both "Menu Submit" and "Jump", or both B and Y buttons being "Punch") then it can become quite difficult to determine what exactly should be displayed to the player in help screens, tutorials, or in-game prompts.

Conclusion

As you can see, despite years of developers doing so, implementing input for a game is not yet a "solved" problem. While some aspects are solved by the engine or by third-party assets, many of the things that are essential for a good user experience are still being reinvented by each developer/studio confronting it. Working on this for my prototype has led me to understand just how out-of-scope solving all of this is, and also helped me realize that I must allocate time down the road, to make sure I give working on this for the final game the proper time and focus it deserves. Additionally, the excercise of trying to do all this myself has helped me to better appreciate just how much work goes into something I ordinarly take for granted in the games I play. Hopefully reading this has also helped you as well.