Optimizing User Interfaces for Human Performance

Antti Oulasvirta

Optimizing User Interfaces for Human Performance

Despite decades of research and enormous investments by the industry, our most commonly used user interfaces are age-old. For example, the QWERTY keyboard was invented in the 19th century, the menu in the 1950s, and the mouse and touchscreen in the 1960s. In soft- ware engineering projects, the user inter- face is among the most difficult things to get right. We believe that these problems are due to the fact that the space of alternative designs is too enormous to be explored by trial and error. A designer can entertain only the minutest fraction of all designs. Let us consider the case of designing a simple menu, one of the most commonly used user interface. The number of possible designs for a menu with 20 items is 20! = 2432902008176640000 – more than there are stars in the observable universe (10²⁴).

We believe that computational methods can be exploited for interface design. The automatization of recurring problems will allow a designer to focus on truly novel aspects. Instead of generating and trying out one or only a few instances at a time, the designer defines optimization goals, assumptions about the user and use, and sets constraints, and the computer explores the best designs. Our interface optimizer allows a designer to answer two critical questions:

1) How close to the optimal is a given design? 2) What is the optimal design?

The four general steps of this approach are:

1) representing the design space in terms of continous/discrete variables that are free/fixed,

2) operationalizing the desirable effects/outcomes of interaction as optimization goal(s),

3) constructing a predictive model that can take as an input any instance of the design space, and

4) identifying a suitable optimization method.

We explore this approach in a number of cases ranging from classic problems like text entry and menu selection to those novel multimodal interfaces such as computer vision based full-body control.

In our first case, we developed an optimized keyboard layout two-thumb text entry on tablet devices. To derive a predictive model, we devised a bimanual tapping task. The derived model accurately predicts the time to coordinate thumb movement for any given layout. The model was used in an optimizer that considered millions of alternative layouts. The rate with the predicted keyboard is the fastest reported for two-thumb text entry on a touchscreen device and improved the users’ performance by 34 %.

The group presently works to optimize keyboards for different languages, challenging the hegomony of QWERTY as the universal keyboard. Over a century after its invention, the qwerty layout is ubiquitous: in smartphones, PCs, and large displays alike, despite the fact that the new form factors and input technologies may require radically different movement ranges of fingers. The optimization approach to design critically relies on the availability of valid predictive models that reliably estimate user performance for any given design from the design space. For simple sensorimotor tasks, such as reaching a finger to press a button, there are a few models that are robust enough to make such predictions, for example, for typing. The hard problem has been to acquire new models to extend the approach to other user interfaces. The group consults the behavioral sciences and biomechanics to derive such models.

For instance, in another case, we study full-body control, as in the now popular Kinect games. Here, human movement can be mapped to virtual movement in almost infinite ways, and each mapping is associated with different performance vs. fatigue properties. The light pen, for example, was touted as a serious alternative to the mouse but was never adopted because it could not be used by information workers. We have developed a novel method that allows searching for optimal mappings by combining biomechanics simulations with analyses of speed-accuracy-tradeoff in a single experiment. The method allows designers to identify optimal gestures for games and applications.