Human-Computer Interaction

Special Thanks to Mingyu Chen

The advance of the motion tracking and 3D display technologies impacts the input and output devices of the general human-computer interaction framework nowadays. The motion sensing devices can provide great tracking accuracy at relatively low prices, which make motion-based interactions affordable and popular for general use.

In human-computer interaction, WIMP, stands for "window, icon, menu, pointing device", is so intuitive and user friendly that it is widely adopted for the graphic user interface everywhere in our daily life. Most of the time, 2D interfaces suffice because most of the content we deal with either is flat or resides in planar forms, e.g., text, tables, and pictures. However, any interaction related to reality, either virtual or augmented, is three-dimensional in nature.

The WIMP GUI may still be irreplaceable for tasks that are two-dimensional in nature, but 3D UIs will be superior when the interaction is taking place within a 3D spatial context. As a result, a universal user interface should be a mixture of 2D and 3D. In addition, motion-based control provides another interaction modality, i.e., motion gestures. We can further elevate the gesture recognition to a higher level with semantic meaning, the air-handwriting recognition.

Traditional keyboard and mouse-oriented input devices work well for 2D GUI, and separating degrees of control extends the control capability to 3D operations. This paradigm prevails in 3D modeling and animation packages, CAD systems, video games, and many others. However, these 3D interactions can be more intuitive, immersive and perhaps more efficient with the use of 6 DOF input devices.

In sum, the goal is to combine motion control and the universal user interface to define a unified gateway (control/input device) to consume and manipulate all sorts of information and services which may be involved in the living room or any working environments.