A Unified Framework for the Development of Desktop and Immersive User Interfaces
Withindows is a theoretical framework for developing user interfaces that can operate in both dekstop and full 3D immersion without redevelopment. It simultaneously addresses problems with usability, efficiency, fatigue and fexibility associated with the development of immersive applications. The framework not only provides a route for existing desktop applications to make the transition into 3D but also optimizes several tasks commonly undertaken in virtual environments.
The framework is based on the novel combination of two existing techniques; image-plane selection and through-the-lens.
|Image-plane, or occlusion selection, involves occluding the selection object with the hand or a virtual cursor attatched to it.||Traditional 2D user interface elements can be used for long periods with low fatigue when placed on a window positioned below the hand.|
|Through-the-lens techniques a) place a virtual camera at the eye to create a view within the window. b) Objects can be grabbed and manipulated by c) reaching into the resulting view frustum.||Image-plane selection on TTL windows facilitates 2½D constrained object manipulations without reaching. a) Spatial movements of the virtual cursor translate into b) constrained manipulations at a distance.|
Ygdrasil IDE - A proof of concept graphical interface for the Ygdrasil VR development platform.
A series of design sketches illustrating a number of interaction techniques.
A paper on the framework presented at the 3DUI Symposium at VR 2008.
The full dissertation document.
There are a number of problems common to 3D user interfaces:
Applications and their interfaces must be developed separately for desktop and 3D environments.
Users are routinely asked to stand, bend, sway or reach for objects placed out in front of them.
Users frequently spend their limited time having to learn a new interface modality.
Simple 3D search and object manipulation tasks are generally less efficient in 3D.
The Withindows framework addresses each of these problems:
Because they encapsulate primary virtual environment tasks of global search, object selection and object manipulation, through-the-lens windows can be presented unaltered on the desktop and behave as traditional 3D applications with no redevelopment efffort
Through-the-lens windows can be positioned directly below the hand in 3D environments in order to reduce fatigue and allow longer term work to take place.
Users can be exposed to and learn the immersive interface on the desktop and then concentrate on aspects of the application unique to 3D when using the system in an immersive setting. Exposure to the 3D interface on the desktop is also likely to provide resiliency to the frequent low-resolution display conditions associated with 3D display technologies .
In the absence on true haptic feedback, constrained 2D manipulations from optimal alternate viewpoints overcome inefficiencies associated with methods that rely solely on first-person perspective.
The Withindows framework has a number of additional features including:
Encapsualting common virtual environment tasks within TTL windows allows those windows to be used on the desktop without modification. On the other hand, using image-plane selection on 2D windows in 3D allows desktop applications to migrate easily to 3D. Moreover, there are a number of best-in-class techniques used on trasitional configurations between desktop and immersion that can be understood or even improved when formulated using image-plane techniques. A technique called perspective cursor uses head tracking and screen location to smoothly move the cursor across multiple displays. When developed using an image-plane approach it becomes trivial to do full registered image-plane or clutched virtual-mouse selection on either mono or stereo displays. Projector-based augmented reality typically uses a standard mouse or touch interaction at the surface. An image plane approach allows interactions at a distance without introducing additional complexity or cost (figure at right).
Advantages of Image-Plane Selection