Research Interests

Virtual Reality, Human Computer Interaction and User Interfaces in Virtual Environments, Biomedical Imaging, Scientific Visualization, Geometric Modelling, Computer Graphics, Linux and Free Software

Projects

Cardiovascular Structure Exploration

Virtual biomedical models can help to better understand how diseases affect the function of parts and organs of the human body. By thoroughly exploring such a model, scientists and surgeons are able to analyze the effects of different treatment options, and ultimately find more appropriate ways to prevent diseases. Specifically a virtual model of the vascular system of the heart can aid in finding out more about the way the heart functions, and might give insights in what causes conditions such as coronary heart disease (CHD), which still is the number one killer in the United States. We have developed a system which allows the exploration of a detailed geometric 3D model of the cardiovascular tree in a scalable virtual environment in real time.
The implemented system offers both an exterior and an interior view of the data. The exterior view of the cardiovascular tree allows for a global examination and measurement of global distances. Analysis and measuring tools allow for precise determination of, for instance, distances between different vessel segments, the length of such segments, and angles at bifurcation points.In addition, by means of visual inspection, efficient validation of the simulated vascular tree helps to quickly identify areas of interest and potential problem zones. The interior of the cardiovascular tree can be inspected during a dynamic virtual fly-through simulation, which provides a view comparable to a traditional endoscopic examination. This interactive, guided or free fly-through provides detailed information about the interior of the cardiovascular structure, for example bifurcation angles and diameters of vessels. At any time, the navigation mode can be switched to manual. Here, the user has complete manual control over the position and orientation of the camera and/or the viewpoint, while the vascular tree stays fixed. The user is free to roam the current vessel segment. Thereby, the system provides an experience similar to steering a submarine.

Since we want to effectively explore the 3D model, a 2D input device, such as a desktop mouse, obviously has its limitations. An additional cordless USB input device supplements our system and provides unrestricted navigational control in 3D. As a result, the system provides a flexible low-cost 3D virtual environment, which enables efficient and intuitive 3D navigation. In the current implementation, the Logitech WingMan Cordless Rumblepad was chosen as the main 3D interaction device, because it provides several digital and analog joysticks that emulate six-degrees-of-freedom input. The software is scalable to various virtual environments (VEs). Such a VE facilitates the simulation of computer-aided diagnoses similar to those obtained in real-world surgery environments.

MolVis: Virtual Exploration of Proteins

The rapid growth in publically available macromolecular data sets has created the need for improved distribution and representation of biological data. Since the function of a molecule is directly related to its three-dimensional (3D) structure, techniques for investigating molecular structure, function and interaction are essential for cellular biologists and drug designers in particular.
This prototype study explores the possibilities and potentials of surround-screen projection-based virtual reality environments for interactive visualization of multiple molecular models.
MolScript, a program for textbook quality imaging, was enhanced with CAVE-functionality to facilitate stereoscopic real-time rendering for displaying and manipulating realistic 3D structural models of macromolecular data.
An intuitive and natural user interface was added to enable exploration and side-by-side comparison as well as superimposition of visually appealing and scientifically meaningful visualizations.

This work was supported by the members of the NSF Engineering Research Center and the Department of Computer Science at Mississippi State University, the members of the Center for Image Processing and Integrated Computing (CIPIC) at the University of California, Davis, and the AG Graphische Datenverarbeitung und Computergeometrie at the University of Kaiserslautern, Germany.

Virtual Clay: Rapid Prototyping in VR - Towards Virtual Clay Modeling

Virtual Environments (VEs) have finally matured from proof-of-concept studies performed at university laboratories into fully featured applications. They are now applied to engineering design, research and development, manufacturing, medicine, architecture, marketing, geophysical explorations, and a variety of other fields.
VEs have the potential to revolutionize traditional industrial product design by enabling the transition from conventional keyboard and mouse-based computer-aided design (CAD) to fully virtual product design. A car design, for example, traditionally originates from a clay model that forms the basis for a numerical CAD description in Bezier, B-Spline, or NURBS format after digitization.
Furthermore, physical conceptual models, so called mock-ups, still play a key role in the otherwise CAD-centered development cycles. Our clay modelling project aims at closing this technology gap by merging classical design concepts with state-of-the-art visualization and interaction device technology, while emphasizing the creation of an intuitive and natural work environment.

The virtual clay modelling project explores the use of virtual environments (VEs) for the simulation of two-handed clay modelling and sculpting tasks.
Traditional clay modeliling concepts are implemented and enhanced with new digital design tools leveraging from virtual reality (VR) and new input device technology. In particular, the creation of an intuitive and natural work environment for comfortable and unconstrained modeling is emphasized.
VR projection devices, such as the Immersive WorkBench, shutter glasses, and pinch gloves, equipped with six-degree-of-freedom (6DOF) trackers, are used to apply various virtual cutting tools to a volumetric data structure (octree). The employment of an octree as underlying data structure for volume representation and manipulation in immersive environments allows real-time modeling of solids utilizing a suite of either geometrically or mathematically defined cutting and modeling tools.
A virtual clay model is encoded as an octree, preserving its volumetric and physical properties and design history. Incremental undo/redo functionality for rapid transitions between different modeling states increases the efficiency and flexibility, while advanced features of the primitive and wire cutting tools, such as the removal of single layers from a clay model, enhance the modeling procedure to a level of precision hardly achievable at this level of detail in real clay modeling.
Models can be passed to a raytracer or exported as either triangular or tetrahedral meshes. The resulting work environment can be utilized beyond the targeted application of virtual clay modeling, for example for the visualization and analysis of medical data.
The project was supported by the Visualization Thrust at the Center for Image Processing and Integrated Computing (CIPIC) at the University of California, Davis, U.S.A, and the AG Graphische Datenverarbeitung und Computergeometrie at the University of Kaiserslautern, Germany.

Visualization and Interaction in Immersive Environments

A virtual environment (VE) is characterized by the display mode and the visualized information. Monoscopic and stereoscopic rendering provide a virtual depth perception to immerse the viewer in a virtual world. The degree of immersion depends on the projection type and nature of the information which is supposed to be visualized.

Head Mounted Displays (HMDs)
These single user display devices are mounted on a helmet-like structure. The field of view is usually very limited. For multiple users, multiple units are needed. Since exact synchronization is necessary, collaborative work environments usually don't employ HMDs. Some displays are mounted on full helmets, which are too heavy for long time use. HMDs are often used in Augmented Reality projects.
The specific HMD on the picture below was used at Mississippi State University.

Immersive Workbench
Single-screen projection-based systems such as ImmersaDesk and Responsive Workbench (RWB) consist of one screen, usually several feet in diameter, which is mounted at a variable angle. These Virtual Model Displays (VMDs) lack complete immersion, but guarantee excellent object presence.
While the ImmersaDesk utilizes a near-vertical pitch of the display surface, the RWB uses a tabletop metaphor, in which virtual objects appear to lie on the table's surface. This ensures easy accessibility for interaction with the data and allows intuitive interfaces to be shared by several users.
The field of vision is limited by the screen dimensions, since the displayed data can be explored only by looking down and in a forward direction. No surrounding view of the environment is possible, and only small or a reasonable number of a very small set of large structures fit on the screen. Therefore, navigation in the whole scene is not necessary, only navigation methods for object manipulations must be available.
The picture below was taken at CIPIC at UC Davis.

CAVE
The CAVE is a multiple-screen projection-based virtual reality system with four to six screens that are arranged in a cube for total immersion. The most common setup consists of three walls and a floor. The major advantage of the CAVE is the fact that it enables collaborative exploration of several large-scale objects through manipulation and navigation within an immersive virtual world. The viewer navigates the virtual scene by naturally moving around inside the cube, while his field of vision is completely covered by the images.
Spatially Immersive Displays (SIDs) like the CAVE enable simultaneous access to a greater amount of information than any other display devices and help escaping the bias towards 2D computing by organizing objects more effectively in 3D.
Intensive work for the MolVis project was done in the COVE at the NSF-Engineering Research Center at Mississippi State University.

Immersive Interaction

In a Virtual Environment the user should feel comfortable and unconstrained when interacting with the environment and have access to traditional interaction tools, augmented by new virtual gadgets.

Wands
Wands are easy to use interaction devices for single handed input. Most wands support digital and very often also analog input data. Object positioning is easy, but gesture recognition for sophisticated interaction and object manipulation is, of course, not possible. The custom wand in the picture below was taken from a Nintendo 64 system, features four digital and two analog controls, and is used in the COVE at the NSF-Engineering Research Center at Mississippi State University.

Pinch Gloves
Gloves are used to emulate intuitive and unencumbered two-handed interaction in VEs Unfortunately, most glove models do not permit tracking of the exact finger positions. Nevertheless, basic gesture recognition for navigation, interaction, and modelling tasks is possible through adequate pinch combinations. Pinch gloves were used as primary interaction devices in the VirtualClay project at CIPIC at UC Davis.

Haptic Input
Haptic input and force feedback allow sophisticated interaction with data in VR systems. The exact finger position is tracked and can be incorporated in the application. Since not only the equipment itself but also the data transfer between hardware and application is expensive, haptic devices usually only allow single-handed input, which is also encumbered by the external skeleton structure.
A haptic device as shown in the picture below is available at the Computer Graphics Group at the University of Kaiserslautern.