Multi-sensory Surface Deformation
Mike Oren, Jon Sheller & Chris Harding
3D surfaces, such as Digital Elevation Models (DEM), form the basis of many geoscientific tasks. This NSF funded project aims to create natural and intuitive ways for geoscientists, to perceive and interact with surface data via vision, touch and sound. The Haptic/Auditory Surface Manipulation System, currently developed at Iowa State University’s Virtual Reality Applications Center (VRAC) uses a Omni phantom haptic (force-feedback) device to feel the surface with a “virtual fingertip” and to deform and cut it via virtual tools.
When touching the surface, various types of force effects are generated, such as hardness or friction. In addition, a sound is generated that adds additional data about the current point of contact - for example, elevation (height) values can be “sonified” via musical tones of a certain pitch, if a low elevation is touched (colored blue) a low pitched note is played. As the user moves the virtual fingertip to higher elevations (colored yellow and then red) an increasingly higher pitched note is played. In effect the color-scale blue-yellow-red and its implied increase in elevation is transformed into a musical scale with its implied increase in pitch.
A deformation tool changes the shape of the surface, acting similar to rubber-skin. The scale (size) of deformation is dependent on the size of the tool. During deformation, an opposing force is used to indicate the “severity” of the deformation while a sound indicates the elevation. However, this type of deformation is very unrealistic
The graph window on the left lets the user define a more complex data-to-sound transfer function. Given a known min and max it transforms a incoming value (elevation) into a sound-property (pitch).  The user can shape the transfer function by creating, moving and deleting function points(black squares) and so change the sound while the application is running). This allows us to go beyond a simple linear increase and adjust the sound parameters (such as pitch) to a specific data set and to the user's needs.
This improved deformation tool uses a more realistic approach. As long as the user moves  the virtual fingertip, it calculates an area around the tip that moves with the tip. There are several parameters that define how this area is calculated and, more generally, how the surface behaves during deformation.  
This sequence of images shows the effect of cutting the surface (which is internally made from triangles) with a cutting tool and then grabbing it with the deformation tool and pulling to the right.
The cut was made between the two red points - you can see how the deformation propagates the further the tool is moved to the right.
This shows the effect of performing an incision in the interior of the surface and pulling one side away.
last modified: Aug 8, 2007