We approach the problem of creating haptic simulators that effectively impart skill without requiring high-fidelity devices by identifying perceptually salient events that signal transitions in the interaction. By augmenting these events, we seek to overcome deficiencies in the fidelity of the rendering hardware. We present an extension of event-based haptic rendering to noncollision events, and we describe a user-study of the training effectiveness of passive force-field haptic simulation vs. active eventaugmented simulation in a tool-manipulation task. The results indicate that active augmentation improves skill transfer without requiring an increase in the quality of the rendering device.

We describe an architecture for coping with latency and asynchrony of multisensory events in interactive virtual environments. We propose to decompose multisensory interactions into a series of discrete, perceptually significant events, and structure the application architecture within this event-based context. We analyze the sources of latency, and develop a framework for event prediction and scheduling. Our framework decouples synchronization from latency, and uses prediction to reduce latency when possible. We evaluate the performance of the architecture using vision-based motion sensing and multisensory rendering using haptics, sounds, and graphics. The architecture makes it easy to achieve good performance using commodity off-the-shelf hardware.

We describe an efficient algorithm for the simulation of large sets of non-convex rigid bodies. The algorithm finds a simultaneous solution for a multi-body system that is linear in the total number of contacts detected in each iteration. We employ a novel contact model that uses mass, location, and velocity information from all contacts, at the moment of maximum compression, to constrain rigid body velocities. We also develop a new friction model in the configuration space of rigid bodies. These models are used to compute the feasible velocity and the frictional response of each body. Implementation is simple and leads to a fast rigid body simulator that computes steps on the order of seconds for simulations involving over one thousand non-convex objects in high contact configurations.

To explore the effectiveness of novel methods of human-computer interaction, we implemented a human-like avatar that dynamically responds to the actions of its audience.

Enacting, capturing, and processing real motion for all all potential character animation scenarios is prohibitively expensive. The motion-graph method (resequencing pieces of a large pool of recorded motions to create a desired motion) has become a popular method for motion synthesis. To construct the graph of feasible transitions, however, all feasible transitions must be fixed in advance; we present a method that leverages the capabilities of a database management system to allow for run-time determination of transition feasibility. This approach allows a dynamic tradeoff between the motion feasibility (in terms of its appearance to the audience) and the degree to which the animation performs the desired action.

Realistic domains for learning possess regularities that make it possible to generalize experience across related states. This paper explores an environment-modeling framework that represents transitions as state-independent outcomes that are common to all states that share the same type. We analyze a set of novel learning problems that arise in this framework, providing lower and upper bounds. We single out one particular variant of practical interest and provide an efficient algorithm and experimental results in both simulated and robotic environments.

As a part of the data processing pipeline for the analysis of haptic rendering algorithms, we use the HAVEN to acquire laser-scanned surface data and user-guided force-probe data. These data are combined to create models for haptic rendering of the real-world objects.

With Prof. Jennifer Groh of Dartmouth (now Duke), we are investigating how information about sound location is encoded in the spike trains of neurons in the inferior colliculus, auditory cortex, lateral intraparietal cortex, and superior colliculus during saccades to sounds. We are experimenting with methods of visualizing both the raw recorded neuron activity and processed overviews of the data trends