Tracking Embodiment

My research examines how feeling present in a mediated interaction affects perceptions of the physical world. As such, I look at how immersive media, especially virtual and augmented reality, can be used to detect states of mind, affect interpersonal interactions, and address chronic pain. Immersive media is embodied–it requires the tracking of the user’s body and movements to create an interactive experience. The tracking data provided by these platforms can be used to determine elements of the user’s state of mind, or predict outcomes of an interaction. For example, the head movements of a participant wearing a head-mounted display and viewing a virtual classroom can be used to determine his or her anxiety. The synchrony of two conversational partners standing in front of a Microsoft Kinect can be used to predict the success of their collaboration.

Visualizing Embodiment

avatar and participant schematic

Changing the relationship between these tracked movements and the way they appear to the user, whether on a screen or through the lenses of an HMD, can change how he or she understands the physical world. For example, if a user suffering from chronic pain sees his or her limb moving freely in response to effort, this may affect both immediate and long-term pain sensation. If users feel present in a virtual world, they may feel less present in the real world, and thus respond less to stimuli in their physical environment. While these effects can be very marked in virtual reality, they also exist in sparser forms of mediated interactions, like text communication, on interpersonal interactions. Thus, my investigation of these effects spans multiple media that are used for interpersonal interaction.