The core of Research Area S is the spatial interaction of humans with the digital world. We consider a variety of different spatial environments that expose digital representation in different ways. In virtual reality (VR), users experience the world using specialized devices worn on the body. For VR, we focus on conceptualizing the medium, that is, adapting concepts that have been developed over decades for other presentation media such as film. A particular problem is the autonomous movement of users, since unlike in film the viewpoint cannot be directed. This also has implications for the space necessary for the user environment, and as such, we investigate ways to decouple the physical environment used for VR from that of the depicted (virtual) space. Mixed reality (MR) allows users to interact with digital elements in real space.
We consider how some of the affordances of the virtual world can be carried over into the real. This includes exploiting latest generation 3D printing techniques, dynamic objects, projection, but also a better understanding of human perception in order to close the gap between the apparent fidelity of virtual and real objects. Our goal is to perfect the illusion of the reality of a virtual object. We will also investigate how the ubiquity of sensors and the ever-growing connectedness of simple devices together with machine learning can be used to provide interaction techniques that are seamless and learn to react to the user’s intent, rather than users learning the interaction metaphors. We think of this as the next level of (spatial) natural user interfaces (NUI). Part of this interaction is also to exploit input queues that are continuously provided by humans, such as vital signs or gaze direction. This type of input can help to understand the intent of users without them even engaging in an interaction. Since this type of processing also raises questions of digital sovereignty and ownership of user data, we will critically reflect on all issues in the context of human-computer interaction.
Gesche Joost, UdK Berlin
Marc Alexa, TU Berlin
Marc Alexa: Computer Science
Patrick Baudisch: HCI
Alberto de Campo: Computational Art
Thomas Düllo: Cultural Studies
Gesche Joost: Design Research
Kora Kimpel: Digital Media
Norbert Palz: Experimental Design
Joachim Sauter: New Media Art
Hito Steyerl: Experimental Film
Develop abstractions for the creation of, and interaction in, virtual spaces. Abstraction is necessary to enable the joint development of virtual worlds without limiting the interaction to particular devices. Abstraction of the creation process should draw from metaphors established for other media.
Enrich real objects to enable behavior so far attributed to virtual ones using upcoming display and projection technologies for all modalities, vision, sound, and others.
Push direct digital manufacturing technology to quickly provide accurate physical incarnations of virtual objects, seamlessly embedded in the existing physical context, yet exhibiting properties uncommon for everyday objects.
Go beyond NUI and allow arbitrary objects and materials as interaction devices as well as arbitrary interaction metaphors for communication. User intent should be learned using machine learning, often mapping from low-dimensional physical interaction spaces to high-dimensional parameters driving complex processes.
Consider user data and body-related communication not consciously provided for inter- action to learn about users’ intentions. Critically reflect on the use of such data.