Spatial Interfaces
A deep dive on "spatializing" interfaces
Date
2024
Duration
4 months
Project Type
Interaction design and prototyping
Keywords
Unity, mixed reality, Gaussian Splats
Role
Sole designer and prototyper
The series of prototypes below are based on my ACM Interactions cover story, "Where Is 'Spatial' in Spatial Design?: How Design in the Age of Spatial Computing Can Leverage Paradigms from Physical Spatial Design."
The meaning of “spatial design” is changing, and its implications are far-reaching. If you google the term today, there is a Wikipedia entry defining it as a discipline that “crosses the boundaries of traditional design specialisms such as architecture, landscape architecture, landscape design, interior design, urban design, and services design as well as certain areas of public art.” You don’t need to scroll too far to perhaps see Apple’s dedicated WWDC video titled “Principles of Spatial Design.” Obviously, these two “spatial design” mean different things, right? Or do they?
One of the ways we can leverage paradigms from physical spaces for spatial computing interfaces is situatedness. How can spatial interfaces collocate us and other people, objects, and spatial conditions? From the perspective of the user, we can create a situatedness S, M, L t-shirt sizing showing how much of another space is situated within our space.
Situatedness
💡
The key idea is to integrate the sense of colocated, embodied interaction with people, objects, and environmental conditions.
Small - Situated Objects
S - In this prototype, an object is transported into the physical space and stays securely anchored to the floor. The grab-and-pull gesture is leveraged to show the virtual object, like pulling up the curtain.
Gaussian splat of a fire hydrant.
Gaussian splat of a Meta Quest 3 on a table.
Medium - Partially Situated Spaces
M - As we move along the situatedness spectrum, the physical interior room is partially blended with the virtual balcony. With recent developments in space-time gaussian splats, we can already experience moving virtual environments and are getting closer to a future of real-time blending of the physical and the virtual.
Gaussian splat of a partial staircase at Harvard University.
Large - Fully Situated Spaces
L - Here we arrived at the other end of the spectrum - we are fully immersed and are free to traverse the virtual environment.
Gaussian splat of a backyard at Harvard University.
We can prototype the spectrum of situatedness thanks to advancements in gaussian splats.
Ambient Continuity
💡
The key idea is to create a flexible interface that morphs from peripersonal to extrapersonal, foreground to background, attention-seeking to ambient, object-centered to subjectcentered, flat to immersive, point-ofview to god’s eye.
004 - Touching a blank wall evokes memories embedded in a different wall and brings them back to infuse new meaning to the current one. The interface, being the interactive height markings, bridges architectural elements across time.
005 - How would this look in a working prototype? Utilizing a Meta Quest 3, Meta Interaction SDK, and Meta Quest Scene API again, we can test interfaces mapped to architectural conditions. Imagine a generic interior space, where the interface reveals latent cultural meaning specific to an architectural element. In this case, the Chinese couplets, or Duilian emerge as one looks at an otherwise undecorated door.
006 - Interfaces can seamlessly morph from our body to the form of the contextual objects, obtaining its purpose specific to that object. The interface establishes a relationship between the object and our body. In this prototype, the interface morphs to the side table and shows that the user hasn’t had snacks here with family for a while.
007 - Does the space understand us or maybe reflect ourselves back to us? Looking up at the ceiling, an abstracted line traces our pacing and projects that back at us. The ceiling becomes a reflection of the floor and our movement.
Toward a "Spatial Interface Design"
By drawing insights from decades of research, recognizing that architecture and HCI are not monoliths, and leveraging architectural paradigms, we shed light on actionable concepts around presence, embodiment, intimacy gradients, elasticity of function, ambient transitions, and activity-centered design. They invite us to see the entire environment as an interface and embrace the human capacity to inhabit spaces physically and digitally.