Spatial Ward
Spatial Ward is a project in collaboration with the National University Hospital Emergency Medicine Department (NUH EMD). It empowers clinicians and healthcare workers to design hospital spaces expressively, incorporating their own expertise and lived experience into the design process.
Why?
When a hospital space is proposed, clinical staff are expected to evaluate floor plans on paper. This demands significant mental effort to visualize scale and spatial flow, and frequently produces feedback that misses critical usability issues. The predominant alternative — physical mockups — is expensive, time-consuming, and inflexible.
How?
A high-fidelity digital visualization system offers a low-cost alternative that matches physical mockups in spatial accuracy while expanding the range of feedback that can be captured. By immersing clinicians in the space before it is built, we can surface nuanced, experience-driven insights earlier in the design cycle.
What?
We built an interactive digital twin application on the Apple Vision Pro. The app features geometrically accurate floor plan reconstructions generated from floor plan drawings, a library of common hospital equipment models, and a spatially anchored feedback system. Teams can walk through, furnish, and critique a space before it is built.
Key Challenges & Design Considerations
Geometric Accuracy of Spatial Scene
- A hallmark of an interactive digital twin is that everything is true to size. This lets users — in our case, clinicians and hospital staff — experience and interact with the space as it actually is, with the confidence that "what you see is what you get." Achieving this from a 2D floorplan alone, however, is no easy feat: the amount of spatial information that can be reliably inferred from a single image is limited.
- We started by experimenting with a generative world model (GWM), WorldLabs' Marble. At the time, we believed its global understanding of scene structure and texture would let us lift a 2D floorplan image directly into a photorealistic 3D Gaussian Splat scene. We built a pipeline that extracted semantic information from the floorplan image using a vision-language model (VLM) and passed it to Marble.
- We found, however, that while Marble generates photorealistic spatial scenes, these scenes are not geometrically accurate. The system first diffuses a monocular panoramic view of the scene, then lifts it into 3D Gaussian Splatting — meaning the resulting world geometry depends on monocular depth estimation, which is prone to error. Geometric accuracy is essential here: clinical staff need a true sense of spaciousness, and the application needs to support reliable measurement tools.
- After numerous revisions, we concluded that fully automating the 2D-to-3D conversion was too ambitious for now, and reverted to a human-in-the-loop process. First, we used a PDF editing software to highlight walls in the floorplan document, then extruded them with a custom Python script and scaled the result to life size using the scale ratio specified in the floorplan. We then imported the resulting GLB model into Blender, verified its scale, and used Claude Code with the Blender MCP to texture it according to spec. This significantly accelerated model construction while ensuring the application delivers on its core promise: an interactive digital twin for hospital floor design.
Intuitive Spatial User Interface
- The primary users of this application are clinicians. They are experts in their field, but with limited experience interacting with spatial media. An effective UI design is therefore crucial to minimize friction and let them express their ideas as intuitively as possible.
- visionOS gives us an infinite canvas — the world around us — to place UI windows and volumetric content. One might be tempted to populate the space like a summer analyst's desk. However, from prior experience building my own visionOS app, more is often counterproductive: users become distracted, the hierarchy of information grows opaque, and the cognitive load of navigating the app increases.
- We decided that less is more, and limited everything to a single screen. This was made possible by first fleshing out the UI wireframe and defining the content hierarchy at each stage of the app's lifecycle.
- Another challenge unique to spatial interfaces is input. On a laptop, users have a physical keyboard — built-in or external — and ten fingers to type with. On a spatial computer, that keyboard becomes a virtual floating one, poked at with two index fingers, with no tactile feedback to confirm each keystroke. Since our app is designed to let clinicians move freely around an empty room, carrying a physical Bluetooth keyboard wasn't an option.
- Left unresolved, this would have drastically increased the time and cognitive cost of giving feedback. Voice input offered a way out. This input problem is most pronounced precisely when clinicians are prototyping and reviewing a space in an empty room — but an empty room also means minimal background noise, which is ideal for audio capture. We used visionOS's built-in SFSpeechAnalyzer to handle voice input sessions, and it turned out to offer a more natural way to give feedback than typing ever could.