A mixed reality co-op shooter for Apple Vision Pro built around shared-space play, portal encounters, and new visionOS 26 features.
My Contribution
With a tight deadline of 1-2 months, I focused on preparing the VR-ready 3D assets for the game. This involved sourcing or designing the assets, optimizing them, and integrating them into USDA scenes along with the required material design.
Integrating into RCP requires an understanding of the software to create a functional asset. In addition to the .usdc or .usdz format typically needed, all animations must be prepared on a single timeline prior to setup in RCP. The built-in shader graph is very convenient but may appear basic compared to more advanced node graphs like those in Unity.
Some assets were fully developed using an accelerated AI-driven pipeline, taking them from concept to integration with a straightforward process:
1. Generate high-quality art using GPT or Midjourney, editing as needed through prompts or manual design. 2. Convert the image to a 3D model using Image to Mesh AI. 3. Refine the mesh in your DCC to optimize it and correct texture maps. 4. Use an AI Auto Rigger to obtain animations from Mixamo. 5. Integrate into RCP.
The project relied on a range of real-time VFX, including custom shaders, particle systems, procedural mesh animation, and material-driven effects. While these techniques are common across game development, AR/VR introduces much tighter rendering and performance constraints. Overdraw, transparency, particle fill rate, and fragment shader cost quickly become bottlenecks, requiring constant trade-offs between visual quality and frame budget.
Rather than pursuing highly realistic effects, the focus was on stylized, performant solutions that remained convincing at real-world scale, where users can inspect assets from very close distances.
The portal system presented an additional technical challenge: it had to adapt to doors of arbitrary dimensions at runtime. The solution used modular geometry with tileable materials and UV mapping driven by the mesh's world-space dimensions, allowing the frame to scale while preserving texel density and avoiding texture stretching.
Some initially prototyped effects had to be redesigned or discarded because they relied heavily on transparent materials and screen-space masking. In AR, these techniques often produced inconsistent results due to varying viewpoints, making it essential to design VFX around the user's perspective and the constraints of the rendering pipeline.
These environments differ from those I've made in other Vision Pro projects, as the ones in Spatial Rift exist within a real-time game scene (thus, I could not rely only on baked unlit rendering). They require careful design to balance detail and performance, especially since some game sessions allow users to open up to four portals simultaneously.
It was also essential to recognize that these scenes need only be visible from a 180° viewpoint, but users can open large portals, even though interiors typically have narrow doors. Thus, the wide curved background was designed to ensure no empty areas are visible from the user's perspective.