Spectator: a look inside an AR experience

When using XR devices, only you can see what going on. With a spectator app, you can connect to the XR device and see the real and virtual world from the position of your phone. You can make a professional recording for marketing using advanced video controls, or share a short clip with your friends.

This project had many contributors. I mostly worked on:

• Functional app UI in Unity UI toolkit, with animations
• Connection negotiation using Bluetooth LE, and video connections using WiFi aware and WiFi direct
• Zero copy ARCora camera pipeline (mapping Kotlin images, Vulkan textures and Unity textures to the same memory block) 4K@30FPS
• Professional camera controls (physical and post)
• Person segmentation for advances compositing (Unity Sentis)