top of page

Building for Apple Vision Pro - Insights from our Journey

We adapted our Hold The World with Sir David Attenborough Unity VR experience to make the launch line up for the Apple Vision Pro in February 2024. We learned a huge amount and want to share some of those lessons with the wider community.

Lesson #1: Quality 3D assets pay off in the long run

3D Environments and Specimens in our VR experience were captured in high fidelity almost seven years ago, using photogrammetry and LiDAR scanning techniques. It’s only now, with Apple Vision Pro, that the full majesty of those visual assets are able to really shine and the decision to invest in quality has proven its worth - nothing beats the sensation that you are actually in the Natural History Museum London.

Lesson #2: Choose your Unity PolySpatial paradigm wisely

There are many options to choose from when deciding what “mode” your Unity PolySpatial application will inhabit on Apple Vision Pro, and this abundance of choice can be a little overwhelming. Each mode comes with its own constraints, tradeoffs and feature set so it pays dividends later on to spend time up front thinking about what you need for your application to avoid costly switches midway through your project.

Lesson #3: Adapt your workflow for slower build times

Developing for XR devices generally incurs an overhead of longer build times compared to other modern software platforms, and while Unity has helpful tooling to reduce this overhead you will find that generating builds to run on the Apple Vision Pro (both simulator and device) can take some time. We had limited success with the Unity PolySpatial Play to Device feature, so adjusted our expectations and factored in the build times to task planning and our development build & testing cycles.

Lesson #4: Lean into the visionOS design philosophy

The Apple Human Interface Guidelines (HIG) are a staple information source for building applications for Apple platforms, and the Apple Vision Pro HIG helped inform how best to adapt our experience to fit the platform. Leaning into the indirect control paradigm of “gaze & pinch” gave our audience the superpower of interacting with, and manipulating, rare specimens providing a magical user experience.

Lesson #5: Passthrough isn’t for every experience

After WWDC23 and the Apple Vision Pro reveal we were excited to explore adapting our experience to take advantage of the full colour, high fidelity passthrough of the platform and this was one of the first features we prioritised as we began our Apple Vision Pro development journey. It proved to be a cul-de-sac however, as we quickly realised that significant additional work would be required to solve the numerous edge cases that arose when our specimens (such as the Blue Whale) came to life inside the audience’s physical environment - the key takeaway here is that passthrough experiences are designed from the start with those challenges in mind, and adaptation later on can be a large hurdle to clear!

Lesson #6: Spend the time to make gestures feel magical

As a new user coming to the Apple Vision Pro it takes a minute for you to get comfortable with the gaze and tap, and gaze and pinch mechanics, but once your brain gets acclimated it quickly becomes second nature. The smoothness and responsiveness of navigating throughout the visionOS system are quintessential Apple design, and set an expectation with your audience that your experience will also deliver the same feelings. We found a large proportion of our development effort was focused on polishing gestures to make them feel as magical and intuitive as possible.

Lesson #7: Bring your main character to life Part I

As we mentioned in our earlier posts, the high visual fidelity of Apple Vision Pro with 23 million pixels across two displays enables an incredible level of realism for spatial content. This high visual quality provides both opportunities and challenges when including digital humans in an experience:

  • The enhanced realism offers a deeper feeling of connection with the character, so in our case you feel that Sir David Attenborough is in the room with you; 

  • The human brain is particularly sensitive to these characters, making crossing the “uncanny valley” a harder task to achieve.

Our VR experience used volumetric video capture of Sir David Attenborough, so our challenge was to make these assets shine on Apple Vision Pro which required some lateral thinking …

Lesson #8: Bring your main character to life Part II

Unity PolySpatial support for a variety of video formats is improving all the time, however volumetric video is a niche area with a number of different 3D video formats that often require additional software SDK support from the performance capture companies. With a brand new platform like Apple Vision Pro this support had yet to land, so we needed to find our own solution which included using the virtual camera and screen recording support built into Unity to bring 4K captures of Sir David Attenborough from our original VR experience into the final Apple Vision Pro application to make the one-on-one audience with Sir David feel magical.

Lesson #9: Time spent in headset is crucial

The experience we have built up as a company over the last 9 months has shown how valuable time spent in the Apple Vision Pro has been delivering a successful application. The tools provided by Unity and Apple (such as the device simulator) aid a modern development workflow, but nothing beats testing, iterating and polishing your experience inside the headset especially when considering elements that cannot be simulated adequately, such as gesture controls or fully immersive 3D content.

Lesson #10: Savour that “first time” feeling of Apple Vision Pro

Our final insight to share from our journey with Apple Vision Pro is a suggestion to always remember that feeling you had when putting the headset on for the first time, and embarking on a new spatial computing adventure. As arguably the best “first time” XR headset experience we’ve ever had, your audience will be excited and curious as they discover what the Apple Vision Pro has to offer.

Get in touch

For anyone waiting to get hands on with the device, or wondering how they can jump into launching products on this platform, contact us and we can help you out with both!


bottom of page