The Impact of Apple Vision Pro on Vision Science

Thursday - December 21 2023, 12:09 UTC - 11 months ago

Apple Vision Pro is a mixed reality headset released at the Worldwide Developers Conference on June 5, 2023. It incorporates sophisticated blend of AR and VR technologies, including personalized micro-OLED screens delivering 23 million pixels across two displays. This headset utilizes advanced facial recognition, cameras and sensors, and laser light to measure distances, generating detailed 3D maps, and offers precise environmental awareness, hand recognition, audio spatialization and real-time 3D mapping capabilities. Its impact on the field of ophthalmology is being explored by NASA-funded projects and a Nature study.

Apple's latest venture, Apple Vision Pro, was unveiled at the Worldwide Developers Conference on June 5, 2023, this innovative mixed-reality headset integrates digital elements with the real world, providing users with an immersive experience combining augmented reality (AR) and virtual reality (VR) technologies.

The headset could have a big impact in many areas, including gaming and education, but could it also be the future of ophthalmology and vision science? Artificial intelligence is already being used with platforms like Google's DeepMind, which reviews optical coherence tomography (OCT) scans to predict 50 different types of ocular diseases. Can we expect the same for the Apple Vision Pro? .

Interesting Engineering (IE) posed the same question to Dr. Ethan Waisberg, an Academic Foundation Doctor at the University of Cambridge. He replied, "I think this is a fascinating moment for the field of ophthalmology! The release of the most advanced mixed reality headset by one of the world's best companies will help to accelerate the development of VR and AR in ophthalmology," Dr. Waisberg and his colleagues explore AR and VR technologies and their impact on the field of ophthalmology. They have previously worked on a NASA-funded project to develop a head-mounted visual assessment system for astronauts during spaceflight. Now, they are exploring how the Apple Vision Pro might impact vision science in a Nature study. The co-author of the study, Dr. Joshua Ong, an Ophthalmology Resident Physician at the University of Michigan Kellogg Eye Center, told IE, "The Apple Vision Pro showcases some revolutionary advances in the field of mixed reality and will further allow us to use this technology for human health." .

But first, let's explore the actual technology behind the Apple Vision Pro.

The science behind the Apple Vision Pro .

The Apple Vision Pro integrates a sophisticated blend of AR and VR technologies. This fusion offers users an immersive experience, with the goal of seamlessly blending digital and physical realms. At its core, the Apple Vision Pro boasts personalized micro-OLED screens, delivering 23 million pixels across two displays — more than a 4K TV for each eye. Apple claims that these screens, coupled with eye-tracking technology and the new R1 chip, offer unparalleled visual clarity and a virtually lag-free, real-time view.

The incorporation of cameras and sensors, including front-facing, downward-facing, and side cameras, facilitates precise environmental awareness, hand recognition, and audio spatialization, providing users with a comprehensive and immersive sensory experience. The camera system, assisted by infrared flood illuminators and LiDAR scanners, utilizes laser light to measure distances and generate detailed 3D maps. Additionally, TrueDepth sensors contribute to precise head and hand tracking, facilitating real-time 3D mapping capabilities.The TrueDepth sensors utilize advanced facial recognition technology, enabling the Apple Vision Pro to recognize hand gestures from a wide range of positions, with the aim to further enhance the overall user expereince.