HomeTech NewsApple's Vision Pro monitors your eyes and predicts clicks

Apple’s Vision Pro monitors your eyes and predicts clicks

The first model of the “Spatial Computing Device” called Vision Pro, as referred to by Apple, has finally made its official debut. Prior to its official release and arrival in the hands of users, there has been much curiosity and imagination surrounding it.

With the unveiling of Vision Pro by Apple, Sterling Crispin, who previously served as a neurotechnology prototype researcher at the company, tweeted about his contributions to the development of Vision Pro and revealed the powerful technology it has yet to fully demonstrate to users.

Vision Pro seamlessly integrates digital content into the real world and features a fully 3D user interface. It can be controlled in a natural and intuitive manner through the user’s eyes, hands, or voice, as if everything can be controlled by mere thoughts. With its built-in high-efficiency eye-tracking system, Vision Pro uses high-speed cameras and projects a ring of invisible light images to the user’s eyes, enabling real-time intuitive feedback.

Sterling Crispin’s work at Apple is mostly about the user’s immersive experience, based on the data of the user’s body and brain response to detect their psychological state. Although most of the work has signed a confidentiality agreement and must not be disclosed to the public, some content has been disclosed to the public through patents.

He gave an example that Vision Pro can predict the click behavior before the user clicks something. It looks like mind reading, but it can actually be done through Vision Pro.

Vision Pro will Observe the User

When a user is in a VR (Virtual Reality) or MR (Mixed Reality) environment, the AI model tries to predict whether the user is curious, wandering, fearful, focused, recalling past experiences or Other states, these mental states can be inferred from measurements such as eye tracking, brain electrical activity, heartbeat and rhythm, muscle activity, blood density in the brain, blood pressure, skin conductance, etc.

It is definitely a daunting challenge to be able to predict the user’s click behavior. One observes that the user’s pupils respond before clicking, partly because the user expects something to happen after the click, so by monitoring the user’s eye movement behavior to establish the brain’s biofeedback and let the UI adapt to the user .

Other ways of inferring mental states, like rapidly flashing visuals or playing sounds to users, and then monitoring their reactions.

Apple's Vision Pro monitors your eyes and predicts clicks 2

Another patent details how to use machine learning and signals from the body and brain to predict the user’s attention, relaxation or learning, and then update the virtual environment to enhance the current state, such as by changing the background. Content to help users get work done, study or relax.

However, it is worth contemplating that Apple emphasizes building Vision Pro on the foundation of privacy and security. They claim that users’ browsing content and eye-tracking information will not be shared with Apple itself, third-party websites, or services.

Data from the camera and other sensors is processed directly on the system side. Does this mean that users have nothing to worry about? The fact that users’ eyes are constantly monitored, and such intimate and real-time behavioral records, raises concerns about whether they could be secretly used for other purposes. There are voices of skepticism from the outside world.

At the end of this Apple conference, the long-rumored Vision Pro was unveiled as the “One more thing,” leading people to take a big leap from “PC computing” and “mobile computing” into the new realm of “Spatial Computing.” With some time remaining before its release next year, Apple not only needs to work hard to provide users with a compelling experience but also should provide more concrete responses to concerns about security.

Steven L. Werner
Steven L. Werner
Steven has been writing about technology and science since he graduated from University. He is mostly focused on finding and researching for cutting-edge tech and most interesting innovations.

Follow us on Social Media!


Related Articles

Future trends and innovations in Healthcare app development

IntroductionList of Future trends and Innovations in Healthcare app development●    Advantages●    DisadvantagesConclusionIntroductionAs technology continues to rapidly evolve, healthcare app development is also experiencing some...

Intel Publishes 12-Qubit Chip new Quantum processor

Intel on the 15th announced the latest quantum processor called "Tunnel Falls", which is a 12-qubit (quantum bit, qubit) silicon chip, which is a...

Apple’s Vision Pro battery appears to be replaceable

Apple unveiled the long-rumored new Vision Pro at last week's WWDC conference. It is not difficult to see from the design of the product...

The world’s first manned “UFO” was tested in Shenzhen

Many companies are currently developing pure electric vertical take-off and landing vehicles, but this Chinese company had a sudden idea to create this "flying...

Explore More Articles

Proofing Your World 8 Best Strategies for Ultimate Technology Security-min

Proofing Your World: 8 Best Strategies for Ultimate Technology Security

Expanding the size of the attack surface inevitably amplifies the risk of cybersecurity breaches. Consequently, reducing the attack surface logically diminishes cybersecurity risk. It...
Starfield How to Store items in your Ship-min

Starfield: How to Store items in your Ship?

Starfield's massive open-world comes with unlimited items to farm and collect. However, it might be little bit annoying to over encumber time to time....
superconducting material Uranium Ditelluride-min

A breakthrough in the new superconducting material “Uranium Ditelluride”

Scientists have discovered a new crystalline superconducting state in a new and unusual superconductor called uranium ditelluride, a wonder material that can make extremely...
Starfield How to use Scanner-min

Starfield: How to use Scanner?

Scanner is a tool that you'll be needing a lot of in Starfield. It allows you to see which resources and materials a planet...