CUPERTINO, Calif.—After years of speculation, leaks, rumors, setbacks, and rumblings of amazing behind-the-scenes demos, Apple has made its plans for a mixed reality platform and headset public. Vision Pro is "the first Apple Product you look through, not at," Apple's Tim Cook said, a "new AR platform with a new product" that "augments reality by seamlessly blending the real world with the digital world."

"I believe augmented reality is a profound technology. Blending digital content with the real world can unlock new experiences," Cook said.

The headset, which looks like a pair of shiny ski goggles, can be controlled in a "fully 3D interface" without a handheld controller. It solely uses your eyes, hands, and voice as an interface, and the unit lets you "control the system simply by looking." Icons and other UI elements react to your gaze, and you use natural gestures like tapping your fingers or a gentle flick to select them—no need to hold your hands awkwardly in front of you constantly.

In video demonstrations, Apple showed users walking around and grabbing things from a fridge without taking the headset off. And to further keep you from feeling isolated while wearing the headset, a system called EyeSight will display your eyes when someone is nearby, conveying "a critical indicator of connection and emotion."

Advertisement

Floating 2D apps can be placed to float around your "real world" space, which remains visible through the semi-transparent display. Elements in this interface will cast shadows in the real room around them and respond to light, Apple said. These apps can also "expand fully into your space," like a pulsating 3D animation in a mindfulness app.

A floating 4K Mac display will appear when users glance at their MacBook display while in the Vision Pro. From there, users can interact with use a virtual keyboard or their voice to type, or make use of a physical Magic Trackpad and/or Magic Keyboard.

While watching movies (including 3D movies) on a virtual floating screen, the device will automatically dim your surroundings to be less distracting. The headset can take spatial photos or videos with the click of a button, which you can re-experience as panoramas that you feel like you're actually inhabiting, Apple said.

Over 100 Apple Arcade titles will be available to play via a floating screen and a handheld controller via Vision Pro "on Day One."

This is a developing story and will be updated.