Disclaimer: This page reflects my personal perspective and does not represent Apple. All visuals are AI-generated for demonstration purposes and do not include proprietary material.

Overview

Apple Vision Pro introduced a new interaction paradigm without controllers.

The goal was to create an input system that is natural, precise, and scalable across spatial interfaces.

At Apple, I contributed to defining and validating the interaction model for Apple Vision Pro, now foundational to visionOS.

The Challenge

Traditional input methods do not translate well to spatial computing. Physical controllers are cumbersome and limit interaction.

We needed a system that:

• Works across UI navigation, text input, and object manipulation

• Minimizes fatigue during extended use

• Maintains precision without physical feedback

• Feels intuitive with minimal learning curve

Approach

The system was designed to feel invisible. Users focus on content, not controls.

We explored gaze as the primary targeting mechanism and micro-gestures for confirmation.

The system was designed to:

• Separate intent (gaze) from action (pinch)

• Reduce physical effort

• Provide continuous visual feedback

Use Research

Mixed-method approach combining quantitative performance and qualitative feedback.

Key findings:

• Eye gaze is fast but requires calibration and stabilization

• Micro-gestures reduce fatigue compared to large movements

• Visual feedback is critical for confidence and accuracy

These insights informed interaction thresholds, feedback design, and system behavior.

Impact

• Shipped as the core interaction model for Apple Vision Pro

• Enabled hands-free navigation, text entry, and object interaction

• Influenced hardware decisions (sensor placement, field of view)

• Adopted across system and third-party applications

My Role

• Prototyped interaction concepts

• Evaluated user experience through research and testing

• Conducted user studies and behavioral analysis

• Collaborated with design, engineering, and hardware teams

• Presented prototypes and findings to leadership

Reflection

Designing for spatial computing requires rethinking interaction from first principles.

The success of gaze and pinch comes from balancing human behavior, technical constraints, and system scalability.

Spatial interaction succeeds when complexity disappears. The goal is not to teach users a system, but to make it feel obvious.