Goal

Design a gesture-based interface for annotating 3D point clouds that feels natural and fluid without requiring high precision from the user.

Challenge

• 3D point clouds are unstructured and unlabeled — manual annotation is necessary but existing tools relied on 2D input devices (touchscreens), creating a mismatch with inherently 3D data

• Free-hand gesture systems struggle with precision, occlusion, and ambiguity — especially when tracking multiple fingers simultaneously

Approach

• Designed around a chef's knife metaphor: users iteratively slice the dataset and swipe away unwanted points using 2 gestures (slice + swipe), until the desired region remains

• Used progressive refinement — eliminating the need for precise input by letting users narrow down selection across multiple rough steps

• Tracked only a single fingertip to work within Leap Motion's hardware constraints (multi-finger tracking was unreliable)

• Split tasks across 2 hands: dominant hand for point selection, non-dominant hand for camera control via 3D mouse

My Role

Co-researcher contributing to interaction design, metaphor development, and prototype design sessions.

Findings & Reflections

• The chef's knife metaphor required 0 parameters (no shape, size, or volume to specify), significantly lowering interaction complexity

• 4 key design advantages identified: no precision required, easy to understand, no volume parameters, single-finger tracking only

• Visual feedback for slice preview identified as a critical gap — users needed clearer indication of where the cut would land

• No formal user study at publication; usability testing and performance measurement planned as next steps