Hand tracking allows you to interact with a VR environment without separate controllers. Cameras on the VR headset capture and analyze the hands of the user. Usually, the hands are replicated in the VR space with virtual avatars.
The user can interact with virtual elements by touching them. There are also gestures for special actions. E.g., pinching the thumb and index finger together is used by many headsets. Find supported hand gestures for the Apple Vision Pro and for Quest.
The benefits over controllers are a lower barrier to start, less hardware requirements and a better communication with other users in social apps.
The main drawback is the lack of haptic feedback. Controller vibrations are very useful for indicating when you’re hovering over grabbable or clickable items in VR. With hand tracking, tapping in the air without any feedback feels disconnected and leaves you unsure whether the action has worked.
