Vuer

Hands

Warning: This example is still under construction. now reference: Hand Tracking

The Hands component enables WebXR hand tracking for VR/AR applications. This is essential for:

  • Natural hand interaction in VR
  • Building gesture-based interfaces
  • Creating immersive teleoperation systems
  • Developing hand-tracking research applications

Basic Usage

A minimal example that enables hand tracking:

import asyncio
from vuer import Vuer, VuerSession
from vuer.schemas import Hands

app = Vuer()

@app.add_handler("HAND_MOVE")
async def handler(event, session):
    print(f"Hand Event: key-{event.key}", event.value)

@app.spawn(start=True)
async def main(session: VuerSession):
    session.upsert(
        Hands(
            stream=True,
            key="hands",
        ),
        to="bgChildren",
    )

    while True:
        await asyncio.sleep(1.0)

Key Parameters

Parameter Type Default Description
key str - Unique identifier for the hands component
stream bool False Enable streaming of hand pose data
hideLeft bool False Hide left hand visual but still stream data
hideRight bool False Hide right hand visual but still stream data
disableLeft bool False Disable left hand tracking entirely
disableRight bool False Disable right hand tracking entirely

Hand Data Format

The HAND_MOVE event returns data in the following format:

type HandsData = {
  left?: Float32Array;    // 25 * 16 values (25 joints, 4x4 matrices)
  right?: Float32Array;   // 25 * 16 values
  leftState: HandState;
  rightState: HandState;
};

type HandState = {
  pinch: boolean;
  squeeze: boolean;
  tap: boolean;
  pinchValue: number;
  squeezeValue: number;
  tapValue: number;
};

Hand Landmarks

Following the XR Hand API conventions, 25 joints are tracked per hand:

Joint Index Joint Index
wrist 0 middle-finger-phalanx-distal 13
thumb-metacarpal 1 middle-finger-tip 14
thumb-phalanx-proximal 2 ring-finger-metacarpal 15
thumb-phalanx-distal 3 ring-finger-phalanx-proximal 16
thumb-tip 4 ring-finger-phalanx-intermediate 17
index-finger-metacarpal 5 ring-finger-phalanx-distal 18
index-finger-phalanx-proximal 6 ring-finger-tip 19
index-finger-phalanx-intermediate 7 pinky-finger-metacarpal 20
index-finger-phalanx-distal 8 pinky-finger-phalanx-proximal 21
index-finger-tip 9 pinky-finger-phalanx-intermediate 22
middle-finger-metacarpal 10 pinky-finger-phalanx-distal 23
middle-finger-phalanx-proximal 11 pinky-finger-tip 24
middle-finger-phalanx-intermediate 12

SSL Requirement

WebXR hand tracking requires HTTPS. Use ngrok or localtunnel for development:

Learn More

For detailed examples of using Hands, see: