VR

A VR project creates fully immersive virtual reality experiences that run on WebXR-compatible headsets. Users enter an immersive session where they can look around, move through the scene, grab objects, and interact with 3D content using tracked hands or controllers.

Use this for games, training simulations, virtual showrooms, social spaces, architectural walkthroughs, and any experience that places the user inside a virtual world.

What's different in this project type

When you create a VR project, the editor provides:

1. XR Rig entity - the player's movable origin that contains the camera. The rig handles locomotion (teleport or smooth movement), tracking origin (floor vs. eye-level), and comfort settings.
2. XR Interactor entities - configured hand/controller representation with pointer rays for grabbing and pointing at objects.
3. Teleport-first locomotion - the default is teleport with snap turning, the most comfort-safe movement style recommended by all platform guidelines.
4. Immersive preview - the preview button launches a WebXR session on a connected headset or simulator.

Supported platforms

• Meta Quest (Quest 2, 3, Pro) - hands and controllers, full feature set
• Pico - hands and controllers
• Android XR - hands-first interaction
• Apple Vision Pro - gaze + pinch only (no controllers), immersive-vr mode only (no passthrough on web)
• Desktop VR (SteamVR, Windows Mixed Reality) - controller-based

Workflow

1. Set up the XR Rig - configure locomotion mode, tracking origin, and comfort settings in the inspector.
2. Add scene content - models, lights, interactable objects. Everything is authored at its world position.
3. Mark grabbable objects - add an XR Interactable component to objects users should be able to pick up, and optionally a Physics component for throwing.
4. Define teleport areas - add VR Teleport Area components to surfaces where users can teleport (if using teleport locomotion).
5. Customize hands/controllers - configure the XR Interactor to show hands, controllers, or both, with preset visual styles.
6. Wire up interactions - use Behaviour scripts with XR hooks (onGrab, onRelease, onSelectStart, etc.).
7. Preview - enter the immersive session on your headset to test interactions and comfort.

Core Components

XR Rig

The rig is the player's movable origin. Key settings:

• Tracking Origin:
  • floor - Y=0 is the user's real floor (for standing experiences)
  • eye - Y=0 is where the headset started (for seated/cockpit experiences)
• Locomotion:
  • teleport (default) - point and teleport, most comfort-safe
  • smooth - continuous movement with thumbstick
  • none - no movement (for fixed-position experiences)
• Snap Turn - rotate in fixed steps instead of continuously (reduces nausea)
• Teleport Arc Colors - visual feedback for valid vs. invalid destinations

Getting tracking origin right

The classic "I spawned inside/under the geometry" bug comes from mismatched tracking origin. If your scene is authored with content sitting on Y=0 and you use eye origin, the user's eyes will be at ground level. Use floor for standing experiences where content sits on the ground.

XR Interactor

Controls what users see as their hands and how they interact:

• Interactor Type: hands, controllers, both (default - shows whatever the device reports), or none
• Hand Preset: visual style for tracked hands
  • default - realistic skin tone
  • ghost - translucent
  • neon - glowing edges
  • skeleton - bone structure
  • custom - use your own shader
• Ray Visibility:
  • interacting (default) - shows only when pointing at something actionable
  • always - always visible
  • never - hidden
• Ray Color/Thickness - customize the pointer beam

XR Interactable

Makes objects respond to hands and pointers:

• Grabbable - can be picked up
• Pointable - responds to pointer selection without grabbing
• Reach:
  • physical - hand must touch it
  • distance - grab along the ray from across the room
  • both (default) - accepts either
• At Distance:
  • snap - brings object to hand (Half-Life: Alyx style)
  • remote - manipulate where it stands
• Gesture: pinch (trigger/finger), full (grip squeeze), or any
• Hands:
  • single - one hand only
  • handOff (default) - pass between hands
  • two - both hands hold it simultaneously (for aiming, resizing)
• Apply Velocity - thrown objects inherit hand motion (for physics-enabled objects)
• Attach Points - authored hand-holds (position, rotation, hand/role restrictions)
• Hover Behavior - visual feedback when ray lands on object (color, scale, both, or none)

VR Teleport Area

Marks surfaces as valid teleport destinations. Teleport is an allow-list - only entities with this component can be teleported onto.

• Shape: plane (rectangular) or circle
• Width/Depth - physical size in metres
• Enabled - toggle without removing the component

Grabbing and Throwing

To make an object throwable:

1. Add XR Interactable component
2. Add Physics component with Dynamic rigidbody type
3. Ensure Apply Velocity is enabled (it's on by default)

The engine automatically:

• Disables physics while held
• Tracks hand motion
• Applies release velocity from the peak of recent hand movement
• Makes throws feel natural and match the gesture

import { Behaviour, type XrInputState } from "@relu-interactives/spatial-ecs";

export default class Basketball extends Behaviour {
  onGrab(input: XrInputState) {
    console.log("picked up by", input.handedness);
  }

  onRelease(input: XrInputState) {
    // Velocity already applied - react here
    console.log("thrown!");
  }
}

Two-Hand Interactions

Objects can be held with both hands for:

• Aiming (like a rifle) - primary hand anchors, secondary hand steers
• Resizing - move hands apart/together to scale uniformly
• Hand-off - pass between hands

Attach points define where hands should grip:

• Set role to primary (anchoring hand) or secondary (steering hand)
• Restrict to left, right, or any hand
• The nearest eligible point wins when a hand grabs

For a two-handed rifle:

• Hands: two
• Second Hand: Aims the object
• Two attach points: one primary at trigger grip, one secondary at foregrip

Locomotion

Teleport (Default)

• Point at a surface with VR Teleport Area component
• Arc shows valid (green) or invalid (red) landing
• Release to teleport
• Only commits if you aimed first (letting go of a grabbed object won't teleport you)
• Grabbing an object or pointing at a grabbable object suppresses teleport on that hand

Smooth

• Continuous thumbstick movement
• Higher nausea risk - use teleport as the default and smooth as an option
• Speed configurable on XR Rig

None

• No movement system (for fixed cockpit, turret, or observation experiences)

Apple Vision Pro Considerations

Vision Pro uses gaze + pinch interaction with specific limitations:

• No controllers - only hand tracking via pinch gestures
• Gaze-driven aiming - look at an object, then pinch to select
• No hover before pinch - the input source is created when pinch starts, so pointer rays appear during pinch, not while looking
• No grip/squeeze - set interactable Gesture to Pinch or Any, never Full grip
• VR only - Safari on visionOS does not support immersive-ar (no web passthrough)
• Hand tracking requires permission - on visionOS 2+, but asks the user separately

Vision Pro fallback

VR projects automatically fall back to immersive-vr mode on Vision Pro. Mixed-reality projects offer "View in your room" via AR Quick Look (a .usdz export placed natively) as an alternative.

Mixed Reality Support

Add passthrough, depth occlusion, and real-world physics by enabling Mixed Reality settings on the XR Rig:

• Passthrough - show the real room (Quest and Android XR only; switches session to immersive-ar)
• Depth Occlusion - real objects hide virtual ones
• Environment Physics - make the real room solid (planes, mesh, or auto)
• Environment Debug - visualize detected geometry on-device
• Hit Test - allow scripts to raycast against the real room
• Anchors - pin content to real-world features

Room scan required

Plane and mesh detection replay a scan captured earlier through the headset's room setup. If the user hasn't scanned their room, detected surfaces will be empty. On Quest: Settings > Guardian > Mixed Reality > Space Setup.

See Mixed Reality for full details.

Scripting VR Interactions

Use Behaviour hooks to react to XR events:

Session Lifecycle

import { Behaviour, type XrCapabilities } from "@relu-interactives/spatial-ecs";

export default class SessionManager extends Behaviour {
  onXRSessionStart(capabilities: XrCapabilities) {
    // Check what was actually granted
    if (capabilities.handTracking) { /* … */ }
  }

  onXRSessionEnd() { /* … */ }
}

Interaction Hooks

Hook	When it fires
onSelectStart(input)	Trigger pressed / finger pinched
onSelectEnd(input)	Released
onGrab(input)	This entity was picked up
onRelease(input)	This entity was let go
onSecondaryGrab()	Second hand grabbed (two-hand mode)
onSecondaryRelease()	Second hand released
onHandPoseStart(event)	A configured hand pose began
onHandPoseEnd(event)	That pose ended

input is an XrInputState containing: handedness, kind, rayOrigin, rayDirection, gripPosition, gripQuaternion, select, squeeze, axes, buttons, and joints (for tracked hands).

See VR, Grabbing and Hand Poses for comprehensive interaction patterns.

Tips

Comfort and Safety

• Start with teleport locomotion - it's the most comfortable for the widest audience
• Enable snap turn - rotating in fixed steps greatly reduces nausea
• Use floor tracking origin for standing experiences where content sits on the ground
• Test on actual hardware - comfort and presence cannot be validated in desktop preview
• Respect the guardian boundary - users define their safe space, don't fight it

Performance

• VR must maintain 72fps minimum (90fps or 120fps on newer headsets) or users will feel sick
• Reduce draw calls - combine meshes where possible, use instancing for repeated objects
• Bake lighting when you can - real-time shadows are expensive
• Test on Quest if targeting mobile VR - it's the most constrained platform
• Profile early - don't wait until polish phase to check frame rate

Interaction Design

• Show hover feedback - users need to know what they're pointing at before they grab
• Make important objects grabbable from both physical and distance reach - seated and standing users have different comfort zones
• Avoid forcing physical reach for small objects - distance grab makes fine pickups accessible
• Use attach points for tools and weapons - hands should land in the right place
• Teleport is an allow-list - deliberately mark where users can stand, don't make every surface walkable

Cross-Platform

• Set XR Interactor to both - Quest users may switch from controllers to hands mid-session, Android XR leads with hands
• Support pinch gesture - required for Vision Pro, optional elsewhere
• Test gaze + pinch - Vision Pro interaction feels different from controller pointing
• Provide controller and hand alternatives - not everyone has hand tracking enabled

Related

• VR, Grabbing and Hand Poses - comprehensive interaction patterns and scripting examples
• Mixed Reality - passthrough, environment physics, depth occlusion
• Spatial UI - in-world panels, buttons, and controls for VR menus
• Custom Behaviours - scripting interactive XR logic