Spatial UI

Menus, HUDs, labels and settings panels that live in the world. UI is built from five widget entities — panel, text, button, toggle, slider — each a real entity with its own transform: move it with the gizmo, parent it like anything else, save it with the scene.

Widgets render on both XR headsets (point with the ray or gaze, click with trigger or pinch) and flat 3D experiences (hover and click with the mouse). Hover and press feedback is built in — you never script your own highlight.

The one sizing rule

Widget size is authored in metres on the component — never with transform scale. The widget's face is a texture rasterised for the authored size; scaling the entity stretches those texels into blur. To make a panel bigger, raise its width/height fields and it re-renders crisply.

Building a menu

1. Create a UI Panel — the backdrop. A comfortable read is 1–2 m from the player, centred around eye height (y ≈ 1.2–1.6).
2. Create Text, Button, Toggle, Slider entities and parent them under the panel. Place them with local transforms, a hair in front of the backdrop (local z ≈ 0.01) so they draw over it.
3. Give each interactive widget an actionId (empty means "use the entity's name").
4. Attach one Behaviour to the panel. UI events fire on the widget's own entity first, then bubble up the parent chain — so the panel script hears every child widget and can switch on actionId:

import { Behaviour, type UiInteractionEvent } from "@relu-interactives/spatial-ecs";

export class MenuController extends Behaviour {
  onUiPress(event: UiInteractionEvent) {
    switch (event.actionId) {
      case "start-game":
        // A button under this panel was clicked.
        break;
      case "quit":
        break;
    }
  }

  onUiToggle(event: UiInteractionEvent) {
    if (event.actionId === "music") {
      const on = event.value === true;
      console.log("music enabled:", on);
    }
  }

  onUiChange(event: UiInteractionEvent) {
    if (event.actionId === "volume" && typeof event.value === "number") {
      console.log("volume:", event.value);
    }
  }
}

Bubbling — not broadcast — is what keeps two copies of the same menu prefab independent: each panel script hears only its own children.

The widgets

Widget	Component	What it is
Panel	UIPanelComponent	The flat backdrop. Rounded corners, border, translucency, follow mode.
Text	UITextComponent	Wrapped, aligned text. fontSize is metres of glyph height.
Button	UIButtonComponent	Pressable. Fires onUiPress on release.
Toggle	UIToggleComponent	On/off switch. Fires onUiToggle; value persists with the scene.
Slider	UISliderComponent	Draggable value in [min, max]. Fires onUiChange.

Images need no widget: parent an ordinary image entity under the panel.

Press semantics

A click is press, then release while still pointing at the same widget — sliding off before releasing cancels, exactly like every platform's buttons. While a press or slider drag is held, that hand cannot accidentally teleport (suppressed the same way grabbing suppresses it).

onUiChange streams while a slider is dragged (only on real value changes) and always fires once more with the final value on release.

Text sizing

fontSize is how tall the glyphs stand in the world, in metres — 0.035 reads like comfortable body text at arm's length, independent of the panel's texture density. Lines wrap automatically to the component's width; overflow past height is clamped with an ellipsis.

Follow modes

• follow: "world" (default) — the panel stays where you put it.
• follow: "billboard" — the panel yaw-rotates to face the player every frame at runtime (in the editor it holds still so you can work on it).

For a wrist menu, don't billboard — parent the panel under an XR Interactor's Left or Right slot entity (the same slots used for custom hand meshes). It then rides the hand like a watch.

Reading and driving widgets from scripts

Widget state is ordinary component data:

import { Behaviour } from "@relu-interactives/spatial-ecs";
import {
  UISliderComponent,
  UITextComponent,
} from "@relu-interactives/spatial-ecs/components";

export class ScoreHud extends Behaviour {
  private score = 0;

  addPoint() {
    this.score += 1;
    // This entity is the HUD text; editing the component re-renders it.
    const text = this.world.getComponent(this.entityId, UITextComponent);
    if (text) text.text = `Score: ${this.score}`;
  }

  readVolume(sliderEntityId: number): number {
    const slider = this.world.getComponent(sliderEntityId, UISliderComponent);
    return slider ? slider.value : 0;
  }
}

Style/text edits re-render the widget's texture on the next frame. Toggle and slider value are authored state — what you set in the editor is the state the experience starts with.

Event payload

Every UI hook receives the same shape:

import { Behaviour, type UiInteractionEvent } from "@relu-interactives/spatial-ecs";

export class Inspect extends Behaviour {
  onUiPress(event: UiInteractionEvent) {
    event.targetEntityId; // the widget that was activated
    event.actionId;       // its actionId, or its entity name
    event.input;          // the XR input source, or null for mouse clicks
    event.value;          // boolean (toggle) / number (slider); absent for presses
  }

  onUiHoverStart(event: UiInteractionEvent) {
    // The pointer moved onto the widget (also fires for mouse hover).
    console.log("hovering", event.targetEntityId);
  }
}

Performance notes

• A widget's face redraws only when its style or content changes — an idle menu costs nothing per frame.
• Slider drags and toggle flips move geometry (the knob and fill quads), not pixels, so dragging never re-renders the texture.
• pixelDensity (panel) trades texture memory for sharpness: the default 1024 px/m is crisp at arm's length; raise it only for panels the player inspects up close.