WebXR Overlay API Specification

Specification Status: Proposal Draft
Latest Draft: https://docs.vros.cat/specs/webxr-overlay-api
Version: 0.1.0
Date: 2025-01-18
Editors:

• catnet Team
• [Additional editors to be added]

Abstract

This specification defines an API for creating and managing overlay interfaces in WebXR applications. The WebXR Overlay API enables web applications to display persistent 2D and 3D content that remains visible across immersive sessions, following the VR Overlay Protocol (VROP) principles while integrating naturally with web platform security and privacy models.

Status of This Document

This section describes the status of this document at the time of its publication. This is a draft document and may be updated, replaced or obsoleted by other documents at any time.

1. Introduction

WebXR currently lacks a standardized method for creating overlay interfaces—persistent UI elements that remain visible across different immersive experiences. This specification introduces the WebXR Overlay API to address this gap.

1.1 Goals

• Enable web applications to create overlay interfaces
• Ensure security and privacy through web platform integration
• Provide performant rendering of overlay content
• Support both 2D DOM content and WebGL/WebGPU rendering
• Maintain compatibility with existing WebXR APIs

1.2 Use Cases

1. Virtual Monitors: Display web content as floating screens in VR
2. Communication Tools: Persistent chat or video call overlays
3. System Utilities: Performance monitors, media controls
4. Productivity Tools: Note-taking, task lists, calendars
5. Accessibility Features: Subtitles, visual indicators

2. Conformance

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

3. Dependencies

This specification relies on the following specifications:

• WebXR Device API
• WebGL
• WebGPU
• Permissions API

4. Security and Privacy Considerations

4.1 Permissions

Overlay creation requires explicit user permission:

const permission = await navigator.permissions.query({ name: 'xr-overlay' });
if (permission.state === 'granted') {
  // Can create overlays
}

4.2 Security Restrictions

1. Origin Isolation: Overlays are isolated by origin
2. Visibility Control: Users can hide/show overlays globally
3. Resource Limits: Browsers enforce overlay count and size limits
4. Input Sanitization: Overlay input is sanitized like regular web input

4.3 Privacy Protections

1. No Screenshot API: Overlays cannot capture other content
2. Position Limits: Overlays cannot track user precisely
3. Telemetry Restrictions: Limited performance data exposure

5. API

5.1 XROverlayManager Interface

[SecureContext, Exposed=Window]
interface XROverlayManager : EventTarget {
  readonly attribute FrozenArray<XROverlay> overlays;
  
  Promise<XROverlay> createOverlay(XROverlayInit init);
  Promise<void> requestPermission();
  
  attribute EventHandler onoverlayactivated;
  attribute EventHandler onoverlaydeactivated;
};

partial interface Navigator {
  [SameObject] readonly attribute XROverlayManager xrOverlay;
};

5.2 XROverlay Interface

[SecureContext, Exposed=Window]
interface XROverlay : EventTarget {
  readonly attribute DOMString id;
  readonly attribute XROverlayType type;
  readonly attribute XROverlayState state;
  
  // Positioning
  attribute XRRigidTransform transform;
  attribute boolean trackingSpace; // true = world-locked, false = head-locked
  
  // Visibility
  attribute boolean visible;
  attribute float opacity; // 0.0 - 1.0
  
  // Content
  Promise<void> setDOMContent(Element element);
  Promise<void> setCanvasContent(HTMLCanvasElement canvas);
  XROverlayLayer createLayer(XROverlayLayerInit init);
  
  // Lifecycle
  Promise<void> show();
  Promise<void> hide();
  Promise<void> destroy();
  
  // Events
  attribute EventHandler onshow;
  attribute EventHandler onhide;
  attribute EventHandler oninput;
  attribute EventHandler onfocus;
  attribute EventHandler onblur;
};

enum XROverlayType {
  "2d",
  "3d",
  "dashboard"
};

enum XROverlayState {
  "created",
  "visible",
  "hidden",
  "destroyed"
};

5.3 Initialization

dictionary XROverlayInit {
  required DOMString id;
  XROverlayType type = "2d";
  float width = 1.0;  // meters for 2D
  float height = 0.75;
  boolean trackingSpace = false;
  XROverlayPermissions permissions;
};

dictionary XROverlayPermissions {
  boolean input = true;
  boolean haptics = false;
  boolean audio = false;
};

5.4 Input Handling

[SecureContext, Exposed=Window]
interface XROverlayInputEvent : Event {
  readonly attribute XROverlay overlay;
  readonly attribute XRInputSource inputSource;
  readonly attribute DOMHighResTimeStamp timestamp;
  readonly attribute XRRay ray;
  readonly attribute DOMPointReadOnly intersection;
};

5.5 Rendering Layers

[SecureContext, Exposed=Window]
interface XROverlayLayer {
  readonly attribute XROverlay overlay;
  
  // WebGL rendering
  XRWebGLBinding createWebGLBinding(XRSession session, 
                                   WebGLRenderingContext context);
  
  // WebGPU rendering
  XRGPUBinding createGPUBinding(XRSession session, 
                               GPUDevice device);
  
  void requestRedraw();
};

6. Examples

6.1 Basic 2D Overlay

// Request permission
await navigator.xrOverlay.requestPermission();

// Create overlay
const overlay = await navigator.xrOverlay.createOverlay({
  id: 'my-overlay',
  type: '2d',
  width: 1.2,
  height: 0.8
});

// Set content
const div = document.createElement('div');
div.innerHTML = '<h1>Hello VR!</h1>';
await overlay.setDOMContent(div);

// Position in front of user
overlay.transform = new XRRigidTransform(
  { x: 0, y: 0, z: -2 },
  { x: 0, y: 0, z: 0, w: 1 }
);

// Show overlay
await overlay.show();

6.2 WebGL Overlay

const overlay = await navigator.xrOverlay.createOverlay({
  id: 'webgl-overlay',
  type: '2d'
});

const canvas = document.createElement('canvas');
const gl = canvas.getContext('webgl2');

// Set up WebGL rendering
await overlay.setCanvasContent(canvas);

// Render loop
function render() {
  // WebGL drawing commands
  gl.clearColor(0.0, 0.0, 0.0, 1.0);
  gl.clear(gl.COLOR_BUFFER_BIT);
  
  requestAnimationFrame(render);
}
render();

6.3 Input Handling

overlay.addEventListener('input', (event) => {
  const { ray, intersection } = event;
  
  // Handle ray intersection
  console.log('Input at:', intersection.x, intersection.y);
  
  // Trigger haptic feedback
  if (event.inputSource.hapticActuator) {
    event.inputSource.hapticActuator.pulse(0.5, 100);
  }
});

7. Integration with VROP

The WebXR Overlay API can use VROP as an underlying protocol:

// Browser implements VROP client internally
class XROverlayManager {
  #vropClient;
  
  async createOverlay(init) {
    // Send VROP CREATE_OVERLAY message
    const response = await this.#vropClient.send({
      type: 'CREATE_OVERLAY',
      payload: {
        id: init.id,
        type: init.type,
        width: init.width,
        height: init.height
      }
    });
    
    return new XROverlay(response.overlayId);
  }
}

8. Feature Detection

if ('xrOverlay' in navigator) {
  // WebXR Overlay API is supported
  const permission = await navigator.permissions.query({ 
    name: 'xr-overlay' 
  });
  
  if (permission.state === 'granted') {
    // Ready to create overlays
  }
}

9. Accessibility

Overlays must support accessibility features:

1. Screen readers: Overlay content is exposed to AT
2. Keyboard navigation: Full keyboard support required
3. High contrast: Respect system contrast settings
4. Captions: Support for subtitle overlays

10. Performance Considerations

1. Frame budget: Overlays share frame time with main content
2. Memory limits: Browsers enforce per-origin memory quotas
3. GPU resources: Texture memory is limited and shared
4. Culling: Invisible overlays should not consume resources

11. Privacy Questionnaire Answers

1. What information does this feature expose?

   • Overlay positions and content
   • Limited performance metrics

2. What is the minimum necessary data?

   • Overlay transform for rendering
   • Content for display

3. How does this specification deal with personal information?

   • No personal information is collected
   • Overlays cannot access other overlays' content

4. How does this specification deal with high-value data?

   • Overlays are origin-isolated
   • No cross-origin data access

12. References

Normative References

• [WebXR] W3C WebXR Device API
• [WebGL] Khronos WebGL Specification
• [WebGPU] W3C WebGPU Specification
• [HTML] WHATWG HTML Living Standard

Informative References

• [VROP] VR Overlay Protocol Specification
• [OpenXR] Khronos OpenXR Specification