Embeddable Session Replay runtime API

Once the embed is running, the parent page and the Embeddable Session Replay iframe exchange messages over the postMessage Web API. Use these messages to deliver and refresh the user-level JWT, change settings, control the camera, and scrub the timeline without reloading the viewer.

Prerequisites

A running embed. See Embeddable Session Replay for partners.

Always post to the viewer's exact origin, never '*', and check both event.source and event.origin on every message you receive.

Messages from the parent page to the iframe

Message	Effect
`{ config: {…} }`	Apply a viewer config, using the same shape as the URL `config`. A message whose only key is `apiKey` is a token update and merges into the current config. Any other `config` message replaces the whole config. A replacement with no `apiKey` keeps the current token.
`{ roomData: {…} }`	Load MR Room Layout anchors: `{ roomManifest: [{ id, label, anchors: [{ id, label, shape: "plane" or "volume" }] }], roomData: [{ id, time, enabled, p:[x,y,z], r:[x,y,z,w], s:[…] }] }`. Send `{ roomData: null }` to clear.
`{ jumpToTimestamp: 12345 }`	Scrub playback to that timeline position, in milliseconds. Same as the playback API's `seek`.
`{ camera: { action, requestId, … } }`	Read or move the camera. The viewer accepts it only from the iframe's direct parent window. It needs no credential. See Controlling the camera.
`{ playback: { action, requestId, … } }`	Play, pause, seek, or change speed, and read the playback state. The viewer accepts it only from the iframe's direct parent window. See Controlling playback.

To toggle fullscreen at runtime, send a config message with renderConfig.fullscreen set to true or false.

Messages from the iframe to the parent page

Message	Meaning
`{ iframeReady: true }`	The viewer is initialized and accepting config, camera, and playback messages. Sent once.
`{ clientConfig: {…} }`	The resolved config after init and after each accepted `config` message, including defaults the parent did not set. It also echoes the current `apiKey`.
`{ eventList: [{ name, time, uid }],` `sensorList: [{ name, category, label, tooltip }] }`	Sent once the session's events, and sensors if any, are loaded. Use it to build your own timeline or event menu.
`{ tokenRequired: true }`	`apiTokenMode` is on and no credential arrived in the URL. Boot is blocked until you push a token. The viewer repeats this message every 400 milliseconds for about 10 seconds, then stops.
`{ tokenRefreshRequired: true, status: 401 }`	The API rejected the token with a 401. Mint a fresh token and push it. See Handling token initialization and refresh.
`{ roomError: "…" }`	The room payload was rejected, with the reason.
`{ viewerError: { error } }`	Initialization failed.
`{ cameraState: {…} }`	Reply to a `camera` message, echoing its `requestId`. After a `subscribe` action, camera changes also arrive this way, without a `requestId`.
`{ cameraError: { requestId, code, message } }`	The `camera` message with that `requestId` failed. See Camera errors.
`{ playbackState: {…} }`	Reply to a `playback` message, echoing its `requestId`. After a `subscribe` action, playback changes also arrive this way, without a `requestId`.
`{ playbackError: { requestId, code, message } }`	The `playback` message with that `requestId` failed. See Playback errors.
`{ elevationClip: { verticalSliceAmount } }`	The viewer's own elevation clip slider moved. Copy the value into your stored config so your next `{ config }` message does not put the old value back.

Handling token initialization and refresh

The viewer never reads a token from the page. It asks for one, and your page answers with a token from your backend.

Startup. The viewer posts tokenRequired until a token arrives, for up to about 10 seconds.
Delivery. Your page fetches a token from your backend's token route and posts { config: { apiKey: token } }.
Expiry. When the API answers a request with 401, the viewer posts tokenRefreshRequired, at most once every 5 seconds. A 403 does not trigger a refresh.
Hold. The viewer holds its API requests for up to 30 seconds while it waits for a new token.
Timeout. If no new token arrives in time, the viewer shows a session-expired dialog. Pushing a new token later still recovers the replay.

The viewer ignores a pushed token identical to the one it rejected, and keeps waiting. Your backend must mint a new token for each refresh, so do not cache tokens on the server.

The Example Code: Embed iframe button in the Embed Configurator generates the full handshake. Its version also shows setup errors inside the frame. This trimmed version shows the core flow:

(() => {
  const IFRAME_ID = 'session-replay';
  const VIEWER_ORIGIN = 'https://replay.cognitive3d.com';   // exact origin, never '*'
  const TOKEN_ENDPOINT = '/api/session-replay-token';      // your backend route

  const frame = document.getElementById(IFRAME_ID);
  let pending = null;   // one token request at a time
  let current = null;

  // Fetch a token from your backend. Keep it in memory only: no localStorage,
  // cookies, analytics, or logs.
  async function getToken() {
    const url = new URL(TOKEN_ENDPOINT, location.href);
    if (frame.dataset.participantId) url.searchParams.set('participantId', frame.dataset.participantId);
    if (frame.dataset.sessionIds) url.searchParams.set('sessionIds', frame.dataset.sessionIds);
    const res = await fetch(url, { credentials: 'same-origin' });
    if (!res.ok) throw new Error('token endpoint answered ' + res.status);
    return (await res.json()).token;
  }

  const pushToken = (isRefresh) =>
    (pending ??= getToken()
      .then((token) => {
        if (isRefresh && token === current) console.warn('backend returned the rejected token again');
        frame.contentWindow.postMessage({ config: { apiKey: token } }, VIEWER_ORIGIN);
        current = token;
      })
      .catch((err) => console.error('could not get a token:', err.message))
      .finally(() => { pending = null; }));

  window.addEventListener('message', (e) => {
    if (e.source !== frame.contentWindow || e.origin !== VIEWER_ORIGIN) return;   // both checks required
    const d = e.data;
    if (!d || typeof d !== 'object') return;
    if ('tokenRequired' in d) pushToken(false);
    else if ('tokenRefreshRequired' in d) pushToken(true);
    else if ('viewerError' in d) console.error('viewerError', d.viewerError?.error);
  });
})();

For the token route this script calls, see Add a token route to your backend.

Updating rendering settings at runtime

Each { config } message replaces the whole config, so the parent page keeps a copy of the current config and sends all of it again with only the changed fields. The viewer posts clientConfig after init and after every accepted change, which keeps that copy current.

Remove apiKey from the stored copy. clientConfig echoes the token that was current when it was sent. Sending that copy back can replace a newer token with an expired one. A replacement config with no apiKey keeps the current token.

This snippet keeps the copy in sync and exposes three helpers:

(() => {
  const IFRAME_ID = 'session-replay';
  const VIEWER_ORIGIN = 'https://replay.cognitive3d.com';

  const frame = document.getElementById(IFRAME_ID);
  let currentConfig = null;

  // Keep a copy of the viewer's resolved config, without the token.
  window.addEventListener('message', (e) => {
    if (e.source !== frame.contentWindow || e.origin !== VIEWER_ORIGIN) return;
    if (!e.data?.clientConfig) return;
    currentConfig = { ...e.data.clientConfig };
    delete currentConfig.apiKey;   // never repost a stored token
  });

  const post = (patch) => {
    if (!currentConfig) return console.warn('the viewer has not reported its config yet');
    currentConfig = { ...currentConfig, ...patch };
    frame.contentWindow.postMessage({ config: currentConfig }, VIEWER_ORIGIN);
  };

  // Show or hide the built-in UI controls: header, menus, and timeline chrome.
  window.SR_setUIControls = (show) =>
    post({ renderConfig: { ...currentConfig?.renderConfig, ui: !!show } });

  // Switch the camera control type. See Camera control types in the configuration reference.
  window.SR_setCameraControlType = (controlType) => post({ controlType });

  // Set the elevation clip amount, where 1 is 100 percent and clips nothing.
  window.SR_setElevationClip = (amount) =>
    post({ verticalSliceAmount: Math.min(1, Math.max(0, amount)) });
})();

Call the helpers from your own UI to change the embed's display without reloading it:

// Switch the camera to the participant's first-person perspective.
SR_setCameraControlType('firstPerson');

// Set the elevation clip to a value between 0 and 1.
SR_setElevationClip(0.68);

For every setting you can change this way, see Embeddable Session Replay configuration.

Controlling the camera

The viewer answers each { camera: { action, … } } message with { cameraState } or { cameraError }, echoing the requestId you sent. Unlike { config }, a camera message changes only the camera and needs no credential.

Send camera messages only after the viewer posts iframeReady. The viewer drops earlier camera messages without a reply.

Positions are in the viewer's world space, and angles are in degrees. A pose read with get can be passed back to setPose unchanged, which is all that saving and restoring a view needs. To open the embed on a saved view, pass the pose as initialCamera in the URL config.

Camera actions

Every request can carry a string requestId, which the reply echoes. The viewer clamps durationMs to 0–10,000 milliseconds.

Action	Fields	Effect
`get`	none	Returns the current camera state.
`setMode`	`mode`	Switches the camera mode. Accepts every `controlType` value and the older names it lists, plus `follow`.
`setPose`	`payload`, `durationMs`	Moves the camera to the pose in `payload`, animating over `durationMs` milliseconds (default `0`). In `firstPerson` or `follow` mode, the camera switches to `orbit` first.
`frame`	`target`, `durationMs`	Frames a target, animating over `durationMs` milliseconds (default `600`).
`reset`	none	Returns the active mode to its default view and clears any framing lock. The mode does not change.
`cancelTransition`	none	Stops an animated move where it is.
`subscribe`	`throttleMs`	Starts a stream of `cameraState` messages on each camera change, at most one every `throttleMs` milliseconds (default `100`, minimum `50`). Mode changes and the end of a move arrive immediately.
`unsubscribe`	none	Stops the stream.

A pose is { position, target, fovDeg, orthoHeight }. position is where the camera is, and target is the point it looks at; both are [x, y, z] positions. fovDeg is the vertical field of view, for perspective modes only. orthoHeight is the visible height in world units, for top-down modes only. Both are optional in setPose; leave them out to keep the current zoom.

The frame action accepts these targets:

Target	Frames
`{ kind: 'object', sdkId }`	A Dynamic Object, by its SDK ID.
`{ kind: 'point', point, radius }`	A point, with an optional radius around it.
`{ kind: 'bounds', min, max }`	A box between two corners.
`{ kind: 'scene' }`	The whole scene.
`{ kind: 'hmd' }`	The participant's HMD.

Camera state

cameraState describes the camera after a request, or after a change while subscribed.

Field	Definition
`version`	Camera API version. Currently `1`.
`space`	Coordinate space of the positions. Always `viewer`.
`mode`	Active mode: `orbit`, `fly`, `firstPerson`, `follow`, or `topDown`. Every `controlType` reports as one of these. For example, `followCamera` reports as `follow`, and `topdown` as `topDown`.
`modeOptions.followHmd`	In `topDown` mode, whether the view stays centered on the participant.
`pose`	The current pose, plus `projection`: `perspective` or `ortho`.
`autoZoom`	The `fovDeg` or `orthoHeight` the mode would choose itself. When `pose` has the same value, the user has not zoomed, and you can leave the zoom out of a saved pose.
`transitioning`	Whether an animated move is in progress.
`sceneScale`	The scene's scale factor.
`requestId`	The `requestId` of the request this state answers. Absent on stream updates.

Camera errors

cameraError carries a code and a readable message.

Code	Cause
`badRequest`	A field is missing or has the wrong type.
`unknownAction`	The `action` is not one of the actions above.
`unknownMode`	`setMode` received a mode name the viewer does not recognize.
`unknownObject`	No Dynamic Object has the given `sdkId`.
`notReady`	The target is not available yet, for example `hmd` before the participant loads.
`unsupported`	The request does not apply to the current mode, such as `fovDeg` in a top-down mode.

Camera snippets

The Camera API box in the Embed Configurator's Camera section holds a ready-to-paste message. The Position tab shows a setPose message for the preview's current view and updates as you move the preview camera; it includes the zoom only when it differs from autoZoom. The Control tab shows a setMode message for the Camera Controls selection. The box targets the configurator's own preview iframe, so change the ID to your iframe's ID when you copy it. This is an example of its output:

const iframe = document.getElementById('session-replay');
const viewerOrigin = new URL(iframe.src).origin;

iframe.contentWindow.postMessage(
  {
    "camera": {
      "action": "setPose",
      "payload": {
        "position": [4.215, 6.830, -2.104],
        "target": [0.512, 0.000, 1.377]
      },
      "durationMs": 600
    }
  },
  viewerOrigin,
);

This snippet wraps each action in a promise that resolves when the matching reply arrives, and exposes the helpers as SR_camera. A request with no reply within 5 seconds rejects:

(() => {
  const IFRAME_ID = 'session-replay';
  const REPLY_TIMEOUT_MS = 5000;

  const frame = document.getElementById(IFRAME_ID);
  const origin = new URL(frame.src, location.href).origin;

  // Every request carries a requestId. The reply, { cameraState } or
  // { cameraError }, echoes it, which is how send() resolves the right promise.
  let nextId = 1;
  const pending = new Map();
  const send = (camera) =>
    new Promise((resolve, reject) => {
      const requestId = 'sr-' + nextId++;
      const timer = setTimeout(() => {
        pending.delete(requestId);
        reject(new Error('no camera reply: send camera messages after iframeReady'));
      }, REPLY_TIMEOUT_MS);
      pending.set(requestId, { resolve, reject, timer });
      frame.contentWindow.postMessage({ camera: { ...camera, requestId } }, origin);
    });

  const onMessage = (e) => {
    if (e.source !== frame.contentWindow || e.origin !== origin || !e.data) return;
    const { cameraState, cameraError } = e.data;
    const reply = cameraState || cameraError;
    if (!reply) return;
    const waiter = reply.requestId && pending.get(reply.requestId);
    if (waiter) {
      clearTimeout(waiter.timer);
      pending.delete(reply.requestId);
      if (cameraError) waiter.reject(cameraError);
      else waiter.resolve(cameraState);
    } else if (cameraState) {
      window.SR_camera.onChange(cameraState);           // change-stream update
    }
  };
  window.removeEventListener('message', window.__srCameraOnMessage);
  window.addEventListener('message', (window.__srCameraOnMessage = onMessage));

  let saved = null;
  window.SR_camera = {
    get: () => send({ action: 'get' }),
    setMode: (mode) => send({ action: 'setMode', mode }),
    setPose: (pose, durationMs = 600) => send({ action: 'setPose', payload: pose, durationMs }),
    frameObject: (sdkId, durationMs = 600) =>
      send({ action: 'frame', target: { kind: 'object', sdkId }, durationMs }),
    frameScene: (durationMs = 600) => send({ action: 'frame', target: { kind: 'scene' }, durationMs }),
    reset: () => send({ action: 'reset' }),
    saveView: async () => (saved = (await send({ action: 'get' })).pose),
    restoreView: () =>
      saved
        ? send({ action: 'setPose', payload: saved, durationMs: 600 })
        : Promise.reject(new Error('saveView() first')),
    subscribe: (throttleMs = 200) => send({ action: 'subscribe', throttleMs }),
    unsubscribe: () => send({ action: 'unsubscribe' }),
    onChange: (state) => console.log('[SR] camera', state.mode, state.pose.position),
  };
})();

Call the helpers from your own UI. To handle the change stream, replace SR_camera.onChange before you subscribe:

// Bookmark the current view, then return to it later.
await SR_camera.saveView();
await SR_camera.restoreView();

// Frame a Dynamic Object by its SDK ID.
SR_camera.frameObject('YOUR_DYNAMIC_OBJECT_SDK_ID');

// Stream camera changes, at most one every 200 milliseconds.
SR_camera.onChange = (state) => updateMinimap(state.pose);
SR_camera.subscribe(200);

Controlling playback

Use { playback: { action, … } } messages to drive the replay from your own transport controls. The viewer answers each one with { playbackState } or { playbackError }, echoing the requestId you sent. Like camera messages, playback messages are accepted only from the iframe's direct parent window, after iframeReady.

Playback actions

Every request can carry a string requestId, which the reply echoes.

Action	Fields	Effect
`get`	none	Returns the current playback state.
`play`	none	Starts playback. At the end of the session, playback starts over from `0`.
`pause`	none	Pauses playback.
`toggle`	none	Plays when paused, pauses when playing.
`seek`	`timeMs`	Moves the playhead to `timeMs` milliseconds from the start, clamped to the session.
`setSpeed`	`speed`	Sets the playback speed: `1`, `2`, or `4`.
`subscribe`	`throttleMs`	Starts a stream of `playbackState` messages on each change, at most one every `throttleMs` milliseconds (default `250`, minimum `50`). Play, pause, buffering, speed, duration, and loop changes arrive immediately.
`unsubscribe`	none	Stops the stream.

Playback state

Field	Definition
`version`	Playback API version. Currently `1`.
`isPlaying`	Whether playback is on.
`buffering`	Whether playback is held while the data under the playhead loads. `isPlaying` stays `true` meanwhile.
`currentTimeMs`	Playhead position, in milliseconds from the start of the replay.
`durationMs`	Length of the replay, in milliseconds. `0` until the session has loaded.
`speed`	Playback speed: `1`, `2`, or `4`.
`loop`	Whether playback restarts at the end, from `renderConfig.loopSession`.
`requestId`	The `requestId` of the request this state answers. Absent on stream updates.

Playback errors

playbackError carries a code and a readable message.

Code	Cause
`badRequest`	A field is missing or has the wrong type, such as a `speed` other than `1`, `2`, or `4`.
`unknownAction`	The `action` is not one of the actions above.
`notReady`	The session has not loaded yet, or another session is loading. Applies to `play`, `toggle`, and `seek`.
`unsupported`	The request names a `version` this viewer does not support.

This example toggles playback from your own button and keeps a progress bar in step:

const frame = document.getElementById('session-replay');
const origin = new URL(frame.src, location.href).origin;
const playback = (body) => frame.contentWindow.postMessage({ playback: body }, origin);

window.addEventListener('message', (e) => {
  if (e.source !== frame.contentWindow || e.origin !== origin) return;
  const state = e.data?.playbackState;
  if (!state) return;
  playButton.textContent = state.isPlaying ? 'Pause' : 'Play';
  progress.max = state.durationMs;
  progress.value = state.currentTimeMs;
});

// Send these after the viewer posts iframeReady.
playback({ action: 'subscribe', throttleMs: 250 });
playButton.onclick = () => playback({ action: 'toggle' });
progress.oninput = () => playback({ action: 'seek', timeMs: Number(progress.value) });