Skip to content

Three.js Integration

Cognitive3D integrates with Three.js through the @cognitive3d/analytics NPM package and its C3DThreeAdapter. The Three.js integration supports the full feature set of the WebXR SDK: automatic WebXR gaze tracking, performance profiling, dynamic object tracking with per-object heatmaps, object engagements, scene export, and individual object export.

Requirements

  • Cognitive3D Account: You'll need an active Cognitive3D account to obtain your API keys and set up your project. You can sign up at the Cognitive3D platform website.

Note

You can quickly find your API keys on Cognitive3D with ctrl/cmd + K to pull up the search menu and then searching for Manage Developer Key. Note the difference between the Developer Key (Used for uploading assets to Cognitive3D) and the Application Key (Used in your Applications to allow your project to send data to our servers).

  • Three.js Project: An existing Three.js project with WebXR enabled (renderer.xr.enabled = true). three >= 0.150.1 is the supported peer version.
  • Node.js 20+ for the build toolchain.

Step 1: Install the Cognitive3D NPM Package

In your Three.js project's terminal, install the SDK from NPM:

npm install @cognitive3d/analytics

Step 2: Create a Settings File

Create a settings.js (or settings.ts) file in your project. This holds your API key, the scenes your app knows about, and any batching overrides. You'll fill in the sceneId and versionNumber after you upload your scene in Step 6 — leave them blank for now.

// settings.js
export default {
  config: {
    APIKey: "YOUR_APPLICATION_API_KEY",
    allSceneData: [
      {
        sceneName: "MyThreeJSScene",
        sceneId: "",        // filled in after scene upload
        versionNumber: "1", // filled in after scene upload
      },
    ],
    // Optional overrides (defaults shown)
    // gazeTrackingSource: "webxr",   // or "engine" for locomotion-heavy apps
    // GazeInterval: 0.1,             // seconds between gaze samples
    // customEventBatchSize: 256,
    // sensorDataLimit: 512,
    // dynamicDataLimit: 512,
    // gazeBatchSize: 256,
  },
};

Step 3: Initialize the SDK and the Three.js Adapter

Import both the main C3D class and C3DThreeAdapter, create the SDK instance, then pass it to the adapter. Passing the THREE.WebGLRenderer as the second argument to C3D enables automatic performance profiling (draw calls, memory, frame time).

import C3D from "@cognitive3d/analytics";
import C3DThreeAdapter from "@cognitive3d/analytics/adapters/threejs";
import settings from "./settings";

// 1. Initialize the main SDK. Passing the renderer enables the profiler.
const c3d = new C3D(settings, renderer);

// 2. Initialize the Three.js adapter.
const c3dAdapter = new C3DThreeAdapter(c3d);

// 3. Tell the SDK which scene the session will be running in.
c3d.setScene("MyThreeJSScene");

// 4. Set the required app version property.
c3d.setUserProperty("c3d.app.version", "1.0"); // REQUIRED

The adapter automatically sets AppEngine = "Three.js" and AppEngineVersion = THREE.REVISION on the session's device properties.

Step 4: Session Lifecycle

To capture data reliably, hook into Three.js's native WebXR session events. The SDK starts recording when the participant enters VR, updates every frame, and finalizes the session when they exit.

  • Starting the session: on the renderer's sessionstart event, call c3d.startSession(xrSession). Passing the active XRSession lets the SDK automatically record gaze, HMD orientation, controller tracking, boundary events, and input-source changes.
  • Render loop: call c3dAdapter.update() every frame. This records dynamic object transforms, FPS sensors, and (if configured) engine-driven gaze samples.
  • Ending the session: on the renderer's sessionend event, call c3d.endSession(). This flushes all batched data and closes the session on the Cognitive3D dashboard. Skipping this causes data loss.
// Start session
renderer.xr.addEventListener("sessionstart", async () => {
    const xrSession = renderer.xr.getSession();

    // Initialize tracking BEFORE startSession so the gaze raycaster is wired up.
    c3dAdapter.startTracking(renderer, camera, interactableGroup);

    await c3d.startSession(xrSession);
    console.log("Cognitive3D: session started");
});

// Render loop
renderer.setAnimationLoop((timestamp, frame) => {
    c3dAdapter.update(timestamp, frame); // REQUIRED every frame
    renderer.render(scene, camera);
});

// End session
renderer.xr.addEventListener("sessionend", () => {
    c3d.endSession().then(status => {
        console.log("Cognitive3D: session ended with status", status);
    });
});

About startTracking

c3dAdapter.startTracking(renderer, camera, trackableTarget) initializes gaze raycasting, dynamic object scanning, and XR-synced FPS tracking. trackableTarget can be:

  • A THREE.Scene — the adapter traverses the entire scene graph, picking up any object with userData.c3dId.
  • A THREE.Group — only the group's descendants are considered.
  • A THREE.Object3D — a single trackable mesh or node.
  • An Array<THREE.Object3D> — an explicit list (useful when you don't want a parent group).

A convenient pattern is to use a dedicated interactableGroup:

const interactableGroup = new THREE.Group();
scene.add(interactableGroup);
// ...add any trackable objects to interactableGroup instead of scene directly

Step 5: Marking Dynamic Objects

The Cognitive3D platform distinguishes static geometry (the environment) from dynamic objects (objects that move or that you want heatmap data on). Tag any object you want tracked by setting fields on its userData.

  • isDynamic (boolean, true): flags the object for automatic transform tracking.
  • c3dId (string): the unique identifier that must match the mesh you upload to the dashboard. Gaze ray-hits report their intersection point in this object's local space, which is what drives per-object heatmaps.
  • positionThreshold (number, default 0.01 m): minimum position delta to trigger a new snapshot.
  • rotationThreshold (number, default 0.5 degrees): minimum rotation delta.
  • scaleThreshold (number, default 0.05 units): minimum scale delta.
const myObject = new THREE.Mesh(geometry, material);
myObject.name = "MyTrackedObject";
myObject.userData.isDynamic = true;
myObject.userData.c3dId = "my-tracked-object-01";
myObject.userData.positionThreshold = 0.005; // optional
interactableGroup.add(myObject);

Increasing thresholds reduces snapshot frequency and network traffic, at the cost of less precise session replays. The defaults are a good starting point.

Step 6: Register Dynamic Objects with the Backend

Tagging the object tells the adapter what to track in the engine. You also need to register the object so the Cognitive3D backend knows which uploaded mesh this instance corresponds to. Do this when the object is first added to your scene, typically right after tagging it.

c3d.dynamicObject.registerObjectCustomId(
    myObject.name,                 // human-readable name, e.g. "PlayerCar"
    "car_mesh",                    // mesh name matching the uploaded asset
    myObject.userData.c3dId,       // unique custom id for this instance
    myObject.position.toArray(),   // initial world position
    myObject.quaternion.toArray(), // initial world rotation
    myObject.scale.toArray()       // (optional) initial scale
);

Imported glTF/GLB models

For loaded models, attach the userData and register against the root of gltf.scene.

import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader.js";

const loader = new GLTFLoader();
loader.load("models/car.glb", (gltf) => {
    const car = gltf.scene;
    car.name = "PlayerCar";
    car.userData.isDynamic = true;
    car.userData.c3dId = "car-unique-id-001";

    c3d.dynamicObject.registerObjectCustomId(
        car.name,
        "car_mesh",
        car.userData.c3dId,
        car.position.toArray(),
        car.quaternion.toArray(),
        car.scale.toArray()
    );

    interactableGroup.add(car);
});

Runtime-spawned objects

For objects created mid-session (projectiles, enemies, loot), call c3dAdapter.addInteractable(object) after registering so the adapter picks it up for gaze raycasting and transform tracking.

function spawnEnemy() {
    const enemy = new THREE.Mesh(geometry, material);
    enemy.name = "Enemy_orc_01";
    enemy.userData.isDynamic = true;
    enemy.userData.c3dId = "enemy-orc-" + Date.now();
    scene.add(enemy);

    c3d.dynamicObject.registerObjectCustomId(
        enemy.name,
        "orc_mesh_01",
        enemy.userData.c3dId,
        enemy.position.toArray(),
        enemy.quaternion.toArray()
    );

    c3dAdapter.addInteractable(enemy);
}

Step 7: Exporting Scene and Dynamic Objects

Before the dashboard can visualize a session, it needs your scene geometry and the meshes for every dynamic object. The Three.js adapter generates glTF + bin + screenshot bundles directly from the running app.

Exporting the scene

c3dAdapter.exportScene(scene, "MyThreeJSScene", renderer, camera);

Typically this is wired to a key press (e.g. Shift + E) that you only enable during development. The adapter:

  1. Clones the scene and removes every object tagged with userData.c3dId or userData.isDynamic so only the static environment is exported.
  2. Uses Three.js's GLTFExporter to produce scene.gltf and scene.bin.
  3. Writes a settings.json with the scene name, scale, and SDK version.
  4. Captures a screenshot.png from the provided camera/renderer for the dashboard thumbnail.

If your browser supports the File System Access API (modern Chrome/Edge), you'll be prompted to pick a directory and the files are written into a scene/ subfolder. Otherwise the adapter falls back to a scene-export.zip download.

Exporting a single dynamic object

c3dAdapter.exportObject(myObject, "car_mesh", renderer, camera);

This produces <objectName>.gltf, <objectName>.bin, and cvr_object_thumbnail.png. The thumbnail is rendered against a plain gray background with a temporary ambient light so the mesh reads cleanly.

Warning

Only export during development. Don't ship a build that prompts the user with a directory picker.

Step 8: Uploading Data to Cognitive3D

Upload your scene and dynamic object files using the Cognitive3D Upload Web App. This requires your Cognitive3D Developer Key.

After a successful scene upload, the web app displays the Scene ID and version, for example:

✅ Scene uploaded successfully!
   Scene ID:      a1b2c3d4-e5f6-g7h8i9j0
   Version:       1
   Scene Name:    MyThreeJSScene

Note

You can preview your exported dynamic objects with the glTF viewer by dragging the .zip or folder onto the page before uploading.

Step 9: Enter Your Cognitive3D Scene Data

Update settings.js with the sceneId and versionNumber returned by the Upload Web App:

allSceneData: [
  {
    sceneName: "MyThreeJSScene",
    sceneId: "a1b2c3d4-e5f6-g7h8i9j0",
    versionNumber: "1",
  },
],

Analytics data is only attached to the uploaded geometry when these values match, so keep them in sync whenever you re-upload.

Step 10: Events and Sensors

With the core plumbing in place, you can record custom events and sensor streams anywhere in your app code.

Custom events

Record something that happened at a specific moment, with an optional world-space position and a properties object:

c3d.customEvent.send("tutorial_step_completed", [0, 1, 0], { step: 3 });
c3d.customEvent.send("enemy_hit", enemy.position.toArray(), {
    weapon: "plasma_rifle",
    damage: 75,
    critical: true,
});

Sensors

Record a named value (number or boolean) over time. Sensors render as graphs on the session timeline.

c3d.sensor.recordSensor("heartRate", 85);
c3d.sensor.recordSensor("playerStamina", 92.5);
c3d.sensor.recordSensor("isMoving", true);

For a deeper dive on both features, see the Custom Events and Sensors pages. Performance sensors (draw calls, memory, FPS, frame time) are recorded automatically as long as you passed the renderer to the C3D constructor.

Step 11: Object Engagements

Engagements record high-level interaction states on a dynamic object — grabbing, pointing at, being in proximity to — including which controller or hand is doing it. Use these instead of custom events when you want the interaction to appear directly on the object in Session Replay.

// When the right hand grabs the lever
c3d.dynamicObject.beginEngagement("lever_01", "grab", "right_hand");

// When the right hand lets go
c3d.dynamicObject.endEngagement("lever_01", "grab", "right_hand");

Parameters:

  • objectId — the c3dId of the dynamic object being engaged with.
  • engagementName — a string you define (e.g. "grab", "point", "proximity").
  • parentId — typically the controller or hand id doing the engagement.

Step 12: Making Changes to Your Cognitive3D Project

As your experience evolves, you may add scene geometry or new dynamic objects. Re-export, re-upload using the Upload Web App, and update the versionNumber in settings.js to the new version. Existing sceneIds stay the same — only the version bumps.

intercom If you have a question or any feedback about our documentation please use the Intercom button (purple circle) in the lower right corner of any web page or join our Discord.