Mattercraft Integration

Cognitive3D integrates with Mattercraft through a dedicated NPM package (@cognitive3d/three-mattercraft) that provides native Mattercraft behaviors with a visual properties panel, automatic session management, dynamic object tracking, and scene/object export.

Requirements

• Cognitive3D Account: You'll need an active Cognitive3D account to obtain your API keys and set up your project. You can sign up at the Cognitive3D platform website.

Note

You can quickly find your API keys on Cognitive3D with ctrl/cmd + K to pull up the search menu and then searching for Manage Developer Key. Note the difference between the Developer Key (Used for uploading assets to Cognitive3D) and the Application Key (Used in your Applications to allow your project to send data to our servers).

• Mattercraft Project: An existing Mattercraft project with a WebXR scene. You can sign up on Mattercraft.

Step 1: Install the Cognitive3D NPM Package

1.1 In the Mattercraft project editor, click Add-ons and Dependencies in the left sidebar.

1.2 Click Browse.

1.3 Select From NPM, then search for @cognitive3d/three-mattercraft and press Install.

Step 2: Add the Cognitive3D Behavior to the Scene Hierarchy

Select your Scene (scene.zcomp), select an object in the hierarchy, and under the Behaviors tab press the (+) button. Find and add the Cognitive3D behavior.

This behavior is the Manager. It should exist once in your scene and handles communication with the Cognitive3D platform. The properties panel exposes the following fields:

• Api Key: Your Cognitive3D Application API key.
• Scene Id: The Scene ID from the Cognitive3D dashboard.
• Scene Name: The name of your scene.
• Scene Version: The version number of your uploaded scene.
• App Version: Your application's version string (default: "1.0").
• Enable Export: Toggle to enable scene and dynamic object export at runtime.
• Enable Debug: Toggle to enable verbose debug logging in the browser console.

Note

Initially you will not have all of this data. You can access an API key immediately if you have a Cognitive3D account. The Scene ID, Scene Name, and Scene Version are obtained after uploading your scene geometry.

Step 3: Marking Dynamic Objects

The Cognitive3D platform distinguishes between static geometry (your environment) and dynamic objects (objects that move or that you want heatmap data on).

Mark dynamic objects by attaching the Cognitive3DDynamicObject behavior to them.

Where to Attach the Behavior

Where you attach the behavior depends on whether the object is a ZComponent or not. You can identify a ZComponent by the lightning bolt icon next to the object in the hierarchy. You can think of a ZComponent in Mattercraft as the equivalent of a Prefab in Unity or a Blueprint in Unreal Engine.

3.1 Dynamic Objects on ZComponents

For ZComponents (objects with the lightning bolt icon), attach the behavior inside the ZComponent definition in the Project window, not in the scene hierarchy. Ensure you enter a matching C3D Mesh Name into the field (e.g., WarehouseBox).

3.2 Dynamic Objects on non-ZComponents

For standard scene objects (no lightning bolt), attach the behavior directly to the object in the scene hierarchy. Again, ensure you enter a matching Mesh Name into the field (e.g., Forklift).

3.3 Dynamic Objects on Attachment Points (children of glb files)

Cognitive3D does not track animations directly, but you can track the transform of animated sub-components (e.g., the moving platform of a forklift). To do this:

1. Right-click the animated glb node in the hierarchy.
2. Select New > AttachmentPoint.
3. In the AttachmentPoint's Node Properties, set Attach To to the child object name you want to track (e.g., hydraulics_front_13).
4. Attach a Cognitive3DDynamicObject behavior to this AttachmentPoint.

3.4 Mesh Name, CustomID, and Transform Thresholds

The Cognitive3DDynamicObject behavior exposes four properties:

• C3D Mesh Name: Must match the mesh name uploaded to the Cognitive3D dashboard. This tells the platform which 3D model this object represents. After attaching the behavior, always enter a C3D Mesh Name.

• C3D Custom ID: Must be unique between objects of the same mesh. If left blank, a deterministic ID is automatically generated using a hash of the scene name and mesh name. You can also set your own Custom ID for clarity — for example, if you have 10 identical boxes they share the same C3D Mesh Name, but you can give each a unique Custom ID such as 1, 2, 3... 10.

• Position Threshold (default: 0.1): Minimum position change before a new snapshot is sent.

• Rotation Threshold (default: 0.1): Minimum rotation change before a new snapshot is sent.

Increasing these thresholds reduces the frequency of snapshots, which may slightly improve performance but results in less precise Cognitive3D session replays. The default values of 0.1 are recommended for most use cases.

Step 4: Exporting Scene and Dynamic Objects

Once all dynamic objects are marked, you can export the scene geometry and dynamic object models for upload to the Cognitive3D dashboard.

1. On the Cognitive3D Manager behavior, check the Enable Export toggle, then save.
2. Start the game in Live Preview (Open In Simulator).
3. Press Shift + E to export the scene.
4. Press Shift + D to export the dynamic objects.

The scene and dynamic object files are saved as compressed .zip archives into your Downloads folder.

Warning

Ensure you disable the Enable Export toggle and save after exporting.

You can visualize your dynamic objects before uploading your scene/ dynamic objects data to Cognitive3D using

Step 5: Uploading Data to Cognitive3D

You can upload your scene and dynamic objects data using the Cognitive3D Upload Web App. This will require your Cognitive3D Developer Key.

Note

Before uploading, you can visualize your Dynamic Objects using the glTF viewer. Drag and drop your .zip files onto the viewer.

Step 6: Enter Your Cognitive3D Scene Data

Once you have uploaded your scene via the Cognitive3D website, go back to the Cognitive3D Manager behavior and enter your API Key, Scene ID, Scene Name, and Scene Version.

Step 7: Events and Sensors

Within your other behaviour scripts/ game logic files, you can implement sensors and events by importing Cognitive3DContext with

import { Cognitive3DContext } from "@cognitive3d/three-mattercraft";

Then get the Context once in your constructor or started() with:

const ctx = this.contextManager.getExisting(Cognitive3DContext)

Sensors

Sensors record a named value over time (e.g., speed, rotation, temperature):

ctx?.recordSensor("lever.rotation", 45); // value can be a number or boolean
// The value can be a number or a boolean.

Custom Events

Events record something that happened at a specific moment, with an optional 3D position and custom properties:

ctx?.sendEvent("StepCompleted", [0, 1, 0], { step: 1 });

Step 8: Making changes to your Cognitive3D Project

As your game/ experience evolves, you may add scene geometry or new Dynamic Objects. When this happens, re-upload the scene and any new Dynamic Object data with the Cognitive3D Upload Web App.

If you upload new scene data to an existing Cognitive3D project, update the Scene Version to the latest version under Cognitive3D Behaviour in Mattercraft.

If you have a question or any feedback about our documentation please use the Intercom button (purple circle) in the lower right corner of any web page or join our Discord.