Skip to content

Dynamic Objects

The Dynamic Object component allows you to track the position and state of GameObjects during the user's session. These can be used to track controllers, AI characters, and interactive objects.

This page covers using the Dynamic Object component, Recording Engagements, Uploading meshes and Aggregating data.

Getting Started

For most implementations of Dynamic Objects, simply attaching a Dynamic Object component to your Unity GameObject is all you need to do.

Gaze tracking on a Dynamic Object uses Physics raycasts. Make sure you have colliders attached, and these colliders closely represent the surface of the object to record accurate gaze data.

If you have any VR controllers in your scene, please read the Controllers section of this page.

After you finished attaching the component to all relevant objects, you can start or continue the Scene Setup process.

If you wish to customize your implementation, feel free to read on, but you are likely done.

Controllers

Dynamic Objects for controllers can display the user's inputs in a popup window in SceneExplorer:

controllers

It is highly recommended that you use the Scene Setup process to automatically set up the required components for recording controller inputs. See Troubleshooting for a detailed manual setup.

Extra Dynamic Object Settings

There is generally no reason need to modify these settings, but they are available for specific edge cases:

Basic Settings

  • Use Custom Mesh (default: Exported Mesh Name) - Choose a mesh name to represent this gameobject on SceneExplorer. A default mesh name is automatically chosen when you add a Dynamic Object component. Make sure you share mesh names between Dynamic Objects where appropriate. For example: multiple car GameObjects might have the mesh names "car","car (1)","car (2)",etc. If these all use the same mesh renderer and materials, simplify the mesh name to "car" to help SceneExplorer load quickly.
  • Use Custom ID (default: GUID) - Identifies this specific Dynamic Object instance. This will be used for aggregating data across multiple sessions.

Mesh

  • Export Mesh - Exports the MeshRenderers from the selected gameobject to a temporary directory
  • Thumbnail from SceneView - Takes a screenshot from the Unity scene viewport to represent this mesh on the Dashboard.
  • Upload Mesh - Uploads the mesh and thumbnail from this directory to the current level (if the level has been uploaded to SceneExplorer).

Snapshot

  • Is Controller - Appends additional properties to this dynamic object to display controller inputs on SceneExplorer. See Troubleshooting for a detailed manual setup.
  • Sync with Gaze - Causes this Dynamic Object to record snapshots on the same interval as Gaze. For example, if the player is in a vehicle, this can make movement appear smoother on SceneExplorer.
  • Update Rate - How frequently this dynamic object will check if it has moved/rotated/scaled beyond a threshold and should record its new transformation.
  • Position Threshold - Meters the object must move to write a new snapshot.
  • Rotation Threshold - Degrees the object must rotate to write a new snapshot.
  • Scale Threshold - Percent the object must scale to write a new snapshot.

File Upload Management

A mesh representing a Dynamic Object is uploaded to SceneExplorer and saved to your scene on our cloud. If you have multiple scenes, you will have to upload mesh data to each scene or version. This will happen automatically during the normal Scene Setup upload process.

Updated Dynamic Objects

The Scene Setup wizard covers the conventional way to upload Dynamic Objects, however there are cases where you want to update or upload new Dynamic Objects without going through the entire Scene upload process. These cases include times during development where you may have forgot to tag certain objects, or if object meshes were incorrect.

When you have added Dynamic Object components to everything you want, open the Manage Dynamic Objects window from the cognitive3D menu. It will show all the Dynamic Object components in the scene. There may be warnings about colliders for gaze tracking. Select Upload Selected or Upload All to export these meshes and upload them to your scene.

For a Dynamic Object gaze data to be properly aggregated, a manifest of all objects must exist on our servers. This is done automatically when you upload Dynamic Object meshes, but you can also press Save & Sync with Server in the Manage Dynamic Objects window in case you run into issues.

Prefab Dynamic Objects

Having GameObject prefabs with Dynamic Object components works correctly with one exception - if you spawn multiple instances of one prefab, the ID used to identify it will be split between multiple instances and may not display correctly. In this case, it is important to uncheck 'UseCustomId'.

Aggregation over multiple sessions will not be calculated on Dynamic Objects that do not use custom ids.

Currently unsupported

  • Visualizing material or texture changes on Dynamic Objects
  • Particle Systems
  • Skeletal Animations
  • Mesh Deformation

Proxy Meshes

Some visual elements in Unity cannot be easily exported and reconstructed in SceneExplorer. One example is particle systems. You might also have a very detailed model that doesn't need to be rendered in high fidelity on SceneExplorer. Both of these could benefit from exporting a Proxy Mesh.

First set up the Dynamic Object you want to represent. Here, we want to record a 'fireball' made from a particle system.

fireball

Second, create a Dynamic Object to be the proxy. Then open the proxy Dynamic Object's Advanced Options and press Export. In this example, the proxy is a red sphere named 'fireball_proxy'. You can delete the proxy mesh from your scene afterward.

fireball_proxy

In the CognitiveVR_SceneExplorerExport/Dynamic/"ProxyMeshName" folder, rename the folder to match the desired Dynamic Object's Mesh Name. In this example, the folder would be renamed from 'fireball_proxy' to 'fireball'. In Unity, open the desired Dynamic Object's Advanced Options, press Upload, and you are done!

filerename

Dynamic Object Engagements

Custom Engagements allow you to record precise information about how a user manipulates the scene. An Engagement could represent a state such as grabbing an object, proximity to an object, pointing at an object, etc.

Engagements are used to represent a duration. See Custom Object Events for recording an immediate event related to a Dynamic Object.

To begin or end an Engagement, call the BeginEngagement or EndEngagement function on the Dynamic Object component. The example below shows how you could implement an Engagement into a grab event - this will differ depending on how you implement your interactions.

void OnControllerGrabBegin(GameObject controllerGameObject)
{
    //Begin Engagement of this grabbed object with a unique ID from the controller that instigated the grab
    var thisDynamic = GetComponent<CognitiveVR.DynamicObject>();
    var controllerDynamic = controllerGameObject.GetComponent<CognitiveVR.DynamicObject>();
    int id = -1;

    //including this id is optional. it is used to identify an engagement if there are multiple engagements with the same type occuring
    if (controllerDynamic != null) { id = controllerDynamic.GetId(); }
    if (thisDynamic != null) { thisDynamic.BeginEngagement("grab"); }


    //your code to 'grab' the object
    transform.SetParent(controllerGameObject.transform);
}

void OnControllerGrabEnd(GameObject controllerGameObject)
{
    //End Engagement of this grabbed object with a unique ID from the controller that instigated the grab
    var thisDynamic = GetComponent<CognitiveVR.DynamicObject>();
    var controllerDynamic = controllerGameObject.GetComponent<CognitiveVR.DynamicObject>();
    int id = -1;

    //including this id is optional. it is used to identify an engagement if there are multiple engagements with the same type occuring
    if (controllerDynamic != null) { id = controllerDynamic.GetId(); }
    if (thisDynamic != null) { thisDynamic.EndEngagement("grab", id); }


    //your code to 'drop' the object
    transform.SetParent(null);
}

Engagement statistics are available on both the Dashboard and SceneExplorer for both aggregated viewing and session-by-session statistics.

engagementstats

Code Reference

In almost every case the basic options on the component will be sufficient. However, you can also send 'snapshots' of the dynamic object manually.

void OnSomeEvent()
{
    GetComponent<CognitiveVR.DynamicObject>().RecordSnapshot();

    var properties = new Dictionary<string,object>(){{"key","value"}};
    GetComponent<CognitiveVR.DynamicObject>().RecordSnapshot(properties);
}

SceneExplorer

Dynamic Objects on SceneExplorer

Each Actor with a Dynamic Object component needs a visual representation for SceneExplorer. Each Dynamic Object with a different Mesh Name is uploaded during the Scene Setup. You can add new Dynamic Objects, upload new meshes and replace existing meshes without uploading a New Scene Version.

Uploading Dynamic Meshes - Component

upload component

Using the Dynamic Object component, you can set manually set a Mesh Name. Taking a Screenshot from the Unity Viewport will allow you to quickly identify the Dynamic Object on the Dashboard. This screenshot is shared between all Dynamic Objects with the same Mesh Name.

Uploading Dynamic Meshes - Preferences

From the cognitive3D->Manage Dynamic Objects, the Dynamic Mesh Upload section has options to quickly export multiple Dynamic Object meshes.

upload settings

You can either Upload X Selected Meshes, or Upload All Meshes. This will upload each Dynamic Mesh to your currently open Unity scene. This also happens automatically when you upload a New Scene or New Scene Version from the Cognitive Scene Setup Window.

Aggregating Data

Dynamic Object Gaze Data, and Dynamic Object Engagements can be aggregated per GameObject across multiple sessions. This happens automatically based on a list of all Unique Dynamic Object Ids in the level.

Simply press Upload Ids to SceneExplorer for Aggregation in this window if you add new Dynamic Objects to your level. This also happens when you upload a New Scene or New Scene Version from the Cognitive Scene Setup Window.