Dynamic Objects

The Dynamic Object component allows you to track the position and state of GameObjects during the user's session. These can be used to track controllers, AI characters, and interactive objects.

This page covers using the Dynamic Object component, Recording Engagements, Uploading meshes and Aggregating data.

Getting Started

For most implementations of Dynamic Objects, simply attaching a Dynamic Object component to your Unity GameObject is all you need to do.

Gaze tracking on a Dynamic Object uses Physics raycasts. Make sure you have colliders attached, and these colliders closely represent the surface of the object to record accurate gaze data.

If you have any VR controllers in your scene, please read the Controllers section of this page.

After you finished attaching the component to all relevant objects, you can start or continue the Scene Setup process.

If you wish to customize your implementation, feel free to read on, but you are likely done.

Controllers

Dynamic Objects for controllers can display the user's inputs in a popup window in SceneExplorer:

controllers

It is highly recommended that you use the Scene Setup process to automatically set up the required components for recording controller inputs. See Troubleshooting for a detailed manual setup.

Extra Dynamic Object Settings

There is generally no reason need to modify these settings, but they are available for specific edge cases:

Basic Settings

  • Use Custom Mesh (default: Exported Mesh Name) - Choose a mesh name to represent this gameobject on SceneExplorer. A default mesh name is automatically chosen when you add a Dynamic Object component. Make sure you share mesh names between Dynamic Objects where appropriate. For example: multiple car GameObjects might have the mesh names "car","car (1)","car (2)",etc. If these all use the same mesh renderer and materials, simplify the mesh name to "car" to help SceneExplorer load quickly.
  • Track Gaze on Dynamic Object (default: True) - Gaze on this object will be tracked. You must set a collider on this GameObject for the camera to correctly track the gaze on the object.

Mesh

  • Export Mesh - Exports the MeshRenderers from the selected gameobject to a temporary directory
  • Thumbnail from SceneView - Takes a screenshot from the Unity scene viewport to represent this mesh on the Dashboard.
  • Upload Mesh - Uploads the mesh and thumbnail from this directory to the current level (if the level has been uploaded to SceneExplorer).

ID

  • Use Custom ID - This identifies a specific instance of an object.
  • Group Name - Group Names are used to identify similar items. If you have a number of dynamic objects that share a Group Name, each will still behave as seperate unique objects. However, when you can view heatmaps aggregated from all objects in the group.
  • Release ID OnDisable - Allow other dynamic objects to use this Id when this gameobject becomes disabled.
  • Release ID OnDestroy - Allow other dynamic objects to use this Id when this gameobject becomes destroyed.

Snapshot

  • Requires Manual Enable - Disables recording this Dynamic Object until Init() is called
  • Continually Record Transform - This will continually record the position and rotation of the object when it moves past a certain threshold.
  • Sync with Player Update - This is the 'Interval for Player Snapshots' in the Tracker Options Window. Alternatively, you can set an interval for this Dynamic Object.
  • Position Threshold - Meters the object must move to write a new snapshot. Checked each 'Tick'.
  • Rotation Threshold - Degrees the object must rotate to write a new snapshot. Checked each 'Tick'.

Video Player

  • Video Player - Automatically records events from a video player component. If you wish to track gaze on a Video Component, see the Media page.

Tracking Controllers

Note

The SDK does attempt to automatically attach Dynamic Object components to controllers found in the Scene hierarchy, however this step is required when controller names or hierarchy position are different than default.

File Upload Management

A mesh representing a Dynamic Object is uploaded to SceneExplorer and saved to your scene on our cloud. If you have multiple scenes, you will have to upload mesh data to each scene or version. This will happen automatically during the normal Scene Setup upload process.

Updated Dynamic Objects

The Scene Setup wizard covers the conventional way to upload Dynamic Objects, however there are cases where you want to update or upload new Dynamic Objects without going through the entire Scene uplaod process. These cases include times during development where you may have forgot to tag certain objects, or if object meshes were incorrect.

When you have added Dynamic Object components to everything you want, open the Manage Dynamic Objects window from the cognitive3D menu. It will show all the Dynamic Object components in the scene. There may be warnings about colliders for gaze tracking. Select Upload Selected or Upload All to export these meshes and upload them to your scene.

For a Dynamic Object gaze data to be properly aggregated, a manifest of all objects must exist on our servers. This is done automatically when you upload Dynamic Object meshes, but you can also press Save & Sync with Server in the Manage Dynamic Objects window in case you run into issues.

Prefab Dynamic Objects

Having GameObject prefabs with Dynamic Object components works correctly with one exception - if you spawn multiple instances of one prefab, the ID used to identify it will be split between multiple instances and may not display correctly. In this case, it is important to uncheck 'UseCustomId'.

Aggregation over multiple sessions will not be calculated on Dynamic Objects that do not use custom ids.

Currently unsupported

  • Visualizing material or texture changes on Dynamic Objects
  • Custom scale for dynamic objects (such as room scale) in SceneExplorer

Dynamic Object Engagements

Custom Engagements allow you to record precise information about how a user manipulates the scene. An Engagement could represent a state such as grabbing an object, proximity to an object, pointing at an object, etc.

Engagements are used to represent a duration. See Custom Object Events for recording an immediate event related to a Dynamic Object.

To begin or end an Engagement, call the BeginEngagement or EndEngagement function on the Dynamic Object component. The example below shows how you could implement an Engagement into a grab event - this will differ depending on how you implement your interactions.

void OnControllerGrabBegin(GameObject controllerGameObject)
{
    //begin Engagement
    var thisDynamic = GetComponent<CognitiveVR.DynamicObject>();
    var controllerDynamic = controllerGameObject.GetComponent<CognitiveVR.DynamicObject>();
    int id = -1;

    //including this id is optional. it is used to identify an engagement if there are multiple engagements with the same type occuring
    if (controllerDynamic != null) { id = controllerDynamic.Id; }
    if (thisDynamic != null) { thisDynamic.BeginEngagement("grab", id); }


    //your code to 'grab' the object
    transform.SetParent(controllerGameObject.transform);
}

void OnControllerGrabEnd(GameObject controllerGameObject)
{
    //end Engagement
    var thisDynamic = GetComponent<CognitiveVR.DynamicObject>();
    var controllerDynamic = controllerGameObject.GetComponent<CognitiveVR.DynamicObject>();
    int id = -1;

    //including this id is optional. it is used to identify an engagement if there are multiple engagements with the same type occuring
    if (controllerDynamic != null) { id = controllerDynamic.Id; }
    if (thisDynamic != null) { thisDynamic.EndEngagement("grab", id); }


    //your code to 'drop' the object
    transform.SetParent(null);
}

Engagement statistics will now be available on SceneExplorer for both aggregated viewing and session-by-session statistics.

engagementstats

Code Reference

In almost every case the basic options on the component will be sufficient. However, you can also send 'snapshots' of the dynamic object manually.

void Start()
{
    var snap = GetComponent<CognitiveVR.DynamicObject>().NewSnapshot();

    snap.UpdateTransform(); //updates the position of the dynamic object
    snap.SetTick(false); //start or stop the Tick coroutine that tries to automatically update the position

    snap.SetProperties(Dictionary<string,object>(){{"key","value"}}); //you can add addititional information to snapshots. this is currently unused
    snap.SetEnabled(false); //show or hide the dynamic object. This is automatically disabled when the gameobject becomes disabled or destroyed. It is automatically enabled when the gameobject becomes enabled and SnapshotOnEnable is true
}

SceneExplorer

Dynamic Objects on SceneExplorer

Each Actor with a Dynamic Object component needs a visual representation for SceneExplorer. Each Dynamic Object with a different Mesh Name is uploaded during the Scene Setup. You can add new Dynamic Objects, upload new meshes and replace existing meshes without uploading a New Scene Version.

Uploading Dynamic Meshes - Component

upload component

Using the Dynamic Object component, you can set manually set a Mesh Name. Taking a Screenshot from the Unity Viewport will allow you to quickly identify the Dynamic Object on the Dashboard. This screenshot is shared between all Dynamic Objects with the same Mesh Name.

Uploading Dynamic Meshes - Preferences

From the cognitive3D->Manage Dynamic Objects, the Dynamic Mesh Upload section has options to quickly export multiple Dynamic Object meshes.

upload settings

You can either Upload X Selected Meshes, or Upload All Meshes. This will upload each Dynamic Mesh to your currently open Unity scene. This also happens automatically when you upload a New Scene or New Scene Version from the Cognitive Scene Setup Window.

Aggregating Data

Dynamic Object Gaze Data, and Dynamic Object Engagements can be aggregated per GameObject across multiple sessions. This happens automatically based on a list of all Unique Dynamic Object Ids in the level.

Simply press Upload Ids to SceneExplorer for Aggregation in this window if you add new Dynamic Objects to your level. This also happens when you upload a New Scene or New Scene Version from the Cognitive Scene Setup Window.