Skip to content

Dynamic Objects

The Dynamic Object component allows you to track the position and state of GameObjects during the Participant's session. These can be used to track controllers, non player characters (NPC), and interactive objects.

The Dynamic Object Id is an important concept - it uniquely identifies an object so it can be displayed in SceneExplorer. If the Id is consistent between multiple sessions it can be uploaded to the scene so Dynamic Object data can be aggregated and displayed on the Dashboard.

This page covers using the Dynamic Object component, Uploading Meshes, and advanced topics including Recording Engagements and Spawning Dynamic Objects.

Currently unsupported

  • Visualizing material or texture changes on Dynamic Objects
  • Particle Systems (see Proxy Meshes)
  • Skeletal Animations
  • Mesh Deformation

Getting Started

For most implementations of Dynamic Objects, simply attaching a Dynamic Object component to your Unity GameObject is all you need to do.

Usually you will add this component to all relevant objects in your scene during the Scene Setup process. You can often configure or change dynamic objects afterwards without having to re-upload the scene.

Tracking gaze and fixations on a Dynamic Object uses Physics raycasts. Make sure you have colliders attached, and these colliders closely represent the surface of the object to record accurate gaze data.

If you wish to customize your implementation, feel free to read on, but you are likely done.

Basic Settings

  • Use Custom Mesh (default: Exported Mesh Name) - Choose a mesh name to represent this GameObject on SceneExplorer. A default mesh name is automatically chosen when you add a Dynamic Object component. Make sure you share mesh names between Dynamic Objects where appropriate. For example: multiple car GameObjects might have the mesh names "car","car (1)","car (2)", etc. If these all use the same mesh renderer and materials, simplify the mesh name to "car" to help SceneExplorer load quickly.
  • Id Source (default: GUID) - Identifies this specific Dynamic Object instance. This will be used for aggregating data across multiple sessions. Other options exist for when spawning objects during runtime. See Spawning Dynamic Objects for details.

Mesh

  • Export Mesh - Exports the MeshRenderers from the selected GameObject to a temporary directory, and generates a basic thumbnail.
  • Thumbnail from SceneView - Takes a screenshot from the Unity Viewport to represent this mesh on the Dashboard.
  • Upload Mesh - Uploads the mesh and thumbnail from this directory to the current level (if the level has been uploaded to SceneExplorer).

Data Snapshot

  • Is Controller - Appends additional properties to this dynamic object to display controller inputs on SceneExplorer. See Troubleshooting for a detailed manual setup.
  • Sync with Gaze - Causes this Dynamic Object to record data on the same interval as Gaze. For example, if the participant is in a vehicle, this can make movement appear smoother on SceneExplorer.
  • Update Rate - How frequently this dynamic object will check if it has moved/rotated/scaled beyond a threshold and should record its new transformation.
  • Position Threshold - Meters the object must move to record new data.
  • Rotation Threshold - Degrees the object must rotate to record new data.
  • Scale Threshold - Percent the object must scale to record new data.

Setup and Uploading

Dynamic Object Meshes

A mesh representing a Dynamic Object is uploaded to SceneExplorer and saved to your scene on our cloud. If you have multiple scenes, you will have to upload mesh data to each scene or version. This will happen automatically during the Scene Setup. You can add new Dynamic Objects, upload new meshes and replace existing meshes without uploading a New Scene Version.

Uploading Dynamic Meshes - Component

upload component

Using the Dynamic Object component, you can set manually set a Mesh Name. Taking a Screenshot from the Unity Viewport will allow you to quickly identify the Dynamic Object on the Dashboard. This screenshot is shared between all Dynamic Objects with the same Mesh Name.

Uploading Dynamic Meshes - Preferences

From the cognitive3D->Manage Dynamic Objects, the Dynamic Mesh Upload section has options to quickly export and upload multiple Dynamic Object meshes.

upload settings

You can either Upload X Selected Meshes, or Upload All Meshes. This will upload each Dynamic Mesh to your currently open Unity scene. This also happens automatically when you upload a New Scene or New Scene Version from the Cognitive Scene Setup Window.

Prefab Dynamic Objects

Having GameObject prefabs with Dynamic Object components works correctly with one exception - if you spawn multiple instances of one prefab, the ID used to identify it will be the same between multiple instances and will not display correctly in SceneExplorer. In this case, it is important to set Id Source to not 'UseCustomId'. See Spawning Dynamic Objects for details.

Advanced Use Cases

Aggregating Data between scene

If a Dynamic Object has identical Dynamic Object Ids between sessions, this data can be aggregated and displayed on the Dashboard. These will appear in the Dynamic Objects sidebar. The list of which objects should be aggregated must be set manually, but with the tools in the SDK this takes virtually no time.

First is the Scene Setup Window. This will add all Dynamic Objects in the scene when the scene is uploaded to this Dynamic Object Dashboard list.

Second (and more convenient) is the Manage Dynamic Objects Window. This can be accessed from the cognitive3D->Manage Dynamic Objects menu.

manage dynamic object window

This displays a list of all Dynamic Objects in the scene, including the Mesh Name, GameObject and if the Dynamic Object has been exported. You can click on any of the names to select the GameObject.

The options at the bottom allow you to select the texture resolution when exporting the Dynamic Object Mesh. There are buttons for Exporting (if not exported already) and Uploading a Dynamic Object Mesh to the Dashboard to be used in the Scene that is currently open.

Finally, there is a button to append the IDs of all Dynamic Objects currently enabled in the scene. These will be displayed on the dashboard.

Proxy Meshes

Some visual elements in Unity cannot be easily exported and reconstructed in SceneExplorer. One example is particle systems. You might also have a very detailed model that doesn't need to be rendered in high fidelity on SceneExplorer. Both of these could benefit from exporting a Proxy Mesh.

First set up the Dynamic Object you want to represent. Here, we want to record a 'Fire' made from a particle system.

fire_proxy

Second, create a Dynamic Object to be the proxy. Then open the proxy Dynamic Object's Advanced Options and press Export. In this example, the proxy is a red sphere named 'Fire_Proxy'. You can delete the proxy mesh from your scene afterward.

In the CognitiveVR_SceneExplorerExport/Dynamic/ folder, rename the folder to match the desired Dynamic Object's Mesh Name. In this example, the folder would be renamed from 'fire_proxy' to 'fire'. In Unity, open the desired Dynamic Object's Advanced Options, press Upload, and you are done!

filerename

Spawning Dynamic Objects

The Dashboard includes a list of all know Dynamic Objects in each scene. These can share meshes, but every instance will have a unique ID. This 'manifest' of unique ids is required for aggregating information between sessions.

If a Dynamic Object is not in the scene when this manifest is being uploaded to the Dashboard, it will not have it's data aggregated. In some cases this may be fine - the Dynamic Object might only be relavent in SceneExplorer within the context of the scene. If this is true, you can set Id Source to 'Generate Id' from the Dynamic Object Component on your prefab.

dynamic id source

If you need to spawn Dynamic Objects and need to have their data aggregated, you should consider using a Dynamic Object Id Pool. Set Id Source to 'Pool Id' from the Dynamic Object Component on your prefab. You can create a new DynamicObjectIdPool asset that will hold several Ids that will be used when new instances of this prefab are instantiated. Press the Upload Ids button on the Dynamic Object Id Pool asset to add all of the Ids to the scene's manifest.

dynamic id pool

In rare cases, it may be best to avoid spawning prefabs and simply enable existing Dynamic Objects in the scene. These can use the 'Custom Id' option from the Dynamic Object Component.

Engagement Events

Engagements offer a different way to record interactions with your Dynamic Objects. Internally, they are implemented as Custom Events, but automatically keep a reference to active Custom Events and can be easier to manage.

Custom Engagements allow you to record precise information about how a participant manipulates the scene. An Engagement could represent a state such as grabbing an object, proximity to an object, pointing at an object, etc.

Engagements are used to represent a duration. See Custom Object Events for recording an immediate event related to a Dynamic Object.

To begin or end an Engagement, call the BeginEngagement or EndEngagement function on the Dynamic Object component. The example below shows how you could implement an Engagement into a grab event - this will differ depending on how you implement your interactions.

void OnControllerGrabBegin(GameObject controllerGameObject)
{
    //Begin Engagement of this grabbed object with a unique ID from the controller that instigated the grab
    var thisDynamic = GetComponent<CognitiveVR.DynamicObject>();
    var controllerDynamic = controllerGameObject.GetComponent<CognitiveVR.DynamicObject>();
    int id = -1;

    //including this id is optional. it is used to identify an engagement if there are multiple engagements with the same type occuring
    if (controllerDynamic != null) { id = controllerDynamic.GetId(); }
    if (thisDynamic != null) { thisDynamic.BeginEngagement("grab"); }


    //your code to 'grab' the object
    transform.SetParent(controllerGameObject.transform);
}

void OnControllerGrabEnd(GameObject controllerGameObject)
{
    //End Engagement of this grabbed object with a unique ID from the controller that instigated the grab
    var thisDynamic = GetComponent<CognitiveVR.DynamicObject>();
    var controllerDynamic = controllerGameObject.GetComponent<CognitiveVR.DynamicObject>();
    int id = -1;

    //including this id is optional. it is used to identify an engagement if there are multiple engagements with the same type occuring
    if (controllerDynamic != null) { id = controllerDynamic.GetId(); }
    if (thisDynamic != null) { thisDynamic.EndEngagement("grab", id); }


    //your code to 'drop' the object
    transform.SetParent(null);
}

Controllers

Dynamic Objects for controllers can display the participant's inputs in a popup window in SceneExplorer:

controllers

It is highly recommended that you use the Scene Setup process to automatically set up the required components for recording controller inputs. See Troubleshooting for a detailed manual setup.

Engagement statistics are available in aggregate on the Dashboard and per-session in SceneExplorer.

Manually recording data

In almost every case the basic options on the component will be sufficient. However, you can also send data 'snapshots' of the dynamic object manually.

void OnSomeEvent()
{
    //records the postiion, rotation and scale of the object
    GetComponent<CognitiveVR.DynamicObject>().RecordSnapshot();
}