Dynamic Objects

The Dynamic Object component allows you to track the positions and states of actors during the Participant's Session. These can be used to track controllers, AI characters, interactive objects and even environments.

Currently unsupported

• Visualizing material or texture changes on Dynamic Objects
• Particle Systems
• Skeletal Animations
• Mesh Deformation

Dynamic Object Component

Buttons

• Generate Mesh Name and Unique ID - Quickly set a Mesh Name from a Static Mesh Component on this actor. Also sets a unique Custom Id.
• Export Mesh - Export the Static Mesh Component to represent this Actor on SceneExplorer. Disabled if the Export folder has not been set from the Scene Setup Window or Preferences.
• Take Screenshot - Takes a screenshot from the Unreal viewport to represent this mesh on the Dashboard.
• Upload Mesh - Uploads the exported Mesh to the current Level (disabled if the Level has not been uploaded to SceneExplorer).

Basic

• Use Custom Mesh Name - Enables a custom mesh to represent this Actor on SceneExplorer. Otherwise this Actor will use the Common Mesh Name to represent this Actor.
• Mesh Name - The name of the mesh to represent this actor on SceneExplorer.

Advanced

• Common Mesh Name - If you have not enabled Use Custom Mesh Name, you can select one of these common VR meshes. You do not need to upload this to SceneExplorer.
• Snapshot On Begin Play - When BeginPlay is called on the component in Unreal, this will record a 'snapshot' of the Actor at the current position and rotation.
• Snapshot On Interval - This will continually check if the object has moved and needs to record a snapshot of the current position and rotation.

• Release Id on Destroy - Allow other Dynamic Objects to use this Id when this actor is destroyed. This can optimize performance on SceneExplorer and should generally be enabled.

• Id Source Type - Determines how this Dynamic Object is uniquely identified. In most cases, Custom Id is the simplest option.

• Custom Id - This identifies a specific object using a GUID.

• Id Pool - This references an asset with an array of GUIDs. See Spawning Dynamic Objects below.

• Snapshot Interval - Used if Snapshot On Interval is true. This is the delay in seconds before checking if the Actor moved beyond its threshold.

• Position Threshold - Centimeters the Actor must move to record a new snapshot. Checked each SnapshotInterval.
• Rotation Threshold - Degrees the Actor must rotate to record a new snapshot. Checked each SnapshotInterval.
• Scale Threshold - Percent of scale change that must be reached to record a new snapshot. Checked each SnapshotInterval.

Advanced Dynamic Objects

Manually Recording Data

In most cases the options on the Component will be sufficient. However, you can also record 'snapshots' of the Dynamic Object Component manually. This is mostly useful if you are not automatically updating the position and rotation by using the SnapshotOnInterval option. This can also be useful if you are setting custom parameters before sending the first snapshot.

BlueprintC++

//.h
#include "CognitiveVR/Public/CognitiveVR.h"

//.cpp
void AMyCustomActor::StartupComplete()
{
    TArray<UDynamicObject*> Comps;
    GetComponents(Comps);
    if (Comps.Num() > 0)
    {
        UDynamicObject* FoundComp = Comps[0];
        FDynamicObjectSnapshot snapshot = FoundComp->MakeSnapshot(false);
        FoundComp->SnapshotFloatProperty(snapshot, "initproperty", 5.0);
        FoundComp->SendDynamicObjectSnapshot(snapshot);

        //if you want to record changes on an interval after this custom startup
        //you can simply set SnapshotOnInterval = true
        FoundComp->SnapshotOnInterval = true;
    }
}

Engagements

Engagements allow you to record precise information about how a participant manipulates the scene. An Engagement could represent a state such as grabbing an Actor, proximity to an Actor, or pointing at an Actor. Internally, these are implemented as Custom Events with this Dynamic Object as the target.

BlueprintC++

    UDynamicObject* dynamic = AActor::FindComponentByClass<UDynamicObject>();

    UDynamicObject::BeginEngagement(dynamic, "Grab");
    UDynamicObject::EndEngagement(dynamic, "Grab");

Controllers

Motion Controllers can be configured to record Participant inputs in addition to position and rotation.

Setup Controller

If you are using the VR Bluepring template from Unreal Engine, call Setup Controller in the VRPawn blueprint after each controller is spawned. Connect the MotionControllerObject reference as the Target, configure the controller as the right/left hand, and choose the type of controller. Repeat this for the Motion Controller for the other hand.

If you are using the OpenXR template in 4.27, call Setup Controller in EventOnBeginPlay targetting the MotionController components on the VRPawn

Input Tracker Actor

You will need to add an Input Tracker Actor to your level to record inputs from your controllers. In the Content Browser, select the option to show C++ classes and Plugin Content to see this actor.

Key Bindings

You will need to add several bindings to your Config/DefaultInput.ini file. If you are using Unreal 4.24 or later, copy this into your Default Input file. For older versions, copy this

You will see the controllers as Dynamic Objects in SceneExplorer. You will also see the Participant's inputs in the popup in the bottom left corner.

Spawning Dynamic Objects

By default, GUIDs are used to uniquely identify the same Dynamic Object across multiple Session. These are set in the Unreal Editor. If you spawn blueprints during runtime, the Custom Id will be incorrect because the same Id will be representing two different objects. In this case, choose Dynamic Object Id Pool. This is an asset that holds a number of GUIDs. When a blueprint is spawned, it will take an unused value from an array on this asset. This will allow spawned Dynamic Object to have consistent Ids so data can be aggregated.

To configure this, first create a Dynamic Id Pool Asset in your project:

Set the Mesh Name. This mesh will be used when visualizing the object in SceneExplorer or Object Explorer. Make sure you upload the Mesh Geometry to the dashboard. The best method is to add the spawned actor (with the Dynamic Object component) to the level and press the Export and Upload buttons. Then you can delete this actor afterward.

The Prefab Name will be displayed as the name of the object in the Dynamic Object gallery view on the Dashboard.

Press Add Id to create a new GUID. When Dynamic Objects are spawned, they will take a value from this array. You should include as many as you expect will spawn in the session. If you include too few, some objects will not aggregate data.

Press Upload Ids for Aggregation. This will ensure data recorded on these objects will be aggregated on the Dashboard.

In Unreal's Blueprint Editor, select the Dynamic Object component and select Pool Id from the Id Source Type. Then set the ID Pool property to the asset you created.

Proxy Meshes

Some visual elements in Unreal cannot be easily exported and reconstructed in SceneExplorer. One example is particle systems. Or you might also have a very detailed model that doesn't need to be rendered in high fidelity on SceneExplorer. Both of these could benefit from exporting a Proxy Mesh.

First set up the Dynamic Object you want to represent. Here, we want to record 'my_particle' made from a particle system.

Second, create a Dynamic Object to be the proxy. Then open the proxy Dynamic Object's Advanced Options and press Export Mesh. In this example, the proxy is a red sphere named 'my_particle_proxy'. You can delete the proxy mesh from your scene afterward.

In your Export/dynamic/ folder, rename the folder to match the desired Dynamic Object's Mesh Name. In this example, the folder would be renamed from 'my_particle_proxy' to 'my_particle'. In Unreal, open the desired Dynamic Object's Advanced Options, press Upload Mesh, and you are done!

SceneExplorer

Dynamic Objects on SceneExplorer

Each Actor with a Dynamic Object Component needs a visual representation for SceneExplorer. Each Dynamic Object with a different Mesh Name is uploaded during the Scene Setup. You can add new Dynamic Objects, upload new meshes and replace existing meshes without uploading a New Scene Version.

Uploading Dynamic Meshes - Component

On a Dynamic Object Component, you can automatically or manually set a Mesh Name. Taking a Screenshot from the Unreal Editor Viewport will allow you to quickly identify the Dynamic Object on the Dashboard. This screenshot is shared between all Dynamic Objects with the same Mesh Name. If the Export Mesh button is disabled, you likely need to choose an Export Directory in Preferences.

Uploading Dynamic Meshes - Dynamic Object Manager Window

Open the Cognitive Dynamic Object Manager Window from the Windows menu. This window lists all the Dynamic Objects in your scene and provides tools to sort, export and upload Dynamic Objects to the Dashboard

At the top is a collapsable Settings section. This is configured automatically when you complete the Scene Setup Window. If some of the buttons are disabled, check that these settings are correct.

In the middle is a list of all Actors with Dynamic Object Components in your scene. This displays the Actor Name, Mesh Name, Id and if the Mesh has been exported.

At the bottom are buttons to Upload Meshes to the Dashboard (this will also export the meshes to your temporary folder if they have not been already). Uploading meshes allows the Dynamic Object to be represented in SceneExplorer and on the Object Explorer page on the Dashboard.

At the very bottom is a button to Validate Mesh Names and Unique Ids. This is used for consistently identifying Dynamic Objects across multiple Sessions so data can be aggregated.

Aggregating Data

Fixation Data, Gaze Data and Custom Events on Dynamic Objects can be aggregated across multiple Session. This requires that a Scene has a list of all Dynamic Objects that can appear. In Dynamic Object Manager Window (above), press Upload Dynamic Object Ids to SceneExplorer for Aggregation.

If you add new Dynamic Objects to your level, you should repeat this step.