Unreal Notifications & Annotations Guide

This guide was written as a quick reference step by step guide for setting up a simple Unreal notification/annotation. For more in depth information on each step please check the documentation at https://www.genvidtech.com/for-developers/sdk-latest/

C++ Implementation

1. In BeginPlay we need to add 2 more streams. One for notifications and one for annotations. Create two functions. One called ClockNotificationStream and one called ClockAnnotationStream.

   • Both return a FGenvidStream and have no inputs

2. ClockNotificationStream should do the following:

3.     FGenvidStream ClockNotification;
       ClockNotification.Name = "ClockNotification";
       ClockNotification.OnGenvidStreamDelegate.BindUFunction(this, "OnSubmitClockNotification");
       ClockNotification.Framerate = 30;
   
       return ClockNotification;

4. We create a new genvid stream named “ClockNotification”

5. We bind it to “OnSubmitClockNotification” which we will create later

6. We set it’s framerate to 30

7. ClockAnnotationStream should do the following:

8.     FGenvidStream ClockAnnotation;
       ClockAnnotation.Name = "ClockAnnotation";
       ClockAnnotation.OnGenvidStreamDelegate.BindUFunction(this, "OnSubmitClockAnnotation");
       ClockAnnotation.Framerate = 30;
   
       return ClockAnnotation;

9. This function works exactly the same as above but instead binds to “OnSubmitClockAnnotation” which we will create later

10. Create two properties on the GenvidStream class of type bool called “ShouldSendNotification” and “ShouldSendAnnotation”.

    • When these booleans are true we will submit an notification or annotation

11. Create another property on the GenvidStream class of type int32 called “PreviousSecondUsed”

    • This will help track the notification or annotation to send

12. Override the “TickComponent” function of UGenvidStreams.

    • It has a return type of void

    • It has 3 parameters: float DeltaTime, enum ElevelTick TickType, and FActorComponenetTickFunction* ThisTickFunction

13. Due to the tick component on UGenvidStreams being private. We cannot call Super::TickComponent on it. This is scheduled to be changed soon. For now we can get around it by going to the function and copy and pasting all of its contents and replacing it’s super call with a call to UActorComponent::TickComponenet:

14. // UGenvidStreams Tick component is private. so have to do this for now (It will be protected in the future)
    UActorComponent::TickComponent(DeltaTime, TickType, ThisTickFunction);
    if (bIsCreated)
    {
          
        ReceiveTick(DeltaTime);
        for (auto& stream : Streams)
        {
          
            stream.TriggerStream(DeltaTime);
        }
    }

15. Add the following code after the above copy and pasted code from its parent:

16.     // Actual new code:
        int32 CurrentSecond = FDateTime::Now().GetSecond();
        if (PreviousUsedSecond != CurrentSecond && CurrentSecond % 15 == 0)
        {
          
            PreviousUsedSecond = CurrentSecond;
            ShouldSendNotification = true;
            ShouldSendAnnotation = true;
        }

17. We get the current second every tick. The first time it’s 0, 15, 30, or 45 we flip the booleans created. We also track what second flipped the booleans.

18. Create two new functions “OnSubmitClockNotification” and “OnSubmitClockAnnotation”

    • Both take in an FString named Id as a parameter and have a return type of void

19. OnSubmitClockNotification should do the following:

20.     if (!ShouldSendNotification)
        {
          
            return;
        }
    
    
        ShouldSendNotification = false;
    
    
        FClockTime ClockTime;
        ClockTime.Second = PreviousUsedSecond;
        FString ClockTimeString;
        FJsonObjectConverter::UStructToJsonObjectString(ClockTime, ClockTimeString);
    
    
        SubmitNotification(Id, ClockTimeString);

21. If we should send a notification we proceed and flip the bool to false

22. We send a JSON object where only the second value matters. Note you can create a smaller structure to do this and not reuse FClockTime if desired. The second sent is the second that flipped the boolean.

23. We then call “SubmitNotification”

24. OnSubmitClockAnnotation should do the following:

25.     if (!ShouldSendAnnotation)
        {
          
            return;
        }
    
    
        ShouldSendAnnotation = false;
    
    
        FClockTime ClockTime;
        ClockTime.Second = PreviousUsedSecond;
        FString ClockTimeString;
        FJsonObjectConverter::UStructToJsonObjectString(ClockTime, ClockTimeString);
    
    
        SubmitAnnotation(Id, ClockTimeString);

26. It is doing exactly the same thing as the notification function. But with it’s own boolean and using “SubmitAnnotation” instead

Web Implementation

1. Go back to your Web/public folder. Open index.html

2. Add two divs. One for notification and one for annotation:

3.         <div class="child" id="notification"></div>
           <div class="child" id="annotation"></div>

4. Open overlay.js. Create two new functions. One called notificationUpdate and one the other called annotationUpdate. Both take in a parameter called data:

5. function notificationUpdate(data) {
        
       document.getElementById("notification").innerHTML = "Notification: " + Math.round(data.second);
   }
   
   
   function annotationUpdate(data) {
         
       document.getElementById("annotation").innerHTML = "Annotation: " + Math.round(data.second);
   }

6. We set the divs created with the second value we sent in the above steps

7. In the existing genvidClient.onDraw function add the following code after the existing “ClockTime” stream code:

8. if ("ClockAnnotation" in frame.annotations && frame.annotations["ClockAnnotation"].length)
   {
             
   for (let gameAnnotation of frame.annotations["ClockAnnotation"])
       {
         
           if (gameAnnotation && gameAnnotation.user)
           {
         
               annotationUpdate(gameAnnotation.user);
           }
               }
   }

9. We check to see if a ClockAnnotation data value exists. If we pass the parsed JSON value to annotationUpdate. (It is already being parsed in onStreamsRecieved)

10. Similarly to genvidClient.onDraw we will define another function on genvidClient called “onNotificationsReceived” that triggers whenever a notification is sent:

11. genvidClient.onNotificationsReceived(function (notifications)
    {
          
        for (let notification of notifications.notifications) 
    {
          
            try 
    {
          
                notification.user = JSON.parse(notification.data);
                notificationUpdate(notification.user);
            }
            catch (e) {
          
                console.log(e, notification);
            }
        }
    });

12. We parse the notification received and pass in the data to notificationUpdate

Putting it all together

1. Package your project making sure that .exe ends up in the same location as before so that it can be picked up by the web code

   • If you need to - recreate your bastion instance, setup the genvid-sdk, and load using the python script

2. Open up your genvid monitor page using genvid-sdk monitor. Hit start all and open up your stream

3. Note that the annotation updates in sync with the video whenever the seconds hand hits a 15 second interval

4. Note that the notification updates out of sync with the video ASAP when the second hand hits a 15 second interval