Keyframe Interpolation

Interpolation of an annotation is a process of labelling a moving object by tool estimating its position across frames between two defined frames. These frames in which the user sets the position of the annotation are called Keyframes. A keyframe can also be created with the Shortcut K.

On creating a keyframe, the tool automatically calculates and repositions the annotations in the interim frames between defined keyframes.

The keyframes are visible on the timeline (blue line under the frames) for a given selected annotation indicating that the position was set by the annotators themselves in those frames.

For instance, in the example below, the object was in motion from Frame 1 to Frame 25 and moved out of the scene. Frame 1, 5, 10, 15 & 25 are keyframes where the annotator manually set the annotation positions, while the intermediate frames have positions calculated by the tool.

Benefits

• Efficiency: Reduces the time and effort required to annotate frames individually by automatically propagating annotations across intermediate frames.

• Consistency: Ensures uniformity in annotations across sequential frames, minimizing discrepancies and improving accuracy.

Steps to use Keyframe Interpolation

Keyframe Interpolation feature is only used for labelling object(s) in motion. Once a moving object has been identified, the process for Keyframe Interpolation can begin.

1. Identify the frame where the object is most clearly visible, in most cases it is the frame where the object is most closest to the ego vehicle.

2. Create the annotation for the object in the same frame. Modify its dimensions to annotate accurately and provide the attribute values.

3. The first time the annotation is created, inherently, becomes a Keyframe.

4. The annotation is propagated across all frames (forward and backward) on the same world coordinates (provided the ego pose data is available in the dataset).

5. Track the object by navigating in one direction, and after approximately 5-7 frames from the initial frame, reposition the annotation on the object accurately.

6. As soon as the cuboid is repositioned the current frame becomes a Keyframe.

7. The tool automatically adjusts the positions of all the annotations between the two Keyframes.

8. This process should be repeated till the object is visible in the sequence.

9. Deleting the annotation in any frame will indicate the immediate next/ previous frames in which the Annotation is present as a Keyframe. Eg:

   • The object is available between Frame 1-9 and is no longer visible on Frame 10. The annotation is created on Frame 1. Then, the annotation is deleted from Frame 10 since it is no longer visible. Therefore, Frame 9 becomes a Keyframe.

   

   • The object is available between Frame 11-14 and leaves the scene. The annotation is created on Frame 11. Then, the annotation is deleted fromFrame 15 onwards since it's no longer visible. Therefore, Frame 14 becomes a Keyframe.

   

Adding Keyframes

A frame turns into a Keyframes when the user does the following:

• Creates an annotation on any frame.

• Makes any position modification to an annotation manually.

• Deletes the annotation on a frame which makes the immediate next and/or previous frames the annotation occurs in a Keyframe.

• Uses the Shortcut K while an annotation is selected. This makes the frame a Keyframe without making any changes to the positioning of the annotation.

For example, an object is stationary from Frame 1-10 and starts moving on Frame 11:

• The user will first create the annotation on Frame 1

• The annotation gets propagated across all frames in the same position.

• On Frame 10, the user will press K Key and mark it as a Keyframe

• On Frame 11 where the object has started to move, the user will reposition the annotation accurately.

  • Hence Frame 11 will become a Keyframe.

Hence, to prevent the tool from interpolating the annotation beyond Frame 1-11, the user created a Keyframe on Frame 10.

Removing a Keyframe

Keyframes can be removed by selecting an annotation on a keyframe and pressing the Shortcut K. This shortcut is a toggle for both adding and removing the Keyframe.

Label Occluded Objects

Sometimes an object might be occluded in the interim frames. In such cases follow this:

1. Create the annotation on the object (Frame 1) .

2. Go to the frame where the object is visible (Frame 8) before it gets occluded (i.e.Frame 9-11).

3. Press K (Frame 8 bcomes a Keyframe)

4. Go to the frame where the object reappears again (Frame 12).

5. Right-click on the selected annotation and select the option to Delete annotation between a frame range.

6. Select the frame range as Frame 9 - 11 and apply action. The annotation will get deleted from this range without re-positioning it in frames that were not a keyframe.