Tool Layout

Tool Overview

There are primary sections of the tool:

1. Header: Green

2. Footer: Purple

3. Class List: Blue

4. Point Cloud: Yellow

5. Image Panel: Red

6. Annotation Detail (On Selecting an Annotation): Orange

1. Header

The header consists the following actions:

BEV-3D Mode

• This component acts like a toggle to view the point cloud in 2D (Top view) or in its default 3D environment.

• When 2D is toggled ON, the annotator can mark objects in the point cloud from the top view using Rectangle interactions for creating a Cuboid.

• When 3D is toggled on, the annotator can mark objects in a point cloud from any angle by dropping a Cuboid.

Rotation

• This icon is enabled upon selecting any Cuboid.

• On using it, the Cudoid Direction get rotated in an anti-clockwise manner.

• This helps if the direction of the object has been indicated incorrectly after setting the dimension.

• Shortcut: R Key

  

Autofit

The autofit feature gets enabled only after a cuboid is selected. This feature allows the user to remove extra space between the end most point and the edge of the cuboid. This feature ensures that no point is excluded while creating a much tighter box from all sides.

Merge Point Cloud

This feature is helpful in sequential tasks to identify and mark objects in their stationary state. Learn how to use it here.

Outlier

• This feature allows annotators to locate unlabelled points.

• On toggling this on, all the points in the point cloud that have not been labelled will start to blink to aid in locating them. Learn how to use it here.

  

Isolate

• This feature is used to focus on specific areas of the point cloud to avoid background distraction while labelling. Learn how to use it here.

  

Navigation Controls

There are 2 Modes available to Navigate in the point cloud:

1. Helicopter Mode:

   • Move (front, left, back, right) - Key W, A, S, D

   • Fly (up, down) - Key Q, E

   • Looking Around - 4 Arrow Keys

   • Zoom Control: Mouse Wheel or double-click to zoom in

   
2. Orbital Mode Shortcuts:

   • Move (up, down, right, left) - 4 Arrow Keys

   • Orbit around:

     • Key W, S (Orbiting top to down)

     • Key A, D (Orbiting right to left)

   • Zoom Control: Mouse Wheel or double-click to zoom in

Fit to Screen

This fits the entire point cloud within the browser screen. Shortcut: Key Ctrl + F

Jump to Origin

The coordinates at which the user is positioned by default when the task loads is the point of Origin. On clicking this, the user is navigated back to this point. Shortcut: Key O (alphabet)

Toggle Fullscreen

Hides the browser bar and makes the tool fullscreen.

Sensor

The ego vehicle has multiple LiDAR sensors installed on it. Because the sensors are not time-synced, a staggering effect of the objects around the ego vehicle can be observed in the point cloud (refer to the image below). The user can use the sensor filter and select the sensor points they would like to consider.

Status

The Status link on the top right corner is enabled on selecting a Cuboid. On clicking it, a box opens with the following details.

1. Class

2. Dimension

3. Instance (if Class has been marked with Instance)

Issues

• This link gets enabled in the Audit Tool after the batch passes through the Nodes defined in the Taskflow, it moves to the Audit stage and gets reviewed by the auditor.

• The Auditor raises Issues observed in the form of issue cards as and when they are spotted.

• Once auditing is completed, the batch can be reopened by the experts on the Annotation tool with the issue cards visible for them to refer to and resolve the issues.

Validation

The Validation executes a logical validation on each task to ensure

1. Better quality is delivered

2. The annotator can be retrained

Help

On the top left corner of the tool is the help icon to view Shortcuts and Batch-related information.

a. Shortcuts

Selecting the 'Shortcuts' in the help menu, a panel opens from the right, listing the labelling shortcuts.

View the list of all the Annotation Tool shortcuts here.

b. Batch Info

On selecting 'Batch Info' in the help menu, information about the batch is provided in a box. Using the 'Copy to Clipboard' you can copy the entire contents in a click.

2. Footer

The footer consists of the following actions:

Time Tracker

Displays the time spent on a task in real time.

Timeline

• The timeline displays all the frames within the frame sequence.

  • To further see the frames, drag the horizontal white bar below the timeline view

• Navigation across the timeline can be done by

  • clicking on the frame

  • dragging the timeline marker

• entering frame number on its right

• clicking the first, previous, play, next and last buttons

Menu Option

a. Tool Setting

Point Cloud visibility can be configured here

Timer Range

• Each point captured by each LiDAR sensor has a timestamp. This timestamp is the time difference between the reference time of the frame and the time of an individual point. This use case occurs when the LiDAR sensors placed on the ego vehicle are not synchronised.

• In the below example, '0' is the frame's reference time and each frame is 100milli seconds apart. Using the Timer Ranger filter, the user can select any range between -49 up to +49 and filter the points in between. Keeping the controls of the slider at the end will show the entire point cloud.

Point Budget

• The number of points to be loaded for frames is set here.

• By default, the value is set to 5 million.

• When the frame contains more than 5 million points, the tool loads the 5 million points in the viewable area. Panning and zooming in loads the other points. Please note, the tool also loads the previous and next frames for the users benefit and smooth experience.

• With increasing the limit of points, latency is to be expected.

Enable Eye Dome Lighting

• EDL is a lighting model that accentuates the shapes of objects within a point cloud

• This is done by visually grouping objects that are close in distance and shading their outlines, enhancing the perception of depth.

Class Point Brightness

• The points that have been classified appear in their class colour.

• With the help of this slider, the user can reduce the brightness of those coloured points.

Background

Change the background colour to make use of the different light situations.

Point size modifier

Increases/decreases the size of the points on the screen.

Point Colours

View the points in different colour schemes:

• Plain Colour: The points are displayed in a single colour which the user can choose from the palette.

• Elevation Gradient: The points are displayed based on their elevation in the point cloud. Cooler colours indicate lower elevation on the Z-axis.

• RGBA: When the point cloud data contains an RGB field, this information is used to set the colour of the points.

• Intensity (B-G-Y-R): When the point cloud files include an intensity field, the intensity values are used to determine the color of the points. Points with intensity values near 0 are assigned a Blue color, while higher intensity values transition through Green and Yellow, with values closest to 1 appearing as Red. Points falling between these thresholds will display a gradient mix of the adjacent colors.

• Intensity (B-C-T-G-Li-Y-O-R): Similar to the Intensity (B-G-Y-R), when the point cloud files contain an intensity field, this information is used to set the colour of the points. If the point's intensity lies closer to 0, the colour of the points will be Blue, next threshold is Cyan, Teal, Green, Lime, Yellow, Orange and intenstiy values closest to 1 are Red.

• Intensity (Rainbow): When the point cloud files contain an intensity field, the colours are set in the rainbow shade card pattern.

• Intensity Grayscale: When the point cloud files contain an intensity field, this information is used to set the colour of the points between white and black and everything between gets a different shade from gray.

• Classification: Displays the points based on any pre-labeling segmentation that is loaded on the tool. Currently, by default, based on the segragation done prior to uploading, the tool displays ground and non-ground points using either Patchwork+. Based on customers acceptance this information can also be segmented using in-house ML.

Speed Controls

• The speed of a mouse scroll can be calibrated.

• The higher the value the further distance is traveled in one scroll.

Regions of Interest

• ROI is the area visually defined for the labelling experts to know the areas that need labelling covered.

Learn how to use

Attributes: Task & Batch Level

The task level attributes and the batch level attributes defined will get reflected here for labelling experts to input.

Adding Comments

Annotators can add any doubts or comments in this section. The expert in the pipeline is notified about it at the next node.

Validate Task (AutoQA)

Any frame-specific instructions or constraints will appear here for the labelling expert to refer to. Read more to understand.

3. Class List

The Class List reflects all the classes defined during the step of creating a class.

• This panel can be collapsed from the right corner to have a full-screen view of the point cloud.

• This list consists of all the classes with and without instances.

Learn how to use it here.

Other Controls

Below the Class List, are two checkboxes for better visibility and checking.

Filled Boxes

Fill the cuboid with the colour of the Class.

Object Names:

Displays the name of all annotations on both the point cloud and reference image

4. Point Cloud

The central area of the annotation tool is dedicated to the Point Cloud sensor. This is a 3D Space where the user navigates to view the point cloud scene.

Ground and Ceiling Mover

On the bottom right of the point cloud is a two-way slider that allows the user to hide points from the ceiling to the ground and vice-versa. This allows annotators to control the visibility of unwanted/ obstructing points. Learn how to use it here.

Show all points

This component allows the user to filter out points and keep only ground points visible. This is calculated with or without ML algorithm.

5. Image Panel

Below the point cloud is a collapsible Image panel that consists of all the cameras linked to the point cloud. Referring to the images aids the user in identifying the objects that need to be marked in the point cloud.

• These Images have the name of the camera mentioned in the corner.

• The panel acts like a carousal with arrows for the user to glide through.

• On double-clicking the image, it gets toggled to fullscreen and replaces the point cloud. Press the Esc key to toggle back to the original state.

6. Annotation Detail

Annotation Hover

The top right of the point cloud displays a label of the annotation being hovered upon with its attribute values. Use Shortcut Key L to enable/disable the visibility of this label.

Annotation Click (Cuboid Only)

• The status link automatically displays the class name and dimension details on the top right of the point cloud.

• Upon selecting an annotation, the timeline displays the occurrence of the annotation across the frames by the colour given to its Class.

• On Expanding the timeline, one can further see the values given to all the time-varying and constant attributes linked to the class of the selected annotation.

Annotation Right Click

• Users can View details of the annotation entered at the time of creation.