3D Multi Sensor Fusion - User Documentation
  • Introduction
  • Account Activation
  • QuickServe Platform
  • Project Setup
    • Create Recipe
      • 1. Basic Details
      • 2. Classes
      • 3. Attributes
      • 4. Associations
      • 5. Publish Recipe
    • Create Taskflow
      • 1. Taskflow Details
      • 2. Taskflow Preview & Edit
      • 3. Publish Taskflow
    • Build Jobs
      • 1. Job Details
      • 2. Data Import
      • 3. Data Upload Status
      • 4. Launch Task
    • Batch Export
    • Reports
    • Pre-process Data
    • Storages
  • Annotation Tool
    • Tool Layout
    • Steps to Label
    • Drawing Tools
      • Cuboid
      • Polyline 3D
      • Polygon 3D
      • Brush Sphere
      • Rectangle
      • Polyline 2D
      • Polygon 2D
    • Key Features
      • Progress Bar
      • Keyframe Interpolation
      • Raycaster and Frustum
      • Focus Mode
      • Merged Point Cloud
      • Isolate
      • Outlier
      • Ground and Ceiling Mover
      • Project Points on Image
      • Task Level Attribute Propagation
      • Relationship
      • Intensity Filter and Picker
      • Image Settings Lock
      • Unify Dimension
      • Auto-Grounding for Cuboids and Polylines
    • Tool Shortcuts
  • Audit Tool
  • Visualization Tool
  • API Documentation
    • API Documentaion
Powered by GitBook
On this page
  • Tool Overview
  • 1. Header
  • BEV-3D Mode
  • Rotation
  • Autofit
  • Merge Point Cloud
  • Outlier
  • Isolate
  • Navigation Controls
  • Fit to Screen
  • Jump to Origin
  • Toggle Fullscreen
  • Sensor
  • Status
  • Issues
  • Validation
  • Help
  • 2. Footer
  • Time Tracker
  • Timeline
  • Menu Option
  • Attributes: Task & Batch Level
  • Adding Comments
  • Validate Task (AutoQA)
  • 3. Class List
  • Class Row
  • Other Controls
  • 4. Point Cloud
  • 5. Image Panel
  • 6. Annotation Detail
  1. Annotation Tool

Tool Layout

PreviousAnnotation ToolNextSteps to Label

Last updated 1 month ago

Tool Overview

There are primary sections of the tool:

  1. : Green

  2. : Purple

  3. : Blue

  4. : Yellow

  5. : Red

  6. (On Selecting an Annotation): Orange

1. Header

The header consists the following actions:

BEV-3D Mode

  • This component acts like a toggle to view the point cloud in 2D (Top view) or in its default 3D environment.

  • When 2D is toggled ON, the annotator can mark objects in the point cloud from the top view using Rectangle interactions for creating a Cuboid.

  • When 3D is toggled on, the annotator can mark objects in a point cloud from any angle by dropping a Cuboid.

Rotation

  • On using it, the Cudoid Direction get rotated in an anti-clockwise manner.

  • This helps if the direction of the object has been indicated incorrectly after setting the dimension.

  • Shortcut: R Key

Autofit

The autofit feature gets enabled only after a cuboid is selected. This feature allows the user to remove extra space between the end most point and the edge of the cuboid. This feature ensures that no point is excluded while creating a much tighter box from all sides.

Merge Point Cloud

Outlier

  • This feature allows annotators to locate unlabelled points.

Isolate

Navigation Controls

There are 2 Modes available to Navigate in the point cloud:

  1. Helicopter Mode:

    • Move (front, left, back, right) - Key W, A, S, D

    • Fly (up, down) - Key Q, E

    • Looking Around - 4 Arrow Keys

    • Zoom Control: Mouse Wheel or double-click to zoom in

  2. Orbital Mode Shortcuts:

    • Move (up, down, right, left) - 4 Arrow Keys

    • Orbit around:

      • Key W, S (Orbiting top to down)

      • Key A, D (Orbiting right to left)

    • Zoom Control: Mouse Wheel or double-click to zoom in

Fit to Screen

This fits the entire point cloud within the browser screen. Shortcut: Key Ctrl + F

Jump to Origin

The coordinates at which the user is positioned by default when the task loads is the point of Origin. On clicking this, the user is navigated back to this point. Shortcut: Key O (alphabet)

Toggle Fullscreen

Hides the browser bar and makes the tool fullscreen.

Sensor

The ego vehicle has multiple LiDAR sensors installed on it. Because the sensors are not time-synced, a staggering effect of the objects around the ego vehicle can be observed in the point cloud (refer to the image below). The user can use the sensor filter and select the sensor points they would like to consider.

Status

  1. Class

  2. Dimension

Issues

  • Once auditing is completed, the batch can be reopened by the experts on the Annotation tool with the issue cards visible for them to refer to and resolve the issues.

Validation

The Validation executes a logical validation on each task to ensure

  1. Better quality is delivered

  2. The annotator can be retrained

Help

On the top left corner of the tool is the help icon to view Shortcuts and Batch-related information.

a. Shortcuts

Selecting the 'Shortcuts' in the help menu, a panel opens from the right, listing the labelling shortcuts.

b. Batch Info

On selecting 'Batch Info' in the help menu, information about the batch is provided in a box. Using the 'Copy to Clipboard' you can copy the entire contents in a click.

2. Footer

The footer consists of the following actions:

Time Tracker

Displays the time spent on a task in real time.

Timeline

  • The timeline displays all the frames within the frame sequence.

    • To further see the frames, drag the horizontal white bar below the timeline view

  • Navigation across the timeline can be done by

    • clicking on the frame

    • dragging the timeline marker

  • entering frame number on its right

  • clicking the first, previous, play, next and last buttons

Menu Option

a. Tool Setting

Point Cloud visibility can be configured here

Timer Range

  • Each point captured by each LiDAR sensor has a timestamp. This timestamp is the time difference between the reference time of the frame and the time of an individual point. This use case occurs when the LiDAR sensors placed on the ego vehicle are not synchronised.

  • In the below example, '0' is the frame's reference time and each frame is 100milli seconds apart. Using the Timer Ranger filter, the user can select any range between -49 up to +49 and filter the points in between. Keeping the controls of the slider at the end will show the entire point cloud.

Point Budget

  • The number of points to be loaded for frames is set here.

  • By default, the value is set to 5 million.

  • When the frame contains more than 5 million points, the tool loads the 5 million points in the viewable area. Panning and zooming in loads the other points. Please note, the tool also loads the previous and next frames for the users benefit and smooth experience.

  • With increasing the limit of points, latency is to be expected.

Enable Eye Dome Lighting

  • EDL is a lighting model that accentuates the shapes of objects within a point cloud

  • This is done by visually grouping objects that are close in distance and shading their outlines, enhancing the perception of depth.

Class Point Brightness

  • The points that have been classified appear in their class colour.

  • With the help of this slider, the user can reduce the brightness of those coloured points.

Background

Change the background colour to make use of the different light situations.

Point size modifier

Increases/decreases the size of the points on the screen.

Point Colours

View the points in different colour schemes:

  • Plain Colour: The points are displayed in a single colour which the user can choose from the palette.

  • Elevation Gradient: The points are displayed based on their elevation in the point cloud. Cooler colours indicate lower elevation on the Z-axis.

  • RGBA: When the point cloud data contains an RGB field, this information is used to set the colour of the points.

  • Intensity (B-G-Y-R): When the point cloud files include an intensity field, the intensity values are used to determine the color of the points. Points with intensity values near 0 are assigned a Blue color, while higher intensity values transition through Green and Yellow, with values closest to 1 appearing as Red. Points falling between these thresholds will display a gradient mix of the adjacent colors.

  • Intensity (B-C-T-G-Li-Y-O-R): Similar to the Intensity (B-G-Y-R), when the point cloud files contain an intensity field, this information is used to set the colour of the points. If the point's intensity lies closer to 0, the colour of the points will be Blue, next threshold is Cyan, Teal, Green, Lime, Yellow, Orange and intenstiy values closest to 1 are Red.

  • Intensity (Rainbow): When the point cloud files contain an intensity field, the colours are set in the rainbow shade card pattern.

  • Intensity Grayscale: When the point cloud files contain an intensity field, this information is used to set the colour of the points between white and black and everything between gets a different shade from gray.

  • Classification: Displays the points based on any pre-labeling segmentation that is loaded on the tool. Currently, by default, based on the segragation done prior to uploading, the tool displays ground and non-ground points using either Patchwork+. Based on customers acceptance this information can also be segmented using in-house ML.

Speed Controls

  • The speed of a mouse scroll can be calibrated.

  • The higher the value the further distance is traveled in one scroll.

Region of Interest

  • ROI is the area visually defined for the labelling experts to know the areas that need labelling covered.

  • This is defined by setting the radius from the ego vehicle.

  • Multiple concentric ROI can be defined under this section.

Attributes: Task & Batch Level

Adding Comments

Validate Task (AutoQA)

3. Class List

  • This panel can be collapsed from the right corner to have a full-screen view of the point cloud.

Class Row

  1. Create Annotation: The '+' icon appears on the row of the class. To create an instance under the class, click this > select drawing tool > after creation, the instance appears under it. eg: car_43

  2. No. of instances: The no. in the circle indicates the number of annotations or 'instances' created under that Class.

  3. Instance ID: Name defined in the recipe reflects here. The instance will have the class name with a unique number. For example, class 'Car', Instance 'car_43'

  4. Object Clipping: This action hides all the labelled points while keeping the boundary visible to aid checking if any points are missed for labelling for that Object or if points have been incorrectly labelled. This can be applied to all or individual instances of a Class.

  5. Lock/ Unlock: This action helps in controlling accidental movements of annotations by locking the displacements and dimension modification of the Annotation.

  6. Show/ Hide: This will hide the labelled points as well as the boundary

  7. Jump to Object: This icon allows one to jump to the specific annotation created from anywhere in the point cloud.

  8. Delete Annotation: Annotation when created gets projected in its respective sensors (in a frame) & propagated across all frames. Hence the object can be deleted in:

      • From the sensor the object is selected from

      • Across all sensors the object was annotated (Image & point cloud)

      • From the sensor the object is selected from

      • Delete in all Sensors it was projected in

        .

Other Controls

Below the Class List, are two checkboxes for better visibility and checking:

Filled Boxes

Fill the cuboid with the colour of the Class.

Object Names:

Displays the name of all annotations on both the point cloud and reference image

4. Point Cloud

Ground and Ceiling Mover

Show all points

This component allows the user to filter out points and keep only ground points visible. This is calculated with or without ML algorithm.

5. Image Panel

Below the point cloud is a collapsible Image panel that consists of all the cameras linked to the point cloud. Referring to the images aids the user in identifying the objects that need to be marked in the point cloud.

  • These Images have the name of the camera mentioned in the corner.

  • The panel acts like a carousal with arrows for the user to glide through.

  • On double-clicking the image, it gets toggled to fullscreen and replaces the point cloud. Press the Esc key to toggle back to the original state.

6. Annotation Detail

Annotation Hover

The top right of the point cloud displays a label of the annotation being hovered upon with its attribute values. Use Shortcut Key L to enable/disable the visibility of this label.

  • Upon selecting an annotation, the timeline displays the occurrence of the annotation across the frames by the colour given to its Class.

Annotation Right Click

  • Users can View details of the annotation entered at the time of creation.

This icon is enabled upon selecting any .

For optimal use of this feature, it is advised to ensure that the object is fully visible and not partially obscured from any side.

This feature is helpful in sequential tasks to identify and mark objects in their stationary state. Learn how to use it .

While MPC is ideally used for static objects, any user can also understand the entire scene by smartly using the 'Frame Jump Length'

On toggling this on, all the points in the point cloud that have not been labelled will start to blink to aid in locating them. Learn how to use it .

This feature is used to focus on specific areas of the point cloud to avoid background distraction while labelling. Learn how to use it .

The Status link on the top right corner is enabled on selecting a . On clicking it, a box opens with the following details.

Instance (if Class has been marked with )

This link gets enabled in the after the batch passes through the Nodes defined in the Taskflow, it moves to the Audit stage and gets reviewed by the auditor.

The Auditor raises observed in the form of issue cards as and when they are spotted.

View the list of all the Annotation Tool shortcuts .

The and the defined will get reflected here for labelling experts to input.

Annotators can add any doubts or comments in this section. The expert in the pipeline is notified about it at the next .

Any frame-specific instructions or constraints will appear here for the labelling expert to refer to. Read to understand.

The Class List reflects all the classes defined during the step of .

This list consists of all the classes with and without .

The class rows indicate the properties set up while .

Colour of the Class: The colour allocated to the class in the is reflected here.

Classes that do not have , will only have 1 occurrence. Hence the + icon will be disabled after it's marked the first time. Users can add to this annotation.

Current

Across Seqeunce/

The central area of the annotation tool is dedicated to the Point Cloud sensor. This is a 3D Space where the user to view the point cloud scene.

On the bottom right of the point cloud is a two-way slider that allows the user to hide points from the ceiling to the ground and vice-versa. This allows annotators to control the visibility of unwanted/ obstructing points. Learn how to use it .

Annotation Click (Only)

The link automatically displays the class name and dimension details on the top right of the point cloud.

On Expanding the timeline, one can further see the values given to all the attributes linked to the class of the selected annotation.

Cuboid
here
here
here
Cuboid
Audit Tool
here
Cuboid
more
navigates
here
status
Header
Footer
Class List
Point Cloud
Image Panel
Annotation Detail
Issues
node
Instance
creating a class
instances
creating a class
recipe
instances
task level attributes
batch level attributes
time-varying and constant
frame
batch