# Audit Tool

Audit Tool allows Customers to have access to submitted [batches](https://docs.imerit-prod.io/project-setup/pre-process-data#batch) of the job after they have undergone all the [nodes](https://docs.imerit-prod.io/project-setup/create-taskflow/1.-taskflow-details#taskflow-templates) of the [taskflow](https://docs.imerit-prod.io/project-setup/create-taskflow/1.-taskflow-details#taskflow-templates).

## How to get here?

### From the Landing Dashboard

1. Go to the ‘Audit’ from the Solution listed on the [QuickServe Dashboard](https://docs.imerit-prod.io/quickserve-platform) (from the gear icon) of the Solutions listed.
2. User is taken to the Audit Dashboard, where they can view all the submitted Batches.

   <figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXcYqzdsM-QLAWzBTjIaOuoau4wswK00aDF9Cs4DUNIDWK91o598tIaRGEnsW9JAZRTzq7x3RQPvdrblCtCLh7lQBZ2QsyPdrzQuwgfCVPwdVsPfPdfnujFq8PieHn1h9smzR0pBlXaLXn7EMYrXMpHG1Nm3?key=lGhAQLv5xC3Ciuwf6zBTGQ" alt="" width="188"><figcaption></figcaption></figure>

### From Job Builder

1. Navigate to the Job Builder UI
2. Click on the 'Audit' button at the top right

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2F0kdTJI3ymN0HvBfuVXnB%2Fimage.png?alt=media&#x26;token=3885be4a-a7bd-4be9-a6c6-19d664afa859" alt=""><figcaption></figcaption></figure>

### From All Batches

1. Go to 'Job Details' under the gear icon beside a job on Job Builder
2. Select 'All Batches' from the left navigation.
3. Select the 'Audit' button that appears above the batch table.&#x20;

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2FDkPO0jjcqbT3jQV11uZn%2Fimage.png?alt=media&#x26;token=bcac96de-3373-465e-a51a-59291bc96b55" alt=""><figcaption></figcaption></figure>

## Audit Dashboard

1. Select the Job from the dropdown to see all the batches with their status in the table below.

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2FnelDZzsdwWUrMBna3Zuo%2Fimage.png?alt=media&#x26;token=4d09bb6b-3327-43f4-a46e-98dbd3d40ddb" alt=""><figcaption></figcaption></figure>

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2FcP9kAthBUBaWdahUK4Ey%2Fimage.png?alt=media&#x26;token=16ffd58b-cda4-43fd-97e2-86f9f08c69c8" alt=""><figcaption></figcaption></figure>

All batches will be listed under the dashboard. The table displays the following details:

* **Tasks**: Number of frames in one Batch
* **Annotations**: Total no. of Annotations created in a Batch&#x20;
* **Issues**: No. of Mistakes marked by the auditor
* **Issue Status (Open/ Closed/ Ignored):** No. of Issues under each Type
* **Status** (of the Batch)
  * LIP (Label in Progress) - Cannot be accessed by the auditor till labelling is complete
  * RFA (Ready for Audit) - Auditor can select this sequence provided no other sequence is assigned to them
  * AIP (Audit in Progress) - Auditor is currently reviewing the sequence
* **Assigned To** - Auditor Name
* **Last Updated** - Last time the sequence was submitted. A sequence can go through multiple rounds of audit
* **Action** - Launch the Audit Tool

## Audit Tool Interface

The tool opens in a new tab. The interface is similar to the [annotation tool](https://docs.imerit-prod.io/annotation-tool/tool-layout) with all the annotation data&#x20;

The 'Issues' Link on the Header is enabled in the Audit Tool. Here is where the users can document issues as and when observed.

Note:

* The Annotations are non-editable
* The only action that takes place is marking issues

On the Issues Panel, auditors can see a summary of the issues created and also filter Issues from the Issue Title row.

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2FNdGQaMmihW0IYQAYy2AR%2Fimage.png?alt=media&#x26;token=55ec941f-64f4-4438-9a16-64511d85c7bc" alt=""><figcaption></figcaption></figure>

### How to Raise Issues

To create an issue user can do either of the two:&#x20;

1. Clicking on 'Issue' on the [Header](https://docs.imerit-prod.io/annotation-tool/tool-layout#id-1.-header) opens the Issue Panel. Click on the '+' icon on the issue title row.&#x20;

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2FlWc5VzaaCincVaLWIRYX%2Fimage.png?alt=media&#x26;token=b41d6122-87bc-49af-97f1-9e07a0302e3e" alt=""><figcaption></figcaption></figure>

2. Right-clicking on the annotation and select the ‘Create Issue’ option.

### Create Issue Card Details

Create Issue Card appears on the Issue panel with the following data required for input before saving:

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2F6d94ZpkuUbfAUbwPKyRf%2Fimage.png?alt=media&#x26;token=e8f95938-4935-417d-a0f5-53608ed363cb" alt="" width="188"><figcaption></figcaption></figure>

#### Frame&#x20;

* Start: Frame number where the issue is observed. The tool will automatically populate this with the current frame and will not prompt for an input
* End: Frame number till where the issue is observed if it spans across multiple frames. The tool will automatically populate this with the current frame assuming the error doesn't span beyond the current frame. The user can update to the frame till where it is applicable.

#### Class, Instance, Attribute Selection

Dropdown selection from the list defined in the Project Recipe.

#### Severity

Leverage this field to indicates the level of impact of an issue, ranging from critical to minor.&#x20;

#### Issue Type Selection

All the issues that can be observed are listed under [default](https://docs.imerit-prod.io/project-setup/create-taskflow/1.-taskflow-details#default-template). The user must select whichever is applicable. The list of options for Issues is [customisable](https://docs.imerit-prod.io/project-setup/create-taskflow/1.-taskflow-details#new).

Each error category is assigned a specific color, making it easier for the Auditor to quickly identify the type of issue they are looking for within the list. For example, 'blue' represents wrong annotations, 'green' indicates missing annotations, 'orange' signifies inconsistency in Track ID, and 'purple' refers to miscellaneous issues.

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2Fv3Q28GjLKJX44YQ7SoCq%2Fimage.png?alt=media&#x26;token=218dd642-2950-48ff-8df9-5ea7bde89412" alt="" width="297"><figcaption></figcaption></figure>

#### Comments (if any)

### Issue Menu

Each Issue Card created has a few options which can be pulled up by clicking on the 3 vertical dots on the top right corner.

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2FObnnczYTOlnoApReYMAq%2Fimage.png?alt=media&#x26;token=f7a4159a-172e-4a95-be49-f118857c6833" alt="" width="227"><figcaption></figcaption></figure>

* Reply to the issue
* Ignore/ Close the Issue (the same gets reflected on the [Audit Dashboard](#audit-dashboard))
* 3D/ 2D markers to redirect the annotator's attention to a location.&#x20;
  * For a Missed Annotation: In this case Select the Class & Issue type as 'Missing Object'. After saving, click on the three dots to mark the location of the expected Annotation on the Image/ point cloud using 2D/ 3D Marker respectively.

> Quick Tip: To associate the card to the existing annotation, provide the class and instance details. This will allow users to click on the card to be redirected to where the issue lies.

### Auditing Complete

Once reviewing is completed, click on Accept or Reject on the top right of the tool.

1. Accept will change the status of the batch to Audit Accepted in the batch lifecycle
2. Reject will change the status of the batch to Audit Rejected in the batch lifecycle
3. WithHold is used when the Auditor is unsure and would like to revisit the batch.  The status of the batch turns to Withhold, letting all know that the decision is still pending.

<figure><img src="https://254647936-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzGccvqLYeyaJ6Dgazh7K%2Fuploads%2FKhSGAVNx1iXNw43WHFVb%2Fimage.png?alt=media&#x26;token=5e04c81b-ffcc-4f13-af98-3d0ffe90d205" alt="" width="323"><figcaption></figcaption></figure>