Competition

The evaluation is based on measuring the accuracy between the predicted bounding boxes and the ground-truth bounding boxes with the following formula:

accuracy = tp / (tp + fp + fn - nr_misclass)

• TP = true positive bounding boxes
• FP = false positive bounding boxes
• FN = false negative bounding boxes
• NR_MISCLASS = number of misclassified bound boxes

For computing the final accuracy score, we only use the ground-truth bounding boxes which have both the width and the height greater than 25 pixels. 

A prediction is considered correct if it has an IoU (intersection over union) greater than 0.25 and has the same sign category as the ground-truth bounding box. For more details regarding the IoU, check this. 

Model accuracy: Prize money will be awarded only if the score from the leaderboard is bigger or equal to 94%.

Performance rule: The image processing time should not be higher than 0.5 seconds per GPU (for Nvidia 1080Ti).

Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.

Clarifications regarding NR_MISCLASS

In the situation when you accurately detected the location of the bounding boxes but you predicted another sign category (you misclassified the sign), we don't want to add both an FP (your sign) and an FN (the ground-truth sign) to the metrics when computing the accuracy. As a result, we use the NR_MISCLASS to deal with this situation and add only +1 penalty and not +2.

Breach of any contest rule specified below is ground for disqualification.  In the event of any dispute in connection with the contest or with the interpretation or implementation of these rules, the decision of the organizers shall be final.

1 Training Sets

The training sets in this dataset belong to Telenav, Inc. (“Telenav”) and are licensed under a Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.

2 Software

The Telenav source code is subject to the terms of the Mozilla Public License, v. 2.0. A copy of the MPL can be found at http://mozilla.org/MPL/2.0/.

3 Eligibility

The contest is open to everyone above the age of eighteen (18) and is void where prohibited by law.

4 Specific rules:

• One account per participant
• You cannot and should not submit data from multiple accounts;
• If the case, the participant and all its accounts will be removed from the competition
• Team size
• There is no maximum team size
• Submission limit
• The submission limit is detailed in the competition phases.
• Winner(s) license
• The code should be based on Telenav common or based on commercially open source technology stack;
• You will grant the competition sponsor the source code and datasets used;
• External data, allowed only if dataset can be used on commercial products. The evaluation will be done based on the Telenav provided datasets.
• Winner(s) obligations:
• deliver to Telenav the final model’s software code as used to generate the winning submission and associated documentation.
• the delivered software code must be capable of generating the winning Submission and contain a description of resources required to build and/or run the executable code successfully;
• No private sharing outside teams
• Privately sharing code or dataset outside of the team is not permitted.
• Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.
• Model accuracy:
• Prize money will be awarded only if the score from the leaderboard is bigger or equal to 94%;
• Performance rule:
• The image processing time should not be higher than 0.5 seconds per GPU (for Nvidia 1080Ti);
• Leaderboard:
• Only the participants that publicly provided the link to the code repository will be listed in the leaderboard at the end of the competition;
• The code should contain entire functionality in order to replicate the results, starting from the data; code should be publicly available;
• The participants should add the link in the Forum section specially created for the competition;
• Telenav employees
• This contest is not open to Telenav employees; 

5 Conditions.

By entering this contest, you agree:

(a) To abide by these official rules;

(b) To the extent allowable under applicable law, to release and hold harmless Telenav and sponsors, their respective parents, subsidiaries, affiliates, employees and agents from any and all liability or any injury, loss, damage, right, claim or action of any kind arising from or in connection with this contest or any prize won;

(c) That Telenav’s decisions will be final and binding on all matters related to this contest; and

(d) That by accepting a prize, Telenav and competition sponsors may use your team name, your name, and your place of residence online and in print, or in any other media, in connection with this contest, without payment or compensation to you. The declaration of eligibility, use, indemnity and liability/publicity release provided to the potential winner will make reference to obtaining his/her free consent to use his/her name and place of residence. In any case, the lack of such consent does not prevent the winner from receiving the prize.

(e) This contest will be governed by the laws of the state of California, and you consent to the exclusive jurisdiction and venue of the courts of the state of California for any disputes arising out of this contest.

6 Privacy

During the development phase of the contest and when they submit their final entries, contest participants do not need to disclose their real identity but must provide a valid email address where we can be deliver notifications to them regarding the contest. To be eligible for prizes, however, contest participants will need to disclose their real identity to contest organizers, informing them by email of their name, professional affiliation, and address. To enter the contest, the participants will need to become users of the CodaLab platform. Any profile information stored on this platform can be viewed and edited by the users. After the contest, the participants may cancel their account with the CodaLab and cease to be users of that platform. All personal information will then be destroyed. The CodaLab privacy policy will apply to contest information submitted by participants on the CodaLab. Otherwise, Telenav’s privacy policy will apply to this contest and to all information that we receive from your entry that we receive directly from you or which you have submitted as part of your contest entry on the CodaLab. Please read the privacy policy on the contest entry page before accepting the official rules and submitting your entry. Please note that by accepting the official rules you are also accepting the terms of the Telenav privacy policy: https://www.telenav.com/legal/policies/privacy/privacy-policy.