2019/2020 UZH FPV Leader Board

This is the leader board based on the evaluation used in the 2019 and 2020 UZH FPV competitions. It evaluates position and rotation estimation accuracy on six dataset sequences for which the ground truth is not public.

Sequences

SequenceVideoLengthStart Time SnapdragonStart Time DavisDownload SnapdragonDownload Davis
indoor forward 11 https://youtu.be/mYKStE7e2aI 85.68 1540824001 s 1540824001 s bag, zip bag, zip
indoor forward 12 https://youtu.be/jNlDgN8fdKA 124.07 1540824296 s 1540824296 s bag, zip bag, zip
indoor 45deg 3 https://youtu.be/q6ELgSAjNMY 119.82 1623 s 1545305934 s bag, zip bag, zip
indoor 45deg 16 https://youtu.be/V4OnapxRLD4 58.72 133 s 1545315222 s bag, zip bag, zip
outdoor forward 9 https://youtu.be/ydaMA4Uta9A 314.41 372 s 1540102003 s bag, zip bag, zip
outdoor forward 10 https://youtu.be/G60gls4qeZ4 455.63 674 s 1540102304 s bag, zip bag, zip

Evaluation Method

The submission will be ranked based on the accuracy. We use the same metric as adopted by KITTI. The average translation error (in percentage) over all possible subsequences of a set of certain lengths is used for ranking the submitted results. We use our publicly available trajectory evaluation toolbox to evaluate the estimated trajectories.

For each x in {40m, 60m, 80m, 100m, 120m}, subtrajectories of length x are uniformly sampled (in time) from the entire trajectory. The smaller the subtrajectory, the more samples there are in a given sequence. Each of those subtrajectories is then aligned to the ground truth, optimizing position and yaw that minimizes the position error. For the ranking, the average of relative position error and rotation error is taken across all such samples. The length of the sequence affects the amount of samples in that sequence. For example, outdoor_forward_9 has {964, 916, 862, 807, 762} samples for trajectory lengths {40, 60, 80, 100, 120}. If a trajectory is shorter than x, no samples are generated for x.

Submission

To make a submission, you will need to create an account, follow the instructions below. Each account can make only one submission, but it can be changed afterwards. Submissions are not automatically published – you can review the results and decide whether to publish by yourself (evaluation usually takes around 1 minute). Submissions can also be withdrawn completely.

Click here for the submission format specification.

Error

You must be logged in to upload submissions

Click here to log in or register a new account: Login

Results

Main Sequence
{{ index + 1 }} {{ value.name }} Details
Sequences

{{ sequence }}

{{ index + 1 }} {{ value.name }} Details