Leader Board
These are the sequences used in the UZH FPV competition. We evaluate position and orientation estimation accuracy on nine dataset sequences for which the ground truth is not public. The leader board containing the top 15 submissions is at the bottom of the page.
Sequences
| Sequence | Video | Length [m] | Start Time [s] | Download |
|---|---|---|---|---|
| SplitS Track 1 |  |
246.28 | 1654544051.569510 | bag, zip |
| SplitS Track 2 |  |
246.38 | 1654836213.498711 | bag, zip |
| SplitS Track 3 |  |
249.05 | 1654836718.332698 | bag, zip |
| Sequence | Video | Length [m] | Start Time Snapdragon | Start Time Davis | Download Snapdragon | Download Davis |
|---|---|---|---|---|---|---|
| indoor forward 11 |  |
85.68 | 1540824001 s | 1540824001 s | bag, zip | bag, zip |
| indoor forward 12 |  |
124.07 | 1540824296 s | 1540824296 s | bag, zip | bag, zip |
| indoor 45deg 3 |  |
119.82 | 1623 s | 1545305934 s | bag, zip | bag, zip |
| indoor 45deg 16 |  |
58.72 | 133 s | 1545315222 s | bag, zip | bag, zip |
| outdoor forward 9 |  |
314.41 | 372 s | 1540102003 s | bag, zip | bag, zip |
| outdoor forward 10 |  |
455.63 | 674 s | 1540102304 s | bag, zip | bag, zip |
Evaluation Method
The submission will be ranked based on the position accuracy. We use the same metric as the one adopted by KITTI. The average translation error (in percentage) over all possible subsequences of a set of certain lengths is used for ranking the submitted results. We use our publicly available trajectory evaluation toolbox to evaluate the estimated trajectories.
For each x in {40m, 60m, 80m, 100m, 120m}, subtrajectories of length x are uniformly sampled (in time) from the entire trajectory. The smaller the subtrajectory, the more samples there are in a given sequence. For each of those subtrajectories is then the relative position and rotation error computed as in the trajectory evaluation toolbox. For the ranking, the average of relative position error and rotation error is taken across all such samples. The length of the sequence affects the amount of samples in that sequence. For example, outdoor_forward_9 has {964, 916, 862, 807, 762} samples for trajectory lengths {40, 60, 80, 100, 120}. If a trajectory is shorter than x, no samples are generated for x.
Submission
To make a submission, you will need to create an account, follow the instructions below. Each account can make only one submission, but it can be changed afterwards. Submissions are not automatically published – you can review the results and decide whether to publish by yourself (evaluation usually takes around a few minutes). Submissions can also be withdrawn completely.
Click here for the submission format specification.
Error
You must be logged in to upload submissions
Click here to log in or register a new account: Login