Welcome to the RSS 2024 Robust State Estimation Competition. Participants will run their state estimation algorithms on datasets (including images, IMU measurements, and event data) recorded with an FPV drone racing quadrotor flown by an expert pilot and an autonomous drone with speeds up to 100 km/h. The goal of the competition is to estimate the quadrotor motion as accurately as possible, utilizing any possible sensor combinations. The winner will be selected based on the accuracy of the estimated trajectories and will be rewarded with an invitation to deliver a talk at the workshop.
Sequences
Below are the sequences of the RSS 2024 competition. The entire dataset, containing training sequences with open-source ground truth and calibration files, is available here.
| Sequence | Video | Length [m] | Start Time [s] | Download |
|---|---|---|---|---|
| SplitS Track 1 |  |
246.28 | 1654544051.569510 | bag, zip |
| SplitS Track 2 |  |
246.38 | 1654836213.498711 | bag, zip |
| SplitS Track 3 |  |
249.05 | 1654836718.332698 | bag, zip |
| Sequence | Video | Length [m] | Start Time Snapdragon | Start Time Davis | Download Snapdragon | Download Davis |
|---|---|---|---|---|---|---|
| indoor forward 11 |  |
85.68 | 1540824001 s | 1540824001 s | bag, zip | bag, zip |
| indoor forward 12 |  |
124.07 | 1540824296 s | 1540824296 s | bag, zip | bag, zip |
| indoor 45deg 3 |  |
119.82 | 1623 s | 1545305934 s | bag, zip | bag, zip |
| indoor 45deg 16 |  |
58.72 | 133 s | 1545315222 s | bag, zip | bag, zip |
| outdoor forward 9 |  |
314.41 | 372 s | 1540102003 s | bag, zip | bag, zip |
| outdoor forward 10 |  |
455.63 | 674 s | 1540102304 s | bag, zip | bag, zip |
Evaluation Method
The submission will be ranked based on the position accuracy. We use the same metric as the one adopted by KITTI. The average translation error (in percentage) over all possible subsequences of a set of certain lengths is used for ranking the submitted results. We use our publicly available trajectory evaluation toolbox to evaluate the estimated trajectories.
For each x in {40m, 60m, 80m, 100m, 120m}, subtrajectories of length x are uniformly sampled (in time) from the entire trajectory. The smaller the subtrajectory, the more samples there are in a given sequence. For each of those subtrajectories is then the relative position and rotation error computed as in the trajectory evaluation toolbox. For the ranking, the average of relative position error and rotation error is taken across all such samples. The length of the sequence affects the amount of samples in that sequence. For example, outdoor_forward_9 has {964, 916, 862, 807, 762} samples for trajectory lengths {40, 60, 80, 100, 120}. If a trajectory is shorter than x, no samples are generated for x.
Submission
Check here the submission format.
Submit here.
Deadline
The deadline is Sunday, June 30rd, 2024.
Contact
For support, please contact Giovanni Cioffi at cioffi (at) ifi (dot) uzh (dot) ch
Other
The submissions to this competition will not appear in the Standing Leader board. If you want to submit to the Standing Leader Board please visit the corresponding page here.