A third competition using this dataset was held for IROS 2020. The goal was to estimate the quadrotor motion as accurately as possible, utilizing any desired sensor combinations, and to improve upon the performance of previous competition submissions. The top three submissions outperformed last competition’s best submission.
Results
- Evaluation:The relative pose errors at the sub-trajectory of lengths {40, 60, 80, 100, 120} meters are computed. The average translation and rotation error over all sequences are used for ranking.
- Naming rule: The name in the following table is the combination of the initials of the participant’s last name and the affiliation.
- Sensor coding: S – sensors from the Snapdragon board; D – sensors from the DAVIS.
- References: References (e.g., report) are available upon the participants’ consent.
| Ranking | Name | Sensors | Translation (%) | Rotation (deg/m) | References |
|---|---|---|---|---|---|
| 1 | MEGVII-3D | stereo (S) + inertial | 6.819 | 0.263 | report |
| Can Huang, Ran Yan, and Xiao Liu are with the 3D Team at MEGVII Research. | |||||
| 2 | VCU Robotics | stereo (S) + inertial | 6.891 | 0.265 | report |
| He Zhang and Cang Ye are with the Dept. of Computer Science at Virginia Commonwealth University. | |||||
| 3 | LARVIO | mono (S) + inertial | 6.919 | 0.266 | report |
| Xiaochen Qiu is part of the Navigation and Embedded System Labortory at Beihang University. | |||||
| 4 | Lenovo_LR_ShangHai | stereo (S) + inertial | 7.005 | 0.282 | report |
| LiXin Gao is at Lenovo Research, Shanghai. | |||||
| 5 | Basalt | stereo (S) + inertial | 7.494 | 0.268 | report |
| Nikolaus Demmel is with the Computer Vision Group, TUM. | |||||
| 6 | Xin Zhang | stereo (S) + inertial | 9.140 | 0.219 | report |
| Xin Zhang is with Shanghai Jiao Tong University. | |||||
| 7 | QuetzalC++ | mono (D) + inertial | 34.273 | 1.635 | report |
| J. Arturo Cocoma-Ortega, L. Oyuki Rojas-Perez and Prof. Jose Martinez-Carranza are with the Group of Intelligent Unmanned Aerial Systems (iUAS) at the National Institute of Astrophysics, Optics and Electronics, San Andrés Cholula, Puebla, Mexico. | |||||