Off-line evaluation of indoor positioning systems in different scenarios: the experiences from IPIN 2020 competition

Potorti', F.; Torres-Sospedra, J.; Quezada-Gaibor, D.; Jiménez, A. R.; Seco, F.; Pérez-Navarro, A.; Ortiz, M.; Zhu, N.; Renaudin, V.; Ichikari, R.; Shimomura, R.; Ohta, N.; Nagae, S.; Kurata, T.; Wei, D.; Ji, X.; Zhang, W.; Kram, S.; Stahlke, M.; Mutschler, C.; Crivello, A.; Barsocchi, P.; Girolami, M.; Palumbo, F.; Chen, R.; Wu, Y.; Li, W.; Yu, Y.; Xu, S.; Huang, L.; Liu, T.; Kuang, J.; Niu, X.; Yoshida, T.; Nagata, Y.; Fukushima, Y.; Fukatani, N.; Hayashida, N.; Asai, Y.; Urano, K.; Ge, W.; Lee, N. T.; Fang, S. H.; Jie, Y. C.; Young, S. R.; Chien, Y. R.; Yua, C. C.; Ma, C.; Wub, B.; Zhangc, W.; Wang, Y.; Fan, Y.; Poslad, S.; Selviah, D. R.; Wangd, W.; Yuan, H.; Yonamoto, Y.; Yamaguchi, M.; Kaichi, T.; Zhou, B.; Liue, X.; Gu, Z.; Yang, C.; Wu, Z.; Xie, D.; Huang, C.; Zheng, L.; Peng, A.; Jin, G.; Wangh, Q.; Luo, H.; Xiong, H.; Bao, L.; Zhangi, P.; Zhao, F.; Yuj, C. A.; Hung, C. H.; Antsfeld, L.; Chidlovskii, B.; Jiang, H.; Xia, M.; Yan, D.; Li, Y.; Dong, Y.; Silva, I.; Pendão, C.; Meneses, F.; Nicolau, M. J.; Costa, A.; Moreira, A.; De Cock, C.; Plets, D.; Opiela, M.; Dzama, J.; Zhang, L.; Li, H.; Chen, B.; Liu, Y.; Yean, S.; Lim, B. Z.; Teo, W. J.; Leep, B. S.; H. L., Oh

doi:10.1109/JSEN.2021.3083149

Every year, for ten years now, the IPIN competition has aimed at evaluating real-world indoor localisation systems by testing them in a realistic environment, with realistic movement, using the EvAAL framework. The competition provided a unique overview of the state-of-the-art of systems, technologies, and methods for indoor positioning and navigation purposes. Through fair comparison of the performance achieved by each system, the competition was able to identify the most promising approaches and to pinpoint the most critical working conditions. In 2020, the competition included 5 diverse off-site off-site Tracks, each resembling real use cases and challenges for indoor positioning. The results in terms of participation and accuracy of the proposed systems have been encouraging. The best performing competitors obtained a third quartile of error of 1m for the Smartphone Track and 0.5m for the Foot-mounted IMU Track. While not running on physical systems, but only as algorithms, these results represent impressive achievements.