Positioning evaluation and ground truth definition for real life use cases

Evaluating positioning systems has become a matter of heated debate during the last years. There is no clear standard on how these technologies should be evaluated, and no predominant solution for defining the ground truth in order to compare the position estimates. In this paper, we propose a simple and inexpensive solution for tackling both of these problems in real life use cases. With the proposed methodology, it is possible to measure both static and moving targets, by creating a predefined path with checkpoints. Then, a tester, walking over them, while moving or standing still, indicates when the device was over the aforementioned checkpoints. It is also specified how to evaluate the estimates by comparing them with interpolated points of the ground truth trajectory. Two methods are proposed for performing such interpolation. Finally, in order to evaluate the performance of the positioning system as well as the perceived utility of the position estimates from the end user's point of view, a series of statistical parameters is discussed. Additionally, in the context of perceived utility by the end user, a parameter that measures the occurrence of abrupt changes in the position estimates is proposed.