Simulation Fidelity Distance: A Game-Theoretic Framework

International audience ; The paper addresses one of the fundamental questions in using simulation as a means for system verification and validation, namely, how far the simulation model represents the real system according to the given test objective. Extending studies on quantitative approaches in system refinement based on two player games to the field of system simulation, distance notions for simulation fidelity are proposed. This fidelity distance could be quantified through alternating simulation games between the system model and the simulation model. This coverage metric initially proposed in literature, is still an absolute distance measure and inadequate for the purpose of determining the fitness of a model for an intended use, since a game relation is established only between the simulation model i.e. implementation and system model i.e. specification. In addition, all strategies of the players must be explored to quantify this error not only globally but also with respect to the given test scenario. In this paper, these games between two untimed transition systems are modeled as DEVS in the ProDEVS tool and a quantitative reachability graph is generated using the TINA tool to explore all such player strategies in the game. Further, a relative-weighted fidelity distance metric to account for given test objectives is proposed which penalize cheats on expected simulation model behavior more than the other behavior. This quantitative reachability graph is analyzed for consistency conditions, player strategies, counter examples, distribution and evolution of cheats to gain further insight into the simulation model behavior with respect to the system model and test objectives. These game notions are discussed in the context of derivability of experimental frame formalism, where a global test scenario is proposed and then consistently refined with respect to the model development cycle. Extending this quantitative approach to timed systems and interface automata is briefly discussed.