Input–Output Uncertainty Comparisons for Discrete Optimization via Simulation.

COMBINATORIAL optimization; SIMULATION methods & models; STOCHASTIC processes; EXPERIMENTAL design; DATA analysis

Selecting the optimal policy using simulation is subject to input model risk when input models that mimic real-world randomness in the simulation have estimation error due to finite sample sizes. Instead of trying to find the optimal solution under unknown real-world input distributions by taking a conservative stance or with low statistical guarantee, the input–output uncertainty comparisons (IOU-C) procedure finds a set of solutions that cannot be separated from the best given the resolution decided by the finite sample sizes. The common-input-data (CID) effects measure how differently solutions are affected by the common estimated input models. When CID effects of two systems are positively correlated, the comparison becomes easier than estimating the performance measures of two systems precisely under input model risk; the IOU-C procedure takes advantage of the CID effects to develop a sharp comparison and thereby provides a small subset even in the presence of input model risk. When input distributions to a simulation model are estimated from real-world data, they naturally have estimation error causing input uncertainty in the simulation output. If an optimization via simulation (OvS) method is applied that treats the input distributions as "correct," then there is a risk of making a suboptimal decision for the real world, which we call input model risk. This paper addresses a discrete OvS (DOvS) problem of selecting the real-world optimal from among a finite number of systems when all of them share the same input distributions estimated from common input data. Because input uncertainty cannot be reduced without collecting additional real-world data—which may be expensive or impossible—a DOvS procedure should reflect the limited resolution provided by the simulation model in distinguishing the real-world optimal solution from the others. In light of this, our input–output uncertainty comparisons (IOU-C) procedure focuses on comparisons rather than selection: it provides simultaneous confidence intervals for the difference between each system's real-world mean and the best mean of the rest with any desired probability, while accounting for both stochastic and input uncertainty. To make the resolution as high as possible (intervals as short as possible) we exploit the common input data effect to reduce uncertainty in the estimated differences. Under mild conditions we prove that the IOU-C procedure provides the desired statistical guarantee asymptotically as the real-world sample size and simulation effort increase, but it is designed to be effective in finite samples. The electronic companion of this paper is available at https://doi.org/10.1287/opre.2018.1796. [ABSTRACT FROM AUTHOR]

Copyright of Operations Research is the property of INFORMS: Institute for Operations Research and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)