Comparing Large-Scale Global Optimization Competition winners in a real-world problem

The optimization of thousands of variables, Large-Scale Global Optimization, is a research topic that is obtainingmore and more attention by its applications in engineering andmedical problems. In order to design evolutionary algorithms forthese problems, several specific competitions have been organized,using benchmarks such as the ones proposed in CEC’2010 andCEC’2013, trying to simulate realistic features of real-worldproblems. Several algorithms have been proposed, some of thembeing very competitive on these benchmarks, especially duringthe last years. However, all of them were tested only on thoseartificial benchmarks, so there are no guarantees that theywould obtain good performance in more realistic problems. Inthis paper, we select the best algorithms in these competitionsto optimize a real-world problem, an electroencephalography(EEG) optimization problem. The new benchmark contains noisyproblems and an increasing number of variables (up to 5000)compared to synthetic benchmarks (limited to 1000 variables).Results show that, although thefitness obtained by the majorityof the algorithms is the same, the processing time stronglydepends on the algorithm under consideration. The optimizationtime for afixed number offitness evaluations varies, in themost complex problems, from 3 hours to around 18 minutes,being MOS-2013 the fastest algorithm. However, if we focus ourattention on the time needed to reach the best-known solution,SHADEILS becomes the fastest algorithm (with a maximum ofthree minutes). In our opinion, this should encourage researchersto continue working in more scalable and efficient algorithms forlarge-scale global optimization.