Benchmarking Multi- and Many-Objective Evolutionary Algorithms Under Two Optimization Scenarios

Recently, a large number of multi-objective evolutionary algorithms (MOEAs) for many-objective optimization problems have been proposed in the evolutionary computation community. However, an exhaustive benchmarking study has never been performed. As a result, the performance of the MOEAs has not been well understood yet. Moreover, in almost all previous studies, the performance of the MOEAs was evaluated based on nondominated solutions in the final population at the end of the search. Such traditional benchmarking methodology has several critical issues. In this paper, we exhaustively investigate the anytime performance of 21 MOEAs using an unbounded external archive (UEA), which stores all nondominated solutions found during the search process. Each MOEA is evaluated under two optimization scenarios called UEA and reduced UEA in addition to the standard final population scenario. These two scenarios are more practical in real-world applications than the final population scenario. Experimental results obtained under the two scenarios are significantly different from the previously reported results under the final population scenario. For example, results on the Walking Fish Group test problems with up to six objectives indicate that some recently proposed MOEAs are outperformed by some classical MOEAs. We also analyze the reason why some classical MOEAs work well under the UEA and the reduced UEA scenarios.