Douglas R. Shier

Department of Mathematical Sciences

Clemson University

Clemson, SC 29634-0975

Abstract: The feature article by McGeoch in this issue is a valuable contribution to the operations research/computer science community. It is written at a very accessible level and provides a useful reference for wide-ranging information on the experimental evaluation of algorithms. More importantly, this article surveys a number of methodological issues that arise in algorithm experimentation. These issues are illustrated using the example of bin packing, where weights are drawn from a uniform distribution. This article also provides a host of practical suggestions, as well as pitfalls to be avoided, when carrying out experimentation on algorithms. A major point, and one cogently argued by McGeoch, is that experimental analysis of algorithms can greatly aid our understanding of algorithms. As a consequence, the exploration must be carried out with care and with thoughtful design. This activity is no less challenging than the research tasks more traditionally undertaken within our disciplines. The objective of the present commentary is to reinforce certain issues raised in this feature article, and to suggest a conceptual framework for future exploration.

Key Words: algorithm design, algorithm development, computational experiments