What Good Are Experiments?
Abstract
One way to study an unknown system is to perform experiments on it. How does the ability to control the inputs to a system affect the number of experiments (input/output pairs) needed to determine that system's function? How can a clever experiment selection strategy affect this number? An empirical study was performed using Boolean truth tables as hypotheses. Computer programs were constructed to model several different experimentation strategies. The average number of experiments needed to determine a target theory from a set of hypotheses was measured. Results demonstrated that control of the system inputs gave the most dramatic increase in performance. The most effective experimental strategy tried was a (very expensive) greedy algorithm that attempts to find an experiment that “splits the hypothesis space in half.”However, a much cheaper strategy–that selects any experiment guaranteeing the elimination of at least one hypothesis from the set being considered–was found to be almost as effective.
Cite
Text
Ruff and Dietterich. "What Good Are Experiments?." International Conference on Machine Learning, 1989. doi:10.1016/b978-1-55860-036-2.50036-9Markdown
[Ruff and Dietterich. "What Good Are Experiments?." International Conference on Machine Learning, 1989.](https://mlanthology.org/icml/1989/ruff1989icml-good/) doi:10.1016/b978-1-55860-036-2.50036-9BibTeX
@inproceedings{ruff1989icml-good,
title = {{What Good Are Experiments?}},
author = {Ruff, Ritchey A. and Dietterich, Thomas G.},
booktitle = {International Conference on Machine Learning},
year = {1989},
pages = {109-112},
doi = {10.1016/b978-1-55860-036-2.50036-9},
url = {https://mlanthology.org/icml/1989/ruff1989icml-good/}
}