Comparing Learning Approaches to Language Learning. There is More to it Than Bias

On the basis of results on different Natural Language Processing (NLP) tasks we show that when following current practice in comparing learning methods, we cannot reliably conclude much about their suitability for a given task. In an empirical study of the behavious of representatives of two machine learning paradigms, viz. lazy learning and rule induction we show that the initial dfferences between learning techniques are easily overrules when taking into account factors such as feature selection, algorithm parameter optimization, sample selection and their interaction. We propose genetic algorithms as an elegent method to overcome this costly optimization.