ABSTRACT
This presentation deals with the outline of a research programme for systematically investigating the competence problem of intelligent systems. As a first step towards competence assessment, it seems highly desirable to clarify basic concepts and to resolve some confusion. The focus of the paper is on developing fundamental concepts, although the underlying more general intention is to set the stage for a whole research programme. Some preparatory work on the competence problem for inductive inference devices has been published during the last year.
The development of knowledge-based systems is an exciting area of research and applications. With the growing complexity of application domains for artificial intelligence (AI) systems, there is a growing need for control. But simultaneously, the growth in complexity makes AI systems less controlable. This bears abundant evidence of the need for computer support in system supervision and even for computer self-control. In particular, the competence problems of most current AI systems, i.e., the problem to decide whether or not a certain problem on hand falls into a system's area of competence, is still open. The results reported by van Harmelen, et al. in their Esprit Project report (P3178) are not very satisfactory. The present work is ultimately focussing on the development of AI systems ``knowing'' about their competence.
Reflection denotes someones activity of thinking about oneself as well as about one's relation to the outside world. In particular, reflecting means pondering about ones capabilities and limitations. Reasoning about ones competence is a central issue of reflective behaviour.
A crucial step towards the competence assessment for AI systems is to characterize versions of reflection. Interestingly, this is remarkably more involved than usually expected. Versions of reflection about a system's competence are underlying some scenarios of competence assessment and competent behaviour.
The intention of the presentation on hand is twofold. Mainly, it is aimed at the development of a research programme into the problem of competence assessment for intelligent systems, in general. Thus, it contains a collection of problems and approaches. More specifically, it is introducing a case study in the area of inductive learning systems which enjoy a certain degree of reflective behaviour. Thus, it should contribute to a better understanding of the general phenomenon of reflection, to a more lucid classification of problems and obstacles, and to a more systematic outline of a programme.
In technical terms, there are introduced versions of reflection characterizing a learning system's ability to accept or reject learning data provided. One of the key problems attacked is to exhibit the strength of those requirements. Typical results exhibit either the equivalence of formally different approaches under certain conditions or the distinction of some only slightly different formalizations under some other assumptions.