2000 Jones

The context in which this paper is written is one where many educators feel that government policy on education in the UK is driven by theories from the study of management rather than education. Practitioners, however, are increasingly accepting that in the information age, with a knowledge explosion which threatens our capacity to remember what is available and relevant, the educational endeavour is about the processes of education much less than the content. As Klenowski argues (1996), there has been a move from a banking model of education to a process based model. In a similar vein, Benn and Fieldhouse (1998) suggest that "if learning is seen as the continuous act of making sense and fitting into experience rather than the absorption of pre-ordained knowledge, then teaching is the provision of opportunity to make sense . . ." This paper explores issues of assessment within this process based context, though it may also have relevance to those involved in other forms of educational activity.

I am not concerned here with the ultimate purpose of an assessment. I am not concerned whether the purpose is to rank students in order of merit or to measure them against some criterion or other, as described by Alison Wolf (1993). My concern is with the processes of making judgements. As Rowntree (1987, p.5) suggests, "all these shades of assessment can be practised without any kind of measurement that implies absolute standards; it may be enough simply to observe whether, for each student, some personal, even idiosyncratic, trait or ability appears discernible to a greater or lesser extent than hitherto."

The ideas in this paper started to germinate over twenty years ago. At that time I was involved in research looking at the way in which teachers in Britain assess and evaluate creative art classes in continuing education. In this discussion I am adopting the convention of using the term assess to refer to the process of making judgements about student learning and the term evaluate to refer to making judgements about the success or otherwise of a course of study; one assesses students and evaluates courses.

At that time the conventional approach to assessment was to have an exhibition of the work completed by the students with portfolios of drawings available for inspection. Judgements about success or otherwise were based on an appraisal of this work. At the time it seemed to me that the judgements made about the work were entirely subjective and aesthetic in nature. They said nothing about the learning that had taken place. Indeed, if students had been working creatively, what they produced could well be outside the experience of both teachers and assessors.

The prevailing philosophical basis for this approach to assessment was a belief in aesthetics; a belief that it was possible to make objective aesthetic judgements about works of art and establish their value in absolute terms. The ability to make these judgements depended on training in the prevailing ethos of the time and acceptance into the academy of those who were the arbiters of taste.

To many of us at the time it seemed that this approach ignored the reality of what went on in the art world. Even a cursory look at the history of European art was enough to demonstrate that standards in the arts were not constant. There was a history of avant-garde movements that challenged the established orthodoxy, eventually gaining critical and public acceptance before becoming the new orthodoxy. There were, and still are, fashions in the arts.

We had all been to exhibition openings where a skilful gallery owner would promote the works of a young new painter. The work of the artist would be launched like any other commodity. Influential people would be invited to the opening, critics would be invited to make favourable reviews, press releases would be sent out and attempts made to attract publicity. This was a commercial enterprise where the promoters of this particular artist sought to persuade the art buying public that this newcomer produced work of a high standard that was collectable. Faced with such evidence, it was impossible to say that values in the arts were absolute.

So, if there were no objective standards in the arts, how could one possibly assess students on the basis of making some sort of aesthetic judgement about their work? At best one would be saying that their work conformed to some predetermined view of what constituted ‘good art’. But this idea of what was ‘good art’ was the product of a particular culture and located in a particular time and place. Views of what was good or bad were certainly not international. There was a trend towards internationalism in the art world but this was only amongst a fairly elite group of connoisseurs. I have demonstrated elsewhere (Jones, 2000) that the great majority of the population of the UK often had very different views. They had a different value system. They liked different sorts of art. And so did people in other cultures. There was no absolute aesthetic value system accepted by everyone, including those people from different cultures both in Britain and abroad.

The prevailing value system in the UK art world was white European and driven by that academy of professional arbiters of taste which persuaded us that the work which they themselves esteemed was of high value and status and superior to what the rest of us liked.

This questioning of the prevailing value system can now be seen as part of that philosophical movement which has become known as post-modernism. It was the beginning of that shift from a belief in absolute values to an acceptance of the relativity of values in the arts.

The challenge that this change of perspective presented to educators at the time came to focus on questions of how, in the absence of an absolute aesthetic value system, could one assess the work of our students. Faced with this problem, it became necessary to go back to first principles. Just what was one trying to do in this assessment process?

Reflecting on this it was realised that what we wanted to do was to identify the learning that had taken place and to compare this to expected outcomes. These learning outcomes were derived from an analysis of how artists create, the processes of the creative arts. This model of creative activity had been devised in a research project aimed at developing approaches to assessment and evaluation in creative adult art classes.

During the course of this research I became aware that there were four areas of development with which teachers were concerned. I have written about these more fully elsewhere (Jones, 1988) but will refer to them briefly here. They were:

In terms of conventional educational thinking these could be seen as aims. A list of objectives could be derived from each aim. There were areas of discretion. What counted as ‘relevant’ knowledge, for instance, was open to debate. What should be noted is that this formulation represents a shift from a statement of learning outcomes based on the content of learning to one based on developmental processes. I will return to this point later. This shift from content to process was referred to earlier as a way of restating the adult curriculum (Benn and Fieldhouse, 1988).

Having arrived at this formulation, it became necessary to develop ways of finding out if an ability to become involved in these developmental processes had been learned. Interestingly, at the time, I didn’t think of this as assessment. I simply asked myself the question, how will I know if my students can become involved in the creative process. I realised that simply looking at the paintings they had produced was not enough. I needed more evidence. ‘Evidence’ is the key word here.

During the course of my research, and subsequently, I have collected information about the evidence used for assessments. In the beginning this was in the field of the visual arts but more lately it has been extended to other areas of teaching and learning. As a result of this I have been able to classify the evidence used for assessment into three types. They are

The first type of evidence is product based and is, in a sense, where I started. One has to ask what we can tell about students’ development from looking at what they have produced. Initially I was concerned with paintings and sculptures and other artefacts but similar evidence can be collected from essays and assignments, from videos or audio-tapes, from performances or from field-work, even from answers to examination questions. The focus is on the products of educational endeavour as a source of evidence on which judgements can be made about student learning.

The second source of evidence, that which comes from watching students working, is often neglected. Many teachers of art say that they can tell when students are creatively involved just by looking at them when they are working creatively. In areas of education that involve the learning of practical skills this is also true. But one can also obtain valuable information from watching how students participate in a seminar discussion, or go about planning an essay or conducting an experiment or tackling a problem in mathematics.

The third type of evidence, oral evidence, obtained in conversation with the student is of equal importance. One can learn a lot about students by simply talking to them. The viva voce examination is obviously an extension of this approach to the collection of evidence. But evidence does not need to be collected within the framework of an examination. If opportunities are found to enter into a dialogue with individual students, then the teacher/assessor might gain valuable insights into the thoughts, the mental and psychological processes of that student.

It struck me that all these sources of evidence should be available to those who are making an assessment of a student’s learning. For any learning outcome, we can decide what sorts of evidence will be most appropriate.

But before one can begin to collect evidence of these three types, one needs to ask a more fundamental question – what counts as evidence? What counts as evidence that a student is more perceptive? What counts as evidence that a student is working creatively? What, in other words, are the indicators of creative activity? How do we know when it is taking place? It is one thing to recognise it in ourselves. It is a quite different undertaking to recognise it in other people. It became clear that the next step was to identify the indicators of creative activity.

What emerged from this enquiry was a sequence of steps that formed the basis of an assessment strategy for creative arts classes. Having subsequently discussed this with people both inside and outside the world of arts education, it has become clear that many educationalists from a range of different subject areas see this strategy as having relevance to their work. It is for this reason that I have come to think of it as a generic model of assessment. I want to go on now to outline the stages in this model that have been identified so far.

The model begins, then, with an identification of desired learning outcomes. There is nothing new in this, other than the fact that we are thinking in terms of the processes of learning rather than content. I prefer to express these in developmental terms but they could just as easily be expressed as aims and objectives or as competencies, though this latter term poses problems when applied to anything other than the most basic of skills. There is a danger that the creative process, a process which many people regard as having a spiritual dimension, is reduced to the level of a manual skill. Creating is not the same as making.

We then ask what indicators will tell us whether or not students have attained these outcomes. We may need to go to the literature for this. Psychology and sociology may both help us to identify the indicators for which we are looking. We may also ask practising professionals how they recognise in others the skills and abilities they themselves need for their professional work. How do they tell if another person is competent in their profession? These are the indicators that we need to know about.

We then ask about what sort of evidence we should collect to enable us to make a judgement about whether or not students have achieved the desired learning outcomes. Evidence can come, as suggested earlier, from three sources and we must decide which sources are most appropriate for our purposes. One could draw a matrix with learning outcomes down one axis and sources of evidence along the other with check marks in the boxes to indicate appropriate sources of evidence. The following example might help to explain.

Learning outcome	a. Evidence from products	b. Evidence from observation of students working	c. Oral evidence
1. The development of visual perception	a		a
2. The development of an ability to exploit the potential of a medium	a	a	a
3. The development of an ability to become engaged in the creative process		a	a
4. The acquisition of relevant knowledge	a		a

It is obviously important to ask who carries out these procedures. At any of the stages mentioned above the process can be undertaken by individuals or by groups of individuals. Ideological considerations may ensure that some teachers will want to make these decisions in consultation with students. In other cases an academic board or a group of subject specialists may want to undertake the planning process.

In many institutional settings, and especially when validation and certification are concerns, the question of who makes these decisions is important. So far we have addressed the question of how to arrive at a procedure for assessment and indicated that choices can be made about who designs the procedures, just as they can be made about who carries out the assessment. As well as teachers, one must consider what part students can play in an assessment as well as what part external examiners might play. The answer to these questions usually depends on the purpose of the assessment. One must also make some judgement about when these assessments take place.

Conventionally assessments are either diagnostic, formative or summative (cf. Scriven, 1967, Daines, Daines &Graham 1992). These are seen as reasons why an assessment is carried out, but more importantly they dictate when it is carried out. They can take place at the beginning of a course if diagnostic, during a course if formative and at the end of a course if summative.

Two of the criteria that are usually used to judge the effectiveness of assessment procedures are concerned with validity and reliability. The validity of an assessment procedure depends on being able to demonstrate that the procedure measures or identifies the learning which it purports to measure or identify. This may seem obvious but it is all too easy to devise procedures that do not do this. The procedure for assessing art students outlined at the beginning of this paper is a case in point.

The reliability of any assessment depends on having procedures that ensure a fair and accurate description of a student’s ability and which can be replicated to give the same or similar results with different cohorts of students. The more reliable an assessment procedure is, the more likely it is to give us an accurate picture of student progress.

The model that I am proposing can accommodate concerns about reliability and validity. Indeed, by drawing on several different sources of evidence, it can optimise validity and reliability. If, for a given learning outcome, one can collect evidence from two or three different sources, then one can arrive at a sort of consensual verification that this outcome has been achieved.

This model has been tested in creative art classes and proved to be useful. I am proposing here that it can also work in other areas of educational activity. Having developed the model at a theoretical level, it now needs testing in the field.