Since I first became involved with TOK, now about a decade ago, there
has been a great deal of dissatisfaction with assessment. I had little
involvement with the system employed before the current one, but I heard
enough to sense that it was not highly thought of and that there were
definite problems of lack of comparability between schools. I can still
clearly remember my first TOK workshop in Thailand when the leader gave
us a couple of essays to assess using the criteria in operation at the
time. As was obviously expected, the grades awarded by the various participants
were all over the place, varying by a factor of over two. The workshop
leader then, like Moses coming down from Mount Sinai, unveiled the new
grading system, three criteria with grade descriptors of all the possible
achievement levels. We set to work with fresh essay samples keen to
try out this "Gift from the Gods" (or at least from IBEX [the
old name for IBCA, Ed.], which is a pretty close approximation) The
result? A spread of marks at least as great as those that resulted from
the previous system! Teething troubles?
Apparently not. The dissatisfaction with the assessment system has
been a constant theme of the various TOK workshops that I have since
attended and a recurring topic of conversation whenever I meet with
other TOK teachers or IB coordinators. There have been slight changes
in assessment procedure. Firstly (about 5 years ago?) when the requirement
to submit "extra work" was dropped (this "extra work"
was one of the few saving graces of the assessment in my opinion) because,
as I understand, it was impossible to adequately moderate it, and then
the introduction of the requirement that at least one essay be from
the list of prescribed titles (I would vote this one an improvement).
Even so, the underlying reasons for discontent have not gone away.
As far as I can tell, there are two fundamental problems facing assessment
in TOK. Firstly, the validity of the assessment, i.e. is it actually
testing the things that we are trying to achieve in the TOK course?
Secondly its reliability/reproducibility, i.e. would the assessment
score the same irrespective of where the student is, who is assessing
their work and on what day of the week.
Regarding the validity issue. The present system assesses the students
on their ability to write two essays on rather abstract concepts. I
am far from certain that this is a valid test of the ability of a student
to indulge in open ended reflective thought on the bases of knowledge,
to participate in focused discussion on higher level thoughts concerning
the validity of knowledge claims, or to show an appreciation of the
influences on our personal world view and the way in which these differ
from person to person. There is no doubt in my mind that the native
speaker, or student with near native fluency, in one of the IB languages
starts out at a tremendous advantage. Even in the exemplar pack [published
in 1996, Ed.] the high scoring examples are from students with these
language skills -- there is no example of a 15 [maximum score, Ed.]
from students with very limited English. My experience at Li Po Chun,
where there is a wide range of fluency, would certainly back this up.
In addition, the student who has had extensive, detailed training
in the writing of critical essays in subjects such as English (HL) and
History (HL) is often likely to perform better than those from other
specializations. I remember my great dissatisfaction with the system
when, a few years ago in my previous school, the top two TOK scores
came from a couple of academics who had contributed virtually nothing
in terms of oral participation during the two years of the course.
The reproducibility of marking, in my experience, also leaves a lot
to be desired in spite of the grade descriptors and the set of exemplar
essays. My fellow TOK teacher and I are both very experienced and, in
spite of our best efforts to mark at a consistent level, we still seem
to differ by a consistent 1-2 marks! I am certain that this experience
is not unique. Indeed, as far as I am aware this is recognized by assigning
individual moderation factors to each assessor in the team to whom moderation
samples are sent. I personally find that whilst it is very simple to
recognize an essay as a 14/15 rather than a 9/10 or a 5/6, it is almost
impossible to consistently grade an essay as an 8 or rather a 7 or a
9. Indeed, on odd embarrassing occasions my marking of work resubmitted
by students has shown a significantly larger variation! From discussions
with other TOK teachers and assessors, I believe that I am not unique
in this respect: the criteria are just not precise enough (e.g., "clarified"
and "clarified succinctly" in Clarity) and too many independent
variables are grouped together (depth of analysis, counter arguments,
originality in Critical Thought) to allow for clinical precision in
grading.
A further complication is the actual assistance students receive in
producing the essays submitted. In many centres, the essay is graded
in the form that it is originally submitted, though there are schools
where the commonality of themes seems to betray extensive class discussion
in preparation for the task of writing an essay on a particular title.
There are however places where the initial draft is returned to the
student with corrections and suggestions for improvement, so that the
final marked version can be the result of a number of stages of revision.
At present there is a little guidance about the extent to which these
practices are permitted, but inevitably they will favor institutions
with a limited number of candidates, where more individual advice can
be given.
In summary then, the current situation is that students are being graded
on a very imprecise scale by a technique that varies in its method of
application, which does not really reflect the aims and objectives of
the course and is heavily weighted in favour of a certain group of students.
Can this situation be improved upon? I feel that there is some room
for improving the validity of the assessment, for example by including
a component of internal non-written assessment, but any move in this
direction would almost certainly result in an even greater variation
of standards among centres because of the difficulties of moderation.
Similarly the consistency of assessment could be improved in a number
of ways, but again only at the expense of validity.
Given these obstacles, the obvious question to me is, "Do we need
to assess the course at all?"
I believe that TOK is one of the most (if not the most) important component
of the Diploma and that we only devalue it by this totally inadequate
assessment. What I would like to propose is that TOK should become rather
like CAS is at the moment. Details of the programme would have to be
submitted [to IBCA, Ed.] and if necessary they would have to be amended
to conform to the current guidelines, with regard to content. Details
of assignments set for the students, which would have to include some
written assignments (retaining the prescribed titles?) would also have
to be given. Initially samples of these written assignments, from randomly
selected students, would have to be submitted by all schools until the
standard of their programme was felt to be satisfactory. After this
schools could expect to be sampled every 3/4 years on a random basis.
What about the bonus points? Well I for one think that the amount of
time and effort that many students put into their Extended Essay merit
its being worth up to 3 bonus marks on its own.
Will this proposed change devalue the teaching of TOK? I hope not.
I believe that at present it is done well in some places and less well
in others. This will inevitably continue, but I cannot see the fact
of removing the requirement of every candidate to submit two essays
(a task that could be completed in about five hours) should substantially
affect the situation. There has been a similar discussion about whether
to assess CAS for many years. Perhaps like CAS, TOK is too important
to assess (too difficult to assess some cynics might say!) and that
we should be training students to appreciate intellectual reflection,
like service, for its own sake.
|
