DEVELOPMENT AND RESEARCH IN APPLIED LINGUISTICS
Language assessment
Research
The role of language-testing research as an activity of applied linguistics is to further our understanding of language learning and illuminate the still uncharted space of language use. I do not share the ambition of some applied linguists to map out language use so that it becomes more and more systematized with its own rules of use. After all if that is what were eventually to happen then language use would be more and more taken over by language form, yielding itself to control by the rules of linguistics. That could eventually lead to a situation in which all language behavior and knowledge are rule-governed, with nothing left to chance or to spontaneity. Because I am skeptical of this ambition, I am content with smaller successes, offering partial and temporary understandings.
In the 1980s one of the key research issues in language assessment was that of the so-called unitary competence hypothesis (UCH), or general language factor. This hypothesis, advocated by among others John Oller (1983), stated that underlying all language abilities was one primary factor, the general language factor. There was, of course, a deliberate connection with the ‘g’ of intelligence tests. As it turned out this hypothesis was eventually agreed to be too powerful, as Oller himself recognized. Language ability was not unitary but binary or multifactorial. But if language was not unitary, did that mean that there were separate abilities (and separate knowledges) for different areas of language, known in various contexts as styles, registers, genres, specific purposes, varieties, or rhetorics? The question of the separation of language varieties, itself a special case of the larger language question of the discrete nature of dialects and languages, is basic both to much practical language activity and to our theoretical understanding of what makes up language use.
Language teaching and language assessment in recent years have concentrated much of their efforts on the teaching and assessment of languages for specific purposes (academic, professional, occupational, medical, legal, economic and so on). Such work on English, known as English for Specific Purposes (ESP) took a directly counter-position to that of the unitary competence hypothesis since it seemed impossible that both could be upheld. With the abandonment of the unitary competence hypothesis then, it seemed that the ESP difference proposal must prevail. After all, are they not mirror-images of one another?
From an applied linguistics perspective this is not necessarily the case. It would indeed be possible for both to fail: no UCH and no ESP. If a multiplicity of language factors makes more sense of the evidence than does a unitary hypothesis, then it might also be the case that the multiplicity does not refer to specific purposes, all of which could be informed by one overriding purpose, but to (for example) more internal personal factors such as gender, age and motivation and less external social factors, such as professional academic and occupational purposes.
Research on the stability of specific purpose was initiated by Alderson and Urquhart (1985). Three studies were carried out, all studies used English proficiency tests but only in the third was the test the major investigation instrument:
Study 3 was conducted using parts of the British Council’s ELTS [English Language Testing Service] test, which contains specialized study skills modules aimed at different content area groups, e.g. ‘Medicine’, ‘Life Sciences’. A student sitting ELTS chooses whichever of these modules is appropriate to his course of studies … Three ELTS modules were used … namely Social Studies, Technology and General Academic. (ibid: 196)
Quite deliberately the design of the study was a test rather than an experiment. There were two reasons for this. The first was that the tests were being used in a functionally appropriate way and would be viewed by the candidates as tests rather than as experiments. Artificiality was therefore somewhat controlled. The second reason was that these ELTS tests are ‘social facts’. They are (or were in the mid-1980s) used as part means of determining adequacy in English proficiency levels for overseas students seeking admission to UK universities. Whether or not they are in themselves adequate statements about distinct registers of English, they were used as if they were. What the researchers did was straightforward, as so often good research is. They administered the tests of different ESPs to different groups of candidates with different kinds of ‘background knowledge’ who would normally have taken one of the modules.
Drawing together their results from all three studies the researchers concluded that ‘academic background can play an important role in test performance. However, the effect has not been shown to be consistent … the studies have also shown the need to take account of other factors such as linguistic consistency’ (ibid: 202).
But the most interesting conclusion they reach is the distinction they make between direct and overview questions with relation to accessing the content area under test. They report:
when these students were familiar with the content area, they were able to answer direct and overview questions with equal ease; when this familiarity with the content area was lacking they could still answer direct questions, but their ability to answer overview questions was greatly reduced. (ibid: 202)
Although they do not say so the implication of this finding is that background knowledge matters: direct questions do not in themselves probe sufficiently into background knowledge whereas overview questions do. That is why in some of the comparisons they make there was no distinction on test results between groups who had background knowledge and those who did not, because what was at stake was a preponderance of direct questions.
This finding, properly muted though it is by the researchers, aware of the in adequacy of the tests themselves as valid representations of their content areas, does in fact match the earlier research finding we referred to, that is that the unitary competence hypothesis could not be supported. Similarly, here what was indicated by this research was that there are indeed real differences between language varieties. The researchers carefully point to the need to distinguish between linguistic proficiency (which is the subject of their study) and linguistic competence. On the basis of their study (and perhaps too in the unitary competence research) what is established is that there are different proficiencies not that there are different competences. As Alderson and Urquhart say: ‘the part played … by linguistic competence … remains unknown’ (1985).
