Researching language learning: theories, evidence, claims
Evidence and ILL research: claims and warrants
Experimentation, comparative method and classroom interaction studies
In the context of advocating a 'culling' of proliferating SLA theories, with a view to establishing SLA as a 'normal science', Long (1993) addresses the kinds of methods which research should employ, and he identifies the experiment as 'the highest form' of the 'scientific' approach, and 'third-party replication of experiments' as 'one of the most crucial safety measures for protecting the integrity of a field' (p.233). In our area of interest, it is psychometric investigations in particular which make use of experimental methods, and ILL research may draw on this approach to inform pedagogical practice, even though the research may take place outside any actual classroom (Nunan 1991, cited in Ellis 1994: 566). Thus, issues of cognitive processing, such as whether automatic recognition of spelling errors is linked with learners' general ability in the L2, for example, may generate experiments in laboratory conditions. Learners are tested on their performance in the target language, and then, in a 'controlled' environment such as a computer laboratory, they are presented with stimuli such as specific words to which a response is required. Measurements are taken- of each subject's response times, for example- and the results compared either across groups of subjects or for correlations with test results, and so on.
In practice, many ILL studies are necessarily quasi-experimental, obliged 'to sacrifice some controls because of real-world constraints' (McDonough and McDonough 1997: 160). Gregg (1993) argues that experimental intervention is necessary in research, so that underlying mechanisms may be identified and the construal of laws as nothing more than empirical regularities can be avoided. Whilst we share his view of the importance of distinguishing causal connections from empirical regularities, we are at odds with his optimism that experimental methods can be relied onto do this, for two reasons. Firstly, the conditions for experimental research in the social world are exceedingly difficult to bring about and sustain (not to mention the fact that they are likely to be ethically dubious). Secondly, our view of language as a cultural emergent property would entail a very particular role for experimental research, one which '... should take the form of examining the single empirical relationships in a great many contexts rather than the current practice of examining a great many relationships in a single context' (Pawson 1989: 202). However, this approach is challenging to operationalize.
Comparative method studies aim to establish which of two or more methods of language teaching 'is most effective in terms of the actual learning (the "product") that is achieved after a given period of time' (Ellis 1994: 569). Classroom interaction research focuses on the characteristics of the interactions between teachers and learners (including learner-learner interactions), using various techniques of observation. Such studies may involve pre-coded schedules and the quantification of the occurrence of types of interaction or utterances; it may focus on any number of kinds of discourse produced in the classroom, on error correction or the processes of small group work and so on. ILL evidence derived from the observation of classroom interaction may be collected with the goal of generating 'understanding' rather than 'proof' (van Lier 1988), seeking to be 'interpretive' as opposed to 'normative'. On the other hand, some classroom interaction research 'manipulate[s] instructional variables such as participant (as in studies of small group work) and task (as in studies of different types of pedagogic tasks) experimentally' (Ellis 1994: 573). Evidence resulting from this approach is vulnerable to the critiques of variables-based epistemologies and successionist models of causality which we have discussed above, so we will not repeat those arguments here.
