Where teachers meet and learn

ETAS Journal Editors' Choice, Number 4 (December 2015)

Enjoy our reading selection: 

Judith Mader and Rudi Camerer: Testing intercultural competence in academic and corporate contexts: The WHAT and HOW

ETAS Journal, Volume 33 Number 1 (Winter 2015, pp. 40 - 42)

Introduction

In this outstanding piece, Mader and Camerer present an innovative test of communication skills. We have selected this because we feel that it could be particularly useful for teachers who are interested in providing their in-company clients with relevant objectives and measurable outcomes. As the authors say, “Test results are seen as proof of success of learning […] and as evidence of return of investment”. ELT teachers are very accustomed to language competency assessment. Exams offered by Cambridge, IELTS, and others place learners’ language abilities at a level, now almost always described by CEFR. But many students need more than just language. They need to communicate effectively in English with people from other cultures, and in order to make this intercultural communication effective, they need to gain certain knowledge and develop suitable attitudes.

This is an innovative approach, and is another reason why this article stands out. Combining descriptors from CEFR with descriptors of knowledge and attitudes, Mader and Camerer have created a hybrid system of assessment. It steps outside the traditional belief that cultures can be defined as static and hence, one would simply be able to anticipate particular factors and behave accordingly. As they point out, this view has “been recognised as having serious shortcomings, even possibly to the extent of encouraging interculturally inappropriate views and behavior”. Instead, the authors assert, one needs to become a better communicator in intercultural contexts. This is a competence which includes language skills, but is more comprehensive. Mader and Camerer provide a compelling method which can measure this hybrid competence. If the learner’s goal is to communicate effectively, and we define this as being able to establish trustful relationships through language while avoiding confusion, offense, ridicule, and unintentional humor, then Mader and Camerer’s method of teaching and testing intercultural competence becomes an interesting and necessary tool. 

David Kaufher

___________________________

Testing intercultural competence in academic and corporate contexts: The WHAT and HOW

Why is a test of intercultural competence necessary?

There are many reasons why a test of intercultural competence (IC) may be necessary. Test results are seen as proofof success of learning (and teaching) in academic and corporate contexts as well as by the institution concerned asevidence of return of investment in courses and staff resources. It is not uncommon for at least a certificate ofsuccessful participation to be required and awarded to participants after a course in IC in corporate contexts. Inuniversity contexts courses in IC are generally credit-bearing and thus performance has to be graded in some way.How this can be done will be addressed here.

Language testing, especially in the case of English, is often regarded as following the gold standard as far as testingprinciples are concerned. There are several cases of language testing experts being asked to advise on testing other subjects, e.g. mathematics or medicine. The main features of good language tests are validity, reliability, authenticity, interactiveness, impact, and practicability. Each of these becomes more or less important depending on who is being addressed, but generally validity is considered the most important with all the other features subsumed in or under validity. For a detailed discussion of features of quality tests and a select bibliography, see Mader (2011).

Validity

The most important type of validity is construct validity, so any discussion of IC testing must begin by clarifying the underlying construct. This must imply and include the following.

(A) A convincing demonstration of the underlying concepts of culture, communication, and competence, each of which should be widely accepted in academic and training environments. The practical IC skills to be tested should clearly relate to these.

(B) A holistic concept of IC which includes aspects of knowledge, self-awareness, expression of unbiased/non-judgmental attitudes, and the practical skills necessary (in face-to-face communication and in correspondence).

(C) A combination of assessment methods which have been proven to provide valid and reliable indications of communicative competence, thus allowing forecasts as to the candidate’s communicative performance outside the test situation.

Culture, communication, competence

Two positions are widely held in the current international debate – a phase characterised by SIETAR, the International Association of Interculturalists, by the need to Refresh the Cultural Paradigm http://www.sietareu.org/seucongress2015. Firstly, IC courses are often based on static concepts of culture usually provided by quantitative (i.e. statistics-based) frameworks. However, static concepts of culture, for instance those equating cultures with nations, have been recognised as having serious shortcomings, even possibly to the extent of encouraging interculturally inappropriate views and behaviour. As a result of this development, the findings of Hofstede (2004), Trompenaars and Hampden-Turner (1994), and others, which played a prominent role over the last twenty years, have lost much of their impact.

Secondly, IC training concepts focussing on aspects of personality and the development of these have also led to serious questions being raised with regards to their effectiveness. These concepts draw on a variety of theoretical frameworks such as Humanistic Psychology, Neuro-Linguistic Programming, or what can be seen as Zen Buddhist traditions. The key concepts which have dominated the training market in periodical waves have used such terms as Sensitivity, Empathy, Emotional Intelligence, Resilience, Polycentrism, and Mindfulness. Naturally, Tolerance of Ambiguity, Behavioural Flexibility, Respect for Otherness, Empathy, and so on are excellent features of character and certainly worth having, but even the most sensitive, empathic, and resilient person may falter or fail in unfamiliar environments. The same person may, for instance, produce feelings of irritation and annoyance in others by – usually unknowingly – applying communication strategies unsuitable for the situation. Spencer-Oatey and Xing (2003)provide an excellent example of this in one of their studies. It is worth mentioning that all of the features of personality listed above lack clear definitions, enjoying only to an extremely limited extent the support of relevant sectors of the academic scientific community. For a detailed discussion of tests of intercultural competence available world-wide, see Camerer (2014).

The connection between culture and communication is evident when it is recognised that personality is something co-constructed through communication and interaction, as most contributors to personality theory agree. Omoniyi and White (2006) summarized the six points as shared by most contributors to personality theory: (1) personal identity is not fixed; (2) personal identity is constructed within established contexts and may vary from one context to another; (3) contexts are moderated and defined by intervening social variables and are expressed through language; (4) personal identity is salient in every communicative context; (5) personal identity informs social relationships and also the communicative exchanges that characterize them; and (6) more than one identity may be articulated in a given context in which case there will be a dynamic of identities management. It is precisely communication and interaction which connect with language teaching and testing, fields which have been our professional resort for a long time.

The connection between culture, personality, and communication is aptly demonstrated using a definition of culture provided by the British anthropologist Brian Street (1993). Following a constructivist view, Street (1993) sees culture as a process of collective meaning-making through communication and interaction. The definition ‘Culture is a verb’, also the title of his essay, is useful in this context.This is particularly so since practical communication has been extensively described in a document of singular importance in the field of language teaching and assessment: the Common European Framework of Reference for Languages (CEFR). Without going into detail, one point needs mentioning here: the CEFR describes language in the context of social interaction and defines a great variety of communicative skills using descriptive scales for the competence levels A1 to C2. What has sometimes been overlooked, even by language teachers, is that of the 54 descriptive scales included in the CEFR only four have accuracy as their focus – grammar, vocabulary, pronunciation, and orthography. All other scales refer to genuinely communicative skills. It is for this reason that we use the CEFR’s detailed descriptions as the basis for defining and specifying communicative competence in intercultural contexts.

Lastly, a competence, to quote a definition suggested by the OECD (2003), “is more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilising psychosocial resources (including skills and attitudes) in a particular context. For example, the ability to communicate effectively is a competency that may draw on an individual’s knowledge of language, practical IT skills, and attitudes towards those with whom he or she is communicating.” Thus the term competence includes knowledge, attitudes, and skills, and makes sense only in relation to a particular context (no one can or will be competent in everything). Competence as a hybrid quality is, strictly speaking, neither observable nor testable. All we can do is draw conclusions based on observable performance.

Intercultural communicative competence

Summarising what we have said so far, intercultural competence may be seen as a hybrid construct involving knowledge, features of personality, and – most of all – communicative skills. All of these relate to specific contexts. When testing a person’s intercultural competence, the underlying assumptions are the following: The interlocutors come from diverse cultural backgrounds and do not know each other well. They are, however, interested in establishing and/or maintaining a positive and trustful relationship. This assumes face-to-face contact and/or contact in writing in an early phase of the relationship.

What this construct implies can be represented by the following diagram: (DIAGRAM AVAILABLE IN JOURNAL)

It may be helpful to emphasise that knowledge does not necessarily imply the reading of academic books, but the awareness that a number of ways of behaviour, values, communication styles, and so on may be dealt with differently from what may seem normal to oneself. This awareness can be expressed, implicitly or explicitly, in oral and written communication, and thus allows the assessment of observed performance. Knowledge also means being prepared for country or context-specific details as indicated by the underlying ellipsis in the above diagram. A separate (individualised) subtest can assess the level of this type of preparation. In intercultural encounters certain personality features, e.g. non-biased/non-judgmental attitudes, naturally play an important part. But contrary to the personality-based training approaches mentioned above, assessment can be limited to observing a candidate’s actual performance in terms of verbal and nonverbal utterances in speaking and writing, as it is these which will count in actual intercultural encounters, rather than any assumed features of personality.

Finally, when rating a candidate’s communicative competence it is important to note that this is not identical with the rating of linguistic accuracy. Linguistic features need be rated only in so far as they restrict the effective process of establishing trustful relationships by, for instance, causing confusion, misunderstanding, offence, ridicule, or unintentional amusement.

Valid and reliable assessment methods

Any test attempting to follow all the principles of good tests to the letter is bound to run into difficulties as, in general, each of them restricts the possibilities of the others. For example, a test high on validity may be low on practicability, as it is too long or too expensive to mark. For this reason, item types such as multiple-choice and true/false can and should be used but set with care to reflect the construct as closely as possible.

One example can be given here:

A foreign partner asks you how much you earn. You don’t want to answer the question so you say

a) “Enough to live on. What about you?”

b) “Enough for me and my family” and try and talk about another topic.

c) “I’d rather not talk about this!” because he should know it’s a personal question.

Clearly closed items cannot test all aspects of intercultural communicative competence and so performance in written and oral contexts must play a part. These sections are marked by human raters using marking criteria which require a similar effort in their setting as the closed items mentioned above if they are to fulfil their function. The marking criteria must reflect the importance of successful relationship-building through interculturally appropriate communication and the relative unimportance of linguistic correctness.

Here are the rating criteria, e.g. for oral performance as used in the ICE test. (AVAILABLE IN THE JOURNAL)

In addition to these, detailed descriptors for each number of points awarded are provided, as only so can the criteria be applied by raters in a standardised way. Bearing in mind that even the best test is always a compromise, it is nonetheless governed by rules which should be followed as closely as possible. Once the above questions have been answered satisfactorily, the question of practicability arises, including the test administration, the qualifying of raters, regular item-analysis, and development and confidentiality measures, many of which relate directly to fairness.

The Test of Intercultural Competence in English – ICE

This test attempts to follow all that has been described above and provides a useful instrument to assess whether a user of English is able to deal with possibly difficult situations in intercultural environments. A level of English language competence around B1 is assumed at a minimum, but the test is also available at a higher level (B2 and above). This level refers to the language used in the items and tasks, the relative complexity of the tasks, and the level of linguistic competence expected in the tests of oral and written competence. All the above is to be found in the ICE test. A detailed description of the background and the development of the test, the test specifications and sample items are available as free downloads at http://www.elc-consult.com/49601/home.html

References

Camerer, R. (2014). Testing intercultural competence in (International) English: Some basic from questions and suggested answers.

Language Learning in Higher Education, 4(1), 207-236. Retrieved from http://www.elc-consult.com/96101.html

Hofstede, G. (2004). Cultures and organizations: Software of the mind. New York, NY: McGraw-Hill USA.

Mader, J. (2011). Professional Business English trainers: Testing and assessment in Business English. Berlin, Germany: Cornelsen 

Verlag Gmbh.

OECD (2003). Definition and selection of competencies: Theoretical and conceptual foundations (DeSeCo). Summary of the final 

report Key Competencies for a Successful Life and a Well-Functioning Society. Retrieved from 

http://www.oecd.org/education/skills-beyond-school/definitionandselectio...

Omoniyi, T., & White, G. (2006). The sociolinguistics of identity. London, UK: Continuum.

Spencer-Oatey, H., & Xing, J. (2003). Managing rapport in intercultural business interactions: A comparison of two Chinese-British

welcome meetings. Journal of Intercultural Studies, 24, 33-46. Retrieved August 7, 2015, from

http://wrap.warwick.ac.uk/2684/1/WRAP_Spencer_Oatey_0673125-cal-290110-s...

Street, B.V. (1993). Culture is a verb: Anthropological aspects of language and cultural process. In D. Graddol, L. Thompson, & M.

Byram (Eds.), Language and culture (pp. 23-43). Clevedon, UK: BAAL and Multilingual Matters.

Trompenaars, F., & Hampden-Turner, C. (1994). Riding the waves of culture: Understanding diversity in global business. London:

Nicholas Brealy Publishing.

About the Authors

Judith Mader is Head of Languages at the Frankfurt School of Finance and Management and Senior Advisor to elc – European Language Competence. She is the author of a number of books and essays in the fields of language teaching and testing. Together with Rudi Camerer she co-authored Intercultural Competence in Business English (Cornelsen 2012). 

(J.Mader@elc-consult.com)

Rudi Camerer directs elc – European Language Competence, a private consultancy based in Frankfurt am Main and Saarbrücken. He is the author of various essays on the teaching and testing of intercultural competence. Together with Judith Mader, he co-authored Intercultural Competence in Business English (Cornelsen 2012). 

(R.Camerer@elc-consult.com)

Member, ETAS Journal Editorial Board