Knowledge must come through action; you can have no test which is not fanciful, save by trial.

Sophocles c. 495-406 BC: Trachiniae, l. 592

Introduction: defining the terms

1. A definition of ‘test’ from the American Heritage Dictionary

 

 

test1

?noun

1. A means of examination, trial, or proof.

2. A series of questions or problems designed to determine knowledge or intelligence.

3. A criterion; standard.

4. CHEMISTRY a. A physical or chemical reaction by which a substance may be detected or its properties ascertained. b. The reagent used in such determination. c. A positive result obtained. 5. A cupel.

?verb .tested., testing., tests.

?transitive

1. To subject to a test; examine.

2. a. To determine the presence or properties of (a substance). b. To assay (metal) in a cupel.

?intransitive.

1. To undergo a test.

2. To achieve as a score or rating through testing.

3. To exhibit certain properties under test conditions.

4. To administer a test in order to diagnose: test for acid content. [ME, cupel < OFr., pot < Lat. testum.]

test2 ?n. A hard external covering, as that of certain insects and other invertebrates. [Lat. testa, shell.]
 
 

2. A definition of ‘avaluació inicial’, ‘avaluació formativa’ and ‘avaluació sumativa’ from Ribé & Vidal (1995)

(a) Inicial: Ens dóna a conèixer les actituds, els coneixements previs i les capacitats dels alumnes.

(b) Formativa: Orienta als alumnes. Els hi dóna pautes per ser més sistemàtics en l’aprenentatge i

fomenta l’autocorrecció per tal que l’alumne prengui consciència del seu progrés i estigui motivat.

(c) Sumativa: Mostra a l’alumne i al professor fins a quin punt s’han assolit els objectius fixats.

L’Avaluació Formativa té un caràcter regulador, orientador i autocorrector del procés educatiu. El seu objectiu és formar a la persona i millorar el procés, els materials i el programa. L’avaluació formativa implica al professor i a l’alumne, els fa conscients dels processos duts a terme en la consecució del que s’havia planificat i negociat prèviament. Els seus instruments propiciaran la reflexió i ens mostraran en quin put ens trobem respecte els objectius / continguts que ens havíem proposat.

Els instruments poden fer referència als objectius o als continguts en general o a aspectes més concrets de la programació.

Com a professors hem de trobar respostes a la informació que obtindrem de les avaluacions formatives, orientant l’esforç de cada alumne i optimitzant els seus hàbits d’estudi i de treball. També a partir de les respostes podrem programar activitats d’ampliació, de reforç, alternatives lectures opcionals o addicionals. Segons siguin les respostes o els resultats, ens plantejarem si la dinàmica de la classe és la escaient o si hem de replantejar-nos el treball de grup, el treball individual, si la metodologia és la més convenient al grup, etc. Revisant els objectius amb els nostres alumnes ens adonarem de si hem de fer modificacions o si ens trobem en el camí correcte. A través de l’avaluació formativa facilitem a l’alumne la comprensió i adquisició dels objectius que ens havíem proposat i serveix als professors per pronosticar a temps les possibles errades de la seva planificació pel que fa a una determinada classe.

L’Avaluació Sumativa avalua el nivell assolit per cada alumne en relació al seu compromís inicial, és a dir, en relació amb els objectius (conceptuals, procedimentals i actitudinals) que ens havíem proposat d’assolir. Es consideren tant els objectius del grup com els individuals, tant si aquests són d’ampliació, reforç o una barreja d’ambdós. D’aquesta manera estarem tenim en compte el mètode constructivista, portant a terme una avaluació integradora que té en compte les diferències individuals. Es porta a terme al final d’una o més unitats d’ensenyament-aprenentatge (seqüències, unitats didàctiques, crèdits, ...) i en alguns casos serveix per qualificar. Al terme de cada cicle o etapa serveix per promocionar. Amb la informació que obtinguem d’una avaluació sumativa podrem veure la capacitat o la dificultat que l’alumne haurà d’afrontar en el següent pas, sigui aquest en una nova unitat de programació, en una seqüència o en un nou curs. El tipus d’exercicis d’avaluació sumativa en una llengua estrangera poden arribar a ser molt diversos i respondrà naturalment als objectius establerts, encara que comptem també amb aquells que ens serveixen per reconèixer (professor) o conscienciar (alumne) que l’alumne ha dut a terme tot el que estava en les seves mans per assolir els objectius proposats.
 
 
 
 

3. A comparison of the definition of ‘assessment’ and ‘evaluation’ from different sources

• Ribé R and Vidal N. 1993. Project Work Step by Step. Heinemann pp. 92-93.
• Harris M and McCann P. 1994. Assessment. Heinemann pp. 20-91
• Richards, J., Platt J. and H. Weber 1995, 4th, 1985 1^st. Dictionary of Language Teaching and Applied Linguistics. Longman.
• Sawyer W., Watson K. and A. Adams 1988. English Teaching from A to Z. Oxford University Press.

• Procedures for measuring the extent to which students have achieved their objectives. (R & V)
• Measuring our students’ performance in any one of many different ways, diagnosing the problems and measuring the progress students make. (H & Mc)
• The measurement of the ability of a person or the quality or success of a teaching course, etc. Assessment may be by test, interview, questionnaire, observation, etc. For example, assessment of the comprehension of an immigrant child may be necessary to discover if the child is able to follow a course of study in a school, or whether extra language teaching is needed. Students may be tested at the beginning and, again, at the end of a course of study to assess the quality of the teaching on the course. (R, P & W)

• Student’s and teacher’s ongoing analysis of the process and end result of a task or set of tasks. Decisions are made concerning ways of improving the task(s) as a result of this analysis. (R & V)
• Type of assessment which feeds back into learning and gives the learner information on his/her progress throughout a course thus helping him/her to be a more efficient learner. (H & Mc)
• A Formative Test is a test which is given during a course of instruction and which informs both the student and the teacher how well the student is doing. A formative test includes only topics which have been taught, and shows whether the student needs extra work or attention. It is usually a pass or fail test. If a student fails he or she is able to do more study and take the test again. (R, P & W)

• The gathering of information about the results of learning (concerning language, strategies and attitudinal change) at the end of a project. (R & V)
• Type of assessment which aims to measure students’ performance at the end of a period of study. (H & Mc)
• A Summative Test is one given at the end of a course of instruction and which measures or "sums up" how much a student has learned from the course. A summative test is usually a grade test, i.e. it is marked according to a scale or set of grades. (R, P & W)

• Student’s and teacher’s analysis of the process and end product which provides constant feedback throughout the project and allows for changes to be made. (R & V)
• Consideration of all the factors that influence the learning process such as syllabus objectives, course design, materials, methodology, teachers and assessment. (H & Mc)
• In general, the systematic gathering of information for purposes of decision making. Evaluation uses both quantitative methods (e.g. tests), qualitative methods (e.g. observations, ratings) and value judgments. In language planning, evaluation frequently involves gathering information on patterns of language use, language ability, and attitudes towards language. In language teaching programmes, evaluation is related to decisions to be made about the quality of the programme itself, and decisions about individuals in the programmes. The evaluation of programmes may involve the study of curriculum, objectives, materials and tests or grading systems. The evaluation of individuals involves decisions about entrance to programmes, placement, progress, and achievement. In evaluating both programmes and individuals, tests and other measures are frequently used. (R, P & W)

• The process of providing information to curriculum developers during the development of a curriculum or programme, in order to improve it. Formative evaluation is also used in syllabus design and the development of language teaching materials. (R, P & W)

• The process of providing information to decision makers, after the programme is completed, about whether or not the programme was effective and successful. (R, P & W)

These terms are often used interchangeably, but there is a tendency for assessment to be used in more restricted sense of a measure of performance or achievement, while evaluation is becoming a much broader term, denoting concern not only with pupil performance but with the nature of the course itself and the quality of teaching and learning. (S, W & A )
 
 

Pre-reading activities

The purpose of this section is to help you to discover what you can expect from the two readings in this chapter, on the one hand, and, on the other, to build on your previous experience regarding evaluation of learning and of materials. Apart from understanding the author’s position on this matter we are also interested in describing a picture of what your evaluating experiences were like. We will ask you to work in pairs and interview other students in order to find out more about the things you had in common.
 
 

Task 1. Before reading the two articles for this chapter, read the title of this chapter, the title of the two articles included in this chapter (Escobar’s and Ravera’s et al) and the different sections in them and then complete the following diagram:

• Which article deals with the evaluation of learning and which one with the evaluation of materials?
• What is the criteria that Escobar has chosen to organise her article?
• Teaching and testing, a mismatch. What do you think Escobar might mean by mismatch between teaching and testing?
• As a second school student were there any mismatches between the way English was taught in class and how you were evaluated? Where there any mismatches between the tasks and activities you were asked to do in class and the ones you did for the evaluating sessions?
• Assessment: different moments, different types. Think about the evaluation system in secondary school. What was it like?
• When did it take place? (before, while, after the teaching period)
• Where did it take place?
• How did it take place?
• What did it consist of? (oral and written exams, tests, classroom observation, grammar exercises, translations, reading comprehension questions, writing compositions, reading aloud, carrying out a real-life oriented task, etc.)
• What percentage of your final score was based on a final written exam?
• How often did assessment take place?
• Who was responsible for it?

• Communicative-oriented teaching. Did your previous language classes focus on facilitating the communication between students, that is, were you given plenty of opportunities to communicate with other students in pairs, small groups, to discuss, comment, report, etc? Were info-gap activities commonly used?
• Meaningful activities. (Ensenyament significatiu. Activitats significatives) Did the classes use meaningful activities? A meaningful activity needs to fulfill some of the following features: (a) contextualised, (b) close and familiar to the student, (c) relevant to the student’s needs, (d) not language based only (e) real-like.
• Tasks. Did your teacher provide real tasks (such as buying a ticket in a railways station, filling out application forms, etc) for you to be carried out?
• The importance of receptive skills. Were the four skills (reading, listening, writing and talking) equally practised in class? What percentage of time was devoted to each skill? How was it practised (in class, individually, etc.)? Which skills were evaluated in the exams? How were they evaluated? Were receptive skills (listening and reading) extensively practised in class over talking and writing?

Some important concepts

Match the following notes and key terms with each section of Escobar’s article.
 

1. Teaching and testing, a mismatch         2. Assessment: different moments, different types     3. Sample materials for summative evaluation     4. Summative evaluation: a decalogue

A. Initial Evaluation / Formative evaluation / Summative Evaluation / Self-evaluation Co-evaluation / Evaluation vs. Testing B. Let your students know in advance what the evaluation criteria are / Evaluate capacities rather than knowledge / Evaluate what you teach. C. The monster / The mice and the cat D. There has been a shift in the goals to be achieved in the English class. The aim is no longer the mastering of a set of lexical and grammatical forms but to be able to communicate in a target language regarding abilities.

 
 
 

Post-reading activities

Task 1. Preparing a glossary. Working in pairs prepare a glossary with the key terms used in each section of Escobar’s article. Below you will find some examples of terms that you could define. Complete the list and give a sound example and definition for each term. You may want to leave this task until you have completed the rest of the tasks suggested in this section.

1. Teaching and Testing a mismatch

A. Shift of goals in the EFL. According to Escobar there has been a shift in the goals to be achieved in the English class. What has the shift been like? What were the goals in language teaching some years ago according to the author? What are the goals in the teaching of EFL nowadays according to the author?

B. The four changes in EFL teaching. According to Escobar there have been four changes in EFL teaching in recent years. The emphasis on communication, the need for meaningful activities (‘activitats significatives’), the use of real tasks and the growing importance of receptive skills (listening and reading) versus productive skills (talking and writing).

Characteristics of tests.

What are they like according to Escobar?

D. The two types of difficulties in introducing changes in EFL teaching. According to Escobar, which are they?

2. Assessment: Different moments different types

Comparing definitions. Compare Escobar’s definitions of EVALUATION, TESTING and TESTS to those provided in the introduction. What are the similarities and differences between her description of EVALUATION and that of Ribé, Richards?et al??and Harris and McCann? Give some examples of FORMAL AND INFORMAL EVALUATION too.

Initial Evaluation. What type of evaluation (initial, formal or summative) is being described in the following excerpt ? Why? Compare this definition to Ribé’s in the introduction to this unit.

Now try to think about your own definition of Initial Evaluation.
 
 

Formative Evaluation. How does Escobar define ‘formative evaluation’? What are the similarities and differences between Escobar’s description of formative evaluation and Ribé’s, Richards et al’s and Harris and McCann’s? What are the problems regarding this type of evaluation. Give some examples of how Formative Evaluation is carried out? Give your own definition of Formative Evaluation. What’s the relationship between formative evaluation and self-evaluation and co-evaluation?
 
 

Summative Evaluation. How does Escobar define formative evaluation? What are the similarities and differences between Escobar’s description of formative evaluation and the other authors? Give some examples. Try to think about your own definition of this term.
 
 

You may want to use the summary grid on the next page to answer the above questions.
 

	Initial evaluation	Formative evaluation	Summative evaluation
1. Definition
2. When?
3. How? The formats
4. Why? The goals
5. Difficulties
6. Differences in views among the authors

 
 
 

3. Sample materials for summative evaluation

In her article, Escobar offers four sample activities for summative evaluation. First, fill in the following chart to examine these activities and then decide if each one fulfills the following criteria:

1. User-friendly. Easy to use.

4. Real. Genuine.
 

	The Monster. Read and Draw	The mice and the cat’2	Listen and complete	Classroom  Language
1. Level of English
2. Students (age) Primary/ Second.
3. Purpose of the Activity
4. Skills practised. Integrated/ oral/ written/ receptive/ productive
5. Format
6. Type of activities
7.
8.
9
10.

 
 
 
 

Your evaluation following Escobar’s criteria.	The Monster. Read and Draw	The mice and the cat’2	Listen and Complete	Classroom Language
1. User-friendly. Easy to use.
2. Integrative and Contextualised.
3. Capabilities vs. Knowledge. Using Vs knowing.
4. Real. Genuine.

 
 
 

4. Summative Evaluation: A Decalogue.

Rephrase Escobar’s decalogue and complete the DON’Ts column and, if possible, give examples.
 

	Summative Evaluation Decalogue	DON’Ts
1		Do not check what students have not learnt.
2		Do not evaluate knowledge about the language. Example. Write three sentences in present perfect progressive, one in affirmative, one in negative and one in interrogative.
3
4
5
6
7
8
9
10

 
 
 

Task 3. Materials evaluation. Complete the following diagram to summarize Ravera’s ideas about the ideal textbook for children.
 

1	Approach
2	Skills
3	Learning activities
4	Topics of interest:
5	Class Organization
6	Teaching kit
7.	Role of grammar
8.	Evaluation and Testing

 
 
 

Further reading

On evaluation of learning:

Assignments

Assignment 1. Lesson planning

Find a textbook that includes evaluation activities and choose a number of them. They can be activities for initial, formative or summative evaluation. Then carry out the following activities:

- Examine the extent to which they follow Escobar’s ideas about evaluation.

- Write the teacher’s notes or lesson plan for those evaluation activities. Mention what they measure or assess.

- Suggest changes that you would introduce in order to follow Escobar’s ideas more closely. Adapt, edit or redesign the evaluating activities if appropriate.
 
 

Assignment 2: Evaluation

Using a grid for materials evaluation, evaluate two or more textbooks of your choice and compare them. But before you start, you should have into account the following:

- The two textbooks you choose should aim at a similar age group so that the comparison is more realistic.

- It will help you if you have a context in mind when you look at the textbooks. By context we mean a particular age group, level of language competence, type of institution (school, private classes, etc).

- You may choose a published grid (see the ‘further reading’ section) or you can create your own grid by taking the criteria you find most suitable from a number of published grids and by adding some criteria of your own.
 
 

Assignment 3: Data collection / Literature review

Run a survey or literature review on one of the following areas:

- Students’ opinions on exams and evaluation systems

- Language teachers’ ideas of what evaluation is and what type of exams they use

- Ways of conducting initial, formative or summative evaluation (see ‘further reading’ section).