ESL assessment

Update: Can-do descriptors of language development

One of the questions I often wrestled with as a starting teacher was how to build a logical and developmentally sound curriculum.  I’ve written a blog about it before, but return to this topic as I have since found new descriptors for language development that I thought would be interesting to share.

One set of new documents that I’ve found is a series of grade-leveled booklets in which various levels of language development are described for speaking, listening, reading and writing.  An example of one such chart is shown here:

wida_can_do_1-2_rwAs you can see, these descriptors are still quite general, allowing the teacher to decide what vocabulary to teach in order to help their learners develop towards the next level.

Here, I’ve included links to the booklets with descriptors that the WIDA  (World-class Instructional Design and Assessment) developed.

booklet_prek-k

booklet1-2

booklet3-5

booklet6-8

Besides this, WIDA also provides ready-made Can-do descriptor name charts, so teachers can fill in the names of their own children at the appropriate level, thus creating an overview of language goals to work towards.  I’ve included links to these ready-made name lists here:

Key Use Can Dos Kindergarten

Key Use Can Dos Gr 1

Key Use Can Dos Gr 2-3

Key Use Can Dos Gr 4-5

Key Use Can Dos Gr 6-8

actfl-logo-2011Some teachers may find it a bit daunting, however, to deal with these general descriptors.  Is it possible to connect these descriptors with more concrete language behaviors?  The answer is: yes.  The American Council on the Teaching of Foreign Languages (ACTFL) has put together just such a list of concrete language behaviors in their booklet “Can-do statements: Performance indicators for language learners” (2015)

In this booklet, one finds checklists of behaviors such as “I can say hello and goodbye,” or “I can ask who, what, when, and where questions.”  This booklet is meant to be a self-assessment checklist, but can just as easily be used by teachers to assess their learners and decide what benchmark their learners have achieved.  Besides this, the language skills are divided up into five categories: conversing (interacting), presenting (speaking), listening, reading, and writing.  These categories correspond with the five categories employed by the Common European Framework of Reference (CEFR), making it easier for teachers in Europe to use this document in their own work.

Moreover, the ACTFL has collaborated with sixteen language organizations around the world to define “world-readiness standards” for learning languages, and aligned their own benchmark levels with those of the CEFR.  This alignment makes it easier for teachers around the world to use these documents in informing their own teaching.

So now my question remains, what do other teachers use in designing their curricula?  What checklists, language level descriptors, or other standards do you use?  Please let me know!

Important update to this blog entry: I have recently had my Digital Record of Pupil Progress (DRoPP) program updated.  I have re-written it to include the descriptors from the ACTFL booklet, and the levels are divided up into A0 (pre-A1), A1, A2, and B1 levels for the five language skills areas: listening, presenting, conversing, reading, and writing.  I am including the booklet of instruction here so you can look it through.

How-DRoPP-works

If you are interested in a trial use of DRoPP, please contact me here:

Name(required)
Email(required)
Comment(required)

 


Links to the ACTFL documents cited:

https://www.actfl.org/sites/default/files/pdfs/Can-Do_Statements_2015.pdf

https://www.actfl.org/sites/default/files/publications/standards/World-ReadinessStandardsforLearningLanguages.pdf

https://www.actfl.org/sites/default/files/reports/Assigning_CEFR_Ratings_To_ACTFL_Assessments.pdf

Contemplating Portfolios

Two children sit at the table, ready for their role play. 

“What’s the matter?” one of them says.

“It’s my arm.  It hurts,” the other answers.

The two of them carry on for another exchange or two, and then look at me.  “All done!” they grin.  I turn off the camera, and give them a thumbs up.  “Great!  That’s for your portfolio!” I tell them.

By now, the children know to expect me to show up with the video camera every once in a while.  We make recordings of their work which then go into their portfolio.  It took a few tries to figure out a workable portfolio, and quite frankly, I’m still working on it.  But first, a short history of what I’ve tried out…

cefr_head

The first thing I decided was that I wanted the children’s portfolios to be related to the Common European Framework of Reference (CEFR).  The CEFR had printable checklists that I could easily use to keep track of children’s learning.  From there, I had to decide how to document the children’s learning.  Ideally, I wanted to have some tangible form of proof: videos, pictures, recordings and the like.  A paper checklist alone was not going to be sufficient, that much was clear.

My thoughts turned to digital documentation.  I scoured the internet for a child-friendly language portfolio.  The closest thing I found was the “Europees Taal Portfolio” (European Language Portfolio).  I decided to try it out with the children.

europeestaalportfolio

This online language portfolio had a lot of good aspects to it: nice colors and design, space for documents and images, and checklists the children could fill in for themselves.  It also had room for improvement, such as: videos and sound recordings could not be uploaded and the teacher could not add documentation to the children’s portfolios.

After a year or two of working regularly with children on these online portfolios, I had to throw in the towel.  Too many children lost their passwords, the suggested activities for portfolio development weren’t always complete, and to top it off, I accidentally erased my own, years-old portfolio with a single click of the wrong button.  Recovery would cost an entire day, so my portfolio – including any access to those of the children – was scuttled.  Soon after that, the entire project was unexpectedly taken off the air, so all of the portfolios my children had built were gone as well.

As a result, I renewed my search for an alternative.  Until now, I have yet to find anything that I can really use, and so I find myself making do with an external hard drive and a cd-writer.  Every year, I make certain to get two examples of work from each child.  This work is kept on the external hard drive.  When the children graduate or leave school, I burn their work onto a cd-rom as a good-bye gift.

video4What kinds of work do I put into their portfolios?  I like to have a balanced sample of reading, writing, speaking and listening skills.  Writing, obviously, is the easiest sample to get: I scan samples of their writing from their project notebooks or save copies of power points.  For reading, I make a sound recording of the child reading.  For speaking, I make videos of the child doing a presentation or taking part in a role play.  I’m still not sure what to do for listening, so that’s something I think about on a regular basis.

I’m still on the lookout for a child-friendly online portfolio that actually allows all forms of digital documentation, so if anyone knows of anything I could use, please let me know!


TESTING, TESTING… (CONTINUED)

Three children sat at the table, sorting through six laminated cards.  “All right, everybody,” I started.  “Where is frog hopping?”  This is a toughie, since “hopping” in English and “happing” in Dutch soundsimilar, but the meaning is quite different.  I demonstrate with the frog puppet.  “Look!  Hopping!”  The children catch on, and hold up the picture of frog hopping across the yard.  We continue on, looking at frog eating a fly, frog eating a stick, and frog eating a flower.  “What will happen in the story,” I ask.  The children look at the pictures and start to tell me a story.  I encourage them to use all of the English words they know, resulting in a mix of words one might call “Dinglish,” Dutch and English mixed together.  Then I set up the “walls” – some large books – and pull out my storybook.  It’s time to read aloud so that the children can put the pictures in order.

In this task, I look for language behaviors as they are described in the teacher’s handbook for the ESL program I work with (Note 1).  For instance, can the children understand simple, short sentences?  Can they predict what might happen, by looking at the stories?  Can they put the pictures in the correct order?  And after hearing the story, can they answer questions about the story?

With this task, I assess four separate language behaviors with a group of 2 or 3 children.  Within 10 minutes, I have quite a bit of information already, and we still have time for more fun and games, as I call my tests.

I developed my test after extensive work with the Reynell and Anglia tests.  I noticed that while each test had a number of good points, each also had its weak points which made it unsuitable for my purposes.  Here, I list a few of my considerations regarding each of the tests:

Reynell test, pros:

  1. interesting tasks – the children enjoyed each of the tasks given, the tasks are appropriate for the target age group
  2. norm-referenced – no child can fail, although he can perform above or below the norm for his age

Reynell test, cons:

  1. time-consuming – each test is given individually, so a lot of time is spent introducing each task, and the test can take up to 45 minutes
  2. age limit – the test may only be used for children up to 7 years of age

Anglia test, pros:

  1. efficient use of time – the test is given classically, so the entire class is finished with the listening/reading/writing section within an hour or two
  2. structure – the test is well-structured and easily administered

Anglia test, cons:

  1. criterion-referenced – the test may be passed or failed, but in the case of failing (or superbly passing) it doesn’t give any information about what would have been a more appropriate level for testing
  2. level of testing – all sections tested are tested at the same level, there is no differentiation possible between the levels

I decided I needed a test that combined the good aspects of these two tests, while dealing with the negative aspects.  I ended up with my own system, which I call DRoPP (Digital Record of Pupil Progress).

While testing, I use a paper checklist per child.

How do I test children?  Step-by-step…

  1. I list the children in order of their ability.  The reason I do this is because I test children in small groups of 2 or 3, and it’s easier to test them if I have similar tasks for all of the children in the group.
  2. I pull out the children’s individual checklists of language behaviors.  The tasks on these checklists are based on the Early Bird curriculum.
  3. I give the group a number of language tasks to complete.  I don’t repeat the stuff they’ve already proven they can do, I only look for new information.  If the children attempt a task and cannot complete it, I put a dot in the space next to that task.  If the child completes it successfully, I put a dash.  I use one color for each assessment period.
  4. After each assessment – 10 to 15 minutes later – I give each child a well-earned compliment.
  5. I input the information from the paper form into each child’s DRoPP file, noting the date of the assessment was.  I often input the information directly into the summary screen, but I may also use a more detailed screen if I like.

The summary screen for a child, indicating name, group, and general development.

The summary screen for a child’s DRoPP file

A detailed list of the language behaviors.

A detailed list of language behaviors

Once the input has been done for all of the children, I run the output program to see just how well the children have done – as individuals, but also as a group.  I use the results to write my group plan for the coming semester.

I will write more about group plans another time…


Note 1:  for further reading on the ESL Program I work with:  The Early Bird Curriculum for Primary Schools (www.earlybirdie.nl)

TESTING, TESTING, 1-2-3…

For what must have been the hundredth time, I pulled out the apple, the fish, the key, and a dozen other toy-like attributes.  The kindergartener eyeballed the objects, eager to play.  I pulled out my checklist and pencil, and started: “where is the fish?”  “Where is the chair?”  “Where is the key?”  Each time, the child would point at the object, sometimes uncertain, other times rejoicing in the right answer, gaining in self-confidence each time I nodded at him. So began yet another of one of hundreds of Reynell tests I administered as part of a research project being carried out by my employer.

The Reynell test uses a mixture of attributes that children may handle and colorful pictures.

The Reynell test

The Reynell test itself is well-designed, including practical and interesting tasks for young children in order to assess their listening and speaking skills in English.  The test begins with groups of similar, easy tasks, which become increasinly difficult as the test progresses.  When a child begins to fail at a particular sort of task, the assessor ends that task and moves on to the next set.  If the child fails at that set as well, the test is done.  It is easy, standardized, and informative. The point of the test is to see what developmental age a child has reached in his language abilities; even though a child can perform poorly, he can never fail this norm-referenced test.

This test is, however, also very time-consuming.  Depending on how well a child does, the test may take only 10 minutes, or a full three-quarter hour per child.  After administering the Reynell test a hundred times, I found myself quite ready to throw rabbit and bear out of the window and move on to something – anything – besides rabbit putting the knife under the bed and bear pushing the bed.

One of the dangers of administering the same test a hundred times: the examiner (in this case, me) might start getting bored and make things up, which isn’t allowed, of course, as that would affect the standardized scoring.

Later that year, I administered another sort of test to the older ESL pupils.  This time, there was no teddy or rabbit, but instead, paper, pencil, and a cd with spoken texts.  It was time for the Anglia exam.

The Anglia exam

The Anglia exam isn’t a single exam; instead, it is a series of exams that begin at a very basic level (A1) and graduate to higher levels of skill (C2)*.  The basic Anglia exam includes listening, reading, and writing skills.  Speaking assessments are separate and cost extra, depending on who scores the test.  The Anglia exam is a criterion-referenced test, which means that a child may fail.  If he fails, then the attempted level was too difficult, but if he passes, then perhaps the attempted level was too easy.  Therefore with this test, it is necessary for the examiner to know two things ahead of time:

1) what exactly is tested at each level (described in detail in the Teacher’s Manual on the Anglia website)

2) what each examinee’s general level of English is (for instance, by using the Placement Test on the Anglia website)

The Anglia exam is a series of leveled exams, starting at pre-A1 and building up to C2.

Besides that, all of the sections are tested at the same level, regardless of possible differences between a child’s skills in listening, reading, and writing.  The tests are costly, and since children want to pass this exam, the teacher must make a careful estimation of the highest possible test the child will be able to pass, even if much of the examination might be too easy for the child.

Administering the test itself is simple enough, since that is done classically. The feedback from the assessment is a diploma in which the child (barely) passes, passes well, or passes exceptionally well.  If a child fails, it receives a referral to try again at an easier level.  Personally, I found this feedback to be as effective as measuring the depth of the North Sea with a meter stick:  reliable, but not terribly informative.

A new test

A few years ago, my employee decided it was time to create a new sort of test, an informative assessment that would cover the broad range of ability found at our schools, while being time- and cost-effective.  We spent hours analyzing existing tests, discussing questions like “do we really need to count spelling as part of a listening test?” and “how can we differentiate the material so that we can find a child’s level in each language skills area?”

After that, the test was administered to hundreds of children, as part of the process of creating a normative score.  Basic feedback was given to the teachers, and the children all got certificates stating that they helped in creating this test.  Unfortunately, this test is still not available for regular use, so I am – still – left to my own devices.  Fortunately, I had already been developing my own devices for nearly ten years.

My own assessment

I have been creating my own means of assessing children’s progress in English.  Not only that, but I have developed a system of recording their progress so that I have a long-term picture of children’s development in the ESL program.  I used my experiences with both the Anglia and the Reynell tests to form something more useable for my school.  But more about that in another blog…


* The levels A1, A2, B1, B2, C1 and C2 are part of the Common European Framework of Reference.  More information about the CEFR can be found here:  http://en.wikipedia.org/wiki/Common_European_Framework_of_Reference_for_Languages

More information about Reynell tests:  http://www.ncbi.nlm.nih.gov/pubmed/15587011

More information about Anglia tests:  http://www.anglianetwork.eu/

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s