Aslı Saglam's Blog about CPD in ELT

Reflections on “Language Assessment Course”: Part 2 Assessing Writing


Last week we had a “timed essay writing” practice with my intermediate level students which went really very bad.

We usually write essays in a process which involves multiple drafting and on-going feedback from the teacher and peers. After reading and listening to some input materials that would give my students some ideas for their outlines, we write in class and at times if they can’t finish their writing within due time, they also work at home. Occasionally we have timed writing as well. However this time only two of them were able to finish their writing within the given time frame (70 minutes as in their exam) and I thought “Well, they couldn’t do it because they didn’t want to…because they are not under exam conditions and they are not motivating themselves…etc.” But I have to admit that there were many statements regarding the difficulty of the topic which was gender inequality. Following this experience, last week in our “Language Assessment” course, we focused on assessing writing. Our discussions and Sara Cushing Weigle’s book entitled “Assessing Writing” helped me to view issues related to writing assessment under a different light. Here come the highlights…

Designing writing assessment tasks

According to Weigle (2002) development process for a test of writing involves certain stages such as 1) design, 2) operationalization and 3) administration. I would like to summarize points to consider that are suggested by Weigle (2002, p.78-82) at different stages to avoid potential problems with the test at a later test in the table below.

When Ece (a very dear classmate and a friend) said; “The stimulus material should be picked with respect to the construct definition of writing. Choosing a textual, a pictorial or a personal experience as a prompt in writing tasks should be in accordance with the construct definition and test takers’ characteristics” it rang a loud bell in my mind, explaining the inefficient timed writing experience I told you about at the beginning.

I have to admit that I may have overlooked some of the points listed above. For instance, as a teacher when I give my students a writing task to assess their language abilities I often skip pre-testing the items/ the writing prompt. But I have taken my lesson and you will see that in the coming Metamorphosis section 🙂

Importance of having test specifications

Test specifications are blueprints/ guidelines that give brief information about the tests so that when a group of educators have that in their hands, they can design assessment tasks that would be standard in assessing the constructs. Also test specifications provide a means for evaluating the finished test and its authenticity. There are many suggested formats for specifications but according to Douglas (2000 cited in Weigle) at a minimum they should contain:

  • A description of test content (how the test is organised, description of the number and type of test tasks, time given to each task, & description of items)
  • The criteria for correctness
  • Sample task items

I should also say that Weigle provided a particular format of test specifications in her book that was originally developed by Popham (1978) which entail detailed description and examples of test specifications that could help development of writing tests (2002, p.84-85).

Grading the writing papers

Weigle defines “score in a writing assessment” as the outcome of an interaction between test takers, the test/ the prompt or task, the written outcome, the rater(s), and the rating scale. She categorised three types of scales based on whether the scale is intended to be specific to a single writing task (primary trait score) generalized to a class of tasks (holistic or analytic scores) and whether a single score (primary trait or holistic) or multiple scores (analytic) are given to each written outcome.

In addition we discussed about advantages and disadvantages of using holistic and analytic scales in our class meeting and it was an interesting discussion, reflecting real life difficulties that we all encounter as teachers who need to score students’ written outcomes.

Holistic Scoring

Weigle argues that advantages of holistic scales cover 1) faster grading via assigning a single point rather than assigning different points for different aspects of writing, 2) focusing the reader’s attention to the strengths of the writer, rather than deficiencies in the writing, 3) being more authentic and valid than analytic scoring because it reflects the reader’s natural reaction to the text better. On the other hand some disadvantages of holistic scales are that a single assigned score may not provide useful diagnostic information regarding weaknesses in certain parts of writing ability.

 Analytic scoring

Advantages are that it provides useful diagnostic information about students’ writing abilities, higher level of reliability because the criteria is more detailed and comprises of more items. As for the disadvantages it is argued that it takes a longer time to score compared to holistic scoring and raters may read holistically and adjust their scores analytically based on the criteria.


After our Thursday evening classes of ‘Language Assessment’ with Prof. Farhady and classmates (Ece, Volkan, Ece, Merve and Jerry) focusing on writing assessment, I thought about how we deal with this issue at my school.

This is a picture of me and my lovely colleagues just before the writing standardisation session.

Before grading the papers we come together in a standardisation session and go over our criteria. Then, we grade papers together within groups and assign grades and discuss the rationale behind our grading.

Although at times they take time, I really think that standardisation sessions help me because they refresh my understanding of the scoring and criteria and set the scene.

In standardisation sessions we have the opportunity to talk about how raters should arrive at their decisions independently and then compare and discuss their scores, how to treat students who responded to the writing question partially or fully off topic, what to do about memorized and/or incomplete responses.



Metamorphosis: lessons to be taken

Piloting and pre-testing items with a sample group who represent the target group will become my routine in the future.

I will be much more careful about clarity, validity; (“potential of the writing prompt for eliciting written products that span the range of ability of interest among test-takers” (Weigle, 2002, p.90)), reliability of scoring and the potential of the task for being interesting for the test-takers.

While choosing the writing topic (personal or general topic) it’s always a good idea to keep the homogeneity or heterogeneity of the test takers, the test purpose (general or academic writing), test takers’ interests, abilities, and their background knowledge into consideration

In order to sustain fair practice, one of the requirements should be evaluating scoring procedures involving assessing reliability of scores, validity of scoring procedures and evaluating the practicality of scoring procedures. Scoring and issues related to the procedures should be revisited frequently.

I will definitely work on having a user-oriented scoring rubric and familiarising students with these criteria. I really believe that such an understanding will guide them in their writing.

How do you deal with assessing writing at your institution? What’s the students’ reaction to writing test(s)? Please feel free to comment.

Next week we will deal with Assessment in ESP and I am looking forward to our Thursday class with Prof. Farhady …

Reference: Weigle Cushing, S. (2002). Assessing Writing. CUP, Edinburg.

Reflections on “Language Assessment” Course: Part 1 Assessing Speaking


Lessons in Language Assessment

For the last 3 weeks I have been auditing Prof. Hossein Farhady’s Language Assessment course given as a part of Yeditepe University PhD program in English Language Teaching. Though I have finished taking classes and I am on the verge of writing the proposal of my doctorate thesis, I still enjoy participating in Prof. Farhady’s class for 4 hours on Thursday evenings; asking and answering questions, reading articles and books and reflecting on issues related to fundamental concepts and principles of second language assessment, with a lovely group of classmates.

The professor is also my thesis advisor and I believe that our Thursday classes and discussions will help me to develop a critical view on a variety of existing assessment procedures, establish a better understanding of fair practice, forms, functions, uses, and psychometric characteristics of language assessment procedures, paving the way to my future thesis.

“Surrender is easy but don’t”

Thought provoking questions are flagged, real life scenarios are suggested and Prof. Farhady often plays the devil’s advocate when he corners us with his questions, requiring us to analyse the course content and to screen it against our experience as teachers who give tests to their students. When coming with an intelligent and satisfying answer becomes hard and he sees the question marks in our eyes, he says: “surrender is easy but don’t”. So, we promise him that we will keep our discussions in mind and always have a critical eye to our practices and pursue validation and reliability. After all changing the world starts with changing yourself, isn’t it?

This week on our agenda we had assessing speaking and writing and I would like to reflect on lessons to be taken for me.


Assessing Speaking

Do you have a speaking test in your institution? I think that designing a speaking test, coming up with tasks to be used in the test and devising a scoring scale for the test-takers’ performance requires a lot of hard work. In “Assessing Speaking” Sari Luoma (2004) suggests Hymes’s (1972) SPEAKING Framework to make the initial planning of a speaking test.

  • Situation (Consideration of physical setting and nature of the test- Is it an end of term test of speaking?)
  • Participants (How many examinees to take the test? Will they work in pair work? Group work? What would be the specifications about interlocutor and assessor?)
  • Ends (considerations about the outcomes of the test involving formative or summative use, how to provide feedback, test score and fair assessments)
  • Act Sequence (the form and content of speech acts that will be elicited through the test)
  • Key (How examiners are supposed to conduct their act and presence in assessment situations: Any scripts that will accompany, assessors guide regarding how supportive or impersonal they need to be?)
  • Instrumentalities (Which channels or modes (spoken, written, pre-recorded)  and forms of speech (dialects, accents and varieties) will be used?)
  • Norms (Which norms of interaction, such as initiating conversation, asking clarification questions, elaborating, and (dis)agreeing, will be involved in the test?)
  • Genre

This framework can help design of a speaking test because it raises questions about linguistic, physical, psychological and social dimensions of the situation in which language is used. Consequently, task designer has to take input, goals, roles and settings into consideration.

Also, Prof. Farhady presented types of assessing speaking below:

  • Imitative (focus on repetition and pronunciation. E.G. Phone Pass test 
  • Intensive(production of controlled language use and short phrases via minimum interaction)
  • Responsive (interacting to short conversations)
  • Interactive (transactional and interpersonal)
  • Extensive (oral presentations, story telling…)

He stressed that differentiating and understanding these types will help us gear our speaking test to better cater for the needs of our students.

Types of speaking tasks

Luoma (2004) provided a comprehensive summary of what speakers are asked to do in assessment situations. According to Brown and Yule types of informational talk encompass; description, instruction, story-telling and opinion expressing/justification (cited in Luoma, 2004, p.31). Bygate differentiates speaking tasks into factually oriented (description, narration, instruction, comparison) and Evaluative Talk (explanation, justification, prediction and decision). In addition to informational talk, there are also communicative speaking tasks. Common European Framework (2001) divided functional competence into Macrofunctions (description, narration, commentary, explanation, and demonstration) and Microfunctions (giving and asking for factual information, expressing and asking about attitudes, suasion-suggesting, requesting, warning-, socialising, structuring discourse and communication repair). There is a variety of task types that could be used in assessing speaking. Then how can task designers for a speaking exam decide which one(s) to use? Luoma (2004) suggests that task designers should make the organising principle for the assessment and teaching curriculum coherent (p.35).

Other considerations when designing speaking assessment tasks

This week in our testing class we once more saw that task designer’s burden is heavy. In addition to types of talks and communicative functions, they need to plan about how to operationalize these tasks. They need to rationalise whether individual, pair or group tasks will be used. Also assessment developers will choose whether to use real-life or pedagogical tasks, tape-based or live testing and determine between use of construct-based and task-based assessment. They also need to manipulate the difficulty of speaking task with regards to complexity of task materials, task familiarity, cognitive complexity and planning time. (Luoma, 2004, p.46)

Examples of Speaking Scales

One of the highlights of this week’s classes was having the chance to discuss a variety of both analytical and holistic speaking scales examples as well as rating checklists. Luoma outlined;

  • The Finnish National Certificate Scale
  • The American Council for teaching of foreign languages (ACTFL)
  • The Test of Spoken English Scale
  • The Common European Framework speaking scales
  • The Melbourne medical students’ diagnostic speaking scales (2004, p.60)

Metamorphosis; Lessons to be taken

At the end of each week I reflect on our class discussions and ask myself; “How will your future conduct change?”. Here are points to keep in mind for me to change for the better:

It’s important to prepare various versions of speaking scoring rubrics and scales catering for the needs of raters, teachers and examinees.

Holistic and analytical scales have their pros and cons and therefore, their use should be considered carefully. Holistic ones can be accompanied with rating checklists (detailed lists of features describing successful performances on task) for feedback purposes.

To develop good and clear level descriptors stems from examining performances of test-takers from different levels and describe features that makes them a certain level.

Differences between levels should be clear on the speaking scales and should not be blurred with too much dependence on quantifiers such as: many, few, adequately…etc.

I feel that being able to talk about questions in mind, assessment related issues we encounter in real-life and hearing about different perspectives and settings enrich my personal understanding regarding assessment. I really learn a lot…

Thursday testing classes and reflections will continue. Please stay tuned 🙂

Reference: Luoma, S. (2004). Assessing Speaking.The Cambridge Language Assessment Series, CUP, Edinburgh.



Designing an online program; Relevant Learning Theories


I have enrolled into a very interesting and challenging course offered by Georgie tech Coursera; Fundamentals of On-line Education Planning and Application.

Course objectives cover; (1) understanding on-line learning pedagogy and androgogy, (2) reviewing  on-line learning components, (3) creating on-line learning components, (4) investigating on-line course design, (5) exploring learning managemnt systems, and (6) creating an on-line course. Especially ” creating an on-line course” will be really interesting.

The course has just begun and at the onset we were asked to react to course content which focused on learning theories.

Here is my response…

On-line Teaching has been becoming widespread and it has affected instructional design and implementation profoundly. The terrain of on-line learning environment has changed the notion of classroom, roles of teachers, roles of students, curriculum design, tasks, course content, methodology and assessment of learning objectives.

What’s the ideal learning theory to fit the on-line learning environment?

If I was designing an on-line course, then among constructivist (discovery learning- Bruner, social constructivism-Vygotsky, categories of learningGagne, progressive movement- Dewey) and cognitive approaches (cognitive load-Sweller, info processing approach, Androgogy) I would build the course around the premises of social constructivism and discovery learning to cater for the needs of on-line learners. Social constructivism involves cycles of related activities, dialogue, talk, collaboration and the social context in order to help the individuals to construct his or her own reality and understanding. Learning involves (re)learning, (re/de)construction, reflection and change of the input (assimilation). Also, social constructivism does not abstract the person from socio cultural landscape and ignore social dimensions of experience, learning and communication.

Teaching Approaches:

Peer instruction and The Flipped instruction would suit on-line learning environment because in both methods teacher is nor perceived as the sole authority and traditional teaching principles are not prioritised. Instead students are actively engaged in course content and learn from each other, constructing representations of their learning. Also they are compatible with Kolb’s experiential learning and Bruner’s discovery learning.

An Example On-line Program Model

Roger Schank reports their experience of building an online program model which is based on learning by doing principles in the video entitled “How Does Online Learn By Doing Actually Work?”. 

In this program the instructional design is modelled after ecological validity of real life situations. In other words, the program is project-based, theme-based and curriculum emulates real-life settings. So, when students work together on real-life projects they also get oriented to real-life situations that will be encountered in their professional life later on. In this design students are not tabula-rasa, not empty vessels to be filled in with information. Perception of education is not transmitting knowledge to passive students. On the contrary, in this model students put in greater effort, they are active actors who take on responsibility of their own learning. It’s also stated by the course participants that their retention was higher because they actively engage in class tasks and focus on and interact with course content. In “What We Learn When We Learn by Doing” Schank labelled the long retention and positive backwash of the curriculum design as “The Acquisition of Functional Knowledge”. According to him “learning by doing allows for the natural acquisition of micro-scripts that supply a learner with a set of individual or packaged executable procedures that, if practiced, will be of use for as long as necessary”. Furthermore learn by doing model does not assign directive roles to teachers. Here, teachers are Socrative mentors who monitor and guide students in order to assist them to become independent thinkers. Therefore, teachers have abandoned traditional methods of teaching, especially standing in front of large classes and lecturing. Instead they have increased quality interaction time with their students via project-based, task-based projects which aren’t outcomes of the curriculum but the curriculum itself.


Schank, Roger C.  (1995) What We Learn When We Learn by Doing. (Technical Report No. 60). Northwestern University, Institute for Learning Sciences. (http://cogprints.org/637/1/LearnbyDoing_Schank.html)

Lowes, S. 2008. Online teaching and classroom change: The trans-classroom teacher in the age of the internet. Innovate 4 (3).


Wired In or Out?; 1st International ELT Symposium of Yildiz Technical University


The 1st International Symposium of Yildiz Technical University School of Foreign Languages kicked off today, with a top level programme and super-star speakers. The opening talk was followed by music and performance session.

1 (1)

The Rector of the university and the organising committee thanked the sponsors with plagues.

1 (2)

Plenary by Chuck Sandy, İTDİ Director, kicked off the conference. He started off puzzling the audience with an interesting poem and he went on sharing his philosophy of life; “I can’t do this but I am doing it anyway”. Among many others, he focused on the concept of “Open Space Technology”; “whoever comes are the right people. Whatever happens is the only thing that could have”. Chuck Sandy’s session was full of inspirational pictures, stories and quotations.


Partipants of The 1st International Symposium of Yildiz Technical University School of Foreign Languages are very lucky because the conference will give us the opportunity “to communicate” and “get networked”.


Many thanks to the organising committee and Isil Boy for the event.

by posted under Uncategorized | 1 Comment »    

M-learning from start to finish with Mark Pegrum


Mark Pegrum’s concurrent session examined the terrain of mobile handheld technologies and showcased some learner-centred, innovative and creative web tools and applications. These web tools and applications ranged from podcasts (which stood on the simple, less technologically straightforward side of the spectrum) to QR codes, geosocialnetworking and augmented reality (which are pedagogically and conceptually classified as more sophisticated). Mark’s session was highly informative and interesting because he also provided the audience with suggested in class tasks that could accompany this wide array of mobile technologies. For example, possible uses of polling was outlined and before this presentation it never occurred to me that polling could be used as a pre-reading activity to create a student-generated list of target vocabulary list.


The presenter kindly shared his presentation. You can access it HERE.


Thanks a lot for this highly inspirational session Mark Pegrum.

by posted under M-Learning | tagged under ,  |  No Comments »    

The Edublogs Awards


I am looking forward to this year’s edublog awards. Nominations will end very soon. More info is at the Edublog’s website.

I would like to nominate Sultan Zeydan’s blog as the “Best Teacher Blog”.

Good Luck to all and happy blogging.

by posted under Uncategorized | No Comments »    

Web Tools for Learning Vocabulary; Word Clouds and MindMapping Tools


Last week  I asked the students to revise the newly learned vocabulary by using some web tools such as Text2Mindmap, Wordle, and Image Chef . The idea was to get my students write vocabulary exercises using  Word clouds & Mind mapping Tools.

I wanted the students to work in groups to prepare vocabulary exercises and revise the newly learned words. Their task was to go over the units in student course book and supplementary materials and write a vocabulary revision in groups by following certain specifications outlined in a model exercise.

First students got into groups and I assigned units of our course book to groups. They were asked to choose 10-15 key words to test their classmates. After they decided on vocabulary items they showed me their lists. I introduced word cloud generators and mind-mapping tools and asked them;

1) to create word clouds of their chosen key words

2) group these words using mind-mapping tool. They could group target word list that they prepared based on meanings , antonyms, synonyms, word forms…etc

I showed them examples of vocabulary exercises and asked them to write word formation & fill in the blanks type exercises. It took some time and we couldn’t finish the task in one class hour. So after they were done,  one person in the group was assigned to e-mail their work to me. I brought outcome of the groups’ together and edited the document. The next day, our homemade vocabulary task was ready! Students were amazed by the outcome which consisted of everyone’s contribution.

In sum, Word clouds and mind-mapping could be useful for retention of the newly learned words and  I love when students work harder than me!!






Insights from Richard Kiely


Prof.Richard  Kiely gave a webinar that was organised by British Council Turkey and delivered a plenary speech on “The learning of new teachers in TESOL” at Marmara University International Symposium, “Foreign Language Education and Its Applications in Prep Classes” last week.

His book on course evaluation was one of our course books at my doctorate program at Yeditepe University, and therefore, it was my pleasure to meet him at the conference.

He kindly accepted to answer a few questions.

  1. 1.       You looked relaxed and totally in control when you were presenting virtually on ADOBE Connect and I know that it was your first webinar. What did you think of this virtual event? Do you think webinars, on-line communities of practice and MOOCs  could be the future of teacher education?

 Yes, it was my first webinar, so thanks for observing that I looked relaxed. It was not my first tele-conference, so I am accustomed to speaking to groups via the computer. Perhaps two factors facilitated my relaxed participation: first I was in my office at Marjon in Plymouth, UK, my usual workplace, and second, I was speaking about a topic area and research study with which I am familiar.

2.       What should ideal pre-service and in-service teacher development encompass in your opinion? How do teachers learn best? How can teacher experience turn into expertise?

Difficult questions, and really important ones: all institutiona and governments would like the ‘magic bullet’ to address these questions. In my view there is no magic bullet, in terms of a technique, or single course design. The focus is on developing the people, and this is likely to be achieved in a social learning situation:

where novice and expert do things together,

talk about what they are doing,

the experts shows and talks through,

with the novice observing and asking for tips and explanations,

with opportunities for the novice to practice, which is evaluated in a constructive way,

and a desire for a high level of achievement becomes part of the novice’s plan for life.


This kind of process ‘furnishes the imagination’ (Kiely & Askham TESOL Quartlerly Sept 2012)


3.       In your research study “Trinity College London Cert TESOL Impact Study” you examined the impact of a four-week teacher training programme. You mentioned the significance of “performance in professional identity, which has the potential to lead on-going, situated learning”. Can you further expand this point?

I supppose the key point from our research participants is the extent to which they wanted to know, to be good at TESOL, to identify with the kind of practices they had come to admire. It is these affiliating goals which shape the identity.


4.       What were some of the other striking findings?

Many findings, but one striking one was the challenge of dealing with the range of materials in TESOL work contexts. Especially where e-learning platforms and the internet are taken into account, the task of becoming familiar with available materials is a huge one for novice teachers. Many wanted to select their own materials, not just follow the coursebook, but were overwhelmed by the range available. An additional issue here is the time required for scanning electronic resources: we just cannot do that (listen to or read through  materials such as videos, for example) as efficiently as we can printed materials.


5.       Finally, in your opinion how can research (in-class as well as following the literature) contribute to teacher education?

My basic position – pro teacher research – is informed by two axiomatic points:

  1. We are in a post-method period in ELT. Teachers have to develop their own practice, which involves iunderstanding what works for them in their different classroom contexts. Research is a tool which can help teachers in that understanding.
  2. What teachers do when they teach is complex – too much going on in classrooms for teachers to remember everything and use what happened in the normal cycles of reflection and improvement. Research provides tools to look more closely at classroom interaction, such that teachers become more aware of the features of their practice, and over time develop professionally by extending what works well, and minimising what does not seem to work well.

Thanks a lot for your time and I really hope that you enjoyed your trip to Turkey.

I look forward to returning to Turkey – I met such fascinating people.

Autonomy, Social Networking and Professional Development


Marisa Constantinides is one of project partners of aPLaNet and she gave a concurrent keynote entitled “Autonomous Professional Development Begins at Home”.  In this session, Marisa outlined ample opportunities that could be found on the internet for professional development of teachers and discussed various ways of getting involved in networks of community of practice.

How can social networks help with professional development?

Marisa argued that social networks have great potential for enriching professional development because they are flexible, continuous, multi-dimensional, and available to be used at the convenience of participants. Moreover, other participants of these networks are often generous in providing collegial support in the form of feedback, suggestions, and examples of good practice. Marisa suggested EFL Classroom 2.0 and ELT Teachers’ Network, as well as ELTChat. Also, she mentioned “grou.ps” as a web provider for creating one’s own Ning.

Marisa mentioned several ways of getting connected with other professionals from home and evaluating what’s available on the net. She suggested training videos on YouTube, webcasts of webinars, TED talks (for auditory visual learners), blogs and books (for visual learners) and social bookmarking such as Diigo for sharing information.


At the beginning of the session Marisa focused on “multidisciplinary information” that teachers’ should acquire including methodology, linguistics, sociolinguistics, and semantics, reminded audience about expectations of the students by sharing a research that is conducted about concept of ideal teacher in students’ mind and stressed the importance of life-long learning for the professional growth. Therefore, small doses of personal development that autonomous teachers can get from on-line networks can, in the end, put us in “hall of fame teachers” row.

(cross-posted at: aPLaNet Conference Blog)

by posted under conferences | tagged under  |  No Comments »    


Autonomous ‘Personal learning Networks’ for Language Teachers (aPLaNet) Project Launch Conference
Today I attended project launch conference of PLaNet in Yeditepe University. This is a very inspiring EU funded project with the aim of providing language teachers in Europe with chances of professional development and in the first plenary keynote project members explained different aspects of the Project and the on-line resources that are created.
In aPLaNet project language teachers are assisted in their professional development by aPLaNet mentoring programme. Participating teachers can become mentees and mentors assist them in using social networks and online tools for professional development and building their own personal learning network (PLN).

Project members talked about aims of the project, mentor guides, mentee guides, methodology guide, case studies and results of the project briefly in the plenary keynote.

Here is the link to the project’s self-access webquest for Twitter, Facebook and Ning that outlines numerous useful links. You can reach the Facebook Page of the Project by clicking on the link.
This project is a living proof of the role of on-line Community of Practice (COP) for enhancing professional development of language teachers.
Asli Saglam

Cross posted at aPLaNet Conference Blog

by posted under conferences | tagged under  |  No Comments »    
« Older EntriesNewer Entries »

subscribe for free

e-mail subscription

Enter your email address:

Delivered by FeedBurner

Clustr Maps


Follow Me on Pinterest

Twitter Feed

The EduBlog Awards 2014 Finalist

The EduBlog Awards 2014 Finalist



GEC 2014 Presenter

GEC 2014 Presenter

Skip to toolbar