Thanks to web providers such as Blackboard and Adobe, it’s possible to attend to on-line events that are done with participants from all around the world. These webinars disseminate info and expertise and reinforce the educational networks fostering more cooperation and collaboration.
I came across a webinar entitled “Developing materials and practices for the digital generation” by the glorious ELTon winner Nik Peachey and wanted to share it with you.

Here is the link that would take you to the webcast of the event.

In this last day of EALTA Summer School we started to work on real-life data that some of the course partipants provided.

Thanks to them, we had the opportunity to take a look at authentic data sets which portrayed students’ performance on a Maths test. Norman guided us by making us reflect on the data analysis and showed some short cuts to be utilized when we wanted to use our own data sets (instead off typing the whole data from scratch).

I met lovely people, had great fun and learned a lot. I would like to thank Gudrun, Marianne, course tutors and all partipants of 2nd EALTA Summer School for making this course such a memorable event for me.

Some of the course content and list of references are shared on EALTA’s Website.

Happy testing everyone…

On a beautiful and sunny Goteborg morning 2nd EALTA Summer Course on testing started off with very interesting presentation by John de Jong about setting standards. There are many commercialized exams which argue that they measure the same construct in language development and when we compare these exams one to another it’s often assumed that they have exact same-equivalent scores.

To illustrate, my school accepts couple of external exams to exempt students that are enrolled in language preparatory school. Do these exams expect the same standards in terms of task difficulty, and standards expected form the students? I found this suggested point very interesting and you will be able to follow John de Jong’s point of view and data collection in detail when you refer to the presentation on EALTA’s website. Also, we talked about the standard setting procedures that took place in SurveyLang project and luckily we had Neus, Norman, Gudrun and other colleagues who were involved in the project. As a result, we had the chance to listen to their real life experience with respect to setting standards in a large scale EU project like Surveylang.

I would like to summarize the points that I reckon from these sessions:

• Governments, universities and test developers strive to set standards and screen exams accordingly and the success of standards setting procedure depends on planning, training and rigor. The more judges are guided and trained in the process the better and more smoothly runs the process.

• A variety of techniques including the Angoff Method, Basket procedure and Van den Schoot  aid standard setting.
• Language development is conceptualized in 2 dimensions;

1) Quantity (How much a person can do in a test? How many different tasks?

2) Quality (How well can a person do these tasks? Efficiency?)
Furthermore, in the course of language development, combining these two dimensions (quality and quantity) is not a ladder. Instead, it’s a slippery slope which results in profiled development of students. E.G. a student being better in reading but relatively less able in speaking or listening abilities.

• Self-assessment can be unreliable in determining quality and quantity due to the Dunning-Kruger Effect.
• Instead ploting Rasch difficulty (Theta Values) against judgments (what people think about item difficulty) could give better estimates. A variety of approaches including odd& even ability estimates, split-half estimate, considering multi versus uni dimentionality per skill were mentioned.
• What does “a B1 Exam” mean?

As far as I have understood to have an exam at B1 Level we need to sample from all possible tasks that could be done at B1 level (“sampling from a wide universe of tasks”) and consider a student to be at B1 level if that student can master 50% of the task samples.

• Other Sources that you will not regret checking out  🙂

Neus Figueras & José Noijons (eds.) Linking to the CEFR levels: Research perspectives

Felianka Kaftandjieva. Methods for Setting Cut Scores in Criterionreferenced Achievement Tests A comparative analysis of six recent methods with an application to tests of reading in EFL

In short, setting standards requires meticulous work of considering expectations from students at a certain level, item difficulty and ratings of ability. It’s presented as a long, tiring, challenging  and a valuable process which contributes to fair practice in testing and assessment.

What’s the relationship between storytelling and testing?

Norman (Verhelst)says that when testing we have a narrative but we need to be sceptical and critical towards the story and check whether the story we tell has any fallacies. In other words we need to check whether it’s trustable or not regardless of how beautiful the story is. Therefore, in order to check the narrative testers have to collect information.
So, on the 3rd day of the course we focused on ways of collecting information via one-dimensional and multi-dimensional models, likelihood and probability, Pascal’s triangle, joint maximum likelihood, conditional probability, and independence of probability.

But I would like to tell you another story here 🙂
The dinner we had at Pensionat Styrso Skaret…
It was such a lovely break after a hard day’s work.

2nd EALTA Summer School Dinner on PhotoPeach

I think the most difficult days were Day 2 and the following day- Day 3 because Jan-Eric Gustafsson carried out with classical measurement theory and we were introduced to ‘Item Response Theory’ by Norman Verhelst and both made me regret the days back at school when I tried (and unfortunately managed) to escape from the algebra lessons.

Jan Eric focused on Cronbach’s Alpha as a means to assess reliability score and outlined the assumptions of this measure which include; all components measuring the same underlying dimension, having the same relation to the underlying dimension and supposedly having same residual error variance. While constructing items in a test, if these assumptions are violated then there could be a reliability loss. Then, it was suggested that statistical tests, e.g. conducting a confirmatory factor analysis and checking inter-item correlation matrix and covariance matrix might act as a solution. We were also introduced to “a congeneric- Latent Variable Model”, “Path Analysis- Structural Equation Models (SEM)”, “Analysis of Moment Covariate Structures (AMOS) and Chi-Square Goodness of Fit Test”. Another point that Jan-Eric focused on was possibility of measuring a potential discrepancy between your data (what you observed in terms of test-scores) and your model, taking model complexity into consideration. Apparently the Root Mean Square Error of Approximation (RMSEA) test will yield whether your test has a good fit (if the value of the data analysis is less than 0.05).

As for validity Jan-Eric referred to Messick (1989) while defining ,exemplifying 3 classical forms of validity (content, criterion-related and construct)and conceptualising facets of validity as a “progressive matrix” taking evidential and consequential basis as well as test interpretation and test use into consideration. Then, as a final point that summed up the morning session, sources that can give information about construct validity and potential threats against construct validity were discussed. It was very informational and intense session and I am glad that we had the chance to be introduced to these analysis approaches and the underlying rationale. I felt that I would love to have more hands on tasks in the coming summer courses :)so that in the future we will be able to apply and transfer the course content fully in our local contexts.
I may have shunned from Math classes all through my education but there I was in our class of EALTA summer school, very happily and willingly pursue my professional development. Therefore I will give myself a bright star filled with the buzz words of the session.

In the afternoon, Norman gave us a battery of programs involving OPLM that we used for Rasch Analysis and Item Response Analysis. OPLM is a non-profit product that could be downloadable from the internet.

Let me show you how it looks like;

Then when you run the program it gives you information about probabilities of a student with a certain level of skill in getting an item with a certain level of difficulty right together with anaalysis of items.

At the end of the second day, I was confused a lot but I was also I felt comfortable because I knew that there would be more support; internal (Norman & other participants) and external ( e.g. the free manual of the free program OPLM). This was just an introduction…

 
EALTA Summer School Diaries: Day 1 on PhotoPeach

EALTA’s Testing and Assessment summer school kicked off yesterday with 24 participants coming from various nationalities. Most of the participants got wet under the pouring rain but none of us minded this because the coordinator of the event Gudrun (Ericson) and Marianne (Demarret) and the course tutors Professors Norman Verhelst, John de Jong and Jan-Eric Gustafsson gave us a warm welcome.

After a short introduction and orientation to the course we were introduced to Classical Measurement Theory and we sought answers to essential questions for fair practice in testing including:

• “Why should one measure?”
• “What are the differences between modern theories of measurement- Item response theory (IRT) and classical theory?
• “How to interpret correlation between items of a test?”
• “How to maintain reliability of a test?
• “How to measure reliability of a test?”
• “What are the reasons for reliability loss?”
• “What are the factors which may lead to sources of variance in test scores?”
• “What’s the relationship between text length and reliability?

Professor Gustafsson explained/defined/illustrated the answers by exemplifying the constructs, instruments, research design and efforts made to maintain reliability and validity of a large-scale  research study; IEA Reading literacy Study that was conducted in 1991 with 4500 Swedish students. Thanks to real life examples, statistical tables and figures it was easier to grasp the answers provided in response to the questions listed above.

In addition to the rich content of the course, background and profile of the participants also contributed to the summer school. Some of the colleagues are working for ministry of education of their countries, some of them are involved in EU projects that aim at portraying language competencies across Europe, some of them are conducting research into testing and assessment and all of them are eager to talk about their experiences.

In short, I feel lucky and amazed maybe  due to  EALTA network in terms of collegial support and professional development or maybe because of the gorgeous “Welcome Reception” that we were treated with at the end of a trying but fruitful day.

I am introduced to the concept of “Digital Citizenship” thanks to #DigiFoot12 and our great and inspring PLN.

I feel the buzz words that define being a “Digital Citizen” are; copyrights, netiquette and leaving smart foot prints and digital heritage.

(Poster taken from: Common Sense Media)

In Edorigami wiki page it’s argued that digital Citizenship encompasses some certain characteristic behaviours which involve:

1. Respecting yourself
2. Protecting yourself
3. Respecting others
4. Protecting others
5. Respecting intellectual property
6. Protecting intellectual property

#DigiFoot12 PLN stessed similar opinions and you can take a look at different (and similar perceptions) by clicking here

One of the inspring questions was to think about borders between character development and digital citizenship and think about the interaction between these, making such borders blurry. Also I have to say that like many others I also think that adressing the whole person development would foster a more solid ground for establishing the idea of digital citizenship. Otherwise we-teachers may end up with more students who would say “But I did not plagiarise I just copied some parts from the internet. It’s still me who put those bits and pieces into one work and it’s mine!”. Some concepts that are related to not only character development but also digital citizenship can be found here: Please Click here.

Our lead detective Scott Monahan presented the essence of Digital Citizenship from multiple perspectives, sharing a variety of resources about:

• Digital Citizenship & Students

ISTE & Students

• Digital Citizenship and Teachers

ISTE & Teachers 

At this point I would like to say that one of the main aims of the website of Iste NETs for Teachers is to raise awareness about open and sheltered places that educators can make use of  in the design of their ICT use. The most important factor is to consider advantages and potential harms of using one over the other. I means students can use open tools such as Twitter or sheltered places that are password protected,or where they remain anonymous, e.g. privacy settings-controlled classroom wiki page which may require membership.

• Digital Citizenship & Parents

Scott Monahan’s presentation

Digital Footprints – do you know where your child’s is leading?

The other sources that could help educators involve the following:

• 10 Tests of Good Digital Citizenship
• Digital Citizenship and Creative Content 
• Course on digital citizenship
• Scott’s internet safety presentations
• Posters for Digital Citizenship
• http://www.commonsensemedia.org/educators/middlehigh_poster

Thanks a lot to leading mentors/detectives, participants and organizers of  #DigiFoot12 MOOC for a highly inspirational week which was full of useful hints and creative teaching ideas.

It opened a new door for me.

Objective of last week in #DigiFoot12 course was Twitter and I waited to write this post because there were a lot of things to experiment with and learn about. Gill Ville led us-all the participants- into the world of Twitter which has become a powerful virtual teachers’ room.

She shared a very stimulating presentation which can be seen below.

Many people say that Twitter has great potential not only for providing opportunities for professional development, but also for being able to help educators  gather a battery of activities, lesson ideas and tasks. But, I was still not very confident with using Twitter. I mean, I am not an egg but I am not a Twitter machine either.

Therefore I tried out the suggested tasks of the week with great joy. Following Kim’s lead first I tried to understand where I stand in using twitter. One of the tasks was visualising our Twitter foot print by using some web applications that can analyze one’s data. I made use of Visuall.Ly in order to condense mine into a cool infographic.

Kim Gille helped us to gain insight into “Literacy in Twitter” by her hands on task activities about; profile settings, search-hashtags, linking Twitter with Facebook, encryption of messages in 140 characters,saving and archiving tweets among many other things! (Thanks a million Kim)

So, I started as a “lurker”  and followed tweets of other collegaues, and then, Twitter started to become a virtual teachers’ room. I have seen so many interesting teaching ideas, suggested activities, resources and links.

Twitpic – Share photos and videos on Twitter twitpic.com/ab8a5i via z@TwitPic LOVE this and you will too! @verenanz #Digifoot12

— Shannon Poulsen (@shanpoulsen) July 26, 2012

And that’s not all…!! Twitter can provide a platform for educators engage in discussion (chat) and collaborate on-line.

• Twitter Chat Schedule
• List of Education Chats

Therefore, this module of the course week has been a golden opportunity to explore Twitter but I still have lots of inverstigation to do until I become competent in using its full potential as a wonderful networking tool. I feel that I will be writing about Twitter more…

Please feel free to help me out in discovering Twitter to its full exploitation in education…

“My students DO NOT READ”. Please leave a comment to this post if you have heard this statement or voiced it yourself. Unfortunately one of the common concerns of our modern day is that Turkish L2 learners of English often do not read very much.

My friend/colleague Amy (Erenay) conducted an action reserach to foster reading enrichment among a group of Turkish university students. She tried out a reading intervention which comprised of a battery of reading skills and strategy training to see whether this would lead to any difference in terms of reading comprehension. She kindly accepted to talk about this class-based research in the presentation below.
Enjoy 🙂