ADOS-2 Reliability Workshops to Calibrate Coding

The ADOS-2 is unlike any other assessment I know in that you never stop learning and developing your administration and coding skills.

One of the ways we have to do that is by attending reliability workshops to calibrate coding and avoid drifting from the consensus, which is easy to do.

It is not specified how often you should attend reliability workshops, but I do this as often as I can as it is so important. I volunteered videos of myself administering the ADOS-2 for reliability meetings from early on in my ADOS-2 career. I wanted the chance to learn, to have feedback on my videos from my peers and from trainers. I was willing to expose myself regardless of comments and criticism I might receive as I knew that all feedback would help me develop my ADOS-2 skills. It is not easy to do, but I learned a lot and I still give videos to be shown to groups of other trainers and large reliability meetings because I know I will learn a lot, and this outweighs my fear, anxiety and uncomfortableness. If you can share videos, it can be really helpful.

This is what I have learned over the years:

The ADOS-2 encourages you to notice details, but they do not all code. It takes time to know what counts as evidence for items and what does not.

It can be hard to interpret the descriptors, they cannot be completely prescriptive because of the huge range of different people we apply them to. Unspecified words like frequently, several, occasionally, are difficult to interpret, and we all wish they were more concrete.

The ADOS-2 training is intense, we learn an enormous amount of information in a short space of time, and many questions often remain about what descriptors for items mean, which can affect coding. For example, being unsure of what D1 is looking for, what will code as sensory interests or behaviours, being unclear about what we can code as repetitive behaviours in D2.

It is easy to feel that there are right and wrong answers when you calibrate codes in reliability workshops, but this focus can detract from developing a deeper understanding of what counts as evidence for items and why, increase anxiety and make people feel unconfident.

It is easy to want the ADOS-2 to be more than it is, e.g., to want it to be a sensory assessment, a speech and language assessment, but it can only be what it is, and we need to use other assessments to gather more specific information on different areas.

The emphasis is often on coding, not administration. It is easy to feel worried about administering tasks in the “wrong” way or making mistakes, which makes it hard to find yourself as an ADOS-2 administrator and to develop clinical skills using this assessment with a wide range of different individuals.

The workshops are a chance to share everyone’s experience, to ask questions that might have been on your mind for a while, to reflect on and discuss administration and coding. The ADOS-2 can be hard, participants can be very complex, and it is not always easy to use this tool with so many different people.

I am running a series of 3 reliability workshops focussing on girls & women.

The first is a Module 3, with a complex 6-year-old girl, the second is a Module 4, with a wonderfully creative 18-year-old, who has very high levels of anxiety, and the final video is with a bright, successful woman, seeking to understand herself, differences and difficulties she has experienced in her life.

Links to information on assessing girls and women:

https://www.autism.org.uk/advice-and-guidance/what-is-autism/autistic-women-and-girls

Improving Diagnostic Procedures in Autism for Girls and Women: A Narrative Review

https://pmc.ncbi.nlm.nih.gov/articles/PMC10926859

Barriers to Autism Spectrum Disorder Diagnosis for Young Women and Girls: A Systematic Review

https://link.springer.com/article/10.1007/s40489-020-00225-8

Autistic Girls Network Conversations with Dr Judy Eaton: Missed and Misdiagnosed