Thursday, 7 December 2017

Can we talk about high-stakes failure?

Much has been written and debated this week about the latest Progress in International Reading Literacy Study (PIRLS) results, that have been just been released. As readers of this blog would be aware, PIRLS concerns the reading skills of Year 4 students, and in a nut-shell, the latest results offer some encouragement to us in Australia, given that we've moved up six places internationally. Improvement is always to be applauded, particularly in an area as resource-intensive and contested as early reading instruction.

For an accessible overview of how we did in Australia, readers are referred to Alison Clarke's excellent Spelfabet website, where she parses out the key messages for Australia out of this overall improvement. Notable among these messages is the fact that (a) we are lifting the performance of already advantaged children, but (b) leaving behind those who start from behind. This perpetuates the so-called Matthew Effect in learning to read and remains a wicked problem for us in Australia.

For those of you who have not encountered the Matthew Effect in early reading, it is a New Testament reference to the idea that the rich get richer, and the poor get poorer. It is well known that children from lower socio-economic status (SES) families enter school having experienced less elaborate early language exposure, and are thus less well-prepared for the biologically unnatural process of learning to read. Oral language may well be "natural" but it is not a set-and-forget developmental domain. This means, therefore, that how we approach the learning needs of such children when they enter school (not always via any, let alone high-quality preschool) is critical. Early years teaching actually needs to accelerate the progress of such children, not simply see them developing at a similar rate to their more advantaged peers.

Not surprisingly, students who come from text-rich home environments seem to do better on measures such as PIRLS, affirming the role of early SES in later school achievement. SES is, however, a challenging problem to address, and in the absence of effective ways of doing so, we need to modify variables that influence school success, such as instructional quality.

Notably, England, which introduced a Phonics Screening Check (PSC) back in 2011, seems to be making gains in this very space, if the latest PIRLS results are anything to go by. England has not only improved its position in the international rankings (to now be 8th overall in fact), it has done so by taking with them, those children who start from behind, and narrowing the performance gap between boys and girls.

Interestingly, in the same week that the PIRLS results were released, the debate about introducing a PSC in Australia has intensified, with detractors trying to deflect focus away from the fact that England's first cohort of students to have experienced the PSC are the very ones at the centre of the up-lift described above. The PSC was a large natural experiment, with no other known mass interventions occurring at the same time. If England's PIRLS results had deteriorated in 2016, PSC-detractors would have pounced on this as evidence that "it doesn't work". But the thing about the evidence game is you can't have your cake and eat it too. 

However I want to draw attention to another report that was released yesterday, that very few would be aware of - the (NSW) 2015 Young People in Custody Health Survey

Why is this report significant and why would I raise it here?

It is significant, because for the first time, the health audit of young people in custody in NSW included standardised measures of language and literacy skills, and the results  are a damning indictment of what can only be described as a school-to-prison pipeline.

My own research on the language skills of young people in custody and on community-based orders has identified high rates (around 50%) of unidentified language disorders in such young people, notably among young males. However my research has only been able to include samples of such young people, and of course (as is right) young people can elect to take part or not in research, and it is entirely possible that adolescents who feel that their language skills are brittle will baulk at taking part in such a study. I have long suspected that we may be under-estimating the prevalence of language problems in the youth justice context for this reason.

In the recently released report, however, data are provided on a larger (n=227), more representative sample of young people in custody, with 90.4% agreeing to take part, representing 60% of young people in custody at the time. The findings are worse than even my seasoned research team imagined.

For example:

  • 80.3% scored below the average range on the Clinical Evaluation of Language Fundamentals Core Language Score. 
  • Almost half (48.7%) of the young people scored in the “very low” range on this measure, indicating severe difficulties.
  • On the York Assessment of Reading Comprehension: 73.2% of the young people scored below the average range on single word reading. Over two thirds (68.7%) scored below the average range on reading accuracy, 74% below average on fluency, and 94% scored below average on reading comprehension.

I will update this post when I have processed the findings more fully, but in the meantime, let us be clear that these young people have much to tell us about the long shadow cast by early academic under-achievement (coupled with histories of suspension and expulsion). This is the very stuff of the school-to-prison pipeline, and changing this sits in the hands of schools and policy makers.

It is very difficult and expensive to "back-fill" the knowledge and skills that these young people lack with respect to language and literacy.  They are the living manifestation of high-stakes failure. They are also the later manifestation of the tail end of the achievement curve in the early years of school.

There is some early promising evidence that we can engage young people who are in custody in speech pathology interventions, but we cannot and should not rely on complex, expensive, downstream interventions to address problems for which more effective approaches upstream are being ignored. 

So when I hear and read protests to the introduction of a Phonics Screening Check because it might somehow be "high-stakes" for teacher / school / sector accountability, I reflect on a different meaning of "high-stakes" and wonder how the trajectories of some of these young people might have been altered through early reading instruction approaches that are more faithful to the evidence about what works. Early failure for them continues to be high-stakes into adulthood.

(C) Pamela Snow (2017)


  1. Dear Dr. Snow, I agree with you that critics of phonics should pay attention to all the relevant data and not ignore evidence when inconvenient. But I think proponents of phonics are guilty of this when describing the PIRLS results. Over and over I see comments claiming that the results provide evidence for the effectiveness of phonics in England, highlighting how far up the league tables England has moved up (now tied at 8th place), with scores of 539 in 2006, 552 in 2011, and 559 in 2016. What is ignored in this story is that the England also did really well in 2001 (scoring 553). Indeed, of the 6 countries that completed the PIRLS test on all 4 occasions (England, New Zealand, Russian Federation, Singapore, Sweden, and USA), England has gone from 2nd to 3rd. This raises questions of whether England’s recent excellent results can be attributed to the introduction of phonics.

    It is also worth emphasizing that the PIRLS test assesses reading comprehension, and meta-analyses of randomized controlled trials have not obtained evidence that phonics improves reading comprehension (Torgerson et al., 2006; MacArthur et al., 2012). The current results are certainly interesting, and to the extent that these tests provide a reasonable measure of reading outcomes, good news for England. But I think it is far from clear that the results provide evidence for the efficacy of systematic phonics.

    Jeff Bowers

    1. I d> not agr== with y+=# v=&€.

      If you can't easily decode what's on the page in front of you, then comprehension is more challenging.

    2. Dear Jeff
      Thank you for taking the time to post a comment and I am sorry that it's taken me so long to respond - I'll blame the hectic time of year.

      I think we are on the same page re needing to show due caution in over-extrapolating from data, however as far as I am aware, the PSC is the only identifiable policy/practice change in the UK in recent years, so this would suggest to me that there is at the very least a positive trend worthy of ongoing monitoring. I am not close enough to the UK scene to know why things seem to have been better back in the early 2000s - that's a very good question. I do know, however, that you have the good fortune of having just one education jurisdiction to contend with and not 8, as we do in Australia, and of that we are deeply envious :-)

      It's a while since I read the Torgerson et al. review and I need to go back and re-visit it, however I am aware of this recent critique of some of its findings:

      And of course the Simple View of Reading tells us that effective reading is the product of decoding and language comprehension. By Yr 4, we would expect that children are reading a vast number of words automatically ("by sight") by virtue of their familiarity with the orthographic forms of frequently-encountered words. However they still need to decode unfamiliar words, and hopefully either locate them in their existing lexicon or infer their meaning from contextual cues. So to that extent, I would say that PIRLS tasks favour students who are strong on both elements of the SVR.

      Thanks again for taking the time to comment.


  2. Also (serious question) why did Ireland and Northern Ireland do even better than England? They don't do a phonics check and are very similar culturally to England.....

    The phonics check has helped here in England and is a god idea. But it would have been even more effective if it had also included a phonological awareness check. Reading in England isn't improving as much as phonics advocates thought it would - and I think this is because schools still aren't teaching phonological awareness.

  3. Apologies for my very tardy reply Liz...I'll blame the time of year ;-)
    I agree that phonological awareness is extremely important but I am also conscious of the fact that it's an up-hill battle to get any "formal" assessment in place with children after 18 months of school (mainly from a political perspective). It seems we have to win this unnecessary war one battle at a time, and the current battle is about sorting out phonics knowledge and teaching. Maybe once we make some headway there, we can progress the discussion to PA - there is no doubt that children would benefit from having teachers who are more knowledgeable in this space.