“There is a science to learning and we are finding out more and more about what works best to support the learning processes that make a difference for your learners.“ Advertising for a Visible Learning symposium at the Australian Council for Educational Leadership (ACEL) website
“Assisting practising teachers to maximise their impact on student learning relies on implementing practices that have been shown to benefit students the most – with constructive feedback on educational practices, collaboration and effective professional learning.” From the Australian Institute for Teaching and School Leadership (AITSL) Chair John Hattie’s Statement of Intent
“Hattie’s work is everywhere in contemporary Australian school leadership. This is not to say that educators have no opportunity for resistance, but the presence and influence of brand Hattie cannot be ignored. The multiple partnerships and roles held by Hattie the man and the uptake of his work by systems and professional associations have canonised the work in contemporary dialogue and debate to the extent that it is now put forth as the solution to many of the woes of education.” Scott Eacott
“Unfortunately, in reading Visible Learning and subsequent work by Hattie and his team, anybody who is knowledgeable in statistical analysis is quickly disillusioned. Why? Because data cannot be collected in any which way nor analysed or interpreted in any which way either. Yet, this summarises the New Zealander’s actual methodology. To believe Hattie is to have a blind spot in one’s critical thinking when assessing scientific rigour. To promote his work is to unfortunately fall into the promotion of pseudoscience. Finally, to persist in defending Hattie after becoming aware of the serious critique of his methodology constitutes wilful blindness.” Pierre-Jérôme Bergeron
Since the original publication of John Hattie’s book, Visible Learning, there have been questions raised about the statistical methodology underpinning his research and representation of ‘what works best for learning’. By 2014, the year Professor Hattie became the Chair of AITSL, it was clear, even to tertiary statistics students, that serious mathematical errors had been made. There continues to be a steady flow of journal articles contesting Hattie’s ideas. By 2017, concerns about flawed use of statistics and how the politics of education works in Australia sees many practitioners not really needing to read a journal article to know all about “the cult of Hattie” in our schools.
Hattie continues to rank the “195 Influences And Effect Sizes Related To Student Achievement” without acknowledging the concerns raised by statisticians. After reading the latest paper which derides the methodology, I decided to see what some influential educators thought. A simple tweeted question:
Your thoughts about this new analysis of Hattie’s statistics? http://mje.mcgill.ca/article/view/9475/7229
resulted in Stephen Dinham, Scott Eacott and Dylan Wiliam responding with the latter agreeing the stats are flawed:
In my view, yes. Issues: age dependence of ES; sensitivity to instruction, study selection; publication bias; atheoretical categorization…
Scott Eacott‘s recent paper, School leadership and the cult of the guru: the neo-Taylorism of Hattie places Hattie’s work in an Australian context. It really is essential reading for educational leaders. I urge you to read it and engage with him, perhaps on twitter. Hattie’s reply to Eacott’s paper does not even remotely grapple with the issues raised and I note “no potential conflict of interest was reported by the author”. Eacott tweeted that the journal will not publish his response.
Corwin Australia (see screenshot from their website below) is on a good thing. Google “Visible Learning” + your town and see how many primary and secondary websites links you find back to this business/service. There is a growing coterie of trainers around the world delivering this trademarked professional learning, based on Hattie’s meta-analyses.
Just to be clear. Professor Hattie has had a stellar career and much of his work makes complete sense without an iota of research. Who would argue with Hattie’s point that teachers with impact are:
- passionate about helping their students learn
- able to build strong relationships with their students
- clear about what they want their students to learn
- using evidence-based teaching strategies
- monitoring their impact on students’ learning, and adjust their approaches accordingly
- actively seek to improve their own teaching
- viewed by the students as being credible
However, when flawed statistical analysis is resulting in advice that high-impact, evidence-based teaching strategies include:
- Direct Instruction
- Note Taking & Other Study Skills
- Spaced Practice
- Teaching Metacognitive Skills
- Teaching Problem Solving Skills
- Reciprocal Teaching
- Mastery Learning
- Concept Mapping
- Worked Examples
but there’s little or no impact with:
- giving students control over their learning
- problem-based learning
- teaching test-taking
- catering to learning styles
- inquiry-based teaching
one feels a little less comfortable with the advice considering the statistical analysis of effect size is worse than merely dubious. Research can only tell us what may have happened not what is needed next as we all grapple with the future.
Context is everything. That includes the context, throughly discussed by Dr Eacott, that has led to Australian schools looking for scientific, evidence-based solutions to the educational challenges highlight by PISA and NAPLAN. Dylan Wiliam, since at least 2009, has questioned the use of meta-analysis in education. It seems pretty obvious that Hattie’s number-crunching has appealed to politicians and administrators looking to solve what often feels like a manufactured series of education crises.
It is worthing quoting the conclusions from a 2009 paper that you really should read:
…we want to repeat our belief that John Hattie’s book makes a significant contribution to understanding the variables surrounding successful teaching and think that it is a very useful resource for teacher education. We are concerned, however, that:
(i) Despite his own frequent warnings, politicians may use his work to justify policies which he does not endorse and his research does not sanction;
(ii) Teachers and teacher educators might try to use the findings in a simplistic way and not, as Hattie wants, as a source for “hypotheses for intelligent problem solving”;
(iii) The quantitative research on ‘school effects’ might be presented in isolation from their historical, cultural and social contexts, and their interaction with home and community backgrounds; and
(iv) In concentrating on measureable school effects there may be insufficient discussion about the aims of education and the purposes of schooling without which the studies have little point.
It seems appropriate to close with one of the quotes that opened this brief post and to ask what you think? Your commentary is, as always, highly appreciated.
To believe Hattie is to have a blind spot in one’s critical thinking when assessing scientific rigour. To promote his work is to unfortunately fall into the promotion of pseudoscience. Finally, to persist in defending Hattie after becoming aware of the serious critique of his methodology constitutes wilful blindness.” Pierre-Jérôme Bergeron
Bergeron, Pierre-Jérôme; Rivard, Lysanne, How to Engage in Pseudoscience With Real Data: A Criticism of John Hattie’s Arguments in Visible Learning From the Perspective of a Statistician, McGill Journal of Education / Revue des sciences de l’éducation de McGill, [S.l.], v. 52, n. 1, July 2017. ISSN 1916-0666. Available at: <http://mje.mcgill.ca/article/view/9475/7229>, Date accessed: 22 Aug. 2017.
Eacott, Scott, School leadership and the cult of the guru: the neoTaylorism of Hattie, School Leadership & Management, DOI: 10.1080/13632434.2017.1327428, 2017
Hattie, J., Visible learning for teachers: maximizing impact on learning, London: Routledge, 2012
Siebert J. Myburgh, Critique of Peer-reviewed Articles on John Hattie’s Use of Meta-Analysis in Education, Working Papers Series International and Global Issues for Research, No. 2016/3 December 2016. Availability:<http://www.bath.ac.uk/education/documents/working-papers/critique-of-peer-reviewed-articles.pdf> Date accessed: 26 Aug. 2017
Snook, Ivan; O’Neill, John; Clark, John; O’Neill, Anne-Maree and Openshaw, Roger. Invisible Learnings?: A Commentary on John Hattie’s Book – ‘Visible Learning: A Synthesis of Over 800 Meta-analyses Relating to Achievement’ [online]. New Zealand Journal of Educational Studies, Vol. 44, No. 1, 2009: 93-106. Availability:<http://search.informit.com.au/documentSummary;dn=467818990993648;res=IELNZC> ISSN: 0028-8276, Date accessed: 26 Aug. 2017