The Dunning-Kruger Effect is Autocorrelation – Economics from the Top Down

When I first heard of the Dunning-Kruger effect I immediately created a tag for it - I just knew it was was significant. As the author here points out, it was both "too juicy to not be true. Everyone ‘knows’ that idiots tend to be unaware of their own idiocy".

Unfortunately, it's poor science: "if you carefully craft random data so that it does not contain a Dunning-Kruger effect, you will still find the effect... nothing to do with human psychology.1 It is a statistical artifact — a stunning example of autocorrelation... when you correlate a variable with itself... the statistical equivalent of stating that 5 = 5... the Dunning-Kruger chart ... violates a fundamental principle in statistics. If you’re going to correlate two sets of data, they must be measured independently... The chart mixes test score into both axes, giving rise to autocorrelation".

The post provides an excellent intro to autocorrelation, and then explains how it happened in the original Dunning & Kruger paper, including using real data to demonstrate it.

While the original paper came out in 1999, "it took until 2016 for the mistake to be fully understood... in 2016 and 2017.) In 2020, Gilles Gignac and Marcin Zajenkowski published a similar critique... [nevertheless] the three critique papers have about 90 times fewer citations than the original Dunning-Kruger article" (April 2022).

Moreover, the original critique paper finds that the effect disappears when measured properly: "assessment bias is trivially small... [although] the spread in self-assessment error tends to decrease with more education... professors are generally better at assessing their ability than are freshmen".

And the irony: "Here are two Ivy League professors7 arguing that ... not only are unskilled people ‘incompetent’ … they are unaware of their own incompetence... the situation is actually reversed... [they] are the ones broadcasting their (statistical) incompetence".