You may or may not have read my essay titled 'On Being a Hater - The Loss of Critical Consumption', in which I talked about how I am an advocate for critical consumption and believe people have become too attached to the media they consume to be able to critique it in a healthy way.
One point that I left out, because I didn't want to muddy the essay, was about how this problem was evident in the ratings for media across different social media websites. Letterbox'd and IMDb for movies, Goodreads for books, Aniilist and myanimelist for anime and manga. There are tons of rating websites out there, even outside of these few examples, where people make accounts to rank, track, and discuss the things they consume.
I decided to just do some cursory google searching - and I realize that most of these websites have public APIs I could possibly scrape in the future for more scientific results - to try and find the distribution of ratings across these sites . I wasn't able to find exactly what I wanted for every site, which was distribution graphs, but I did find the average ratings for the following:
- Aniilist average rating: 5.95 and
- MAL average rating: 6.75 as of 2022
- Goodreads average rating: 4/5 stars
- IMDb average rating: 7
If you believe, like me, that quality tends to follow a normal distribution - or more simply that for something to be good, something else has to be bad - then these numbers should raise some eyebrows. In a normal distribution, most content should sit at around 'average', with very few outstanding works and very few truly awful works. The distribution also follows a bell curve, where 8s are rarer than 7s, and 3s are rarer than 4s and so on.
The aniilist average, sitting around a 6/10, seems to be the least egregious if we account for the fact that people don't tend to bother watching anime they truly think is awful. However, the fact that the average score for the other websites sits at around 70% or higher really calls into question the usefulness of these ratings and the reasoning of the users of these sites. This even applies to sites like Amazon, Google, Yelp, and other vendor websites for every niche product you can imagine!
There is a great video, and attached article, from a former audiophile reviewer called Crinacle who covers how even vendor sites have frankly useless product ratings. A lot of his points are much more well laid out than mine, so I encourage you to watch the video.
One critical aspect he covers and I wish to highlight is that when a 5-star rating scale has no scores below 3 stars, it ceases to be useful. If I wish to determine if a book is a must-read, it's not helpful to know that the highest score is a 5 and the lowest score is a 3. That basically only gives me 3, 4, and 5 as possible options, narrowing the scale and making the different between a 4 and a 5 so narrow it might as well be meaningless. For your scale to be useful, you need to use the entire range.
So why do people seem to automatically give something a 7/10 as long as they didn't hate it?
I can tell you that for some of my friends, who tend to do this, it comes from the North American school system having conditioned them to believe that 70% is a C, which is average on a letter scale of A, B, C, D, F. This explains the phenomenon in some ways, but doesn't make it make sense. Tests and grades reflect your understanding of material (if we're being generous). If you only understand 50% of what you should have learned, you're certainly doing a poor job. But quality cannot be viewed the same way. If something is average, it should be in the middle of our distribution. It should get a 5/10, or a 2.5/5 stars, etc.
As much as this is not statistical evidence, I was shocked and also impressed when I browsed Google ratings of restaurants in Japan and found that most places hovered around 3-3.5 stars! In the US, a place rated 3 stars barely gets orders right! So maybe it is the fact that in North America we are brought up believing that when we rate things we are giving them a grade, and therefore 5/10 is failing. But maybe there's just some other cultural force at play.
Regardless, let me encourage you to stop rating things like you're giving them a grade at school. 5/10 should be average, not terrible. 7.5/10 is not 'mid'. And just because you like something, it doesn't mean it's at least a 9, or breaks the scale and gets a 13/10.
I try to do my best to follow this advice in my album ratings, and I believe after running some quick stats on all my ratings I hover around a 5.5 as my average, which I would say is skewed but decent, especially considering that I would tend to only listen and be aware of albums from artists whose previous work I've enjoyed.
_ _ _
Thoughts? Send me an email!
kagumail.uselessly535@passinbox.com
Feel free to remain anonymous and send it from a secure mailer!
