What does the growing divide on Rotten Tomatoes mean?

What does the growing divide on Rotten Tomatoes mean?

October 21, 2022

Originally published at notes on cinema

James McMahon

Rotten Tomatoes (RT) found a way to get every last drop from the well of convenience. Film criticism is already pressured, tacitly by convention, or explicitly by editors and bosses, to give bite-size scores with thumbs (up or down), stars (out of 4 or 5), letter grades (with memories of school report cards) or numbers (7 out of 10!). RT goes one step further by collecting as many reviews as possible, producing aggregate scores that measure the general sentiments about films and TV. And if a simple score about a film is all your looking for, RT got rid of all the pesky inconveniences that stood in your way — such as knowing the names of film critics, reading a magazine, or subscribing to a newspaper or journal. (If you find a curmudgeonly tone in this post about RT, I can confirm that I have one. RT hurts film criticism because RT’s convenience lies in giving its users the ability to separate the judgement from the words of the film review. Find film critics you like and read their reviews.)

A recent Bloomberg newsletter revived my curiosity in RT and its role in Hollywood. The newsletter, authored by Lucas Shaw, looks at the growing divide between critic and audience scores about the most popular blockbusters of the past twenty two years. This divide is not only visible in RT data, but the site is central to the trend that Shaw notices: critics appear to have grown tired of the big blockbusters, and their sinking average pulls away from an audience average that is floating close to 80 percent.

---

This post will try to answer why this growing divide appears before us. The annual flow of consumer attention toward the biggest films, which are often part of franchises, widens the day-to-day difference between critics and the average film audience. Critics still have good cinema to write about, but their positive judgements are flowing the other way, toward the films with the smallest box-office. The answer will be produced with two claims:

1. The Bloomberg post is looking at the top box-office films of each year. By ignoring the majority of the cinematic universe, the post has a difficult time saying much more than what can be seen in Figure 1. A stronger explanation, in my opinion, needs to look at what is happening elsewhere in cinema, especially if there are different trends in the less popular pockets of cinema.
2. Changes to the distribution of film have had a very large impact on the distribution of audience attention. The average moviegoer (and likely the at-home streamer) is giving an increasing share of their attention to franchise cinema, in part because there are growing inequalities in theatrical distribution.

1. Surveying all corners of Rotten Tomatoes

The Bloomberg newsletter struck when the iron was hot. Blockbuster cinema appears to be experiencing a period of critic fatigue, which could be produced from poor spin-offs, like Harry Potter’s Fantastic Beasts films, or tiresome repetition in Marvel franchises. But the Bloomberg newsletter also risks inflating its observations. The newsletter does not always qualify that it is selectively looking at the most popular of the the popular; and top-tier blockbusters, especially ten per year, are just a fraction of the cinematic universe. What is happening beyond the top ten biggest films of each year? Are there biases in selecting films by box-office revenues or streaming popularity? Bloomberg seems uninterested in answering these questions.

To investigate the divide, we need more data. I have been gathering RT data for almost a month now. I filtered the data to keep films that have counts of reviews that are greater than or equal to the 50th percentile. There are others ways to filter the data, to be sure, but the main concern is that we create misleading biases if we use all of the data. Thousands of films on Rotten Tomatoes have only small handfuls of reviews — sometimes only one or two.

When filtered above the 50th percentile, my dataset contains 7,350 films that were released between 1930 and 2022. We do not need such a large dataset for this particular blog post, but a plot of the data clearly shows that the Bloomberg article is missing parts of the story. Figure 1 plots the average score of each decade and shows the count per decade in a subplot. There appears to be growing divide between critics and fans after 2010, but the figure is actually showing that the general trend is the mirrored image of what the Bloomberg post wants to claim. Critic scores are higher than audience scores! If there is a divide in opinion, it is the audiences that have grown “tired” of twenty-first century movies.

---

What is going on? Essentially, Lucas did not check if there was a large or small impact on selecting the top movies of each year in Figure 1. When you select the “top” or the “best” in cinema, the selection is often an effect of money. Thus, what do we know about the relationship between highest grossing films and the job of the film critic? Is the data biased to having high audience scores because a high grossing film needs … a big audience?

Figure 3 shows us how box-office grosses can change our picture of the relationship between critic and fan. The figure is plotting the average difference between the critic and audience score of each film between 2000 and 2022. (For example, Bad Boys II was generally hated by critics and received a Tomato Score of 23. Audiences generally liked the film a lot more, which produced an RT Audience Rating of 78. The difference (Tomato Score – Audience Rating) is -55.) The figure also bins films annually by US box-office gross revenues, where Bin #1 is the smallest box-office revenues of the year and Bin #10 is the biggest. When looking at Bin #10 we see what Lucas sees: fans like the films more than the critics. When looking at the lower bins — Bins #1 and #2 — there is a similar degree of difference but the difference is positive because, in this case, critics are rating smaller box-office successes higher than audiences.

---

2. There is less overlap between critic and the average moviegoer

The trend in Figure 3 — for critics to like films with small box-office grosses more than audiences, and for audiences to like box-office hits more than critics — is likely an effect of both sides, audiences and critics, self-selecting. Neither side is randomly sampling the landscape of cinema. Rather, there is a high likelihood that we can predict which films they are watching. (Critics, of the two groups on RT, are watching a lot of films, but this group still has predictable traits — they are not randomly sampling films to watch, but instead doing their jobs by working with editors, watching screeners, traveling to film festivals and attending premieres. Audiences are the more predictable of the two groups. Neoclassical economic approaches may steadfastly hold to notions that consumer unpredictability is a permanent threat to any film, but the predictability of audiences is an effect of the social circumstances of cinema being shaped by firms that have financial interests in consumers seeing some films, but not others.) Self-selection is, in this case, a symptom of watching what is available to you. For instance, when your multiplex has given the majority of its screens to only one franchise blockbuster, the breadth of your ratings on RT will match the narrowness of your situation.

Figure 4 helps us see why the self-selection of audiences trends in one direction. The purple and yellow series plot the inequality of theatrical cinema by gross revenues and opening theatres. The Gini index, which has a range between 0 and 1, will increase when there is greater inequality in a distribution. The increases in the inequalities of gross revenues and opening theatres happened as we experienced a change in consumer behavior. From the 2000s onward, moviegoers in the United States are increasingly giving more of their theatrical attention to franchise cinema.

---

The effects of theatrical cinema’s transformation can be seen in the behavior of the users that rate films on RT. Figure 5 demonstrates how, overall, the larger numbers of audience ratings are given to the highest-grossing films per year, from 1999 to 2022. Films in Bins #9 and #10 will get more critic reviews than films in Bins #1 and #2, but the difference is not as stark as how the attentions of RT users are distributed. The biggest grossing films of the year are receiving around 4 times the amount of reviews as films with low theatrical revenues.

---

The inequality of attention is also increasing over time. The left axis of Figure 6 plots the Gini indices of the distributions of audience ratings and critic reviews per year from 1999 to 2022. As we can see, the inequality of both have increased over time, which suggests that the growing inequalities of theatrical cinema are affecting how people spend their time on RT. Audience ratings have always been more unequal than critic reviews, but the former has now risen to roughly 0.80 (1 is perfect inequality). Furthermore, the rise in the inequality of audience ratings has occurred in parallel with the growth of franchise cinema’s dominance at the US box-office.

---

Conclusion

Shaw’s newsletter was thought-provoking enough to get me to write this blog post. But I also have been staring at RT data for a month now, and I do think the take-away of the Bloomberg newsletter is too clean.

One final thought for now. Figure 7 shows the punchy headline of the newsletter. Based on the data I gathered, 2022 is not the year when there was the most disagreement between critics and audience. Filtering data above 1998, when RT was created, the left panel of Figure 8 demonstrates that there was greater disagreement between critics and audience around 2004 and 2005. Furthermore, the disagreement of the early 2000s is the “type” that matches the narrative of Bloomberg: audiences liking cinema much more than critics.

---