Visualizing Trends and Patterns in Pitchfork Reviews
By Joe Fregault, Karina Santoso, Ashley Lu, Ishaan Shah
Introduction
Pitchfork, the most popular independent-focused online music publication, is best known for its music reviews. The site was started in 1995 as a blog primarily covering indie music and has expanded to cover all genres of music since then. Now, the site holds an extremely large audience, with more than 240,000 readers per day and over 1.5 million unique visitors per month and claims to be “the most trusted voice in music.”
Those who visit Pitchfork’s site and read their reviews are more or less likely to listen to a particular artist, album, or song based on what they read. Thus, the site’s reviews, both positive and negative, have been considered widely influential in making or breaking musicians’ career, as people who visited Pitchfork’s site and read their reviews were As a result, some view the site as a barometer of the independent music scene; however, Pitchfork has been seen as having a negative impact on the careers of some artists, and the site has been accused of being biased against certain genres. Claims have been made that Pitchfork favors lo-fi, indie rock, and other more obscure music genres, and that they intentionally write reviews with provocative and unconventional opinions for attention.
In an attempt to determine if this was true or not, we analyzed a Pitchfork review dataset from Kaggle, which includes almost 20,000 reviews from Pitchfork’s website ranging from 1999–2017. For each review, the dataset includes the album, artist, and genre of the subject of the review, the date the review was published on, the text of the review, and the score the review was given, on a scale ranging from 0 to 10, where higher scores are more favorable.
General Findings
What is the overall distribution of scores given out by Pitchfork?
The average scores of all the reviews in our data set was 7.03, and the median was 7.3, indicating that Pitchfork generally gives relatively high scores to the albums they review. This is further supported by the left-skewed distribution of scores showing that more reviews are given positive ratings than negative ones. Most scores given fall around the 6–8 range, and scores above or below this range are more rare. This may be due to Pitchfork’s relatively selective process when choosing which albums to review. Most albums reviewed typically come from established artists or albums that are generating a buzz.
How many reviews are there per genre?
Pitchfork is true to their roots and primarily reviews Rock albums over other genres. Global and Jazz have the lowest number of reviews, so each individual review in that category will have a greater influence on its average score. Although our dataset only has one genre per album review, it’s important to note that some albums are listed under multiple genres. The genre inputted in our dataset is the first genre listed in the article, which is usually the more prominent one. For example, the album Junk by M83 is under the Electronic genre in our data despite being both Electronic and Rock in the review proper.
Which genres have the highest and lowest median scores?
The median review score for each genre falls between 7 and 8 and seems to mostly be centered around 7.5. Global and Jazz have the highest median scores, while Pop/R&B has the lowest. Interestingly, Jazz and Global had the lowest number of reviews, and genres median scores seem to generally increase as the number of reviews they have decreases. The median was used instead of the mean to ensure that outliers, such as particularly badly reviewed albums, did not affect the result too much. There does not seem to be too large of a gap between the median scores of any two genres, possibly indicating the lack of a strong bias towards or against any genre as some critics of the site claim.
What is the distribution of scores by genre?
We see that the vast majority of scores given tend to fall within the 4–9 range and that scores outside of this range are distributed a lot more sparsely. There are a few extremely low scores every year up until 2009, but none after that, possibly indicating that Pitchfork stopped reviewing music their authors felt strongly disfavorable about. After 2011, we see that scores below 4.0 become a lot less common as well, further supporting this trend. Thus, it seems that at least in recent years, Pitchfork does not seem to be choosing musical works to review negatively, so their negative impact on new musicians/artists would most likely be very minimal, if any. The abundance of pink dots show again that Rock is by far the most reviewed genre compared to the others.
What are the best and worst performing artists?
Following are visualizations depicting the best and worst performing artists on the platform. Only artists who had 5+ albums reviewed on the platform to avoid any sort of anomalies and to give a fairer representation to consistent performers. Rock bands The Velvet Underground and Pavement top the list, with rapper Kendrick Lamar and the infamous Beatles filling up the other 2 spots. On the other side of the coin, rock bands The Dandy Warhols, Joan of Arc, The Get up Kids and N.E.R.D top the list of worst performing on the platform. It seems like Rock is a genre that really divides people’s opinions.
Is Pitchfork biased towards certain genres?
We defined a high scoring album to have a score greater than 8 because only 16.3% of albums received that distinction. Although at first glance it appears that Pitchfork gives out higher scores for Rock albums, when we account for the total amount of reviews each genre received by dividing the amount of high scoring reviews by the total number of reviews, we obtain the following graph which show Rock to be very middle of the pack.
Genres
As shown by over 2000 reviews under the None genre, music can be difficult to categorize at times. Nevertheless, analyzing different trends in a given genre was a useful way for our group to clearly see patterns in segmented portions of the data.
Rock
The first genre we explored was the most popular genre reviewed by Pitchfork with nearly 7000 reviews during our time frame, Rock.
By looking at the change in scores and the amount of rock reviews over time, there seems to be a negative correlation between the mean Rock score and the number of reviews that year. The Rock genre seems to have higher scores in the years where there weren’t as many Rock reviews. This is especially true in 2002 where the amount of Rock reviews increased by over 100 reviews from the previous year and saw a minimum mean score in this time period. Again in 2007 where Rock received nearly 500 reviews during the year, the most out of any year in our time frame, there was a large dip in mean review score. As the genre rose to prominence, there may have been many new artists who were inspired to create low quality albums due to inexperience or utilizing stale tropes.
Interestingly, 2016 yielded the highest mean review score overall after the resurgence of Rock albums the year prior. Not only is there a plethora of rock albums currently being released, they’re of high quality, indicating that the genre is here to stay.
Using word clouds helped to figure out what were the general criteria that Pitchfork reviewers were looking for in certain genres. By removing common English stop-words as well as common music terminology that can be found across all genres (i.e., “song” and “record”), it becomes clearer that reviewers focus on different criteria for different genres. As mentioned previously, Rock is one of the most popular genres on Pitchfork, with 6,958 reviews. Words found at a higher frequency are represented in larger font, and words of similar frequency are represented in the same color. For example, “band” is by far the most common word found in Rock reviews, appearing 14,706 times. In comparison, the second most frequent word, “bands,” appears 8374 times, and the third most frequent word, “time,” appears 8,124 times. It is reasonable, then, that “band” is represented in a larger font and is the only word in gray while “bands” and “time” are of a smaller font and share the same green color. Rock reviews tend to be focused on “bands” and the “guitar,” which are commonly used for all genres but especially so for the rock genre. There is also a focus on strong vocals and lyrics.
Jazz
Jazz was another interesting genre to explore since it consistently recorded the highest overall score of all other genres, despite being one of the categories with one of the lowest total number of reviews.
This genre follows a different trend from the one illustrated by Rock where the more albums that are reviewed, the higher the score. Overall, the Jazz genre has a very low amount of albums reviewed by Pitchfork. The most Jazz albums reviewed in a single year was 34 in 2016, which also saw the peak in mean score that same year. Since Jazz has a relatively higher barrier to entry than other genres due to the classical training and instrumental knowledge inherently required, more established and trained musicians may yield a higher mean score for albums they produce. In contrast, in other genres such as Rap, an artist can compose a track just utilizing their smartphone with little to no experience.
The golden period for Jazz occurred towards the final years of our data collected where it experienced the highest amounts of albums produced with the highest ratings. Jazz is traditionally associated with being an older genre since it was invented more than a century ago, yet it is still experiencing much popularity.
Jazz has a greater focus on different instruments besides the guitar, such as piano, drums, and bass, in comparison to the Rock genre. Another interesting observation is the significant number of times that Miles Davis is mentioned in the reviews. This goes to show just how much influence Davis has had on the jazz genre, and it is likely that Pitchfork reviewers compare jazz albums with those of Davis.
Global
This genre exhibited the second highest median scores after Jazz. Albums classified under the Global genre are foreign albums outside of English speaking countries in Europe and North America and usually feature different languages.
The genre with the second highest mean score was the Global genre. Just like with Jazz, there weren’t many albums released classified as this genre with the highest number being 24 in 2007. The low amount of album reviews means that if a few albums get negative reviews, it can derail the mean score for the entire year. However, the highest mean score of any year in 1999 was a 9.0 which occurred when a single album was reviewed. Since albums classified as “Global” are foreign albums, it’s possible that only the best selection of foreign albums percolate to Pitchfork’s view which is an American publication that primarily reviews popular English speaking artists. It is also important to note that this genre encompasses a large variety of diverse music, as it includes all albums from non-English speaking countries, so the scores given could easily vary greatly depending on the person reviewing the album and their experience or lack thereof with this particular type of music.
The lowest mean score in a given year is 7 which is much higher than any the lowest mean score of other genres.
Global encompasses a wide range of genres outside of Western music. That is reflected in the word cloud, which mentions genres around the world such as reggae and funk. Dance also seems to be mentioned more frequently in the global genre than in other genres, which makes sense given that dance plays a larger role in a lot of non-Western genres.
Rap
The final genre we took an in depth look at was Rap primarily due to the fact that it began as a very niche genre reviewed by Pitchfork, beginning with reviews in single digits and ending with nearly 200 rap albums reviewed in a single year.
Similar to the trends we saw in the Rock genre, the more Rap albums reviewed in a given year usually corresponds with a lower mean score for that year. Since only ~10 albums were reviewed each year from 1999–2001, this genre will probably not see very high mean scores for a given year ever again due to the explosion in popularity. The more reviews in a year, the greater the chance there is for low scoring albums to decrease the mean score for that year. Although the Rap genre is as popular as it’s ever been, we do not see a decline in mean scores with the increase in popularity like we did with Rock.
The reviews for rap were the most different from the reviews from the other genres, focusing more on the beats and production. In comparison to other genres, rap reviews make no mention of common words such as “band” and “guitar.” From these word clouds, Pitchfork users can make more informed decisions when it comes to venturing into new genres. For example, people who prefer more vocal-heavy songs might lean towards Rock, whereas people who want to be able to dance to what they’re listening to might decide to explore Global music.
Conclusion
This Pitchfork dataset provided an interesting view into the popularity and critical reception of different musical genres. A common theme that was constant throughout our analysis was how a high amount of reviews a genre garnered in a given year generally correlated with a lower mean score. The oversaturation of a genre invites the possibility for many albums to be poorly received. The poor reception may be due in part to critic fatigue where critics grow tired of constantly reviewing similar musical works. Intriguingly however, this trend reversed itself in the final years of our dataset around 2015–2017 where there was an increase of albums reviewed and a higher mean score across the board for all genres. This could be attributed to the wide variety of music available to the populace through the internet and streaming services. Never before has music been so accessible and easy to make which allows the proliferation of many great artists serving different musical niches.
From our analysis, we saw no evidence to suggest that Pitchfork controversially rates albums for publicity or is biased towards a certain genre. Although most of the albums reviewed by the publication are classified as Rock, the genre experiences a variety of different scores including many poor reviews. Furthermore, in recent years, there has been a large upward trend in the number of non-Rock albums being reviewed by Pitchfork, possibly indicating that they are currently further expanding the range and diversity of music they review and engage with. Overall from our findings, Pitchfork emerges as a growing platform for musical reviews that provides a variety of reviews and is an excellent source for listeners who are seeking to find new music in this current golden age of music production.
