Understanding the S&P 500 Through Data Analytics
Authors: Aarnav Sarvaiya (Project Lead), Allen Chen, Sean Gee, Robin Murphy, Max Xu
Introduction
The S&P 500 is often regarded as the barometer of the U.S. economy and a critical indicator of global market trends. Comprising 500 of the largest publicly traded companies in the United States, it spans diverse sectors, reflecting the pulse of economic growth, innovation, and consumer demand.
In this analysis, our group explores the S&P 500 through various lenses to uncover meaningful insights. From sector breakdowns to market cap trends and historical event impacts, our goal is to highlight how data can reveal underlying patterns in the market and inform investment decisions.
Sector Breakdown of the S&P 500
The S&P 500 is divided into 11 primary sectors, ranging from Technology to Utilities. By examining the sector distribution, we see that certain industries — such as Technology and Healthcare — dominate the index, collectively comprising over 40% of its weight. Conversely, sectors like Materials and Utilities are less represented, reflecting their smaller role in the overall economy. However, this distribution also shows the disproportionate impact certain sectors will have on market performance. The recent positive trends of the S&P 500 can be attributed to similar growth shown in the strength of the technology sector.
Consumer Discretionary Sector Analysis
Comprising about 20% of the S&P 500 index, the Consumer Discretionary sector encompasses industries that provide non-essential goods and services — everything from retail and entertainment to automotive manufacturing and transportation. This dynamic sector faces heightened volatility because of changing discretionary spending based on consumer welfare and the current state of the economy.
The COVID-19 pandemic created unprecedented challenges for this sector, disrupting supply chains, altering consumer behavior, and triggering government interventions. Through my analysis of the Consumer Discretionary sector post-covid, I aimed to understand the market trends caused by the pandemic’s aftermath.
The top three industries in the Consumer Discretionary sector reveal an especially sharp upward trajectory over the last five years, largely driven by these companies’ innovative approaches to evolving consumer demands. Amazon, for example, capitalized on the pandemic-driven surge in e-commerce, solidifying its dominance in online retail and specialty distribution. Similarly, Walmart and Costco adapted to shifting purchasing behaviors by expanding their digital platforms and maintaining supply chain resilience, allowing them to thrive during inflationary pressures. Tesla’s rapid ascent redefined the automotive industry, as its market valuation soared amid heightened interest in electric vehicles and Elon Musk bringing a wave of virality. These companies not only drove their respective industries to unprecedented highs but also reinforced their position as leaders within the sector.
Upon closer examination of the time period following the onset of COVID-19, it is evident that global events had drastic impacts on the stock prices within the Consumer Discretionary sector. There are several peaks and dips throughout the graph that highlight the sensitive nature of the sector to external shocks.
March 2020: The initial pandemic lockdown triggered a sharp decline, as uncertainty and disruption to processes across the world left people in a state of panic. However, a massive shift to e-commerce and online retail helped companies like Amazon and Walmart quickly rebound. Around this time, Tesla began gaining traction as well due to the virality of Elon Musk’s endorsements and his dedicated fanbase. This correlates with habits such as investors purchasing “meme stocks” during this time as a result of boredom and interest during the pandemic.
November 2020: The announcement of a vaccine in November 2020 marks the beginning of a strong rally, particularly for Retail and Auto Manufacturing, as optimism about a return to normalcy drove renewed consumer demand.
January 2022: The Omicron surge and poor quarterly results led to a sharp decline in stock performance. This period marks a time of reemerging concern and uncertainty about the future.
March-June 2022: The post-pandemic period saw inflation levels rise to new highs of 9.1%, as the economy struggled with supply chain and production issues. To combat this, the Federal Reserve raised interest rates by half a point, the highest in 15 years. This led to high volatility as events were frequently influencing investors between optimism for inflation control and fears of a potential economic slowdown.
The performance of Consumer Discretionary stocks post-COVID underscores a fundamental truth about long-term investing: while short-term volatility is inevitable, the market historically trends upwards over time. Periods of hardship, such as the pandemic and the ensuing inflationary pressures, can be unsettling, but they often serve as temporary setbacks in the broader growth trajectory of the U.S. economy. The recovery seen since mid-2022, with current stock prices exceeding prior highs, is a testament to the resilience of both the market and the underlying economy.
For prospective investors, this recovery demonstrates the potential for significant gains by maintaining a disciplined approach. While the short term may bring challenges, staying invested and focused on the long game has consistently been a winning strategy. If the economy fails to recover, it would signal affects on our daily life far worse than just poor stock performance. Until then, investing in quality companies within sectors like Consumer Discretionary offers the opportunity to benefit from long-term growth as the economy continues to strengthen.
Case Study: Consumer Goods and Rolling Averages
To continue off of the Industry Performance Trends, let’s have a look at a case study about consumer goods and their rolling averages. We will focus on the performance of some prominent consumer goods companies like Coca-Cola (KO), Procter & Gamble (PG), PepsiCo (PEP), and Colgate-Palmolive (CL). To give some context, Coca-Cola sells a wide range of beverages, including its flagship soda, as well as bottled water and juice, with a global presence and brand recognition that has made it a staple in the consumer goods sector. Similarly, Procter & Gamble is a leader in household products, spanning areas such as cleaning, personal care, and hygiene, offering brands like Tide, Pampers, and Gillette. PepsiCo competes directly with Coca-Cola in the beverage industry but also has a significant portfolio in snacks, including brands like Lay’s and Doritos. Lastly, Colgate-Palmolive focuses on personal care and household products, with well-known products such as Colgate toothpaste and Palmolive dish soap.
First, the distribution of stock prices reveals that most of the stocks for these companies have historically clustered around certain price points, with PepsiCo and Procter & Gamble showing higher price frequencies compared to Coca-Cola and Colgate-Palmolive. This suggests that stocks from PepsiCo and Procter & Gamble have had a tendency to trade at higher levels, potentially reflecting investor confidence in their financial stability and market performance. On the other hand, Coca-Cola and Colgate-Palmolive, while also performing well, have seen a wider distribution of prices, indicating more variability in investor sentiment or market conditions affecting their stock prices.
Now, to understand how these companies are performing in the market, we look at their stock prices over time. Stock prices show how much people are willing to pay for a share of a company. But these prices can go up and down a lot day to day, influenced by news, trends, or market events. To make sense of this, we use something called a 30-day rolling average. A 30-day rolling average takes the average stock price over the last 30 days, smoothing out the short-term ups and downs to show a clearer long-term trend. Think of it like tracking your average temperature over the last 30 days, so you’re not distracted by one unusually hot or cold day.
When we apply this method to companies like Coca-Cola, Procter & Gamble, PepsiCo, and Colgate-Palmolive, we can better understand how they are performing over time. If a company’s stock price steadily increases with a consistent rolling average, it means the company is likely growing, and investors are confident in its future. On the other hand, big fluctuations in the stock price or a declining rolling average could suggest challenges or instability. The line graph shows the stock prices for these companies along with their 30-day rolling averages. Using this, we can compare how their performance over the years changed and spot patterns. For example, if Coca-Cola’s stock price jumps up but the rolling average stays stable, it could indicate a temporary spike in the stock price that might not last long.
One interesting thing to consider is how rolling averages can also be calculated over different time periods. In addition to the 30-day average, the use of a 90-day rolling average gives us an even smoother line that reflects longer-term trends. The 30-day average gives us a quicker, short-term view of the company’s performance, while the 90-day average helps us see more stable, long-term growth or decline. By comparing, we can see that for example, if a company has a strong 30-day rolling average but the 90-day average shows fluctuations, it might be growing fast, but it could also be prone to sudden changes. This helps investors make decisions about whether they want to invest based on quick growth or stable, long-term performance.
Ultimately, this type of analysis helps us see beyond the day-to-day noise in the stock market, giving us a better idea of how these major consumer goods companies are performing over time, and where they might be headed in the future. It’s a key tool for making informed decisions about which companies to invest in, whether for steady growth or long-term stability.
Tech Stocks and Volume
Companies in the technology sector are constantly evolving as technological advancements foster a hyper-competitive market. This is the case for all the technology companies in this case study, which include Nvidia, Apple, Google, and Microsoft. Although these companies sell different products, as industry leaders, any decision made by these companies is a major indicator of where the technology sector is headed.
We will analyze each of these companies’ relationship to the broader technology sector as a whole, before exploring the connections that exist between them. A unique aspect of technology companies is that they often partner with each other and are dependent on other technology companies to source components, so it is essential to factor these in when considering volume spikes or dips in the data.
The company that has the most frequent spikes in trading volume from 2000 to the present is Nivida. While it targeted a niche market in its early years by producing GPUs to target the gaming industry, since then it has pivoted towards manufacturing GPUs for commercial AI and scientific computing applications, a change that has been exacerbated by the ever-growing popularity of AI chatbots and products that use AI. Thus, Nivida’s status as a high-volume stock over the past 20 years shows that it has always garnered investor interest, especially in the present as it continues to partner with companies in all sectors that can benefit from the AI revolution.
Apple historically has a strong showing in trading volume, as the graph indicates a peak whenever a new product is revealed, which is especially prominent in the peak where the iPhone 1 was unveiled in 2008. However, there has been a gradual downward trend in Apple’s trading volume since 2008, which can be attributed to several reasons. Increasing competition in foreign markets, such as from Samsung and Chinese brands, a slowing pace of innovation due to a lack of major upgrades, supply-chain disruptions from COVID-19, and being behind in the AI revolution shows that despite Apple’s market capitalization of $3.5 trillion solidifying its position as the largest company in the world, it could be attributed to not merely investor interest in the stock but rather for other reasons, such as the perception of Apple being a stable stock.
As indicated in the graph, Google initially became a large player in the web search market, with investors expecting good performance after its IPO in 2004. However, its trading volume gradually stabilized over the years. Recent developments may change that, as the US Department of Justice seeks to break up Google for its monopoly in search engines, so its future stock price remains tentative.
The trend in the trading volume for Microsoft exhibits a stable one, as Microsoft has maintained its status as one of the largest companies by trading volume by offering a diverse range of products from cloud computing (Azure) to Office 365.
These stocks show that technology companies that have a trading volume graph with low or high fluctuation do not necessarily reflect a strong stock. However, it rather shows the reason why a technology company’s stock is strong or not, mainly gauged through investor interest.
Tech Stocks Post-COVID: A Case Study in Correlation
Correlation between different stocks in a specific sector can be one of the key indicators in finding what is driving stock prices. During the COVID-19 pandemic (2020–2021), technology stocks exhibited unprecedented levels of correlation. Companies across the sector saw their stock prices move almost in lockstep, driven by shared catalysts such as the global shift to remote work, accelerated digital transformation, and increased reliance on cloud services and e-commerce platforms.
For instance, stocks like Alphabet, Microsoft, and Amazon experienced simultaneous price surges as demand for their services skyrocketed. This high correlation reflected a collective market perception that all major players in the tech sector would benefit from the same macroeconomic shifts.
However, as the pandemic’s impact on the economy began to wane, correlation among tech stocks decreased significantly in 2023–2024. Divergences became more pronounced as companies faced unique challenges and opportunities.
The declining correlation among tech stocks in recent years underscores the importance of granular analysis when evaluating investments in the sector. While external factors once drove uniform movement, the current landscape rewards a focus on individual company performance and market positioning.
By understanding these trends, investors can better navigate the nuances of the tech sector, identifying opportunities based on specific company strengths rather than broader sector movements.
Historical Event Impact: Market Reactions to Tariff Announcements
Another notable example of external events shaping the market is the impact of trade policy announcements by recent presidential administrations, namely Trump and Biden. This is especially significant now that Trump has been re-elected, as his previous tariff decisions under his first presidency had significant impacts on markets and sectors dependent on global trade.
From 2018 to 2021, the Trump administration’s tariff announcements, especially during the U.S.-China trade war significantly influenced the financial markets. These policy shifts introduced uncertainty, leading to sharp fluctuations in stock prices and investor sentiment. The chart above visualizes these stock price movements for Apple (AAPL), Target (TGT), Amazon (AMZN), and Tesla (TSLA).
August 2018: Trump’s steel and aluminum tariffs, along with tariffs on Chinese imports, led to an initial dip in stock prices for companies like Apple, Target, Amazon, and Tesla. The market reacted negatively to fears of increased production costs.
February/March 2020: Trump’s reduction of Chinese tariffs sparked a recovery, as markets responded positively to the easing of trade tensions.
January 2021: Just before Biden’s presidency, tariffs peaked at $360 billion in Chinese imports, worsening supply chain disruptions and raising costs, leading to a market downturn.
May 2023: Biden continued aluminum tariffs, signaling ongoing trade pressures and fueling concerns about inflation and supply chain stability.
It is evident that tariffs can significantly impact industries that rely on imported goods and raw materials, such as manufacturing and technology. These sectors often depend on key international suppliers, such as China, to maintain competitive pricing. When tariffs are imposed, the cost of these imports increases, which can either lead to reduced profit margins if companies absorb the costs or higher prices for consumers if companies pass those costs along.
Conclusion
Analyzing the S&P 500 through various dimensions reveals how data can simplify complex market behaviors and provide meaningful insights. The dominance of sectors like Technology and Consumer Discretionary underscores their outsized influence on market trends, while smaller sectors such as Utilities offer unique risk and reward profiles. The evolution of market dynamics, such as the shift from highly correlated tech stock movements during the COVID-19 pandemic to more individualized performance in recent years, highlights the need to evaluate companies based on their distinct strengths and challenges.
Historical events further illustrate the impact of external factors on market performance. The COVID-19 pandemic accelerated digital transformation, creating shared catalysts for tech companies, while tariff announcements disrupted trade-dependent sectors, reflecting the profound influence of macroeconomic policies. These examples emphasize the importance of understanding both sector-specific trends and broader economic forces.
While the current analysis offers valuable perspectives, it has limitations. Exploring fundamental financial ratios like price-to-earnings ratios or incorporating predictive models could deepen insights and better forecast sector growth. Expanding the scope to include global indices might also enhance understanding of international market interconnections.
Ultimately, data analytics provides a powerful framework to uncover hidden patterns and inform investment strategies. By blending historical context with innovative methodologies, investors can navigate the complexities of financial markets with greater clarity and confidence.
