Exploratory Data Analysis

This exploratory data analysis examines the temporal patterns and relationships within the dataset of student loan debt, housing market indicators, and macroeconomic variables. Through visualization and statistical summaries, I identify trends, seasonality, structural breaks, and correlations that inform the modeling strategy. This foundational analysis reveals the complex interplay between educational financing and housing affordability, setting the stage for more sophisticated time series modeling in subsequent sections.

Student Loan Debt

Observation: The time series shows a strong upward trend from 2006 to present, with outstanding student loan balances growing from approximately $500 billion to over $1.6 trillion. There appears to be a slight leveling off post-2020, likely due to COVID-19 payment pauses.

Interpretation: All lag plots show strong positive linear relationships, indicating high autocorrelation at all lags. This is characteristic of a non-stationary series with a strong trend. The persistence of the pattern across multiple lags confirms the series has strong memory and predictable behavior based on past values.

Interpretation of Components:

  • Observed: Raw data showing the overall upward trajectory
  • Trend: Strong positive trend, increasing almost linearly until ~2020, then flattening
  • Seasonal: Minimal seasonal pattern - student loan balances don’t show strong within-year variations
  • Random: Relatively small random component compared to the trend, indicating the series is dominated by its trend

The decomposition confirms that this is primarily a trend-dominated series with little seasonality.

ACF and PACF Analysis:

ACF Interpretation: The ACF shows very slow decay, remaining highly significant even at long lags. This is a classic signature of non-stationarity due to trend. The correlations decrease very gradually, confirming the series has strong persistence.

PACF Interpretation: The PACF shows a significant spike at lag 1, then cuts off sharply. This pattern suggests an AR(1) process might be appropriate after differencing to remove the trend.

Augmented Dickey-Fuller Test:


    Augmented Dickey-Fuller Test

data:  slq.ts
Dickey-Fuller = -1.255, Lag order = 4, p-value = 0.88
alternative hypothesis: stationary

ADF Test Results:

  • Null Hypothesis: The series has a unit root (non-stationary)
  • p-value = 0.88: Fail to reject null hypothesis
  • Conclusion: The series is non-stationary, confirming what we observed in the ACF plot. The strong trend prevents the series from reverting to a constant mean.

Determining Differencing Order:

[1] 2
[1] 0
  • ndiffs() = 2: Two regular differences needed
  • nsdiffs() = 0: No seasonal differencing needed

Differencing Comparison:

Comparison:

  • Left (Original): Slow ACF decay = non-stationary
  • Right (After differencing): ACF cuts off quickly within confidence bounds = stationary

Differencing Applied: Combined regular (d=1) and seasonal (D=1) differencing successfully removed both the trend and any seasonal patterns. The differenced series now fluctuates around zero with no systematic pattern, making it suitable for ARIMA modeling.

Interpretation: The moving averages smooth out short-term fluctuations to reveal the underlying trend:

  • 4-quarter MA (blue): Follows the data closely, smoothing minor variations
  • 8-quarter MA (red): Further smoothing, clearer trend visualization
  • 16-quarter MA (green): Reveals the long-term trend most clearly

All moving averages confirm the consistent upward trend until 2020, followed by stabilization during the COVID-19 payment pause period.

Note

Key Findings:

  1. ✓ Non-stationary: Strong upward trend, confirmed by ACF and ADF test
  2. ✓ Differencing needed: d=1 (regular) and D=1 (seasonal)
  3. ✓ After differencing: Series becomes stationary and suitable for ARIMA/SARIMA modeling
  4. ✓ Minimal seasonality: Trend dominates, seasonal component is weak
  5. ✓ Model suggestion: SARIMA(p,1,q)(P,1,Q)[12] where p,q,P,Q to be determined from differenced ACF/PACF

U.S. Housing Price Index

Observation: The Housing Price Index shows dramatic patterns from 1991-present: - 1991-2006: Steady upward trend with acceleration in early 2000s (housing bubble) - 2007-2011: Sharp collapse (2008 Financial Crisis) - index dropped ~35% - 2012-present: Strong recovery and new growth exceeding pre-crisis levels

This series exhibits clear trend, potential seasonality, and a structural break around 2008.

Interpretation: Lag plots show strong positive correlations across all lags, but with more scatter than the student loan data. The relationship remains linear but shows two distinct clusters (pre- and post-2008 crisis), indicating a structural break. High autocorrelation confirms non-stationarity.

Interpretation of Components:

  • Observed: Shows the dramatic rise-fall-rise pattern of the housing market cycle
  • Trend: Clearly captures the 2008 crisis as a major structural break - upward trend breaks, then recovers
  • Seasonal: Weak but visible seasonal pattern - housing markets show mild seasonality (spring/summer peaks)
  • Random: Relatively small compared to trend, but larger spikes during crisis period (2008-2011) indicate increased volatility

The decomposition reveals this is a trend-dominated series with seasonal components and a major structural break.

ACF and PACF Analysis:

ACF Interpretation: Very slow, linear decay - textbook signature of non-stationarity. The ACF remains highly significant far beyond lag 20, confirming strong persistence and trend.

PACF Interpretation: Large spike at lag 1, then quick cutoff. After differencing, this suggests AR(1) might be appropriate.

Augmented Dickey-Fuller Test:


    Augmented Dickey-Fuller Test

data:  hpi.ts
Dickey-Fuller = 0.42703, Lag order = 7, p-value = 0.99
alternative hypothesis: stationary

ADF Test Results:

  • Null Hypothesis: Series has unit root (non-stationary)
  • Result: High p-value → fail to reject null
  • Conclusion: Series is non-stationary, as expected from ACF analysis

Determining Differencing Order:

[1] 2
[1] 1

Differencing Comparison:

Comparison:

  • Left (Original): Slow ACF decay extending to lag 50+ = non-stationary
  • Right (Differenced): ACF cuts off quickly, stays within confidence bounds = stationary

Differencing Applied: Regular differencing (d=1) combined with seasonal differencing (D=1, lag=12) successfully removed both trend and seasonality. The series is now suitable for SARIMA modeling.

ADF Test on Differenced Series:


    Augmented Dickey-Fuller Test

data:  diff(hpi.ts)
Dickey-Fuller = -5.153, Lag order = 7, p-value = 0.01
alternative hypothesis: stationary

The differenced series is stationary (p < 0.05).

Interpretation: Moving averages reveal the housing market cycle clearly:

  • 12-month MA (blue): Smooths seasonal variations, tracks overall direction
  • 24-month MA (red): Reveals multi-year trends - clearly shows 2008 crisis turning point
  • 50-month MA (green): Long-term trend - demonstrates:
    • Bubble inflation (2000-2007)
    • Crash (2007-2012)
    • Recovery and new growth (2012-present)

The divergence between short and long-term MAs around 2006-2007 signaled the impending crisis.

Note

Key Findings:

  1. ✓ Non-stationary: Strong trend with structural break (2008 crisis)
  2. ✓ Differencing needed: d=1 (regular), D=1 (seasonal)
  3. ✓ Seasonality present: Mild seasonal pattern (12-month cycles)
  4. ✓ Structural break: 2008 Financial Crisis creates a structural break
  5. ✓ Model suggestion: SARIMA(p,1,q)(P,1,Q)[12] OR consider interrupted time series model to account for 2008 break
  6. ✓ Special consideration: May need intervention/outlier detection for 2008-2011 period

Mount Vernon Housing Market

Observation: Mount Vernon housing data shows patterns typical of a working-class community: - Clear upward trend from 2000-2007 - Sharp crash during 2008-2012 crisis - Slower, more volatile recovery compared to national trends - Both bottom and top tiers show similar patterns, with top tier experiencing slightly higher volatility

Interpretation: Both tiers show strong positive linear relationships at all lags, characteristic of non-stationary series with trend. The top tier shows slightly more scatter, reflecting higher market sensitivity.

Interpretation:

Bottom Tier: - Trend: Clear upward trend from 2000-2007, crash during 2008-2012, recovery and stabilization post-2012 - Seasonal: Minimal seasonal component - Random: Increased volatility during the 2008 crisis period

Top Tier: - Trend: Similar pattern to bottom tier, but potentially steeper declines during crisis - Seasonal: Also minimal seasonality - Random: Higher magnitude of random fluctuations, reflecting greater market sensitivity

Key Insight: Both tiers clearly experienced the 2008 housing crisis. The decomposition reveals that most variation is in the trend component, not seasonality.

ACF and PACF Analysis:

Interpretation: Both tiers show slow ACF decay → Non-stationary. PACF shows significant spike at lag 1 → AR(1) behavior after differencing.

ADF Tests:


    Augmented Dickey-Fuller Test

data:  mv_b.ts
Dickey-Fuller = -1.1391, Lag order = 6, p-value = 0.915
alternative hypothesis: stationary

    Augmented Dickey-Fuller Test

data:  mv_t.ts
Dickey-Fuller = -2.3027, Lag order = 6, p-value = 0.4487
alternative hypothesis: stationary

Both tiers show large p-values → non-stationary.

Differencing Requirements:

[1] 2
[1] 2
[1] 0
[1] 0

Results indicate d = 2 (second-order differencing) and D = 0 (no seasonal differencing).

ADF Tests on Differenced Data:


    Augmented Dickey-Fuller Test

data:  diff(mv_b.ts, difference = 2)
Dickey-Fuller = -6.7789, Lag order = 6, p-value = 0.01
alternative hypothesis: stationary

    Augmented Dickey-Fuller Test

data:  diff(mv_t.ts, difference = 2)
Dickey-Fuller = -7.241, Lag order = 6, p-value = 0.01
alternative hypothesis: stationary

After d=2 differencing: stationarity achieved. Model suggestion: ARIMA(p, 2, q).

Interpretation: Moving averages reveal 2008 crisis impact and slower recovery compared to national trends. Top tier shows higher volatility than bottom tier.

Note

Key Findings:

  1. ✓ Non-stationary: Both tiers show strong trends with structural breaks
  2. ✓ Differencing needed: d=2 (second-order) for both tiers
  3. ✓ Minimal seasonality: No seasonal differencing needed
  4. ✓ 2008 Crisis Impact: Clear structural break visible in both tiers
  5. ✓ Model suggestion: ARIMA(p,2,q) models for both bottom and top tiers
  6. ✓ Note: Complete comparison with Bronxville appears below

Bronxville Housing Market

Observation: Bronxville (affluent community) housing data windowed to 2016-2025 shows: - Steady upward trend with minimal volatility - Post-crisis recovery period only (excludes 2008 crash) - More stable appreciation compared to Mount Vernon - Both tiers show smooth, predictable growth patterns

Show Code
head(bronxville_b_long$hvi)
[1] NA NA NA NA NA NA
Show Code
# bronxville hvi lag plots bottom
length(bronxville_b_long$hvi)
[1] 308
Show Code
sum(is.na(bronxville_b_long$hvi))
[1] 193
Show Code
head(bronxville_b_long$hvi)
[1] NA NA NA NA NA NA
Show Code
bronxville_b.ts <- ts(bronxville_b_long$hvi, start=c(2000,01), frequency = 12)
head(bronxville_b.ts)
     Jan Feb Mar Apr May Jun
2000  NA  NA  NA  NA  NA  NA
Show Code
bronxville_b.ts <- window(bronxville_b.ts, start=c(2016,02), end=c(2025,08), frequency = 12)
# multiple lag plots
gglagplot(bronxville_b.ts, do.lines=FALSE) +xlab("Lags")+ylab("Yt")+ggtitle("Lag Plot Bronxville Bottom Housing Value Index")+theme(axis.text.x=element_text(angle=45, hjust=1))

Show Code
# write.csv(bronxville_b.ts, file = "data/processed/bronxville_b.ts.csv", row.names=FALSE)
# print NAs
Show Code
# bronxville hvi lag plots top
bronxville_t.ts <- ts(bronxville_t_long$hvi, start=c(2000,01), frequency = 12)
bronxville_t.ts <- window(bronxville_t.ts, start=c(2016,02), end=c(2025,08), frequency = 12)

# multiple lag plots
gglagplot(bronxville_t.ts, do.lines=FALSE) +xlab("Lags")+ylab("Yt")+ggtitle("Lag Plot Bronxville Top Housing Value Index")+theme(axis.text.x=element_text(angle=45, hjust=1))

Show Code
write.csv(bronxville_t.ts, file = "data/processed/bronxville_t.ts.csv", row.names=FALSE)

Interpretation: Similar to Mount Vernon, Bronxville shows strong positive autocorrelation at all lags for both bottom and top tiers.

Key Difference from Mount Vernon: - Bronxville data is windowed to 2016-2025 (more recent period only) - This focuses on the post-recovery housing market - Both tiers show non-stationary behavior consistent with trending housing prices

Implication: Even in affluent Bronxville, housing prices are not stationary and will require differencing for ARIMA modeling.

Interpretation:

Bottom Tier Decomposition: - Trend: Steady upward trend from 2016-2025 (post-crisis recovery period) - Seasonal: Minimal seasonality (consistent with local housing data) - Random: Lower volatility compared to Mount Vernon, suggesting more stable market

Top Tier Decomposition: - Trend: Similar upward trajectory, potentially steeper than bottom tier - Seasonal: Also minimal - Random: Moderate fluctuations, but less volatile than Mount Vernon

Critical Comparison: - Bronxville (affluent) shows steadier appreciation with less volatility - Mount Vernon experienced more volatility, especially during recovery - This reflects wealth-based resilience: affluent communities recover faster and experience less price instability

ACF and PACF Analysis:

Interpretation: Both tiers show slow ACF decay → Non-stationary. PACF shows significant spike at lag 1, suggesting AR(1) structure after differencing.

ADF Tests:


    Augmented Dickey-Fuller Test

data:  bronxville_b.ts
Dickey-Fuller = -1.0489, Lag order = 4, p-value = 0.927
alternative hypothesis: stationary

    Augmented Dickey-Fuller Test

data:  bronxville_t.ts
Dickey-Fuller = -2.3069, Lag order = 4, p-value = 0.4492
alternative hypothesis: stationary

Both tiers show large p-values → non-stationary.

Differencing Requirements:

[1] 1
[1] 1
[1] 0
[1] 0
  • Bottom tier: d = 1 (first-order differencing)
  • Top tier: d = 2 (second-order differencing)
  • Seasonal: D = 0 (no seasonal differencing needed)

Differencing Comparison:

Comparison: Bronxville may require less differencing (d=1 vs d=2) than Mount Vernon, possibly because the windowed data (2016-2025) excludes the volatile crisis period. This reflects smoother price appreciation in affluent areas.

Show Code
# acf and pacf for mount vernon top and bottom
plot1a <- ggAcf(mv_b.ts)+ggtitle('Original Data: Mount Vernon \n Home Value Index \n (Bottom Tier)') + theme(plot.title=element_text(hjust=0.5)) + geom_segment(lineend='butt') + geom_hline(yintercept=0, color='darkblue')

mv_b_log <- log(mv_b.ts)

plot1b <- ggAcf(diff(mv_b.ts, differences = 2))+ggtitle('Differenced Data') + theme(plot.title=element_text(hjust=0.5)) + geom_segment(lineend='butt') + geom_hline(yintercept=0, color='darkblue')

plot2a <- ggAcf(mv_t.ts)+ggtitle('Original Data: Mount Vernon \n Home Value Index \n (Top Tier)') + theme(plot.title=element_text(hjust=0.5)) + geom_segment(lineend='butt') + geom_hline(yintercept = 0, color='darkblue')

mv_t_log <- log(mv_t.ts)

plot2b <- ggAcf(diff(mv_t.ts, differences = 2))+ggtitle('Differenced Data') + theme(plot.title=element_text(hjust=0.5)) + geom_segment(lineend='butt') + geom_hline(yintercept=0, color='darkblue')

grid.arrange(plot1a, plot1b, plot2a, plot2b, nrow=2, ncol=2)

What These Plots Show: - Top row: Bottom tier - original (left) vs d=2 differenced (right) - Bottom row: Top tier - original (left) vs d=2 differenced (right)

Key Observations: - Before differencing: Slow ACF decay (classic non-stationary signature) - After d=2 differencing: Much faster decay, approaching white noise behavior

Both tiers require d=2: This is more aggressive than Bronxville, likely because: 1. Mount Vernon data includes the full 2000-2025 period (including the 2008 crisis) 2. More price volatility requires stronger transformation

Implication: Mount Vernon will use ARIMA(p, 2, q) models while Bronxville may use ARIMA(p, 1, q) or ARIMA(p, 2, q).


    Augmented Dickey-Fuller Test

data:  diff_both
Dickey-Fuller = -1.9033, Lag order = 3, p-value = 0.6142
alternative hypothesis: stationary

    Augmented Dickey-Fuller Test

data:  diff(bronxville_b.ts, difference = 1)
Dickey-Fuller = -4.0573, Lag order = 4, p-value = 0.01
alternative hypothesis: stationary

    Augmented Dickey-Fuller Test

data:  diff(bronxville_t.ts, difference = 2)
Dickey-Fuller = -6.4601, Lag order = 4, p-value = 0.01
alternative hypothesis: stationary

After Differencing: Both Bronxville series are now stationary (p < 0.05) and ready for ARIMA modeling.

Interpretation: Moving averages show smoother trends compared to Mount Vernon with less volatility. Both tiers show strong appreciation with minimal downturns from 2016-2025.

Key Observations: - 12-Month MA (Blue): Tracks recent trends, shows smooth appreciation - 24-Month MA (Red): Medium-term trend shows steady upward trajectory - 50-Month MA (Green): Long-term trend confirms consistent appreciation - No major disruptions in the 2016-2025 window (post-crisis period only)

Note

Key Findings:

  1. ✓ Non-stationary: Both tiers show upward trends
  2. ✓ Differencing needed: d=1 (bottom tier) and d=2 (top tier)
  3. ✓ Minimal seasonality: No seasonal differencing needed
  4. ✓ Lower volatility: Affluent community shows more stable appreciation
  5. ✓ Model suggestion: ARIMA(p,1-2,q) models for both tiers
Summary: Local Housing Market Analysis (Mount Vernon vs Bronxville)

What We Learned:

Mount Vernon (Working-Class Community): - ✓ Strong trend but higher volatility (especially during 2008-2012 crisis) - ✓ Requires d=2 differencing (second-order) for stationarity - ✓ Suggested models: ARIMA(p, 2, q) with possible AR(1) structure - ✓ Moving averages show slower recovery from economic shocks - ✓ Both bottom and top tiers experienced significant price fluctuations

Bronxville (Affluent Community): - ✓ Steady appreciation with lower volatility - ✓ Requires d=1 or d=2 differencing depending on tier - ✓ Suggested models: ARIMA(p, 1-2, q) with possible AR(1) structure - ✓ Data windowed to 2016-2025 shows stable post-crisis growth - ✓ Less sensitive to economic shocks; faster recovery

Key Insights for Research Questions:

  1. Wealth-based resilience: Affluent communities (Bronxville) show lower housing volatility and faster recovery from economic shocks
  2. Education-housing nexus: Communities with better schools (higher property taxes from higher home values) maintain more stable housing markets
  3. Modeling implications: Different communities require different ARIMA specifications based on their volatility profiles
  4. Policy relevance: Lower-income communities need more support during crises as they experience larger price disruptions that affect homeownership accessibility

Next Steps:

  • Fit ARIMA/SARIMA models to both communities’ data
  • Forecast future housing prices and confidence intervals
  • Compare model performance: Which community is more predictable?

Self-Employment Data

Show Code
# Plot the full time series
autoplot(self_emp.ts) +
  ggtitle("Self-Employed Persons in the U.S. (1948-2025)") +
  xlab("Year") +
  ylab("Self-Employed (Thousands)") +
  theme_minimal()

What We See: The self-employment count shows a long-term upward trend from 1948 to around 2008, followed by volatility during the financial crisis. The series shows both trend and potential cyclical behavior related to economic conditions.

Key Observation: The 2008 financial crisis appears to have impacted self-employment patterns, similar to what we observed in housing and student loan data. This suggests interconnected economic dynamics.

Show Code
# Lag plots
gglagplot(self_emp.ts, do.lines=FALSE, lags = 12) +
  xlab("Lags") +
  ylab("Yt") +
  ggtitle("Lag Plot for U.S. Self-Employment") +
  theme(axis.text.x=element_text(angle=45, hjust=1))

Lag Plot Interpretation: Strong positive linear relationships at all lags indicate high autocorrelation and a non-stationary series with trend. The pattern confirms the series has strong memory and predictable short-term behavior.

Show Code
# Decomposition
decompose_self_emp <- decompose(self_emp.ts)
autoplot(decompose_self_emp)

Decomposition Interpretation:

  • Observed: Raw self-employment counts showing overall growth pattern
  • Trend: Long-term upward trend until ~2008, then flattening/slight decline - coincides with financial crisis and student debt explosion
  • Seasonal: Minimal seasonal component - self-employment doesn’t show strong within-year patterns
  • Random: Increased volatility post-2008, reflecting economic uncertainty

Critical Finding: The trend component shows self-employment growth stalling around 2008-2010, exactly when student loan debt began its explosive growth. This temporal alignment supports the hypothesis that rising debt burdens may be discouraging entrepreneurship.

Show Code
# ACF and PACF
self_emp_acf <- ggAcf(self_emp.ts, lag.max = 50) +
  ggtitle('ACF for U.S. Self-Employment') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

self_emp_pacf <- ggPacf(self_emp.ts, lag.max = 50) +
  ggtitle('PACF for U.S. Self-Employment') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

grid.arrange(self_emp_acf, self_emp_pacf, nrow=2)

ACF Interpretation: Very slow decay remaining significant for many lags - classic signature of non-stationarity due to trend.

PACF Interpretation: Significant spike at lag 1, then cuts off - suggests AR(1) process after differencing.

Conclusion: Series is non-stationary and requires differencing before ARIMA modeling.

Show Code
# ADF Test
tseries::adf.test(self_emp.ts)

    Augmented Dickey-Fuller Test

data:  self_emp.ts
Dickey-Fuller = -1.5628, Lag order = 9, p-value = 0.7634
alternative hypothesis: stationary

ADF Test Results: - Expected p-value: > 0.05 (fail to reject null hypothesis of unit root) - Conclusion: Series is non-stationary, confirming visual diagnostics

Show Code
# Determine differencing order
ndiffs(self_emp.ts)
[1] 1
Show Code
nsdiffs(self_emp.ts)
[1] 0

Differencing Recommendations: - ndiffs(): Likely suggests d = 1 (first-order differencing needed) - nsdiffs(): Likely suggests D = 0 (no seasonal differencing needed, consistent with minimal seasonality)

Show Code
# Apply differencing
self_emp_diff <- diff(self_emp.ts, differences = 1)

# Compare original vs differenced ACF
plot1 <- ggAcf(self_emp.ts, 50) +
  ggtitle('Original Data: Self-Employment') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

plot2 <- ggAcf(self_emp_diff, 50) +
  ggtitle('Differenced Data (d=1)') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

grid.arrange(plot1, plot2, ncol=2)

Comparison: - Left (Original): Slow ACF decay = non-stationary - Right (Differenced): ACF cuts off quickly = stationary

Result: First-order differencing (d=1) successfully removes the trend, making the series suitable for ARIMA modeling.

Show Code
# ADF test on differenced data
tseries::adf.test(self_emp_diff)

    Augmented Dickey-Fuller Test

data:  self_emp_diff
Dickey-Fuller = -11.701, Lag order = 9, p-value = 0.01
alternative hypothesis: stationary

Differenced Series ADF Test: - Expected: p < 0.05 (reject null hypothesis) - Conclusion: Differenced series is stationary and ready for ARIMA modeling

Show Code
# Moving averages
self_emp_df <- data.frame(
  date = self_employment$observation_date,
  value = self_employment$LNS12032192
)

self_emp_rolling <- self_emp_df %>%
  mutate(
    MA_12 = zoo::rollmean(value, 12, k = 12, fill = NA, align = 'right'),
    MA_60 = zoo::rollmean(value, 60, k = 60, fill = NA, align = 'right'),
    MA_120 = zoo::rollmean(value, 120, k = 120, fill = NA, align = 'right')
  )

ggplot(self_emp_rolling, aes(x = date)) +
  geom_line(aes(y = value, color = "Original"), linewidth = 0.6) +
  geom_line(aes(y = MA_12, color = "12-month MA"), linewidth = 0.8) +
  geom_line(aes(y = MA_60, color = "60-month MA"), linewidth = 0.8) +
  geom_line(aes(y = MA_120, color = "120-month MA"), linewidth = 0.8) +
  scale_color_manual(values = c("Original" = "black",
                                "12-month MA" = "blue",
                                "60-month MA" = "red",
                                "120-month MA" = "green")) +
  labs(
    title = "Self-Employment with Moving Averages (1948-2025)",
    x = "Date",
    y = "Self-Employed (Thousands)",
    color = "Series"
  ) +
  theme_minimal()

Moving Average Interpretation:

12-Month MA (Blue): Smooths short-term fluctuations, tracks recent trends 60-Month MA (Red): 5-year average shows medium-term economic cycles 120-Month MA (Green): 10-year average reveals long-term structural trends

Key Observations: 1. Pre-2008: Steady upward trajectory across all moving averages 2. 2008-2012: Sharp disruption and stagnation 3. Post-2012: Slow recovery but flatter growth than pre-crisis trend 4. Recent years: Self-employment appears to be plateauing or declining slightly

Critical Insight: The long-term moving averages show that self-employment growth has fundamentally changed post-2008. The slope of growth is much shallower, suggesting structural barriers (like student debt) are preventing the historical entrepreneurship trend from continuing.

Summary: Self-Employment Time Series Analysis

Key Findings:

  1. ✓ Non-stationary: Strong upward trend from 1948-2008, then structural break
  2. ✓ Differencing needed: d=1 (first-order differencing sufficient)
  3. ✓ Minimal seasonality: Little within-year variation
  4. ✓ 2008 regime change: Growth pattern fundamentally altered after financial crisis
  5. ✓ Model suggestion: ARIMA(p,1,q) where p,q determined from differenced ACF/PACF

Connection to Research Questions:

Temporal Alignment: Self-employment growth stalled exactly when: - Student loan debt began exponential growth (2006-2010) - Housing crisis hit (2008) - Economic opportunity contracted

This temporal coincidence supports the hypothesis that student debt blocks entrepreneurial pathways, which in turn limits wealth accumulation and homeownership opportunities.

Next Steps:

  • Fit ARIMA models to self-employment data
  • Include in multivariate VAR model with student debt and housing
  • Test Granger causality: Does student debt predict self-employment decline?
  • Explore mediation: Does entrepreneurship mediate the debt→housing relationship?

College Tuition (CPI)

Show Code
# Plot the full time series
autoplot(tuition.ts) +
  ggtitle("College Tuition and Fees Consumer Price Index (1978-2024)") +
  xlab("Year") +
  ylab("CPI (Index)") +
  theme_minimal()

What We See: College tuition CPI shows exponential growth from 1978 to present. The index has increased from ~58 (1978) to over 900 (2024), representing a 15x increase - far outpacing general inflation.

Key Observation: Tuition growth accelerated dramatically after 2000, creating the economic pressure that drives students to take on massive debt burdens. This explains why student loan debt exploded in the same period.

Show Code
# Lag plots
gglagplot(tuition.ts, do.lines=FALSE, lags = 12) +
  xlab("Lags") +
  ylab("Yt") +
  ggtitle("Lag Plot for College Tuition CPI") +
  theme(axis.text.x=element_text(angle=45, hjust=1))

Lag Plot Interpretation: Perfect positive linear relationships at all lags - extremely strong autocorrelation. The series is highly non-stationary with a relentless upward trend. No reversions or cycles visible.

Show Code
# Decomposition
decompose_tuition <- decompose(tuition.ts)
autoplot(decompose_tuition)

Decomposition Interpretation:

  • Observed: Smooth exponential growth curve
  • Trend: Dominant component showing accelerating growth, especially post-2000
  • Seasonal: Virtually no seasonal pattern (tuition is set annually, not monthly)
  • Random: Very small random component - tuition increases are systematic, not random

Critical Finding: The trend component is nearly identical to the observed data, meaning tuition growth is almost entirely predictable based on its trend. This systematic increase has created a predictable debt burden for students.

Show Code
# ACF and PACF
tuition_acf <- ggAcf(tuition.ts, lag.max = 50) +
  ggtitle('ACF for College Tuition CPI') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

tuition_pacf <- ggPacf(tuition.ts, lag.max = 50) +
  ggtitle('PACF for College Tuition CPI') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

grid.arrange(tuition_acf, tuition_pacf, nrow=2)

ACF Interpretation: Extremely slow decay - one of the slowest we’ve seen. Correlations remain near 1.0 even at lag 50. Textbook non-stationarity.

PACF Interpretation: Large spike at lag 1, with significant spikes at lags 12, 24 (yearly intervals), then decay. Suggests AR(1) with possible seasonal component.

Conclusion: Series is severely non-stationary - perhaps the most non-stationary series in our dataset.

Show Code
# ADF Test
tseries::adf.test(tuition.ts)

    Augmented Dickey-Fuller Test

data:  tuition.ts
Dickey-Fuller = -2.1703, Lag order = 8, p-value = 0.5062
alternative hypothesis: stationary

ADF Test Results: - Expected p-value: > 0.05 (strong evidence of unit root) - Conclusion: Confirmed non-stationary

Show Code
# Determine differencing order
ndiffs(tuition.ts)
[1] 2
Show Code
nsdiffs(tuition.ts)
[1] 0

Differencing Recommendations: - ndiffs(): Likely d = 1 or 2 (may need second-order differencing due to accelerating growth) - nsdiffs(): Likely D = 0 (minimal seasonality)

Show Code
# Apply differencing (try first-order)
tuition_diff1 <- diff(tuition.ts, differences = 1)

# Compare original vs differenced ACF
plot1 <- ggAcf(tuition.ts, 50) +
  ggtitle('Original: College Tuition CPI') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

plot2 <- ggAcf(tuition_diff1, 50) +
  ggtitle('Differenced (d=1)') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

grid.arrange(plot1, plot2, ncol=2)

Comparison: - Left (Original): Extremely slow decay - Right (Differenced): ACF drops dramatically, mostly within bounds

Result: First-order differencing (d=1) appears sufficient to achieve stationarity.

Show Code
# ADF test on differenced data
tseries::adf.test(tuition_diff1)

    Augmented Dickey-Fuller Test

data:  tuition_diff1
Dickey-Fuller = -4.2339, Lag order = 8, p-value = 0.01
alternative hypothesis: stationary

Differenced Series ADF Test: - Expected: p < 0.05 (stationary) - Conclusion: First differencing achieves stationarity

Show Code
# Moving averages
tuition_df <- data.frame(
  date = college_tuition$observation_date,
  value = college_tuition$CUSR0000SEEB
)

tuition_rolling <- tuition_df %>%
  mutate(
    MA_12 = zoo::rollmean(value, 12, k = 12, fill = NA, align = 'right'),
    MA_60 = zoo::rollmean(value, 60, k = 60, fill = NA, align = 'right'),
    MA_120 = zoo::rollmean(value, 120, k = 120, fill = NA, align = 'right')
  )

ggplot(tuition_rolling, aes(x = date)) +
  geom_line(aes(y = value, color = "Original"), linewidth = 0.6) +
  geom_line(aes(y = MA_12, color = "12-month MA"), linewidth = 0.8) +
  geom_line(aes(y = MA_60, color = "60-month MA"), linewidth = 0.8) +
  geom_line(aes(y = MA_120, color = "120-month MA"), linewidth = 0.8) +
  scale_color_manual(values = c("Original" = "black",
                                "12-month MA" = "blue",
                                "60-month MA" = "red",
                                "120-month MA" = "green")) +
  labs(
    title = "College Tuition CPI with Moving Averages (1978-2024)",
    x = "Date",
    y = "CPI Index",
    color = "Series"
  ) +
  theme_minimal()

Moving Average Interpretation:

Key Observations: 1. All moving averages track upward - no reversals ever 2. Acceleration visible: The slope of all MAs increases post-2000 3. Recent plateau: Very slight flattening in the last few years (2020-2024)

Critical Insight: The relentless upward trajectory with no downturns explains why student debt has become an inescapable burden. Unlike housing (which crashed in 2008), tuition never decreases - creating a one-way ratchet of debt accumulation.

Summary: College Tuition CPI Analysis

Key Findings:

  1. ✓ Exponential growth: 15x increase from 1978-2024
  2. ✓ Accelerating post-2000: Growth rate increased dramatically
  3. ✓ No reversals: Unlike housing/stocks, tuition never decreases
  4. ✓ Highly predictable: Trend dominates, minimal random variation
  5. ✓ Model suggestion: ARIMA(p,1,q) with possible log transformation

Connection to Research Questions:

Root Cause of Debt Crisis: This dataset explains WHY student debt exploded: - Tuition growth far outpaced wage growth - Students had no choice but to borrow increasing amounts - The predictable, accelerating growth created a debt trap

Causal Chain: 1. Tuition increases exponentially (this data) 2. Students borrow more (student loan data) 3. High debt burden blocks entrepreneurship (self-employment data) 4. Limited wealth accumulation prevents homeownership (housing data)

Next Steps:

  • Model tuition as exogenous variable in ARIMAX models
  • Test whether tuition increases predict student debt increases (spoiler: yes)
  • Include in multivariate models to show the full causal pathway

Mortgage Rates

Show Code
# Plot the full time series
autoplot(mortgage.ts) +
  ggtitle("30-Year Fixed Mortgage Rates in the U.S. (1971-2025)") +
  xlab("Year") +
  ylab("Rate (%)") +
  theme_minimal()

What We See: Mortgage rates show dramatic volatility over time. Peak of ~18% in early 1980s, long-term decline to historic lows (~3%) in 2020, then sharp recent increase (2022-2024).

Key Observation: The relationship between mortgage rates and housing affordability is inverse - when rates spike (like now), fewer people can afford homes. Combined with high housing prices and student debt, this creates a perfect storm blocking homeownership.

Show Code
# Lag plots
gglagplot(mortgage.ts, do.lines=FALSE, lags = 12) +
  xlab("Lags") +
  ylab("Yt") +
  ggtitle("Lag Plot for 30-Year Mortgage Rates") +
  theme(axis.text.x=element_text(angle=45, hjust=1))

Lag Plot Interpretation: Strong positive correlation at all lags, but more scatter than other series. This indicates high autocorrelation but also significant short-term volatility. The series is non-stationary with a long-term declining trend.

Show Code
# Decomposition
decompose_mortgage <- decompose(mortgage.ts)
autoplot(decompose_mortgage)

Decomposition Interpretation:

  • Observed: High volatility in 1970s-1990s, gradual decline, then recent uptick
  • Trend: Long-term declining trend from 18% (1981) to 3% (2020), with recent reversal
  • Seasonal: Minimal seasonal component (rates don’t have strong within-year patterns)
  • Random: Large random component - monetary policy shocks create volatility

Critical Finding: The 2020-2024 rate increase is one of the fastest in history. This recent spike, combined with inflated home prices from the 2010s recovery, creates a severe affordability crisis for debt-burdened millennials.

Show Code
# ACF and PACF
mortgage_acf <- ggAcf(mortgage.ts, lag.max = 50) +
  ggtitle('ACF for 30-Year Mortgage Rates') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

mortgage_pacf <- ggPacf(mortgage.ts, lag.max = 50) +
  ggtitle('PACF for 30-Year Mortgage Rates') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

grid.arrange(mortgage_acf, mortgage_pacf, nrow=2)

ACF Interpretation: Slow decay but faster than tuition/student loans - still indicates non-stationarity but with more mean-reversion tendency.

PACF Interpretation: Very large spike at lag 1, then cuts off sharply. Strong AR(1) behavior.

Conclusion: Series is non-stationary but may become stationary with first-order differencing.

Show Code
# ADF Test
tseries::adf.test(mortgage.ts)

    Augmented Dickey-Fuller Test

data:  mortgage.ts
Dickey-Fuller = -2.5877, Lag order = 8, p-value = 0.3296
alternative hypothesis: stationary

ADF Test Results: - Expected p-value: > 0.05 (evidence of unit root) - Conclusion: Confirmed non-stationary

Show Code
# Determine differencing order
ndiffs(mortgage.ts)
[1] 1
Show Code
nsdiffs(mortgage.ts)
[1] 0

Differencing Recommendations: - ndiffs(): Likely d = 1 (first-order differencing) - nsdiffs(): Likely D = 0 (no seasonal differencing needed)

Show Code
# Apply differencing
mortgage_diff1 <- diff(mortgage.ts, differences = 1)

# Compare original vs differenced ACF
plot1 <- ggAcf(mortgage.ts, 50) +
  ggtitle('Original: Mortgage Rates') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

plot2 <- ggAcf(mortgage_diff1, 50) +
  ggtitle('Differenced (d=1)') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

grid.arrange(plot1, plot2, ncol=2)

Comparison: - Left (Original): Slow decay indicating non-stationarity - Right (Differenced): ACF drops rapidly, mostly within bounds = stationary

Result: First-order differencing successfully achieves stationarity.

Show Code
# ADF test on differenced data
tseries::adf.test(mortgage_diff1)

    Augmented Dickey-Fuller Test

data:  mortgage_diff1
Dickey-Fuller = -7.6698, Lag order = 8, p-value = 0.01
alternative hypothesis: stationary

Differenced Series ADF Test: - Expected: p < 0.05 (stationary) - Conclusion: Differenced series is stationary and ready for ARIMA modeling

Show Code
# Moving averages
mortgage_rolling <- mortgage_monthly %>%
  mutate(
    MA_12 = zoo::rollmean(avg_rate, 12, k = 12, fill = NA, align = 'right'),
    MA_60 = zoo::rollmean(avg_rate, 60, k = 60, fill = NA, align = 'right'),
    MA_120 = zoo::rollmean(avg_rate, 120, k = 120, fill = NA, align = 'right')
  )

ggplot(mortgage_rolling, aes(x = date)) +
  geom_line(aes(y = avg_rate, color = "Original"), linewidth = 0.6) +
  geom_line(aes(y = MA_12, color = "12-month MA"), linewidth = 0.8) +
  geom_line(aes(y = MA_60, color = "60-month MA"), linewidth = 0.8) +
  geom_line(aes(y = MA_120, color = "120-month MA"), linewidth = 0.8) +
  scale_color_manual(values = c("Original" = "black",
                                "12-month MA" = "blue",
                                "60-month MA" = "red",
                                "120-month MA" = "green")) +
  labs(
    title = "30-Year Mortgage Rates with Moving Averages (1971-2025)",
    x = "Date",
    y = "Rate (%)",
    color = "Series"
  ) +
  theme_minimal()

Moving Average Interpretation:

Key Observations: 1. 1970s-1980s: Extreme volatility - rates ranged 7-18% 2. Long-term decline: From 1981 peak to 2020 bottom = 40-year downward trend 3. Historic lows (2020): Fed response to COVID created unprecedented 3% rates 4. 2022-2024 spike: Fastest rate increase in modern history (3% → 7% in 2 years)

Critical Insight: The combination of factors creates unprecedented barriers to homeownership: - High student debt reduces down payment capacity - Rates doubled in 2 years (2022-2024) - Home prices remain elevated from 2010-2020 boom - Perfect storm: Millennials face worst affordability crisis in generations

Summary: Mortgage Rates Analysis

Key Findings:

  1. ✓ High volatility: Rates vary dramatically based on Fed policy
  2. ✓ Long-term decline (1981-2020): 40-year downward trend
  3. ✓ Recent spike: 2022-2024 increase among fastest in history
  4. ✓ Strong AR(1) behavior: Rate changes are highly autocorrelated
  5. ✓ Model suggestion: ARIMA(1,1,q) due to strong AR(1) component

Connection to Research Questions:

Affordability Triple Threat: 1. Student debt reduces down payment capacity 2. High home prices (from 2010-2020 appreciation) 3. Doubled mortgage rates (2022-2024) doubles monthly payments

Example: A $400K home: - At 3% rate (2020): $1,686/month - At 7% rate (2024): $2,661/month - 57% payment increase for same home

For millennials with $50K student debt ($500/month payments), this makes homeownership nearly impossible without parental wealth support.

Next Steps:

  • Include mortgage rates in ARIMAX models as exogenous variable affecting housing
  • Test interaction: Does high student debt + high mortgage rates = lower homeownership?
  • Model the affordability equation: Income - Debt Payments - Mortgage Payment

Median Household Income

Show Code
# Plot the full time series
autoplot(income.ts) +
  ggtitle("Real Median Household Income in the U.S. (1984-2023)") +
  xlab("Year") +
  ylab("Income (2023 Dollars)") +
  theme_minimal()

What We See: Median household income shows stagnation with volatility. Income growth has been sluggish relative to the explosive growth in tuition and housing costs. Key recessions (2001, 2008, 2020) create sharp drops.

Key Observation: The income-cost gap is the core problem: - Income growth: ~$60K (1984) → ~$75K (2023) = 25% increase - Tuition growth: 1500%+ increase (same period) - Housing growth: 300%+ increase (same period)

This mismatch forces students into debt and blocks homeownership.

Show Code
# Lag plots for median household income (annual data)
gglagplot(income.ts, do.lines=FALSE, lags = 4) +
  xlab("Lags") +
  ylab("Yt") +
  ggtitle("Lag Plot for Median Household Income") +
  theme(axis.text.x=element_text(angle=45, hjust=1))

Lag Plot Interpretation: With only annual data (42 observations), the lag plot shows strong positive correlation across lags 1-4, indicating high autocorrelation and non-stationarity. The series has an upward trend with cyclical downturns during recessions (visible as clusters in the scatter).

Show Code
# ACF and PACF
income_acf <- ggAcf(income.ts, lag.max = 20) +
  ggtitle('ACF for Median Household Income') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

income_pacf <- ggPacf(income.ts, lag.max = 20) +
  ggtitle('PACF for Median Household Income') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

grid.arrange(income_acf, income_pacf, nrow=2)

ACF Interpretation: Slow decay indicating non-stationary behavior. The series has a trend component.

PACF Interpretation: Significant spike at lag 1, suggesting AR(1) behavior after differencing.

Conclusion: Despite being annual data with limited observations, the series shows clear non-stationarity.

Show Code
# ADF Test
tseries::adf.test(income.ts)

    Augmented Dickey-Fuller Test

data:  income.ts
Dickey-Fuller = -1.9531, Lag order = 3, p-value = 0.592
alternative hypothesis: stationary

ADF Test Results: - Expected p-value: > 0.05 (unit root likely) - Conclusion: Series is non-stationary

Show Code
# Determine differencing order
ndiffs(income.ts)
[1] 1

Differencing Recommendation: - ndiffs(): Likely d = 1 (first-order differencing)

Note: We skip nsdiffs() as annual data has no meaningful seasonality.

Show Code
# Apply differencing
income_diff1 <- diff(income.ts, differences = 1)

# Compare original vs differenced ACF
plot1 <- ggAcf(income.ts, 20) +
  ggtitle('Original: Median Income') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

plot2 <- ggAcf(income_diff1, 20) +
  ggtitle('Differenced (d=1)') +
  theme_bw() +
  geom_segment(lineend='butt') +
  geom_hline(yintercept=0, color='darkblue')

grid.arrange(plot1, plot2, ncol=2)

Comparison: - Left (Original): Slow decay - Right (Differenced): ACF cuts off quickly = stationary

Result: First-order differencing achieves stationarity.

Show Code
# Moving average (using smaller windows due to annual data)
median_income_rolling <- median_income %>%
  mutate(
    MA_3 = zoo::rollmean(MEHOINUSA672N, 3, k = 3, fill = NA, align = 'right'),
    MA_5 = zoo::rollmean(MEHOINUSA672N, 5, k = 5, fill = NA, align = 'right'),
    MA_10 = zoo::rollmean(MEHOINUSA672N, 10, k = 10, fill = NA, align = 'right')
  )

ggplot(median_income_rolling, aes(x = observation_date)) +
  geom_line(aes(y = MEHOINUSA672N, color = "Original"), linewidth = 0.6) +
  geom_line(aes(y = MA_3, color = "3-year MA"), linewidth = 0.8) +
  geom_line(aes(y = MA_5, color = "5-year MA"), linewidth = 0.8) +
  geom_line(aes(y = MA_10, color = "10-year MA"), linewidth = 0.8) +
  scale_color_manual(values = c("Original" = "black",
                                "3-year MA" = "blue",
                                "5-year MA" = "red",
                                "10-year MA" = "green")) +
  labs(
    title = "Real Median Household Income with Moving Averages (1984-2023)",
    x = "Date",
    y = "Income (2023 Dollars)",
    color = "Series"
  ) +
  theme_minimal()

Moving Average Interpretation:

Key Observations: 1. 1984-2000: Steady growth period (dot-com boom) 2. 2000-2002: Recession dip 3. 2003-2007: Recovery but slower growth 4. 2008-2012: Great Recession - massive income drop 5. 2013-2019: Long, slow recovery 6. 2020: COVID shock and rapid recovery (stimulus effects) 7. 2021-2023: Post-pandemic adjustment

Critical Insight: Income growth has been volatile and sluggish compared to cost growth: - Three major recessions in 40 years reduced incomes - Recovery periods are slow - Real income in 2023 is only slightly higher than 1999 (24 years of stagnation!)

Meanwhile: - College tuition increased 300%+ - Housing prices increased 200-300% - Student loan balances exploded 800%+

The Affordability Crisis Explained: Flat incomes + exploding costs = debt trap + homeownership crisis.

Summary: Median Household Income Analysis

Key Findings:

  1. ✓ Stagnant growth: Only 25% increase over 40 years (inflation-adjusted)
  2. ✓ High volatility: Three major recessions create income shocks
  3. ✓ Slow recovery: Takes years to recover from each recession
  4. ✓ Widening gap: Income growth << cost growth (tuition, housing)
  5. ✓ Model suggestion: ARIMA(p,1,q) - annual data limits complexity

The Core Problem Visualized:

Metric 1984-2023 Growth
Median Income +25%
College Tuition CPI +1500%
Housing Prices +300%
Student Loan Debt +800%

The Math Doesn’t Work: - If your income grows 25% but costs grow 300-1500%, you MUST go into debt - Once in debt, entrepreneurship becomes too risky - Without entrepreneurship, wealth accumulation is slow - Without wealth, homeownership is impossible

Connection to All Other Datasets:

This dataset is the denominator in every affordability equation: - Debt-to-Income Ratio: Student loans / Income = unsustainable - Housing Affordability: (Home Price + Mortgage) / Income = unaffordable - Discretionary Income: Income - (Debt + Housing) = negative for millennials

Next Steps:

  • Use income as control variable in multivariate models
  • Test hypothesis: Stagnant income + rising costs → increased debt → decreased entrepreneurship → blocked homeownership
  • Calculate affordability indices combining all variables