Outliers in regression analysis can significantly impact results. We'll explore methods to identify them using standard deviations and residuals, examining how they affect regression lines and correlation coefficients.

Removing outliers isn't always straightforward. We'll discuss the importance of carefully evaluating their impact and considering distribution characteristics before deciding whether to exclude them from the analysis.

Outliers in Regression Analysis

Outliers using standard deviations rule

• Data points significantly different from other observations in a dataset
• Calculate mean and standard deviation of dataset
• Points more than two standard deviations from mean considered potential outliers
  • Assumes data follows normal distribution (95% within two standard deviations)
• Further investigate outliers to determine validity and impact on analysis
  • Extreme values may be errors (measurement, data entry) or genuine unusual cases
  • Decide to include or exclude based on context and objectives of analysis

Standard deviation of residuals

• Residuals are differences between observed and predicted values from regression model
• Calculate residuals by subtracting predicted from observed values
• Compute standard deviation of residuals using formula: $\sqrt{\frac{\sum(residuals - mean(residuals))^2}{n - 2}}$
• Points with residuals more than two standard deviations from mean considered potential outliers
  • May significantly impact regression line and model fit
• Identify outliers in context of specific regression model
  • Unusual observations relative to predicted relationship between variables
• Studentized residuals can be used to standardize the residuals, making it easier to identify outliers

Impact of outlier removal

• Outliers can substantially influence regression line and correlation coefficient
• Evaluate impact by comparing results before and after removing outliers
  1. Fit regression model with all data points
  2. Calculate regression line and correlation coefficient
  3. Remove identified outliers from dataset
  4. Refit regression model without outliers
  5. Recalculate regression line and correlation coefficient
• Significant changes indicate strong influence of outliers on model
• Minimal changes suggest outliers may not substantially impact overall analysis
• Carefully justify decisions to remove outliers
  • May represent valid data points providing valuable insights
  • Improper removal can lead to biased results and incorrect conclusions
  • Consider context, objectives, and potential consequences of exclusion

Distribution characteristics and outlier treatment

• Skewness measures the asymmetry of a distribution, which can affect the identification of outliers
• Kurtosis indicates the heaviness of the tails in a distribution, influencing the presence of extreme values
• Jackknife resampling can be used to assess the influence of individual observations on statistical estimates
• Winsorization is a technique to limit extreme values in statistical data without removing them entirely