Log-linear models are powerful tools for analyzing multi-way contingency tables in biostatistics. They help uncover complex relationships between categorical variables by expressing cell frequencies as linear combinations of main effects and interactions.

These models are crucial for understanding associations in categorical data, a key aspect of this chapter. By examining main effects and interactions, researchers can gain insights into the intricate relationships between variables in biological studies.

Log-linear models for contingency tables

Introduction to log-linear models

• Log-linear models are a class of statistical models used to analyze the associations and interactions among multiple categorical variables in a contingency table
• Multi-way contingency tables are cross-tabulations of three or more categorical variables, where each cell represents the frequency or count of observations falling into a specific combination of categories (gender, age group, and education level)
• Log-linear models express the logarithm of the expected cell frequencies as a linear combination of main effects and interaction terms, allowing for the examination of the relationships among the variables

Components and assumptions of log-linear models

• The main effects in a log-linear model represent the independent effects of each variable on the cell frequencies, while the interaction terms capture the dependencies or associations between the variables
• Log-linear models assume that the cell frequencies follow a Poisson distribution and that the logarithm of the expected frequencies can be modeled as a linear function of the parameters
• The Poisson distribution is appropriate for modeling count data, such as the number of individuals falling into each cell of a contingency table
• The logarithmic transformation of the expected frequencies allows for the additive decomposition of the effects and interactions, making the interpretation of the model parameters more straightforward

Constructing log-linear models

Defining variables and model formula

• To construct a log-linear model, the first step is to define the variables and the possible categories for each variable in the multi-way contingency table
• For example, in a study examining the relationship between gender, age group, and education level, the variables would be defined as follows:
  • Gender: Male, Female
  • Age group: Young, Middle-aged, Old
  • Education level: Low, Medium, High
• The model formula specifies the variables and the interaction terms to be included in the log-linear model, using a notation similar to that of analysis of variance (ANOVA) models
• The formula includes the main effects of each variable and the interaction terms of interest, such as Gender + Age + Education + Gender:Age + Gender:Education + Age:Education + Gender:Age:Education

Hierarchical models and the principle of marginality

• The saturated log-linear model includes all possible main effects and interaction terms, representing the most complex model that perfectly fits the observed data
• Hierarchical log-linear models are constructed by systematically removing or adding interaction terms to the model based on the principle of marginality, ensuring that lower-order terms are included before higher-order terms
• The principle of marginality states that if a higher-order interaction term is included in the model, all lower-order terms that are subsets of the higher-order term must also be included
• For example, if the Gender:Age:Education interaction is included, the main effects of Gender, Age, and Education, as well as the two-way interactions Gender:Age, Gender:Education, and Age:Education, must also be present in the model

Parameter estimation and model fitting

• The parameters of the log-linear model are estimated using maximum likelihood estimation (MLE) techniques, such as iterative proportional fitting (IPF) or Newton-Raphson algorithms
• IPF is an iterative algorithm that adjusts the cell frequencies to match the marginal totals of the observed data, converging to the maximum likelihood estimates of the model parameters
• Newton-Raphson is a general optimization algorithm that iteratively updates the parameter estimates by minimizing the negative log-likelihood function
• The model fitting process involves estimating the parameters that maximize the likelihood of observing the data given the specified log-linear model

Interpreting log-linear model parameters

Main effects and interaction parameters

• The parameters of a log-linear model represent the effects of the variables and their interactions on the logarithm of the expected cell frequencies
• The main effect parameters indicate the independent contribution of each variable to the cell frequencies, while the interaction parameters capture the associations or dependencies among the variables
• For example, the main effect parameter for Gender represents the difference in the logarithm of the expected frequencies between males and females, assuming all other variables are held constant
• The interaction parameter for Gender:Age represents the additional effect on the logarithm of the expected frequencies due to the combination of specific levels of Gender and Age, beyond their individual main effects

Assessing model fit and goodness-of-fit measures

• Goodness-of-fit measures, such as the likelihood ratio chi-square (G²) and Pearson's chi-square (X²), assess how well the log-linear model fits the observed data
• A non-significant goodness-of-fit test suggests that the model adequately describes the associations and interactions in the data
  • For example, if the likelihood ratio chi-square test for a log-linear model has a p-value greater than 0.05, it indicates that the model fits the data well and captures the important relationships among the variables
• A significant goodness-of-fit test indicates that the model does not fit the data well, and additional interaction terms or alternative models should be considered
• The deviance (G²) and the Akaike information criterion (AIC) are commonly used to compare the fit of nested log-linear models, with lower values indicating better fit
  • Nested models are models where one model is a special case of the other, obtained by setting some parameters to zero or constraining them to be equal

Model selection for log-linear models

Model selection techniques

• Model selection in log-linear analysis involves choosing the most parsimonious model that adequately describes the associations and interactions among the variables
• Backward elimination is a model selection technique that starts with the saturated model and sequentially removes non-significant interaction terms, based on the likelihood ratio test or other criteria, until a final model is obtained
  • The process begins with the most complex model and gradually simplifies it by removing higher-order interactions that do not significantly contribute to the model fit
• Forward selection begins with the simplest model (usually the independence model) and gradually adds interaction terms that significantly improve the model fit
  • The independence model assumes that all variables are independent of each other, and the cell frequencies are determined solely by the main effects of the variables
  • Interaction terms are added one at a time, based on their contribution to the model fit, until no further significant improvements can be made

Assessing the significance of interactions

• The likelihood ratio test is used to assess the significance of the difference in fit between nested log-linear models, determining whether the inclusion or exclusion of specific interaction terms is justified
  • The test compares the deviance (G²) of the simpler model to that of the more complex model, and a significant result indicates that the additional interaction terms in the complex model significantly improve the fit
• Partial association tests examine the significance of individual interaction terms by comparing the fit of models with and without the interaction, while controlling for other variables and interactions
  • These tests assess the conditional independence of the variables involved in the interaction, given the other variables in the model
• The significance of the interaction terms in the selected log-linear model provides insight into the dependencies and associations among the categorical variables, guiding the interpretation of the results
  • Significant interactions suggest that the relationship between two or more variables depends on the levels of other variables, while non-significant interactions indicate that the variables are conditionally independent