GRM Introductory Overview - Basic Ideas: The Need for Simple Models
A good theory is the end result of a winnowing process. We start with a comprehensive model that includes all conceivable, testable influences on the phenomena under investigation. Then we test the components of the initial comprehensive model, to identify the less comprehensive submodels that adequately account for the phenomena under investigation. Finally from these candidate submodels, we single out the simplest submodel, which by the principle of parsimony we take to be the "best" explanation for the phenomena under investigation.
We prefer simple models not just for philosophical but also for practical reasons. Simple models are easier to put to test again in replication and cross-validation studies. Simple models are less costly to put into practice in predicting and controlling the outcome in the future. The philosophical reasons for preferring simple models should not be downplayed, however. Simpler models are easier to understand and appreciate, and therefore have a "beauty" that their more complicated counterparts often lack.
The entire winnowing process described above is encapsulated in the model-building techniques of stepwise and best-subset regression. The use of these model-building techniques begins with the specification of the design for a comprehensive "whole model." Less comprehensive submodels are then tested to determine if they adequately account for the outcome under investigation. Finally, the simplest of the adequate is adopted as the "best."
Keeping it simple also has driven the design of the GRM module. For example, through the use of Quick Spec dialogs, the "whole model" design for most analyses can be specified simply by selecting the variables for the analysis. But comprehensiveness has not been sacrificed for the sake of simplicity. Using general linear model methods, GRM implements stepwise and best-subset regression not just for standard multiple regression designs, but for any Analysis of Variance (ANOVA) design with categorical predictor variables, any Analysis of Covariance (ANCOVA) design with categorical and continuous predictor variables, and any regression design, including designs with powers and products of continuous predictor variables. In short, GRM takes a simple but innovative and comprehensive approach in implementing model-building techniques using the general linear model.
Other GRM Introductory Overview Topics
A detailed discussion of univariate and multivariate ANOVA techniques can also be found in the Introductory Overview of the ANOVA/MANOVA module; a discussion of Multiple Regression methods is provided in the Overviews. Discussion of the ways in which the linear regression model is extended by the general linear model can be found in the Introductory Overview of the GLM module.