Technical Notes: The MARSplines Algorithm

Implementing MARSplines involves a two-step procedure that is applied successively until a desired model is found. In the first step, we build the model, i.e. increase its complexity by adding basis functions until a preset (user-defined) maximum level of complexity has been reached. Then we begin a backward procedure to remove the least significant basis functions from the model, i.e. those whose removal will lead to the least reduction in the (least-squares) goodness of fit. This algorithm is implemented as follows:

  1. Start with the simplest model involving only the constant basis function.
  2. Search the space of basis functions for each variable and for all possible knots, and add those that maximize a certain measure of goodness of fit (minimize prediction error).
  3. Step 2 is recursively applied until a model of pre-determined maximum complexity is derived.
  4. Finally, in the last stage, a pruning procedure is applied where those basis functions are removed that contribute least to the overall (least squares) goodness of fit.

For more information, see also the Introductory Overview.