Lasso Regression - Advanced Tab

The Lasso Regression Advanced tab shows the following options.

Estimation Method

In this group box, select a method of estimation for the algorithm specified in the Algorithm list on Quick tab. The available choices vary based on the Algorithm selection in Quick tab.

Linear Regression. These methods fit the partial residual to the standardized predictors using the simple least-squares approach. The two methods differ in terms of complexity associated with the estimation of loss gradient at each coordinate descent step.

Element Name Description
Covariance update The complexity of coordinate descent stepwise update is proportional to the number of non-zero terms in the model. This method is more efficient than the Naive estimation.
Naive The complexity of coordinate descent stepwise update is proportional to the number of predictor variables.
Logistic Regression These methods fit the partial residuals to the standardized predictors using the iteratively reweighted least squares approach. The two methods differ in terms of hessian computation at each coordinate descent step.
Newton This method uses the exact hessian.
Modified Newton This method uses a bounded hessian. This method can be more efficient.
Include intercept Select this check box to include the intercept in the model.
Max. vars in largest model. Specify the maximum number of variables to be included in the model.
Max. iterations. Specify the maximum number of iterations for the coordinate descent. This will be the maximum number of times data is accessed.
Convergence threshold Specify the convergence threshold for coordinate descent. This value is used to determine whether the iterative estimation procedure has converged; specifically, the integer value entered into this field is used as the (negative) exponent of a base 10 constant. For instance, if the default value 7 is used, the constant will evaluate to 10E-7. This constant is then used to check for convergence of the iterative estimation procedure by comparing it to the absolute value of the difference of the deviance function between two successive iterations.
Penalties Click this button to display the Set penalties for predictors dialog box. Here you can specify separate penalties to be applied to each coefficient. A value of 0 can be used to always include the variable in model.
Note: For categorical variable, the specified penalty will be applied for each level

Options / C / W / By Group. See Common Options.