relgam is a package implementing Reluctant Generalized Additive Modeling (RGAM), a new technique for fitting sparse generalized additive models (GAMs). RGAMs are computationally scalable and work with continuous, binary, count and survival data. For the mathematical details, please see the preprint. For a short tutorial on how to use the package, please see the vignette on CRAN.

At a high level, RGAM is fit in the following way: letting y denote the response variable and X denote the design matrix,

1. Fit the lasso of y on X using the glmnet package. Compute the residuals r at the lambda hyperparameter selected by cross-validation.
2. For each of the original features, fit a smoothing spline with d degrees of freedom of r on the feature to get a new, non-linear feature. Let F denote the matrix of new features.
3. Fit the lasso of y on X and F using the glmnet package.

This section lists the updates that we have since the CRAN version of the package. This development version is v1.1.

• Made Step 2 more generic. Previously Step 2 was restricted to fitting a degree smoothing spline with d degrees of freedom of r on each original feature. Now, the user can specify the univariate fitting method for Step 2 using the new nl_maker option in rgam(). Additional arguments for the nl_maker function can be passed to rgam() as well: they are passed down to nl_maker via the .... The default fitting function is the smoothing spline with d degrees of freedom (as before).
• Added basic tests.
• Simplified code for plot.rgam().
• Fixed bug in getf() (some issue with standardization of variables when computing the non-linear component).

Here is a simple example to illustrate how to use this package. First, let’s generate some data:

set.seed(1)
n <- 100; p <- 12
x = matrix(rnorm((n) * p), ncol = p)
f4 = 2 * x[,4]^2 + 4 * x[,4] - 2
f5 = -2 * x[, 5]^2 + 2
f6 = 0.5 * x[, 6]^3
mu = rowSums(x[, 1:3]) + f4 + f5 + f6
y = mu + sqrt(var(mu) / 4) * rnorm(n)

We can fit an RGAM using the rgam() function:

library(relgam)
fit <- rgam(x, y, verbose = FALSE)
#  init_nz not specified: setting to default (all features)
#  using default value of gamma for RGAM: 0.6

(If verbose = TRUE (default), model-fitting is tracked with a progress bar in the console.)