Andrew Gard, Ph.D

Stabilizing regression coefficients with the lasso

Andrew Gard — Mon, 12 Oct 2026 05:00:00 GMT

Sometimes linear regression gives poor results even when the data comes from a linear relationship. I’ll explain why and then show you what you can do about it.

Cross validation

Andrew Gard — Mon, 05 Oct 2026 05:00:00 GMT

How to tidymodels

Andrew Gard — Mon, 21 Sep 2026 05:00:00 GMT

library(tidymodels)
library(ISLR2)

The tidymodels family of packages is a unified framework for machine learning in R. I recommend it for predictive modeling, where future performance is the primary benchmark for success. For descriptive modeling and statistical inference, tidymodels can be unwieldy.

When using data to estimate the future performance of a model, we always adhere to one essential core principle, which tidymodels works hard to enforce: Never use the same data to both train and assess any model.

The `College` data set

Throughout this demonstration, I’ll use the ISLR2::College data set, which includes information about 777 US colleges in 1995. We’ll view Outstate tuition as the response variable, attempting to model it using the other variables in the set.

glimpse(College)

Rows: 777
Columns: 18
$ Private      Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes…
$ Apps         1660, 2186, 1428, 417, 193, 587, 353, 1899, 1038, 582, 173…
$ Accept       1232, 1924, 1097, 349, 146, 479, 340, 1720, 839, 498, 1425…
$ Enroll       721, 512, 336, 137, 55, 158, 103, 489, 227, 172, 472, 484,…
$ Top10perc    23, 16, 22, 60, 16, 38, 17, 37, 30, 21, 37, 44, 38, 44, 23…
$ Top25perc    52, 29, 50, 89, 44, 62, 45, 68, 63, 44, 75, 77, 64, 73, 46…
$ F.Undergrad  2885, 2683, 1036, 510, 249, 678, 416, 1594, 973, 799, 1830…
$ P.Undergrad  537, 1227, 99, 63, 869, 41, 230, 32, 306, 78, 110, 44, 638…
$ Outstate     7440, 12280, 11250, 12960, 7560, 13500, 13290, 13868, 1559…
$ Room.Board   3300, 6450, 3750, 5450, 4120, 3335, 5720, 4826, 4400, 3380…
$ Books        450, 750, 400, 450, 800, 500, 500, 450, 300, 660, 500, 400…
$ Personal     2200, 1500, 1165, 875, 1500, 675, 1500, 850, 500, 1800, 60…
$ PhD          70, 29, 53, 92, 76, 67, 90, 89, 79, 40, 82, 73, 60, 79, 36…
$ Terminal     78, 30, 66, 97, 72, 73, 93, 100, 84, 41, 88, 91, 84, 87, 6…
$ S.F.Ratio    18.1, 12.2, 12.9, 7.7, 11.9, 9.4, 11.5, 13.7, 11.3, 11.5, …
$ perc.alumni  12, 16, 30, 37, 2, 11, 26, 37, 23, 15, 31, 41, 21, 32, 26,…
$ Expend       7041, 10527, 8735, 19016, 10922, 9727, 8861, 11487, 11644,…
$ Grad.Rate    60, 56, 54, 59, 15, 55, 63, 73, 80, 52, 73, 76, 74, 68, 55…

Splitting the data

The rsample::initial_split() function partitions a data set by randomly assigning rows (prop = 3/4 is the default) to the training set and storing them, along with the full data frame, in a list (technically, an rsplit object).

set.seed(0)

split <- initial_split(College)
split


<582/195/777>

names(split)

[1] "data"   "in_id"  "out_id" "id"

split$in_id |> head(n = 20) # Training set row numbers

 [1] 398 679 129 509 471 299 270 187 307 597 277 494 330 591 725  37 105 729 485
[20] 677

tidymodels provides simple functions to extract both the training and testing sets.

train <- training(split)
test <- testing(split)

glimpse(train) # 582 rows, ~75% of the original set

Rows: 582
Columns: 18
$ Private      Yes, No, Yes, No, Yes, Yes, No, Yes, Yes, Yes, Yes, Yes, Y…
$ Apps         657, 1401, 344, 4216, 427, 2929, 8681, 1457, 1243, 2425, 4…
$ Accept       537, 1239, 264, 2290, 385, 1834, 6695, 1045, 947, 1818, 35…
$ Enroll       113, 605, 97, 736, 143, 622, 2408, 345, 324, 601, 913, 225…
$ Top10perc    37, 10, 11, 20, 18, 20, 10, 27, 50, 62, 13, 24, 25, 57, 42…
$ Top25perc    90, 34, 42, 52, 38, 56, 35, 50, 77, 93, 33, 42, 55, 81, 64…
$ F.Undergrad  1039, 3716, 500, 4296, 581, 2738, 15701, 1109, 1129, 2110,…
$ P.Undergrad  466, 675, 331, 1027, 533, 1662, 1823, 502, 74, 95, 1446, 2…
$ Outstate     12474, 7100, 12600, 5130, 12700, 12600, 7799, 14990, 17163…
$ Room.Board   5678, 4380, 5520, 4690, 5800, 5610, 3403, 4980, 3891, 5150…
$ Books        630, 540, 630, 600, 450, 450, 537, 450, 525, 500, 570, 600…
$ Personal     1278, 2948, 2250, 1450, 700, 3160, 2605, 550, 975, 490, 11…
$ PhD          53, 63, 77, 73, 81, 90, 77, 77, 76, 94, 66, 55, 94, 81, 91…
$ Terminal     71, 88, 80, 75, 85, 90, 84, 98, 92, 96, 83, 60, 95, 91, 91…
$ S.F.Ratio    11.9, 19.4, 10.4, 17.9, 10.3, 15.1, 21.0, 21.5, 10.1, 9.6,…
$ perc.alumni  19, 0, 7, 18, 37, 9, 16, 21, 57, 20, 14, 19, 15, 41, 40, 3…
$ Expend       10613, 5389, 9773, 5125, 11758, 9084, 5569, 7502, 13965, 1…
$ Grad.Rate    72, 36, 43, 56, 84, 84, 54, 64, 77, 93, 66, 67, 65, 70, 86…

We could also consider stratifying the split by the response variable. If the response is quantitative, as is the case here, initial_split() will automatically group it into quartiles.

split <- initial_split(
  College, 
  strata = Outstate
  )

We now leave the test set aside and build the model entirely using train.

Specifying the model

tidymodels distinguishes between model specification, where we make choices about how the data will be modeled, and fitting, where training data is used to construct a concrete algorithm for making predictions.

If you’re used to base R modeling functions like lm(), this may feel awkward at first.

lm(
  Outstate ~ Top10perc  + PhD, 
  data = College
  ) # standard


Call:
lm(formula = Outstate ~ Top10perc + PhD, data = College)

Coefficients:
(Intercept)    Top10perc          PhD  
    5202.71       114.05        28.83

linear_reg(
  Outstate ~ Top10perc  + PhD,
  data = College
  ) # fails

Error in `linear_reg()`:
! unused argument (data = College)

The tidymodels code to obtain regression coefficients is somewhat more verbose.

my_lm <- linear_reg() # parsnip specification

my_lm_fit <- fit(
  my_lm,
  Outstate ~ Top10perc + PhD,
  data = College
  ) # fit the model

my_lm_fit

parsnip model object


Call:
stats::lm(formula = Outstate ~ Top10perc + PhD, data = data)

Coefficients:
(Intercept)    Top10perc          PhD  
    5202.71       114.05        28.83

tidy(my_lm_fit) # results as a tibble

# A tibble: 3 × 5
  term        estimate std.error statistic  p.value
                          
1 (Intercept)   5203.     553.        9.41 5.64e-20
2 Top10perc      114.       7.95     14.4  1.37e-41
3 PhD             28.8      8.59      3.36 8.23e- 4

This is underwhelming because the model is so simple and the lm() syntax so familiar.

Let’s switch to a modestly more complicated -nearest neighbor regression, where the closest points to a new observation are used to estimate an unknown response value. For this, we use parsnip::nearest_neighbor().

my_knn <- nearest_neighbor(
  mode = "regression",
  engine = "kknn",
  neighbors = 27 # a reasonable first choice
  )

my_knn

K-Nearest Neighbor Model Specification (regression)

Main Arguments:
  neighbors = 27

Computational engine: kknn

Fitting this model immediately would give sub-optimal results, however, as the data set is not yet in an appropriate form for knn regression.

Preprocessing

A recipe, as the name suggests, is a documented set of steps to be followed when any data (training, test, or otherwise) is handled by a model. Storing our steps in this way allows us to automate the process so that it be easily applied as needed.

Let’s create a simple recipe appropriate to a knn model.

col_rec <- recipe(
  Outstate ~ ., data = train) |> 
  step_dummy(all_nominal_predictors()) |> 
  step_normalize(all_numeric_predictors())

col_rec

── Recipe ──────────────────────────────────────────────────────────────────────

── Inputs

Number of variables by role

outcome:    1
predictor: 17

── Operations

• Dummy variables from: all_nominal_predictors()

• Centering and scaling for: all_numeric_predictors()

Preprocessing tools generally have the form step_*(). Most common techniques have built-in functions, including the two used above.

You may have noticed that some steps (including step_normalize) require input from the training set. Currently the recipe, which is just a set of instructions, has not performed any such computations. Use prep() to tell tidymodels to incorporate the training data into the recipe.

rec_prepped <- prep(col_rec)
rec_prepped

── Recipe ──────────────────────────────────────────────────────────────────────

── Inputs

Number of variables by role

outcome:    1
predictor: 17

── Training information

Training data contained 582 data points and no incomplete rows.

── Operations

• Dummy variables from: Private | Trained

• Centering and scaling for: Apps, Accept, Enroll, Top10perc, ... | Trained

Once the recipe is trained (prepped), it can be applied to any set with the correct column names, including train and test.

train_baked <- bake(
  rec_prepped,
  new_data = train
  )

glimpse(train_baked)

Rows: 582
Columns: 18
$ Apps         -0.584565901, -0.392108788, -0.665532402, 0.336072357, -0.…
$ Accept       -0.59303713, -0.29578081, -0.70863680, 0.14925677, -0.6574…
$ Enroll       -0.73007850, -0.16151757, -0.74856828, -0.01013245, -0.695…
$ Top10perc    0.53780784, -1.01427892, -0.95679422, -0.43943197, -0.5544…
$ Top25perc    1.718305395, -1.097897433, -0.695582743, -0.192689381, -0.…
$ F.Undergrad  -0.54793004, 0.03889871, -0.66608495, 0.16604129, -0.64832…
$ P.Undergrad  -0.25540460, -0.12116108, -0.34211693, 0.10493328, -0.2123…
$ Room.Board   1.1778784439, -0.0004757491, 1.0344424173, 0.2809493663, 1…
$ Books        0.45651909, -0.07400850, 0.45651909, 0.27967656, -0.604536…
$ Personal     -0.08996942, 2.29903910, 1.30051937, 0.15608415, -0.916823…
$ PhD          -1.21449875, -0.59630890, 0.26915688, 0.02188095, 0.516432…
$ Terminal     -0.60448281, 0.56500349, 0.01465699, -0.32930956, 0.358623…
$ S.F.Ratio    -0.54144438, 1.38707844, -0.92714895, 1.00137388, -0.95286…
$ perc.alumni  -0.30283014, -1.84957177, -1.27971959, -0.38423759, 1.1625…
$ Expend       0.164258830, -0.812840693, 0.007144818, -0.862219382, 0.37…
$ Grad.Rate    0.39369765, -1.70682518, -1.29839018, -0.53986805, 1.09387…
$ Outstate     12474, 7100, 12600, 5130, 12700, 12600, 7799, 14990, 17163…
$ Private_Yes  0.5913933, -1.6880167, 0.5913933, -1.6880167, 0.5913933, 0…

In general, you only need prep() and bake() if you want to explicitly see the transformed predictors. More typically, we just incorporate the preprocessor directly into a tidymodels workflow.

Creating a workflow

A workflow incorporates both a recipe and a model specification.

knn_wf <- workflow(
  spec = my_knn,
  preprocessor = col_rec,
  )

Often, you’ll see this step written using the pipe.

knn_wf <- workflow() |> 
  add_recipe(col_rec) |> 
  add_model(my_knn) # same as before

Importantly, creating a workflow does not actually perform any preprocessing or model fitting.

knn_wf

══ Workflow ════════════════════════════════════════════════════════════════════
Preprocessor: Recipe
Model: nearest_neighbor()

── Preprocessor ────────────────────────────────────────────────────────────────
2 Recipe Steps

• step_dummy()
• step_normalize()

── Model ───────────────────────────────────────────────────────────────────────
K-Nearest Neighbor Model Specification (regression)

Main Arguments:
  neighbors = 27

Computational engine: kknn

We can take this next step using the fit() function.

knn_wf_fit <- fit(
  knn_wf, 
  data = train
  )

The model is now ready to use.

Putting a trained workflow into action

Use predict() to make predictions, duh. To start, we might just look at the training set, checking how far actual results are from the model’s predictions. The output is a tibble, not a vector.

predict(
  knn_wf_fit, 
  new_data = train
  )

# A tibble: 582 × 1
    .pred
    
 1 10953.
 2  6386.
 3 11191.
 4  6389.
 5 12944.
 6 12481.
 7  7213.
 8 12128.
 9 15024.
10 16666.
# ℹ 572 more rows

Predictions made on the same data used to train the model will almost always be overly optimistic. It’s like when your mom tells you that you look nice.

Model tuning and cross validation

It’s unlikely that our choice of neighbors was optimal. Model tuning refers to the process of comparing a range of hyperparameter values and selecting the best one. tidymodels is built to facilitate this.

We start by specifying within our model that a hyperparameter needs to be tuned. We do this with the tune() placeholder.

my_knn_cv <- nearest_neighbor(
  mode = "regression",
  engine = "kknn",
  neighbors = tune()
  )

knn_wf_cv <- workflow(
  preprocessor = col_rec,
  spec = my_knn_cv
  )

The most common way of comparing competing models during the tuning process is using cross-validation, where the training set is split into pieces (often ) of approximately equal size, called folds. The models are all then trained once per fold, with that one fold used as a holdout set while the others are used for training. The results are then averaged to give performance estimates for each of the competing models.

folds <- vfold_cv(
  train, 
  v = 10,
  repeats = 5) # create folds

head(folds)

# A tibble: 6 × 3
  splits           id      id2   
                 
1  Repeat1 Fold01
2  Repeat1 Fold02
3  Repeat1 Fold03
4  Repeat1 Fold04
5  Repeat1 Fold05
6  Repeat1 Fold06

The simplest tool to perform model fitting on cross-validation folds is tune_grid().

tune_results <- tune_grid(
  knn_wf_cv,
  resamples = folds
  )

This function operates by brute force using a default set of 10 values that make sense for the hyperparameter being tuned. You can tweak these values by passing new ones within a data frame.

neighbors <- seq(11, 33, by = 2)
knn_grid <- data.frame(neighbors)
knn_grid

   neighbors
1         11
2         13
3         15
4         17
5         19
6         21
7         23
8         25
9         27
10        29
11        31
12        33

tune_results <- tune_grid(
  knn_wf_cv,
  resamples = folds,
  grid = knn_grid
  )

Another option for modifying the tuning grid is to use one of the dedicated functions from the dials package.

glimpse(tune_results)

Rows: 50
Columns: 5
$ splits    [], [ "Repeat1", "Repeat1", "Repeat1", "Repeat1", "Repeat1", "Repea…
$ id2       "Fold01", "Fold02", "Fold03", "Fold04", "Fold05", "Fold06", "…
$ .metrics  [], [], [], …
$ .notes    [], [], [], [


The result of tune_grid() is a mutated version of the output of vfold_cv() that includes new columns for .metrics and .notes.
Use collect_metrics() to compare performance across parameter values.

tuning_metrics <- collect_metrics(tune_results)
tuning_metrics

# A tibble: 24 × 7
   neighbors .metric .estimator     mean     n  std_err .config         
                                     
 1        11 rmse    standard   1944.       50 30.6     pre0_mod01_post0
 2        11 rsq     standard      0.770    50  0.00895 pre0_mod01_post0
 3        13 rmse    standard   1927.       50 30.6     pre0_mod02_post0
 4        13 rsq     standard      0.774    50  0.00900 pre0_mod02_post0
 5        15 rmse    standard   1917.       50 30.7     pre0_mod03_post0
 6        15 rsq     standard      0.777    50  0.00905 pre0_mod03_post0
 7        17 rmse    standard   1910.       50 30.8     pre0_mod04_post0
 8        17 rsq     standard      0.778    50  0.00911 pre0_mod04_post0
 9        19 rmse    standard   1906.       50 30.8     pre0_mod05_post0
10        19 rsq     standard      0.780    50  0.00914 pre0_mod05_post0
# ℹ 14 more rows


It’s often helpful to plot these results.

tuning_metrics |> 
  filter(.metric == "rmse") |> 
  ggplot(aes(x = neighbors, y = mean)) + 
  geom_line() +
  geom_point() +
  theme_minimal()








Use show_best() to automatically filter out sub-optimal rows from tuning_metrics. By default, the best 5 are shown, though this can be controlled with the optional n argument.

show_best(
  tune_results,
  metric = "rmse",
  n = 7
  ) 

# A tibble: 7 × 7
  neighbors .metric .estimator  mean     n std_err .config         
                                
1        23 rmse    standard   1904.    50    30.7 pre0_mod07_post0
2        21 rmse    standard   1904.    50    30.7 pre0_mod06_post0
3        25 rmse    standard   1905.    50    30.6 pre0_mod08_post0
4        19 rmse    standard   1906.    50    30.8 pre0_mod05_post0
5        27 rmse    standard   1907.    50    30.5 pre0_mod09_post0
6        29 rmse    standard   1909.    50    30.5 pre0_mod10_post0
7        17 rmse    standard   1910.    50    30.8 pre0_mod04_post0


To automatically extract the result that minimizes rmse (or any other metric), use select_best().

optimal_k <- select_best(
  tune_results,
  metric = "rmse"
  )

optimal_k # 23 neighbors

# A tibble: 1 × 2
  neighbors .config         
                  
1        23 pre0_mod07_post0


select_best() has several friends, including select_by_one_std_error(), which chooses the least flexible model with performance comparable to the optimal one.
The output of select_best() can be fed back to the workflow being tuned with finalize_workflow().

knn_wf_final <- finalize_workflow(
  knn_wf_cv,
  optimal_k
  )

knn_wf_final

══ Workflow ════════════════════════════════════════════════════════════════════
Preprocessor: Recipe
Model: nearest_neighbor()

── Preprocessor ────────────────────────────────────────────────────────────────
2 Recipe Steps

• step_dummy()
• step_normalize()

── Model ───────────────────────────────────────────────────────────────────────
K-Nearest Neighbor Model Specification (regression)

Main Arguments:
  neighbors = 23

Computational engine: kknn 


The model is now fully specified and ready to be fit using the training set.

knn_fit_final <- fit(
  knn_wf_final, 
  data = train)



Evaluation using the test set
Once we’ve finalized our model, we can use it to make predictions on new data. The broom::augment() function makes this information readily available in a data frame that includes all the variables relevant to the model.

test_aug <- augment(
  knn_fit_final, 
  new_data = test
  )

glimpse(test_aug)

Rows: 195
Columns: 20
$ .pred        13403.758, 8653.189, 14218.005, 11708.722, 13200.189, 9714…
$ .resid       2191.2416, 1814.8112, 2861.9951, -2018.7216, -628.1892, 14…
$ Private      Yes, Yes, Yes, Yes, Yes, Yes, No, No, Yes, Yes, No, Yes, Y…
$ Apps         1038, 582, 2652, 1179, 1267, 619, 12809, 1734, 1879, 2496,…
$ Accept       839, 498, 1900, 780, 1080, 516, 10308, 1729, 1658, 1402, 8…
$ Enroll       227, 172, 484, 290, 385, 219, 3761, 951, 497, 531, 546, 48…
$ Top10perc    30, 21, 44, 38, 44, 20, 24, 12, 36, 53, 12, 37, 11, 15, 20…
$ Top25perc    63, 44, 77, 64, 73, 51, 49, 52, 69, 95, 36, 68, 28, 55, 50…
$ F.Undergrad  973, 799, 1707, 1130, 1306, 1251, 22593, 3602, 1950, 2121,…
$ P.Undergrad  306, 78, 44, 638, 28, 767, 7585, 939, 38, 69, 824, 49, 74,…
$ Outstate     15595, 10468, 17080, 9690, 12572, 11208, 7434, 3460, 13353…
$ Room.Board   4400, 3380, 4440, 4785, 4552, 4124, 4850, 2650, 4173, 8124…
$ Books        300, 660, 400, 600, 400, 350, 700, 450, 540, 600, 660, 350…
$ Personal     500, 1800, 600, 1000, 400, 1615, 2100, 1000, 821, 850, 180…
$ PhD          79, 40, 73, 60, 79, 55, 88, 57, 78, 83, 57, 80, 63, 66, 76…
$ Terminal     84, 41, 91, 84, 87, 65, 93, 60, 83, 93, 62, 80, 63, 68, 71…
$ S.F.Ratio    11.3, 11.5, 9.9, 13.3, 15.3, 12.7, 18.9, 19.6, 12.7, 10.3,…
$ perc.alumni  23, 15, 41, 21, 32, 25, 5, 5, 40, 33, 16, 17, 13, 19, 19, …
$ Expend       11644, 8991, 11711, 7940, 9305, 6584, 4602, 4739, 9220, 12…
$ Grad.Rate    80, 52, 76, 74, 68, 65, 48, 48, 71, 91, 46, 63, 35, 75, 58…


The yardstick package, included in tidymodels, consists of functions which compute standard performance metrics, including root mean squared error.

rmse(test_aug, 
     truth = Outstate,
     estimate = .pred)

# A tibble: 1 × 3
  .metric .estimator .estimate
               
1 rmse    standard       1904.


You can compute multiple performance metrics at once by creating a metric set.

my_metrics <- metric_set(rmse, rsq)

my_metrics(
  test_aug,
  truth = Outstate, 
  estimate = .pred
  )

# A tibble: 2 × 3
  .metric .estimator .estimate
               
1 rmse    standard    1904.   
2 rsq     standard       0.767


The fact that these outputs are tibbles will be very convenient once we begin comparing various models.
If all you want to do is apply the trained model to the holdout set created with initial_split(), use last_fit(). To extract performance metrics from this object, use collect_metrics().

last_fit(knn_wf_final, split) |>
  collect_metrics()

# A tibble: 2 × 4
  .metric .estimator .estimate .config        
                          
1 rmse    standard    1904.    pre0_mod0_post0
2 rsq     standard       0.767 pre0_mod0_post0


On average, this model misses the true out-of-state tuition by about $1904.


Final thought
If you were expecting that learning tidymodels would be like learning tidyverse, you’re probably disappointed right about now. While you can practice dplyr without touching ggplot2, you can’t meaningfully do this with the individual tidymodels packages. This isn’t a design flaw in tidymodels but rather a fundamental challenge of the machine learning process, which is interconnected and complex by its very nature.
Overall, tidymodels is as about as simple as it can be while still doing what it does. Be patient with yourself, take multiple passes at the content, and you’ll get there sooner than you think.
Want to hear me explain all of this? Here’s the vid corresponding to this post:



Working with factors in R
Andrew Gard — Mon, 07 Sep 2026 05:00:00 GMT
 




Categorical variables (or factors as they are referred to when working in R) are variables that assign observations to specific groups. This post will cover some of the most common problems you may encounter while trying to visualize or model such variables.
Throughout, I’ll refer to the scooby data set, originally sourced from kaggle but modified slightly for use in my vids and in this post. A download link can be found at the end of this tutorial.

# Load and prep data

library(tidyverse)
theme_set(theme_minimal())

library(readxl)
scooby <- read_excel("_internal/scooby.xlsx") 

scooby_sm <- scooby |> 
  filter(format == "TV Series") |>  
  select(title,  
         date_aired,
         imdb,
         motive) 


Factor, character, and other vector types
As a general rule, tidyverse functions treat character vectors as factors. For instance, we can apply dplyr::count to scooby$motive, even though the latter just consists of strings.

glimpse(scooby_sm)

Rows: 374
Columns: 4
$ title       "What a Night for a Knight", "A Clue for Scooby Doo", "Hass…
$ date_aired  1969-09-13, 1969-09-20, 1969-09-27, 1969-10-04, 1969-10-11…
$ imdb        8.1, 8.1, 8.0, 7.8, 7.5, 8.4, 7.6, 8.2, 8.1, 8.0, 8.5, 8.2,…
$ motive      "Theft", "Theft", "Treasure", "Natural Resource", "Competit…

scooby_sm |> 
  count(motive)

# A tibble: 17 × 2
   motive               n
               
 1 Abduction            3
 2 Assistance           2
 3 Competition        130
 4 Conquer             33
 5 Counterfeit          6
 6 Entertainment        3
 7 Extortion            3
 8 Imagination          1
 9 Inheritance          6
10 Natural Resource    19
11 Preservation        10
12 Safety               2
13 Simulation           2
14 Smuggling           19
15 Theft               93
16 Treasure            40
17                  2


Use as.factor() to explicitly coerce variables to factors. This is particularly useful when category levels are numeric, like am (automatic/manual transmission) in the built-in mtcars set, since functions like ggplot will otherwise attempt to treat them as continuous variables. For instance, we can’t get directly get a side-by-side boxplot of mpg vs am.

glimpse(mtcars)

Rows: 32
Columns: 11
$ mpg   21.0, 21.0, 22.8, 21.4, 18.7, 18.1, 14.3, 24.4, 22.8, 19.2, 17.8,…
$ cyl   6, 6, 4, 6, 8, 6, 8, 4, 4, 6, 6, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 8,…
$ disp  160.0, 160.0, 108.0, 258.0, 360.0, 225.0, 360.0, 146.7, 140.8, 16…
$ hp    110, 110, 93, 110, 175, 105, 245, 62, 95, 123, 123, 180, 180, 180…
$ drat  3.90, 3.90, 3.85, 3.08, 3.15, 2.76, 3.21, 3.69, 3.92, 3.92, 3.92,…
$ wt    2.620, 2.875, 2.320, 3.215, 3.440, 3.460, 3.570, 3.190, 3.150, 3.…
$ qsec  16.46, 17.02, 18.61, 19.44, 17.02, 20.22, 15.84, 20.00, 22.90, 18…
$ vs    0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0,…
$ am    1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0,…
$ gear  4, 4, 4, 3, 3, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 4, 4, 4, 3, 3,…
$ carb  4, 4, 1, 1, 2, 1, 4, 2, 2, 4, 4, 3, 3, 3, 4, 4, 4, 1, 2, 1, 1, 2,…

ggplot(mtcars, aes(x = am, 
                   y = mpg)) + 
  geom_boxplot() # not what we want

Warning: Orientation is not uniquely specified when both the x and y aesthetics are
continuous. Picking default orientation 'x'.


Warning: Continuous x aesthetic
ℹ did you forget `aes(group = ...)`?









While it’s fine to follow the recommendation given in this warning message, it may be better to just convert am to a factor once and for all:

mtcars2 <- mtcars |> 
  mutate(am = as.factor(am)) 

ggplot(mtcars2, aes(x = am, 
                    y = mpg)) + 
  geom_boxplot()










Relabeling factors
There are many options for changing the text used to label factor levels, but fct_recode() is a solid choice.

mtcars2 <- mtcars2 |> 
  mutate(am = fct_recode(am, 
                         "automatic" = "0",
                         "manual" = "1"))
mtcars2 |> 
  count(am)

         am  n
1 automatic 19
2    manual 13


Notice the syntax, “new_name” = “old_name”, which is common in tidyverse R.
When you have a large number of different factor levels, it may be helpful to lump the less common ones into an “other” category. For instance, the next chunk combines all categories with fewer than 20 occurrences.

scooby_sm2 <- scooby_sm |> 
  mutate(motive = fct_lump_min(motive,
                               min = 20)) 
scooby_sm2 |> 
  count(motive) |> 
  arrange(-n)

# A tibble: 6 × 2
  motive          n
         
1 Competition   130
2 Theft          93
3 Other          76
4 Treasure       40
5 Conquer        33
6             2


Also check out fct_lump_n() to specify the number of levels retained, fct_lump_prop() to lump low-percentage levels, and fct_lump_lowfreq() to try to force the new “Other” level to have lowest frequency.


Rearranging levels
Sometimes the order of factor levels matters, for instance in a bar chart. The default odering is alphanumeric.

ggplot(scooby_sm2, 
       aes(x = motive)) + 
  geom_bar()








Use fct_relevel() to shift levels to the front. For instance, the next code chunk places the motives Treasure and Theft before Compitition and others.

scooby_sm3 <- scooby_sm2 |> 
  mutate(motive = fct_relevel(motive,
                              "Treasure",
                              "Theft")) 

ggplot(scooby_sm3, 
       aes(x = motive)) + 
  geom_bar()








Often, you just want to put the most common factor levels first. Use fct_infreq() to accomplish this.

ggplot(scooby_sm2, aes(x = fct_infreq(motive))) + 
  geom_bar()








You can also reverse the order of the factor levels with fct_rev.


Reordering levels using a second variable
For some plots, it can be natural to order a categorical variable using a numeric one. For instance, in the next plot, it would be preferable to show the various motives in increasing order of imdb ratings.

ggplot(scooby_sm3, aes(x = motive,
                       y = imdb)) + 
  geom_boxplot()








Use fct_reorder to accomplish this.

scooby_sm4 <- scooby_sm3 |> 
  mutate(motive = fct_reorder(motive,
                              imdb))

ggplot(scooby_sm4, aes(x = motive,
                       y = imdb)) + 
  geom_boxplot()








By default, fct_reorder sorts using the median. You can specify a different function (like mean) by adding a .fun argument.


Ordered factors
Occasionally you may encounter ordered factors, ones of class ord. For the most part, you can treat these like regular factors.

glimpse(diamonds)

Rows: 53,940
Columns: 10
$ carat    0.23, 0.21, 0.23, 0.29, 0.31, 0.24, 0.24, 0.26, 0.22, 0.23, 0.…
$ cut      Ideal, Premium, Good, Premium, Good, Very Good, Very Good, Ver…
$ color    E, E, E, I, J, J, I, H, E, H, J, J, F, J, E, E, I, J, J, J, I,…
$ clarity  SI2, SI1, VS1, VS2, SI2, VVS2, VVS1, SI1, VS2, VS1, SI1, VS1, …
$ depth    61.5, 59.8, 56.9, 62.4, 63.3, 62.8, 62.3, 61.9, 65.1, 59.4, 64…
$ table    55, 61, 65, 58, 58, 57, 57, 55, 61, 61, 55, 56, 61, 54, 62, 58…
$ price    326, 326, 327, 334, 335, 336, 336, 337, 337, 338, 339, 340, 34…
$ x        3.95, 3.89, 4.05, 4.20, 4.34, 3.94, 3.95, 4.07, 3.87, 4.00, 4.…
$ y        3.98, 3.84, 4.07, 4.23, 4.35, 3.96, 3.98, 4.11, 3.78, 4.05, 4.…
$ z        2.43, 2.31, 2.31, 2.63, 2.75, 2.48, 2.47, 2.53, 2.49, 2.39, 2.…

diamonds |> 
  count(cut) # several ordered factors

# A tibble: 5 × 2
  cut           n
       
1 Fair       1610
2 Good       4906
3 Very Good 12082
4 Premium   13791
5 Ideal     21551


Specifying that a factor is ordered (for instance using ordered) can occasionally matter in modeling applications and certain forms of statistical inference. Other than that, the most visible difference with ordered factors in R is how they’re displayed by ggplot.

ggplot(diamonds, aes(x = cut,
                     fill = cut)) + 
  geom_bar(show.legend = FALSE)








By default, ordered factors are colored using the viridis palette, which is has a distinct order to it.


Want more?
If something here isn’t clear, there’s a good chance you’ll find an explanation in my vid on the subject:


  
This post will be cross-listed on R-bloggers, a site that I totally recommend.




 


Regular expressions for R users
Andrew Gard — Mon, 24 Aug 2026 05:00:00 GMT
 




Searching and manipulating strings (those things that make up character vectors) is easy as long as you only need to match specific sequences of letters and numbers. To match patterns rather than specific substrings or to search symbols, you’ll need to use regular expressions (regex for short).
Throughout this post, I’ll make use three stringr functions: str_subset, str_detect, and str_extract, and a subset of the sentences vector.

library(stringr)

sentences <- sentences[1:10]
sentences

 [1] "The birch canoe slid on the smooth planks." 
 [2] "Glue the sheet to the dark blue background."
 [3] "It's easy to tell the depth of a well."     
 [4] "These days a chicken leg is a rare dish."   
 [5] "Rice is often served in round bowls."       
 [6] "The juice of lemons makes fine punch."      
 [7] "The box was thrown beside the parked truck."
 [8] "The hogs were fed chopped corn and garbage."
 [9] "Four hours of steady work faced us."        
[10] "A large size in stockings is hard to sell." 

str_subset(sentences, "of") # keep only elements with a match

[1] "It's easy to tell the depth of a well."
[2] "Rice is often served in round bowls."  
[3] "The juice of lemons makes fine punch." 
[4] "Four hours of steady work faced us."   

str_detect(sentences, "of") # TRUE when an element matches, FALSE otherwise

 [1] FALSE FALSE  TRUE FALSE  TRUE  TRUE FALSE FALSE  TRUE FALSE

str_extract(sentences, "of") # pull out matching substrings

 [1] NA   NA   "of" NA   "of" "of" NA   NA   "of" NA  



Special characters
The following characters (all of which are explained below) have special meanings and can’t be searched literally.

. \ | ( ) [ { ^ $ * + ? 

For instance, a dot matches with any character, which explains the following output.

str_extract(sentences, ".")

 [1] "T" "G" "I" "T" "R" "T" "T" "T" "F" "A"


In order to match these metacharacters, precede them with two slashes, like so:

str_extract(sentences, "\\.")

 [1] "." "." "." "." "." "." "." "." "." "."




Position and relative position
To specify that a substring comes at the beginning of a string, use “^”. To specify that it comes at the end, use “$”.

str_subset(fruit, "^re")

[1] "redcurrant"

str_subset(fruit, "er$") # note order

[1] "bell pepper"  "chili pepper" "cucumber"    


As mentioned above, a dot indicates that substrings are separated by an arbitrary character.

str_extract(sentences, "s.o")

 [1] "smo" NA    NA    NA    "s o" NA    NA    NA    "s o" "sto"




Character sets
Use brackets to allow multiple options for the character to be matched.

str_subset(sentences, "[oe]n")

[1] "The birch canoe slid on the smooth planks."
[2] "These days a chicken leg is a rare dish."  
[3] "Rice is often served in round bowls."      
[4] "The juice of lemons makes fine punch."     

str_extract(sentences, "[oe]n")

 [1] "on" NA   NA   "en" "en" "on" NA   NA   NA   NA  


Ranges of letters and numbers are allowed.

str_subset(sentences, "[a-j]o")

[1] "Rice is often served in round bowls."       
[2] "The box was thrown beside the parked truck."
[3] "The hogs were fed chopped corn and garbage."
[4] "Four hours of steady work faced us."        

str_extract(sentences, "[a-j]o")

 [1] NA   NA   NA   NA   "bo" NA   "bo" "ho" "ho" NA  


Inside of brackets, “^” represents negation.

str_extract(sentences, "[^a-j]o")

 [1] "no" "to" "to" NA   " o" " o" "ro" NA   "Fo" "to"




Character classes
It’s also possible to specify classes of characters. “\d” matches any digit and “\w” matches and word character (a-z, A-Z, 0-9, and underscore), while “\D” and “\W” will match anything except digits and word characters, respectively. Many other classes are available. Enter ?regex for a full list.


The role of “\” in Quarto
If you write “\d” in a Quarto document, it won’t render properly. This is because Quarto interprets the slash as an escape character. To actually render this string, use “\\d” instead.
Similarly, to render a slash, use \\.


Matching whitespace
Use \\s for space, \\t for tab, and \\n for a new line.

str_subset(sentences, "e\\sb")

[1] "The birch canoe slid on the smooth planks." 
[2] "Glue the sheet to the dark blue background."
[3] "The box was thrown beside the parked truck."




Repetition
Use curly braces to indicate how many times in a row a character must appear.

str_subset(sentences, "e{2}")

[1] "Glue the sheet to the dark blue background."


Use “+” to indicate at least one occurrence, possibly more.

str_subset(sentences, "she+t")

[1] "Glue the sheet to the dark blue background."


To indicate a character is optional, use “?”. For instance, the next command detects both “the” and “often”.

str_extract(sentences, "th?e")

 [1] "the" "the" "te"  NA    "te"  NA    "the" NA    "te"  NA   


Finally, “*” indicates an optional character that may occur more than once.

str_extract(sentences, "she*")

 [1] NA     "shee" NA     "sh"   NA     NA     NA     NA     NA     NA    


Notice that these symbols come after the characters they refer to, unlike “\” which comes before.


Combining substrings with |
Use the pipe to indicate an “or” statement. Expressions on either side are grouped by default.

str_subset(sentences, "chicken|hog")

[1] "These days a chicken leg is a rare dish."   
[2] "The hogs were fed chopped corn and garbage."


You can still group using parentheses, though.

str_subset(sentences, "(chicken|hog)s")

[1] "The hogs were fed chopped corn and garbage."




Want more?
If something here isn’t clear, there’s a good chance you’ll find an explanation in my vid on the subject:


 
This post will be cross-listed on R-bloggers, a site that I totally recommend.




 


Understanding the grammar of graphics
Andrew Gard — Mon, 05 Jan 2026 06:00:00 GMT
 






The world of data visualization has been revolutionized in recent years with the introduction of the grammar of graphics and its implementation in R, the ggplot2 package. While other plotting systems set aside a different function for each sort of visualization (histogram, scatterplot, and so on), the grammar of graphics takes the view that the most fundamental characteristics of a plot are the variables it includes and the roles they play (x-axis, y-axis, color, etc). The specific brand of plot is secondary and can be specified later.
To create a visualization, we need to specify three things:

what data is being used.
how the variables are mapped onto the plot (in this case, onto the x- and y-axes).
which geometry should be used to represent the data (scatter plot, line graph, or something else entirely).

Design considerations like axis labels, color palette, and font, can be specified separately after the fact
Notice how all of these elements appear, in order, in the following code chunk.

library(tidyverse)

ggplot(penguins, 
       aes(x = bill_len, y = body_mass)) + 
  geom_point() +
  labs(x = "Bill length (mm)",
       y = "Body mass (g)") +
  theme_minimal()








 



 


Content to support my course, Data Science with R
Andrew Gard — Mon, 10 Nov 2025 06:00:00 GMT
 










Data manipulation with R:
 
Getting started with ggplot:
 
Single-variable plots with ggplot2:
 
Quantitative vs categorical plots with ggplot2:
 
Introducing the pipe:
 
Grouped summaries in R:
 
Grouped filters and mutates
 
Tidying data with pivot_longer:
 
Tidying data with pivot_wider:
 
Mutating joins:
 
Filtering joins
 
Working with dates and times
  
Working with strings
 
Working with factors
  




 


What is Quarto?
Andrew Gard — Mon, 27 Oct 2025 05:00:00 GMT
 




Quarto lets you write and save code while also adding text, images, and other content to contextualize what you’re doing. It makes it easy to create attractive public-facing versions of your data work. My advice is to always work inside of a Quarto document when building your R skills.


 



 


Getting started with R
Andrew Gard — Mon, 13 Oct 2025 05:00:00 GMT
 




I’ve decided to start updating my introductory-level content, much of which was recorded early in the 2020 pandemic. When completed, I expect to have a complete course in data science with R, roughly paralleling what you might find in a sophomore-level university course for students with some statistical knowledge but not necessarily and programming experience.






 


How to set up RStudio for success
Andrew Gard — Tue, 30 Sep 2025 05:00:00 GMT
 




By making a few changes to the RStuio IDE the first time you open it, you can both improve your quality of life and enforce some important data science best practices right from the start. My three most important recommendations:

Put the console pane on the right
Set .RData to automatically clear after every session
Work inside of a project






 


Why haven’t I covered tidymodels?
Andrew Gard — Mon, 22 Sep 2025 05:00:00 GMT
 




The tidymodels framework for machine learning is powerful and about as user-friendly as we can hope for in that space, bringing consistency and clarity to a process that can continue to feel ad-hoc well into the learning curve. Over time, it’s supplanted the older favorite meta-package, carat, which coincidentally (note sarcasm) was written by one the primary tidymodels architects, Max Kuhn.
I love tidymodels and use it consistently for my machine learning needs. Yet when I’ve done vids on ML techniques, I’ve almost always shied away, opting to use individual packages instead. Why?
The short answer is that I haven’t figured out how to teach it in a YouTube-friendly way yet. Vids on Equitable Equations aim to be short, self-contained, clear expositions of individual topics. While these can (and should!) be chained together to build a comprehensive whole, it’s important to me - and to the Almighty Algorithm, whom we all serve - that they stand on their own. Machine learning is hard to do this way.
One of the wonderful things about the core tidyverse is the degree to which it teaches basic data science concepts. When we learn ggplot2, we learn the grammar of graphics. When we learn tidyr, we learn about tidy data and pivoting. As we move deeper into the era of automated coding, this structure for gaining conceptual knowledge is invaluable.
But tidymodels is different. It’s a higher-order system designed to unify a large number of related tasks, some of which have very little meaning outside of a machine learning workflow.
If you’re sitting down to study logistic regression for the first time, for instance, tidymodels will simultaneously want you to consider a data split, preprocessing steps, and performance metrics, to start. This is wonderful and important, ultimately, but puts a huge burden on the learner, not to mention the YouTuber.
Over time, I’ve covered many components of the ML process in my vids, keeping them as separate and simple as possible. This has often come at the expense of data science best practices, for instance when I build models without first splitting data and conclude without computing performance metrics on an independent set. I’ve always felt that logistic regression (to name one technique) is hard enough to learn on its own without simultaneously tackling cross-validation (to name another). It hasn’t been a perfect choice, but it’s the one I’ve made, and I think my reasons are valid.
So, the TL/DR is that tidymodels is a pedagogical challenge. I’ve spent a lot of time wrestling with it, and though I’m not done yet, I hope to be soon. With any luck, you can expect a tidymodels series some time in the fall of 2026.



 


Discrete probability calculations in R
Andrew Gard — Mon, 15 Sep 2025 05:00:00 GMT
 




In R, functions that perform probabilities calculations follow a consistent structure:

Probability mass functions (pmfs) start with the letter “d”. Use such a function to compute  for any  of interest.
Cumulative distribution functions (cdfs) start with the letter “p”. Use such a function to compute .
Inverse cdfs start with the letter “q”. For a given probability , they determine the least  such that .
Functions to generate random values start with the letter “r”.

The rest of the function name is a shortened version of the distribution name:










Distribution
Probability
Cumulative
Inverse
Random




Bin
dbinom()
pbinom()
qbinom()
rbinom()


Geom
dgeom()
pgeom()
qgeom()
rgeom()


NB
dnbinom()
pnbinom()
qnbinom()
rnbinom()


Hyp
dhyper()
phyper()
qhyper()
rhyper()


Pois
dpois()
ppois()
qpois()
rpois()



In case some of these distributions aren’t familiar to you:

The binomial distribution models the total number of success in  Bernouilli trials (identical and independent trials, each with only two possibly outcomes). The probability of success in each trial is .
The geometric distribution models the number of failures in a sequence of Bernouilli trials prior to the first success.
The negative binomial distribution models the number of failures in a sequence of Bernouilli trials prior to the  success.
The hypergeometric distribution models the number of successes in a sample of size  when selection is done without replacement from a pool that includes  successes and  failures.

For instance, imagine a student guessing randomly on a 12-question quiz where each question has 4 possible answers. The probability that they get exactly 3 questions right is computed using dbinom().

dbinom(3, 12, .25)

[1] 0.2581036


Similarly, the probability that they get at most 3 questions right is given by

pbinom(3, 12, .25)

[1] 0.6487786


If you need the probability that they get at least 3 questions right, add a lower.tail argument, like so:

pbinom(3, 12, .25, lower.tail = FALSE)

[1] 0.3512214


qbinom() reverses the logic of pbinom(), finding the number of successes with a specified cumulative probability. Said differently, qbinom() identifies percentiles. For instance, we might wonder how many questions the child would have to get right to do at least as well as 80% of randomly-guessing quiz takers.

qbinom(.80, 12, .25)

[1] 4


Notice how the arguments of these functions follow a common pattern: first the value/percentile of interest, then the parameters, which come in a consistent order. This pattern continues to hold for rbinom(), which generates a specified number of random values

set.seed(0) # For reproducibility

rbinom(10, 12, .25)

 [1] 5 2 2 3 5 2 5 5 4 3


All of R’s probability functions are vectorized, meaning that they can perform multiple calculations at once. For instance, the next line computes the probability of obtaining exactly  Butterfingers when randomly grabbing 5 candy bars from a bowl with 8 Butterfingers and 10 Snickers. Note that .

dhyper(0:5, 8, 10, 5)

[1] 0.029411765 0.196078431 0.392156863 0.294117647 0.081699346 0.006535948


As you’d expect, these probabilities all add up to one.
Some caution is needed when working with the geometric and negative binomial distributions, both of which consider the number of failures occurring before a fixed number of successes. Some textbooks and computational tools define these distributions in terms on the total number of trials needed rather than the total number of failures, so some translation may be needed. Fortunately,
 so the two formulations differ only in the addition or subtraction of the parameter .
Example. A basketball player makes 85% of their free throws. If they shoot until they make 5 shots, what is the probability that 7 shots are needed?
Solution. Making 5 out of 7 shots requires exactly 2 misses, so the probability is

dnbinom(2, 5, .85)

[1] 0.1497505


You might consider trying the next problem on your own before reading my solution.
Example. Every day, a new author has a 10% chance of selling a copy of their ebook. Assume that purchases are independent of one another and that there are never multiple sales in a single day.

Find the probability that the first purchase occurs on the 10 day.
Find the probability that the second purchase occurs during the second week (days 8-14).

Solution. Let  be the number of days that pass before the first purchase and let  be the number of purchase-less days before the second. Then  has a geometric distribution and  has a negative binomial distribution. In each case, .

If 10 trials are needed for a single success, then 9 failures must have occurred, so we compute  dgeom(9, .10) = 0.038742
If between 8 and 14 days are needed for 2 sales, then between 6 and 12 days must have occurred without sales.  pnbinom(12, 2, .10) - pnbinom(5, 2, .10) = 0.2656765

Prefer to have this all clearly explained in a short vid? I’ve got you covered:



This post will be cross-listed on R-bloggers, a site that I totally recommend.



 


Writing prompts using interpolate()
Andrew Gard — Tue, 02 Sep 2025 05:00:00 GMT
 




This week, I’m writing a function that generates a short help file for any stats concept the user specifies. While prompting ChatGPT to do this isn’t particularly hard, the process spotlights some important and interesting aspects of embedding such prompts inside of functions. I had a great time learning about interpolate(), in particular.





 


Teaching an AI to webscrape using the ellmer package
Andrew Gard — Mon, 11 Aug 2025 05:00:00 GMT
 










 


All about system prompts
Andrew Gard — Mon, 28 Jul 2025 05:00:00 GMT
 




I’m generally ambivalent about large language models, but that doesn’t mean I’m going to bury my head in the sand about them. This week I learned how to assign Chat a specific role, require that it respond in the style (here, tidyverse) that I want, and place limitations on its behavior. All of this isn’t generally possible in a browser-based interface, which sent me even deeper into the ellmer package. It’s actually more fun when you understand how these things work.
As usual, code and other supporting materials are available below the vid.


 



 


Advanced counting
Andrew Gard — Fri, 25 Jul 2025 05:00:00 GMT
 




The phrase “advanced counting” is bringing me an unreasonable amount of joy today. No, I don’t just mean listing really big whole numbers (though actually that could be pretty cool, too). The newest upload to my stats-focused channel, Normal Knowledge, features 4 challenging enumeration problems, including this nugget: how many distinct rolls of 12 dice are possible when the order of the rolls doesn’t matter? Listen, the world is in a state, and I’m finding joy wherever I can.





 


How to show a ggplot to an AI assistant
Andrew Gard — Mon, 14 Jul 2025 05:00:00 GMT
 









 


Can using an LLM for coding be ethical?
Andrew Gard — Mon, 07 Jul 2025 05:00:00 GMT
 




Arguably, coding assistance is the most ethical (least unethical?) way to use a large language model.





 


Chatting with Claude AI in RStudio using ellmer
Andrew Gard — Mon, 30 Jun 2025 05:00:00 GMT
 









 


All about facets
Andrew Gard — Tue, 27 May 2025 05:00:00 GMT

Distribution	Probability	Cumulative	Inverse	Random
Bin	dbinom()	pbinom()	qbinom()	rbinom()
Geom	dgeom()	pgeom()	qgeom()	rgeom()
NB	dnbinom()	pnbinom()	qnbinom()	rnbinom()
Hyp	dhyper()	phyper()	qhyper()	rhyper()
Pois	dpois()	ppois()	qpois()	rpois()

Andrew Gard, Ph.D

Stabilizing regression coefficients with the lasso

Cross validation

How to tidymodels

The College data set

Splitting the data

Specifying the model

Preprocessing

Creating a workflow

Putting a trained workflow into action

Model tuning and cross validation

Evaluation using the test set

Final thought

Working with factors in R

Factor, character, and other vector types

Relabeling factors

Rearranging levels

Reordering levels using a second variable

Ordered factors

Want more?

Regular expressions for R users

Special characters

Position and relative position

Character sets

Character classes

The role of “\” in Quarto

Matching whitespace

Repetition

Combining substrings with |

Want more?

Understanding the grammar of graphics

Content to support my course, Data Science with R

What is Quarto?

Getting started with R

How to set up RStudio for success

Why haven’t I covered tidymodels?

Discrete probability calculations in R

Writing prompts using interpolate()

Teaching an AI to webscrape using the ellmer package

All about system prompts

Advanced counting

How to show a ggplot to an AI assistant

Can using an LLM for coding be ethical?

Chatting with Claude AI in RStudio using ellmer

All about facets

The `College` data set

Combining substrings with `|`