tidyverse: arrange, slice, group_by and summarize

Author

Florian Geier

Published

October 7, 2026

Introduction

In this lecture we will continue on the use of tidyverse for exploratory data analysis. We will focus on:

  • ways of ordering data sets

  • combine slicing and ordering to identify extreme cases

  • group data sets by variables

  • perform operations on data groups including group summaries

TipExercise

Create a script (a regular .R script, an .Rmd R Markdown file, or a *.qmd Quarto file) that will be used to save all the code that we will write today.

We first load tidyverse, which is a package bundle that will load several specialized packages for data analysis.

library(tidyverse)
── Attaching core tidyverse packages ─────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.2.1     ✔ readr     2.2.0
✔ forcats   1.0.1     ✔ stringr   1.6.0
✔ ggplot2   4.0.3     ✔ tibble    3.3.1
✔ lubridate 1.9.5     ✔ tidyr     1.3.2
✔ purrr     1.2.2     
── Conflicts ───────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

We use the Palmer Penguins data set from the previous lectures. Assuming the location of the data file is in the data sub-directory of your current working directory (data/penguins.csv), we can load it with:

dat <- readr::read_csv("data/penguins.csv")
Rows: 344 Columns: 8
── Column specification ───────────────────────────────────────────────────
Delimiter: ","
chr (3): species, island, sex
dbl (5): bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g,...

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

Short recap

The previous lecture introduced several commands to subset and modify the data. We will shortly recapitulate some of them by examples.

Example 1

  • What data types are in our data set?
str(dat)
spc_tbl_ [344 × 8] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
 $ species          : chr [1:344] "Adelie" "Adelie" "Adelie" "Adelie" ...
 $ island           : chr [1:344] "Torgersen" "Torgersen" "Torgersen" "Torgersen" ...
 $ bill_length_mm   : num [1:344] 39.1 39.5 40.3 NA 36.7 39.3 38.9 39.2 34.1 42 ...
 $ bill_depth_mm    : num [1:344] 18.7 17.4 18 NA 19.3 20.6 17.8 19.6 18.1 20.2 ...
 $ flipper_length_mm: num [1:344] 181 186 195 NA 193 190 181 195 193 190 ...
 $ body_mass_g      : num [1:344] 3750 3800 3250 NA 3450 ...
 $ sex              : chr [1:344] "male" "female" "female" NA ...
 $ year             : num [1:344] 2007 2007 2007 2007 2007 ...
 - attr(*, "spec")=
  .. cols(
  ..   species = col_character(),
  ..   island = col_character(),
  ..   bill_length_mm = col_double(),
  ..   bill_depth_mm = col_double(),
  ..   flipper_length_mm = col_double(),
  ..   body_mass_g = col_double(),
  ..   sex = col_character(),
  ..   year = col_double()
  .. )
 - attr(*, "problems")=<pointer: 0x55c78d4af1d0> 

Example 2

  • In which years are penguins observed?
years <- select(dat, year)
distinct(years)
# A tibble: 3 × 1
   year
  <dbl>
1  2007
2  2008
3  2009

or more conveniently using the pipe-operator (|>):

dat |>
  select(year) |>
  distinct()
# A tibble: 3 × 1
   year
  <dbl>
1  2007
2  2008
3  2009

Example 3

  • Select the first 3 cases observed in 2007.
dat |>
  filter(year == 2007) |>
  slice(1:3)
# A tibble: 3 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Adelie  Torge…           39.1          18.7               181        3750
2 Adelie  Torge…           39.5          17.4               186        3800
3 Adelie  Torge…           40.3          18                 195        3250
# ℹ 2 more variables: sex <chr>, year <dbl>

Example 4

  • Which animals are “exceptionally” big? We call exceptional, if an animal is bigger than the average body mass plus two times its standard deviation:
dat |>
  drop_na(body_mass_g) |> 
  mutate(big = body_mass_g > mean(body_mass_g) + 2*sd(body_mass_g)) |>
  filter(big) 
# A tibble: 9 × 9
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Gentoo  Biscoe           48.4          14.6               213        5850
2 Gentoo  Biscoe           49.3          15.7               217        5850
3 Gentoo  Biscoe           49.2          15.2               221        6300
4 Gentoo  Biscoe           59.6          17                 230        6050
5 Gentoo  Biscoe           51.1          16.3               220        6000
6 Gentoo  Biscoe           45.2          16.4               223        5950
7 Gentoo  Biscoe           49.8          15.9               229        5950
8 Gentoo  Biscoe           55.1          16                 230        5850
9 Gentoo  Biscoe           48.8          16.2               222        6000
# ℹ 3 more variables: sex <chr>, year <dbl>, big <lgl>

Example 5

  • How many species live on each island?
dat |>
  count(species, island)
# A tibble: 5 × 3
  species   island        n
  <chr>     <chr>     <int>
1 Adelie    Biscoe       44
2 Adelie    Dream        56
3 Adelie    Torgersen    52
4 Chinstrap Dream        68
5 Gentoo    Biscoe      124

count() is an example for a common analysis procedure: first group data into subsets, then calculate summaries of each subset. The way this can be done in tidyverse is one of today’s main topics. But first we will turn to ways of ordering data sets.

Order data sets with arrange()

Why should we care about ordering data sets?

  • Nicer data presentation
  • Find a particular data point
  • Check minimum or maximum values
  • Spot outliers or extreme values
  • Indicate trends/correlations among variables
  • …
TipExercise

Name an ordering criteria for the following data types:

  1. Integer
  2. Character
  3. Factor (=categorical data)
# 1. Size
# 2. Lexicographic order
# 3. Order of categories

The arrange() command orders the rows of our tibble according to some selected variable(s). Lets define a small data summary in order to highlight a few use cases for arrange().

sub_dat <- dat |>
  drop_na(sex) |>
  count(island, species, sex)
sub_dat
# A tibble: 10 × 4
   island    species   sex        n
   <chr>     <chr>     <chr>  <int>
 1 Biscoe    Adelie    female    22
 2 Biscoe    Adelie    male      22
 3 Biscoe    Gentoo    female    58
 4 Biscoe    Gentoo    male      61
 5 Dream     Adelie    female    27
 6 Dream     Adelie    male      28
 7 Dream     Chinstrap female    34
 8 Dream     Chinstrap male      34
 9 Torgersen Adelie    female    24
10 Torgersen Adelie    male      23

We can arrange the data by animal count (n) with:

sub_dat |>
  arrange(n)
# A tibble: 10 × 4
   island    species   sex        n
   <chr>     <chr>     <chr>  <int>
 1 Biscoe    Adelie    female    22
 2 Biscoe    Adelie    male      22
 3 Torgersen Adelie    male      23
 4 Torgersen Adelie    female    24
 5 Dream     Adelie    female    27
 6 Dream     Adelie    male      28
 7 Dream     Chinstrap female    34
 8 Dream     Chinstrap male      34
 9 Biscoe    Gentoo    female    58
10 Biscoe    Gentoo    male      61

By default numerical variables are sorted from smallest to largest value. Sorting in descending order is achieved by wrapping the sort variable with desc():

sub_dat |>
  arrange(desc(n))
# A tibble: 10 × 4
   island    species   sex        n
   <chr>     <chr>     <chr>  <int>
 1 Biscoe    Gentoo    male      61
 2 Biscoe    Gentoo    female    58
 3 Dream     Chinstrap female    34
 4 Dream     Chinstrap male      34
 5 Dream     Adelie    male      28
 6 Dream     Adelie    female    27
 7 Torgersen Adelie    female    24
 8 Torgersen Adelie    male      23
 9 Biscoe    Adelie    female    22
10 Biscoe    Adelie    male      22

Its possible to arrange the data by several variables at once:

sub_dat |>
  arrange(island, species, desc(n))
# A tibble: 10 × 4
   island    species   sex        n
   <chr>     <chr>     <chr>  <int>
 1 Biscoe    Adelie    female    22
 2 Biscoe    Adelie    male      22
 3 Biscoe    Gentoo    male      61
 4 Biscoe    Gentoo    female    58
 5 Dream     Adelie    male      28
 6 Dream     Adelie    female    27
 7 Dream     Chinstrap female    34
 8 Dream     Chinstrap male      34
 9 Torgersen Adelie    female    24
10 Torgersen Adelie    male      23

In the example, the data is first sorted alphabetically on island, within the same island on species, and within each island-species combination on count in descending order.

Sometimes we would like to arrange categorical data in a particular order. Categorical variables are best represented as factors in R where the order of categories (=levels) can be freely chosen. This allows to arrange the data in the order of the categories.

fact_dat <- sub_dat |>
  mutate(species = factor(species, levels = c('Chinstrap','Adelie','Gentoo'))) 
str(fact_dat)
tibble [10 × 4] (S3: tbl_df/tbl/data.frame)
 $ island : chr [1:10] "Biscoe" "Biscoe" "Biscoe" "Biscoe" ...
 $ species: Factor w/ 3 levels "Chinstrap","Adelie",..: 2 2 3 3 2 2 1 1 2 2
 $ sex    : chr [1:10] "female" "male" "female" "male" ...
 $ n      : int [1:10] 22 22 58 61 27 28 34 34 24 23
fact_dat |>
  arrange(species)
# A tibble: 10 × 4
   island    species   sex        n
   <chr>     <fct>     <chr>  <int>
 1 Dream     Chinstrap female    34
 2 Dream     Chinstrap male      34
 3 Biscoe    Adelie    female    22
 4 Biscoe    Adelie    male      22
 5 Dream     Adelie    female    27
 6 Dream     Adelie    male      28
 7 Torgersen Adelie    female    24
 8 Torgersen Adelie    male      23
 9 Biscoe    Gentoo    female    58
10 Biscoe    Gentoo    male      61
TipExercise

Arrange sub_dat by species and with in each species by the following order of islands: 1. Dream, 2. Torgersen, 3. Biscoe.

sub_dat |>
  mutate(island = factor(island, levels = c('Dream', 'Torgersen', 'Biscoe'))) |>
  arrange(species, island)
# A tibble: 10 × 4
   island    species   sex        n
   <fct>     <chr>     <chr>  <int>
 1 Dream     Adelie    female    27
 2 Dream     Adelie    male      28
 3 Torgersen Adelie    female    24
 4 Torgersen Adelie    male      23
 5 Biscoe    Adelie    female    22
 6 Biscoe    Adelie    male      22
 7 Dream     Chinstrap female    34
 8 Dream     Chinstrap male      34
 9 Biscoe    Gentoo    female    58
10 Biscoe    Gentoo    male      61
TipExercise

Use arrange() and slice() to get the male animal from Biscoe island with the largest flipper length.

dat |>
  arrange(island, desc(sex), desc(flipper_length_mm)) |>
  slice(1)
# A tibble: 1 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Gentoo  Biscoe           54.3          15.7               231        5650
# ℹ 2 more variables: sex <chr>, year <dbl>

More on slicing data

The slice() command selects rows by index, but there are more variants of this command. First, slice_head() and slice_tail() lets you select the first or last cases of the data, respectively. Here we select the first two and last two:

dat |>
  slice_head(n = 2)
# A tibble: 2 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Adelie  Torge…           39.1          18.7               181        3750
2 Adelie  Torge…           39.5          17.4               186        3800
# ℹ 2 more variables: sex <chr>, year <dbl>
dat |> 
  slice_tail(n = 2)
# A tibble: 2 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Chinst… Dream            50.8          19                 210        4100
2 Chinst… Dream            50.2          18.7               198        3775
# ℹ 2 more variables: sex <chr>, year <dbl>

Two more slice variants aim at selecting rows with the smallest or largest value for a given variable. Here we select cases with the smallest and the largest bill length, respectively.

dat |>
  slice_min(bill_length_mm)
# A tibble: 1 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Adelie  Dream            32.1          15.5               188        3050
# ℹ 2 more variables: sex <chr>, year <dbl>
dat |>
  slice_max(bill_length_mm)
# A tibble: 1 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Gentoo  Biscoe           59.6            17               230        6050
# ℹ 2 more variables: sex <chr>, year <dbl>

These versions of slice are similar to a combination of arrange() and slice_head() or slice_tail().

dat |>
  arrange(desc(bill_length_mm)) |>    
  slice_head()
# A tibble: 1 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Gentoo  Biscoe           59.6            17               230        6050
# ℹ 2 more variables: sex <chr>, year <dbl>
Caution

If the ordering variable contains missing values (NA), these are ordered at the end.

dat |>
  arrange(bill_length_mm) |>    
  slice_tail()
# A tibble: 1 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Gentoo  Biscoe             NA            NA                NA          NA
# ℹ 2 more variables: sex <chr>, year <dbl>
TipExercise

Explain why these two expressions return different results!

dat |>
  arrange(year) |>    
  slice_head()
# A tibble: 1 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Adelie  Torge…           39.1          18.7               181        3750
# ℹ 2 more variables: sex <chr>, year <dbl>
dat |>
  slice_min(year)
# A tibble: 110 × 8
   species island    bill_length_mm bill_depth_mm flipper_length_mm
   <chr>   <chr>              <dbl>         <dbl>             <dbl>
 1 Adelie  Torgersen           39.1          18.7               181
 2 Adelie  Torgersen           39.5          17.4               186
 3 Adelie  Torgersen           40.3          18                 195
 4 Adelie  Torgersen           NA            NA                  NA
 5 Adelie  Torgersen           36.7          19.3               193
 6 Adelie  Torgersen           39.3          20.6               190
 7 Adelie  Torgersen           38.9          17.8               181
 8 Adelie  Torgersen           39.2          19.6               195
 9 Adelie  Torgersen           34.1          18.1               193
10 Adelie  Torgersen           42            20.2               190
# ℹ 100 more rows
# ℹ 3 more variables: body_mass_g <dbl>, sex <chr>, year <dbl>
# `slice_min()` and `slice_max()` may return more rows than requested in the presence of ties.
TipExercise

From the Adelie penguins select the case(s) with the smallest body mass. How many are there?

dat |>
  filter(species == "Adelie") |>
  slice_min(body_mass_g)
# A tibble: 2 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Adelie  Biscoe           36.5          16.6               181        2850
2 Adelie  Biscoe           36.4          17.1               184        2850
# ℹ 2 more variables: sex <chr>, year <dbl>

Analysis on grouped data

Consider the following set of questions:

  • What is the minimum observed body mass in male penguins?
  • What is the maximum flipper length per species?
  • How many penguins were measured per species on each island?
  • How much heavier or lighter is each penguin than the average of its species?
TipExercise

Which of these questions rely on splitting the data into groups?

# 1. Just filtering
# 2. Analysis per species
# 3. Analysis per species-island combination
# 4. Analysis per case, with analysis per species
TipExercise

Which of these questions above ask for:

  • a result per observation

  • a result per group of observations

# Only the last one asks for a result per observation!

Define data subsets with group_by()

Grouping of our data is done with the group_by() command which accepts a (set of) grouping variable(s):

dat |>
  group_by(species)
# A tibble: 344 × 8
# Groups:   species [3]
   species island    bill_length_mm bill_depth_mm flipper_length_mm
   <chr>   <chr>              <dbl>         <dbl>             <dbl>
 1 Adelie  Torgersen           39.1          18.7               181
 2 Adelie  Torgersen           39.5          17.4               186
 3 Adelie  Torgersen           40.3          18                 195
 4 Adelie  Torgersen           NA            NA                  NA
 5 Adelie  Torgersen           36.7          19.3               193
 6 Adelie  Torgersen           39.3          20.6               190
 7 Adelie  Torgersen           38.9          17.8               181
 8 Adelie  Torgersen           39.2          19.6               195
 9 Adelie  Torgersen           34.1          18.1               193
10 Adelie  Torgersen           42            20.2               190
# ℹ 334 more rows
# ℹ 3 more variables: body_mass_g <dbl>, sex <chr>, year <dbl>

group_by() doesn’t perform any computation, it just returns the data grouped into the subsets defined by the grouping variable, in our case its by species.

We can also group our data by several variables, e.g. species and island:

dat |>
  group_by(species, island)
# A tibble: 344 × 8
# Groups:   species, island [5]
   species island    bill_length_mm bill_depth_mm flipper_length_mm
   <chr>   <chr>              <dbl>         <dbl>             <dbl>
 1 Adelie  Torgersen           39.1          18.7               181
 2 Adelie  Torgersen           39.5          17.4               186
 3 Adelie  Torgersen           40.3          18                 195
 4 Adelie  Torgersen           NA            NA                  NA
 5 Adelie  Torgersen           36.7          19.3               193
 6 Adelie  Torgersen           39.3          20.6               190
 7 Adelie  Torgersen           38.9          17.8               181
 8 Adelie  Torgersen           39.2          19.6               195
 9 Adelie  Torgersen           34.1          18.1               193
10 Adelie  Torgersen           42            20.2               190
# ℹ 334 more rows
# ℹ 3 more variables: body_mass_g <dbl>, sex <chr>, year <dbl>

Note, there are 5 groups in total, i.e., the combination of island and species led to 5 distinct subsets. Combinations that don’t appear in the data such as the Gentoo penguins on Torgersen island are dropped by default.

TipExercise

Group the data by species and island. What do the following functions tell you about the data grouping?

  • group_keys()

  • group_indices()

grp_dat <- dat |>
  group_by(species, island)

grp_dat |> group_keys()
# A tibble: 5 × 2
  species   island   
  <chr>     <chr>    
1 Adelie    Biscoe   
2 Adelie    Dream    
3 Adelie    Torgersen
4 Chinstrap Dream    
5 Gentoo    Biscoe   
grp_dat |> group_indices()
  [1] 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2
 [36] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3 3
 [71] 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1
[106] 1 1 1 1 1 1 1 1 1 1 1 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2
[141] 2 2 2 2 2 2 2 2 2 2 2 2 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
[176] 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
[211] 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
[246] 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 4 4 4 4
[281] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
[316] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
TipExercise

What happens in the following case? Carefully check the returned tibble!

dat |>
  group_by(species) |>
  group_by(island)
# By default any existing grouping will be overwritten by a new `group_by()` command.

Operations on data subsets

The important difference of a grouped tibble to an un-grouped tibble is that subsequent operations are applied to the data groups (when applicable) instead of the whole data set. Compare the results of these two pipelines, first un-grouped, then grouped:

dat |>
  slice_min(bill_length_mm)
# A tibble: 1 × 8
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Adelie  Dream            32.1          15.5               188        3050
# ℹ 2 more variables: sex <chr>, year <dbl>
dat |>
  group_by(species) |>
  slice_min(bill_length_mm)
# A tibble: 3 × 8
# Groups:   species [3]
  species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <chr>   <chr>           <dbl>         <dbl>             <dbl>       <dbl>
1 Adelie  Dream            32.1          15.5               188        3050
2 Chinst… Dream            40.9          16.6               187        3200
3 Gentoo  Biscoe           40.9          13.7               214        4650
# ℹ 2 more variables: sex <chr>, year <dbl>

The grouped case returns the minimum bill length cases per species, instead of the overall dataset minimum!

Lets use the feature of grouping and mutate() to answer one of the above questions:

  • How much heavier or lighter is each penguin than the average of its species?
dat |>
  group_by(species) |>
  mutate(avg = mean(body_mass_g, na.rm = TRUE),
         diff = body_mass_g - avg) |>
  ungroup() |>
  select(-c(bill_length_mm, bill_depth_mm, flipper_length_mm)) # focus output
# A tibble: 344 × 7
   species island    body_mass_g sex     year   avg   diff
   <chr>   <chr>           <dbl> <chr>  <dbl> <dbl>  <dbl>
 1 Adelie  Torgersen        3750 male    2007 3701.   49.3
 2 Adelie  Torgersen        3800 female  2007 3701.   99.3
 3 Adelie  Torgersen        3250 female  2007 3701. -451. 
 4 Adelie  Torgersen          NA <NA>    2007 3701.   NA  
 5 Adelie  Torgersen        3450 female  2007 3701. -251. 
 6 Adelie  Torgersen        3650 male    2007 3701.  -50.7
 7 Adelie  Torgersen        3625 female  2007 3701.  -75.7
 8 Adelie  Torgersen        4675 male    2007 3701.  974. 
 9 Adelie  Torgersen        3475 <NA>    2007 3701. -226. 
10 Adelie  Torgersen        4250 <NA>    2007 3701.  549. 
# ℹ 334 more rows

As you can see, mutate() returns a result for each case, however the grouping allows us to calculate the average per species.

Lets use grouping to calculate the number of cases for each species on each island.

dat |>
  group_by(species, island) |>
  mutate(n = n()) |>
  select(species, island, n)
# A tibble: 344 × 3
# Groups:   species, island [5]
   species island        n
   <chr>   <chr>     <int>
 1 Adelie  Torgersen    52
 2 Adelie  Torgersen    52
 3 Adelie  Torgersen    52
 4 Adelie  Torgersen    52
 5 Adelie  Torgersen    52
 6 Adelie  Torgersen    52
 7 Adelie  Torgersen    52
 8 Adelie  Torgersen    52
 9 Adelie  Torgersen    52
10 Adelie  Torgersen    52
# ℹ 334 more rows

The results is redundant, as expected from using mutate(). However, if our question was:

  • How many penguins were measured per species on each island?

we ask for a non-redundant summary per species/island combination.

TipExercise

Can you modify the example such that we get a non-redundant summary? Hint: make use of the command distinct() introduced in the previous lecture.

dat |>
  group_by(species, island) |>
  mutate(n = n()) |>
  distinct(species, island, n)
# A tibble: 5 × 3
# Groups:   species, island [5]
  species   island        n
  <chr>     <chr>     <int>
1 Adelie    Torgersen    52
2 Adelie    Biscoe       44
3 Adelie    Dream        56
4 Gentoo    Biscoe      124
5 Chinstrap Dream        68

While the above example gives a valid summary of counts, for the purpose of building general data summaries there is a more elegant way using summarize().

Calculate data summaries with summarize()

Summarize will return one row for each group or combination of groups, adding the requested summary. Thus, summarize() is helpful in all cases where you ask for a single number per group.

TipExercise

Which of these functions are problematic when using them to calculate data summaries?

  • n()
  • log()
  • sum()
  • round()
# log() and round() are problematic as they don't summarize their input into one output number

Lets replicate the example on animal counts per species and island using summarize():

dat |>
  group_by(species, island) |>
  summarize(n = n())
`summarise()` has regrouped the output.
ℹ Summaries were computed grouped by species and island.
ℹ Output is grouped by species.
ℹ Use `summarise(.groups = "drop_last")` to silence this message.
ℹ Use `summarise(.by = c(species, island))` for per-operation grouping
  (`?dplyr::dplyr_by`) instead.
# A tibble: 5 × 3
# Groups:   species [3]
  species   island        n
  <chr>     <chr>     <int>
1 Adelie    Biscoe       44
2 Adelie    Dream        56
3 Adelie    Torgersen    52
4 Chinstrap Dream        68
5 Gentoo    Biscoe      124
Caution

summarize() simplifies the output in several ways:

  1. All variables un-related to the grouping and the summary are dropped.
  2. summarize() returns a reduce data set with one row per group. This “modification” is in contrast to mutate() that will preserve all rows.
  3. Finally, the inner most group (island in our case) is dropped from the grouping.

Note that count() from the introduction is basically a convenience function for the above approach, with the difference of returning an ungrouped tibble.

TipExercise

Calculate the mean and standard deviation (sd) of the flipper length for each combination of species and sex. Add a score that measures mean over sd.

dat |>
  drop_na(sex) |>
  group_by(species, sex) |>
  summarize(avg_flipper_len = mean(flipper_length_mm),
            sd_flipper_len = sd(flipper_length_mm),
            score = avg_flipper_len/sd_flipper_len) 
`summarise()` has regrouped the output.
ℹ Summaries were computed grouped by species and sex.
ℹ Output is grouped by species.
ℹ Use `summarise(.groups = "drop_last")` to silence this message.
ℹ Use `summarise(.by = c(species, sex))` for per-operation grouping
  (`?dplyr::dplyr_by`) instead.
# A tibble: 6 × 5
# Groups:   species [3]
  species   sex    avg_flipper_len sd_flipper_len score
  <chr>     <chr>            <dbl>          <dbl> <dbl>
1 Adelie    female            188.           5.60  33.6
2 Adelie    male              192.           6.60  29.2
3 Chinstrap female            192.           5.75  33.3
4 Chinstrap male              200.           5.98  33.4
5 Gentoo    female            213.           3.90  54.6
6 Gentoo    male              222.           5.67  39.1
TipExercise

Using the n() and sum() commands we want to calculate the average body mass per species using the following code:

dat |>
  group_by(species) |>
  summarize(avg = sum(body_mass_g, na.rm = TRUE)/n())
# A tibble: 3 × 2
  species     avg
  <chr>     <dbl>
1 Adelie    3676.
2 Chinstrap 3733.
3 Gentoo    5035.

Why does this workflow result differ? Which one is correct?

dat |>
  group_by(species) |>
  summarize(mean = mean(body_mass_g, na.rm = TRUE))
# A tibble: 3 × 2
  species    mean
  <chr>     <dbl>
1 Adelie    3701.
2 Chinstrap 3733.
3 Gentoo    5076.
# Using `mean` is correct as it accounts for the correct group sizes.

Each summary call removes one grouping level

As mentioned above, a special feature of the summary() command is that it will “peel-off” one level of the grouping.

grp_dat <- dat |>
  group_by(species, island) 

groups(grp_dat) 
[[1]]
species

[[2]]
island
grp_dat |>
  summarize(n = n()) |>
  groups()
`summarise()` has regrouped the output.
ℹ Summaries were computed grouped by species and island.
ℹ Output is grouped by species.
ℹ Use `summarise(.groups = "drop_last")` to silence this message.
ℹ Use `summarise(.by = c(species, island))` for per-operation grouping
  (`?dplyr::dplyr_by`) instead.
[[1]]
species
grp_dat |> 
  summarize(n = n()) |>
  summarize(n = n()) |>
  groups()
`summarise()` has regrouped the output.
ℹ Summaries were computed grouped by species and island.
ℹ Output is grouped by species.
ℹ Use `summarise(.groups = "drop_last")` to silence this message.
ℹ Use `summarise(.by = c(species, island))` for per-operation grouping
  (`?dplyr::dplyr_by`) instead.
list()

This feature can be useful when calculating nested summaries. For example, we calculate the fraction of species per island by first counting animals per island and species (“deepest” summary) and then summing animals over the remaining group (island).

dat |>
  group_by(island, species) |> # grouped by island and species
  summarize(n = n()) |> # grouped by island
  mutate(frac = n/sum(n)) # grouped by island
`summarise()` has regrouped the output.
ℹ Summaries were computed grouped by island and species.
ℹ Output is grouped by island.
ℹ Use `summarise(.groups = "drop_last")` to silence this message.
ℹ Use `summarise(.by = c(island, species))` for per-operation grouping
  (`?dplyr::dplyr_by`) instead.
# A tibble: 5 × 4
# Groups:   island [3]
  island    species       n  frac
  <chr>     <chr>     <int> <dbl>
1 Biscoe    Adelie       44 0.262
2 Biscoe    Gentoo      124 0.738
3 Dream     Adelie       56 0.452
4 Dream     Chinstrap    68 0.548
5 Torgersen Adelie       52 1    
TipExercise

Calculate the faction of males per species and island. As in the previous exercise, exclude all animals with undetermined sex.

dat |>
  drop_na(sex) |>
  group_by(species, island, sex) |>
  summarize(n = n())  |>
  mutate(frac = n/sum(n))
`summarise()` has regrouped the output.
ℹ Summaries were computed grouped by species, island, and sex.
ℹ Output is grouped by species and island.
ℹ Use `summarise(.groups = "drop_last")` to silence this message.
ℹ Use `summarise(.by = c(species, island, sex))` for per-operation
  grouping (`?dplyr::dplyr_by`) instead.
# A tibble: 10 × 5
# Groups:   species, island [5]
   species   island    sex        n  frac
   <chr>     <chr>     <chr>  <int> <dbl>
 1 Adelie    Biscoe    female    22 0.5  
 2 Adelie    Biscoe    male      22 0.5  
 3 Adelie    Dream     female    27 0.491
 4 Adelie    Dream     male      28 0.509
 5 Adelie    Torgersen female    24 0.511
 6 Adelie    Torgersen male      23 0.489
 7 Chinstrap Dream     female    34 0.5  
 8 Chinstrap Dream     male      34 0.5  
 9 Gentoo    Biscoe    female    58 0.487
10 Gentoo    Biscoe    male      61 0.513
TipExercise

For every species we would like to know how much heavier on average males are with respect to females. Since there are a few undetermined sex case, we want to exclude them. What is the correct analysis workflow? Hint: use diff() in order to calculate successive differences.

dat |>
  drop_na(sex) |>
  group_by(species, sex, .drop = FALSE) |> # so we don't loose sex levels
  summarize(avg_mass_g = mean(body_mass_g)) |>
  summarize(diff_mass_g = diff(avg_mass_g))
`summarise()` has regrouped the output.
ℹ Summaries were computed grouped by species and sex.
ℹ Output is grouped by species.
ℹ Use `summarise(.groups = "drop_last")` to silence this message.
ℹ Use `summarise(.by = c(species, sex))` for per-operation grouping
  (`?dplyr::dplyr_by`) instead.
# A tibble: 3 × 2
  species   diff_mass_g
  <chr>           <dbl>
1 Adelie           675.
2 Chinstrap        412.
3 Gentoo           805.

More information

Session info

R version 4.6.0 (2026-04-24)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 24.04.5 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_DK.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_DK.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_DK.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_DK.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Zurich
tzcode source: system (glibc)

attached base packages:
[1] grid      stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
 [1] lubridate_1.9.5  forcats_1.0.1    stringr_1.6.0    dplyr_1.2.1     
 [5] purrr_1.2.2      readr_2.2.0      tidyr_1.3.2      tibble_3.3.1    
 [9] ggplot2_4.0.3    tidyverse_2.0.0  BiocStyle_2.40.0 png_0.1-9       
[13] knitr_1.51      

loaded via a namespace (and not attached):
 [1] utf8_1.2.6          generics_0.1.4      stringi_1.8.7      
 [4] hms_1.1.4           digest_0.6.39       magrittr_2.0.5     
 [7] evaluate_1.0.5      timechange_0.4.0    RColorBrewer_1.1-3 
[10] fastmap_1.2.0       jsonlite_2.0.0      BiocManager_1.30.27
[13] scales_1.4.0        cli_3.6.6           rlang_1.3.0        
[16] crayon_1.5.3        bit64_4.8.2         withr_3.0.3        
[19] yaml_2.3.12         otel_0.2.0          tools_4.6.0        
[22] parallel_4.6.0      tzdb_0.5.0          vctrs_0.7.3        
[25] R6_2.6.1            lifecycle_1.0.5     htmlwidgets_1.6.4  
[28] bit_4.6.0           vroom_1.7.1         pkgconfig_2.0.3    
[31] pillar_1.11.1       gtable_0.3.6        glue_1.8.1         
[34] xfun_0.60           tidyselect_1.2.1    rstudioapi_0.19.0  
[37] dichromat_2.0-0.1   farver_2.1.2        htmltools_0.5.9    
[40] rmarkdown_2.31      compiler_4.6.0      S7_0.2.2