First things first: Make a new project in RStudio, and be sure to place it somewhere where you can easily find it (e.g., Documents, Desktop). Then download this file (the .qmd file) from the syllabus website and place it in the folder associated with that project. Then open this file in RStudio and follow along.
I encourage you to work with/talk to others throughout this tutorial, especially during times where I have you code on your own and use AI. When you code, think about the SPARCK framework:
S: Set up (What is your end goal?)
P: Pseudocode (What are the steps?)
A: AI (What are your AI prompts?)
R: Run (Run it!)
C: Check (Did your code work?)
K: Knowledge (What does your code mean?)
Agenda
Tidyverse syntax
Data Moves
Quarto
For today’s tutorial, we’re going to be working with this dataset on enrollment in the State University of New York (SUNY) system. This data comes from New York State’s open data portal (https://data.ny.gov/). It contains information on enrollment at different types of SUNY colleges every fall since 2011.
Place the data in the folder you created for this R Project. Load the data into R. (Note: The date at the end of the file name is the date the file was downloaded, so your file name may be slightly different from mine.)
library(tidyverse)
Warning: package 'ggplot2' was built under R version 4.4.3
Warning: package 'tidyr' was built under R version 4.4.3
Warning: package 'purrr' was built under R version 4.4.3
Warning: package 'dplyr' was built under R version 4.4.3
-- Attaching core tidyverse packages ------------------------ tidyverse 2.0.0 --
v dplyr 1.2.0 v readr 2.1.5
v forcats 1.0.0 v stringr 1.5.1
v ggplot2 4.0.2 v tibble 3.2.1
v lubridate 1.9.4 v tidyr 1.3.2
v purrr 1.2.1
-- Conflicts ------------------------------------------ tidyverse_conflicts() --
x dplyr::filter() masks stats::filter()
x dplyr::lag() masks stats::lag()
i Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
Rows: 990 Columns: 8
-- Column specification --------------------------------------------------------
Delimiter: ","
chr (3): Term, College or Institution Type, College or Institution Name
dbl (1): Year
num (4): Undergraduate Full-Time, Undergraduate Part-Time, Graduate Full-Tim...
i Use `spec()` to retrieve the full column specification for this data.
i Specify the column types or set `show_col_types = FALSE` to quiet this message.
You can call your data whatever you’d like, but I will call mine suny.
One other thing to note: When using Gemini, you can upload your data file so that Gemini knows exactly what your data looks like. If you don’t upload it, you’ll have to explain a bit what your data looks like so it knows exactly what you’re trying to do and how to help you. So it’s not a requirement, but it’s useful when trying to get help on how to work with your data specifically.
Tidyverse syntax
We’ve already learned a bit about the “syntax” of R - functions, objects, arguments, etc. One important aspect of the tidyverse package is that it comes with its own syntax that differs slightly from base R. This syntax is a little bit more “logical” and easier to read (i.e. “tidier”), which is part of why it’s so popular.
In this class, you’ll primarily be using tidyverse functions to clean and analyze data.
The pipe operator: |>
Before we start moving data around, we need one of the most important pieces of tidyverse syntax: the pipe operator, |>. The pipe lets you chain several operations together, one after another, so that the output of one step becomes the input of the next. (This section is adapted from Ismay et al., 2025.)
Imagine you have an object x and three functions f(), g(), and h(). You want to:
Take x, then
Use x as the input to f(), then
Use the output of f(x) as the input to g(), then
Use the output of g(f(x)) as the input to h().
Without the pipe, you have to nest the functions inside each other:
h(g(f(x)))
This works, but it’s hard to read: you have to start from the inside and work your way out, which is the opposite of the order we’d describe the steps in.
With the pipe, you can write the same thing like this:
x |>f() |>g() |>h()
Now the code reads top-to-bottom, left-to-right, in the same order as the steps. A good way to read |> out loud is “and then.” “Take x, and then do f(), and then do g(), and then do h().”
A few things to notice:
The pipe takes whatever is on its left and passes it in as the first argument of the function on its right. x |> f() is the same as f(x).
Put each step on its own line, with the |> at the end of the line. R knows the command isn’t finished yet because the line ends in a pipe.
The keyboard shortcut for the pipe is Cmd + Shift + M (Mac) or Ctrl + Shift + M (Windows).
You will often see an older version of the pipe, %>%, especially in code written by AI or found online. For our purposes it does the same thing.
In our case, x will almost always be a data frame, and f(), g(), and h() will be tidyverse functions that each do one thing to that data frame. For example, use the clean_names() function from the janitor package on suny. (When prompting Gemini, it may be useful to explicitly say “using the pipe” or “using tidyverse”.)
library(janitor)
Attaching package: 'janitor'
The following objects are masked from 'package:stats':
chisq.test, fisher.test
suny <- suny |>clean_names()
Warning in FUN(X[[i]], ...): unable to translate '<U+00C4>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00D6>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00DC>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00E4>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00F6>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00FC>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00DF>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00C6>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00E6>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00D8>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00F8>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00C5>' to native encoding
Warning in FUN(X[[i]], ...): unable to translate '<U+00E5>' to native encoding
Notice that we saved the result. If you don’t assign the result with <-, R will print it but won’t save it.
The $ (or lack thereof)
Last class we learned about the $ sign, which we use to index columns in R. But in the tidyverse, we don’t really need it. Because we’re “piping in” our dataframe, the tidyverse functions know what dataframe our columns come from, and so we only need to call the column name.
For example, let’s use the pull() function to display the college names. (Don’t forget that you can start typing the column name and use tab to auto-complete! That will save you a lot of time.)
suny |>pull(college_or_institution_name)
[1] "Adirondack" "Broome" "Cayuga County"
[4] "Clinton" "Columbia-Greene" "Corning"
[7] "Dutchess" "Erie" "Fashion Institute"
[10] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[13] "Herkimer County" "Hudson Valley" "Jamestown"
[16] "Jefferson" "Mohawk Valley" "Monroe"
[19] "Nassau" "Niagara County" "North Country"
[22] "Onondaga" "Orange County" "Rockland"
[25] "Schenectady County" "Suffolk County" "Sullivan County"
[28] "Tompkins Cortland" "Ulster County" "Westchester"
[31] NA "Brockport" "Buffalo State"
[34] "Cortland" "Empire State" "Fredonia"
[37] "Geneseo" "New Paltz" "Old Westbury"
[40] "Oneonta" "Oswego" "Plattsburgh"
[43] "Potsdam" "Purchase" NA
[46] "Albany" "Alfred-Ceramics" "Binghamton"
[49] "Buffalo University" "Cornell Stat" "Downstate Medical"
[52] "ESF" "Optometry" "Stony Brook"
[55] "SUNY Poly" "Upstate Medical" NA
[58] NA "Alfred State" "Canton"
[61] "Cobleskill" "Delhi" "Farmingdale"
[64] "Maritime" "Morrisville" NA
[67] "Adirondack" "Broome" "Cayuga County"
[70] "Clinton" "Columbia-Greene" "Corning"
[73] "Dutchess" "Erie" "Fashion Institute"
[76] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[79] "Herkimer County" "Hudson Valley" "Jamestown"
[82] "Jefferson" "Mohawk Valley" "Monroe"
[85] "Nassau" "Niagara County" "North Country"
[88] "Onondaga" "Orange County" "Rockland"
[91] "Schenectady County" "Suffolk County" "Sullivan County"
[94] "Tompkins Cortland" "Ulster County" "Westchester"
[97] NA "Brockport" "Buffalo State"
[100] "Cortland" "Empire State" "Fredonia"
[103] "Geneseo" "New Paltz" "Old Westbury"
[106] "Oneonta" "Oswego" "Plattsburgh"
[109] "Potsdam" "Purchase" NA
[112] "Albany" "Alfred-Ceramics" "Binghamton"
[115] "Buffalo University" "Cornell Stat" "Downstate Medical"
[118] "ESF" "Optometry" "Stony Brook"
[121] "SUNY Poly" "Upstate Medical" NA
[124] NA "Alfred State" "Canton"
[127] "Cobleskill" "Delhi" "Farmingdale"
[130] "Maritime" "Morrisville" NA
[133] "Adirondack" "Broome" "Cayuga County"
[136] "Clinton" "Columbia-Greene" "Corning"
[139] "Dutchess" "Erie" "Fashion Institute"
[142] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[145] "Herkimer County" "Hudson Valley" "Jamestown"
[148] "Jefferson" "Mohawk Valley" "Monroe"
[151] "Nassau" "Niagara County" "North Country"
[154] "Onondaga" "Orange County" "Rockland"
[157] "Schenectady County" "Suffolk County" "Sullivan County"
[160] "Tompkins Cortland" "Ulster County" "Westchester"
[163] NA "Brockport" "Buffalo State"
[166] "Cortland" "Empire State" "Fredonia"
[169] "Geneseo" "New Paltz" "Old Westbury"
[172] "Oneonta" "Oswego" "Plattsburgh"
[175] "Potsdam" "Purchase" NA
[178] "Albany" "Alfred-Ceramics" "Binghamton"
[181] "Buffalo University" "Cornell Stat" "Downstate Medical"
[184] "ESF" "Optometry" "Stony Brook"
[187] "SUNY Poly" "Upstate Medical" NA
[190] NA "Alfred State" "Canton"
[193] "Cobleskill" "Delhi" "Farmingdale"
[196] "Maritime" "Morrisville" NA
[199] "Adirondack" "Broome" "Cayuga County"
[202] "Clinton" "Columbia-Greene" "Corning"
[205] "Dutchess" "Erie" "Fashion Institute"
[208] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[211] "Herkimer County" "Hudson Valley" "Jamestown"
[214] "Jefferson" "Mohawk Valley" "Monroe"
[217] "Nassau" "Niagara County" "North Country"
[220] "Onondaga" "Orange County" "Rockland"
[223] "Schenectady County" "Suffolk County" "Sullivan County"
[226] "Tompkins Cortland" "Ulster County" "Westchester"
[229] NA "Brockport" "Buffalo State"
[232] "Cortland" "Empire State" "Fredonia"
[235] "Geneseo" "New Paltz" "Old Westbury"
[238] "Oneonta" "Oswego" "Plattsburgh"
[241] "Potsdam" "Purchase" NA
[244] "Albany" "Alfred-Ceramics" "Binghamton"
[247] "Buffalo University" "Cornell Stat" "Downstate Medical"
[250] "ESF" "Optometry" "Stony Brook"
[253] "SUNY Poly" "Upstate Medical" NA
[256] NA "Alfred State" "Canton"
[259] "Cobleskill" "Delhi" "Farmingdale"
[262] "Maritime" "Morrisville" NA
[265] "Adirondack" "Broome" "Cayuga County"
[268] "Clinton" "Columbia-Greene" "Corning"
[271] "Dutchess" "Erie" "Fashion Institute"
[274] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[277] "Herkimer County" "Hudson Valley" "Jamestown"
[280] "Jefferson" "Mohawk Valley" "Monroe"
[283] "Nassau" "Niagara County" "North Country"
[286] "Onondaga" "Orange County" "Rockland"
[289] "Schenectady County" "Suffolk County" "Sullivan County"
[292] "Tompkins Cortland" "Ulster County" "Westchester"
[295] NA "Brockport" "Buffalo State"
[298] "Cortland" "Empire State" "Fredonia"
[301] "Geneseo" "New Paltz" "Old Westbury"
[304] "Oneonta" "Oswego" "Plattsburgh"
[307] "Potsdam" "Purchase" NA
[310] "Albany" "Alfred-Ceramics" "Binghamton"
[313] "Buffalo University" "Cornell Stat" "Downstate Medical"
[316] "ESF" "Optometry" "Stony Brook"
[319] "SUNY Poly" "Upstate Medical" NA
[322] NA "Alfred State" "Canton"
[325] "Cobleskill" "Delhi" "Farmingdale"
[328] "Maritime" "Morrisville" NA
[331] "Adirondack" "Broome" "Cayuga County"
[334] "Clinton" "Columbia-Greene" "Corning"
[337] "Dutchess" "Erie" "Fashion Institute"
[340] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[343] "Herkimer County" "Hudson Valley" "Jamestown"
[346] "Jefferson" "Mohawk Valley" "Monroe"
[349] "Nassau" "Niagara County" "North Country"
[352] "Onondaga" "Orange County" "Rockland"
[355] "Schenectady County" "Suffolk County" "Sullivan County"
[358] "Tompkins Cortland" "Ulster County" "Westchester"
[361] NA "Brockport" "Buffalo State"
[364] "Cortland" "Empire State" "Fredonia"
[367] "Geneseo" "New Paltz" "Old Westbury"
[370] "Oneonta" "Oswego" "Plattsburgh"
[373] "Potsdam" "Purchase" NA
[376] "Albany" "Alfred-Ceramics" "Binghamton"
[379] "Buffalo University" "Cornell Stat" "Downstate Medical"
[382] "ESF" "Optometry" "Stony Brook"
[385] "SUNY Poly" "Upstate Medical" NA
[388] NA "Alfred State" "Canton"
[391] "Cobleskill" "Delhi" "Farmingdale"
[394] "Maritime" "Morrisville" NA
[397] "Adirondack" "Broome" "Cayuga County"
[400] "Clinton" "Columbia-Greene" "Corning"
[403] "Dutchess" "Erie" "Fashion Institute"
[406] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[409] "Herkimer County" "Hudson Valley" "Jamestown"
[412] "Jefferson" "Mohawk Valley" "Monroe"
[415] "Nassau" "Niagara County" "North Country"
[418] "Onondaga" "Orange County" "Rockland"
[421] "Schenectady County" "Suffolk County" "Sullivan County"
[424] "Tompkins Cortland" "Ulster County" "Westchester"
[427] NA "Brockport" "Buffalo State"
[430] "Cortland" "Empire State" "Fredonia"
[433] "Geneseo" "New Paltz" "Old Westbury"
[436] "Oneonta" "Oswego" "Plattsburgh"
[439] "Potsdam" "Purchase" NA
[442] "Albany" "Alfred-Ceramics" "Binghamton"
[445] "Buffalo University" "Cornell Stat" "Downstate Medical"
[448] "ESF" "Optometry" "Stony Brook"
[451] "SUNY Poly" "Upstate Medical" NA
[454] NA "Alfred State" "Canton"
[457] "Cobleskill" "Delhi" "Farmingdale"
[460] "Maritime" "Morrisville" NA
[463] "Adirondack" "Broome" "Cayuga County"
[466] "Clinton" "Columbia-Greene" "Corning"
[469] "Dutchess" "Erie" "Fashion Institute"
[472] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[475] "Herkimer County" "Hudson Valley" "Jamestown"
[478] "Jefferson" "Mohawk Valley" "Monroe"
[481] "Nassau" "Niagara County" "North Country"
[484] "Onondaga" "Orange County" "Rockland"
[487] "Schenectady County" "Suffolk County" "Sullivan County"
[490] "Tompkins Cortland" "Ulster County" "Westchester"
[493] NA "Brockport" "Buffalo State"
[496] "Cortland" "Empire State" "Fredonia"
[499] "Geneseo" "New Paltz" "Old Westbury"
[502] "Oneonta" "Oswego" "Plattsburgh"
[505] "Potsdam" "Purchase" NA
[508] "Albany" "Alfred-Ceramics" "Binghamton"
[511] "Buffalo University" "Cornell Stat" "Downstate Medical"
[514] "ESF" "Optometry" "Stony Brook"
[517] "SUNY Poly" "Upstate Medical" NA
[520] NA "Alfred State" "Canton"
[523] "Cobleskill" "Delhi" "Farmingdale"
[526] "Maritime" "Morrisville" NA
[529] "Adirondack" "Broome" "Cayuga County"
[532] "Clinton" "Columbia-Greene" "Corning"
[535] "Dutchess" "Erie" "Fashion Institute"
[538] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[541] "Herkimer County" "Hudson Valley" "Jamestown"
[544] "Jefferson" "Mohawk Valley" "Monroe"
[547] "Nassau" "Niagara County" "North Country"
[550] "Onondaga" "Orange County" "Rockland"
[553] "Schenectady County" "Suffolk County" "Sullivan County"
[556] "Tompkins Cortland" "Ulster County" "Westchester"
[559] NA "Brockport" "Buffalo State"
[562] "Cortland" "Empire State" "Fredonia"
[565] "Geneseo" "New Paltz" "Old Westbury"
[568] "Oneonta" "Oswego" "Plattsburgh"
[571] "Potsdam" "Purchase" NA
[574] "Albany" "Alfred-Ceramics" "Binghamton"
[577] "Buffalo University" "Cornell Stat" "Downstate Medical"
[580] "ESF" "Optometry" "Stony Brook"
[583] "SUNY Poly" "Upstate Medical" NA
[586] NA "Alfred State" "Canton"
[589] "Cobleskill" "Delhi" "Farmingdale"
[592] "Maritime" "Morrisville" NA
[595] "Adirondack" "Broome" "Cayuga County"
[598] "Clinton" "Columbia-Greene" "Corning"
[601] "Dutchess" "Erie" "Fashion Institute"
[604] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[607] "Herkimer County" "Hudson Valley" "Jamestown"
[610] "Jefferson" "Mohawk Valley" "Monroe"
[613] "Nassau" "Niagara County" "North Country"
[616] "Onondaga" "Orange County" "Rockland"
[619] "Schenectady County" "Suffolk County" "Sullivan County"
[622] "Tompkins Cortland" "Ulster County" "Westchester"
[625] NA "Brockport" "Buffalo State"
[628] "Cortland" "Empire State" "Fredonia"
[631] "Geneseo" "New Paltz" "Old Westbury"
[634] "Oneonta" "Oswego" "Plattsburgh"
[637] "Potsdam" "Purchase" NA
[640] "Albany" "Alfred-Ceramics" "Binghamton"
[643] "Buffalo University" "Cornell Stat" "Downstate Medical"
[646] "ESF" "Optometry" "Stony Brook"
[649] "SUNY Poly" "Upstate Medical" NA
[652] NA "Alfred State" "Canton"
[655] "Cobleskill" "Delhi" "Farmingdale"
[658] "Maritime" "Morrisville" NA
[661] "Adirondack" "Broome" "Cayuga County"
[664] "Clinton" "Columbia-Greene" "Corning"
[667] "Dutchess" "Erie" "Fashion Institute"
[670] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[673] "Herkimer County" "Hudson Valley" "Jamestown"
[676] "Jefferson" "Mohawk Valley" "Monroe"
[679] "Nassau" "Niagara County" "North Country"
[682] "Onondaga" "Orange County" "Rockland"
[685] "Schenectady County" "Suffolk County" "Sullivan County"
[688] "Tompkins Cortland" "Ulster County" "Westchester"
[691] NA "Brockport" "Buffalo State"
[694] "Cortland" "Empire State" "Fredonia"
[697] "Geneseo" "New Paltz" "Old Westbury"
[700] "Oneonta" "Oswego" "Plattsburgh"
[703] "Potsdam" "Purchase" NA
[706] "Albany" "Alfred-Ceramics" "Binghamton"
[709] "Buffalo University" "Cornell Stat" "Downstate Medical"
[712] "ESF" "Optometry" "Stony Brook"
[715] "SUNY Poly" "Upstate Medical" NA
[718] NA "Alfred State" "Canton"
[721] "Cobleskill" "Delhi" "Farmingdale"
[724] "Maritime" "Morrisville" NA
[727] "Adirondack" "Broome" "Cayuga County"
[730] "Clinton" "Columbia-Greene" "Corning"
[733] "Dutchess" "Erie" "Fashion Institute"
[736] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[739] "Herkimer County" "Hudson Valley" "Jamestown"
[742] "Jefferson" "Mohawk Valley" "Monroe"
[745] "Nassau" "Niagara County" "North Country"
[748] "Onondaga" "Orange County" "Rockland"
[751] "Schenectady County" "Suffolk County" "Sullivan County"
[754] "Tompkins Cortland" "Ulster County" "Westchester"
[757] NA "Brockport" "Buffalo State"
[760] "Cortland" "Empire State" "Fredonia"
[763] "Geneseo" "New Paltz" "Old Westbury"
[766] "Oneonta" "Oswego" "Plattsburgh"
[769] "Potsdam" "Purchase" NA
[772] "Albany" "Alfred-Ceramics" "Binghamton"
[775] "Buffalo University" "Cornell Stat" "Downstate Medical"
[778] "ESF" "Optometry" "Stony Brook"
[781] "SUNY Poly" "Upstate Medical" NA
[784] NA "Alfred State" "Canton"
[787] "Cobleskill" "Delhi" "Farmingdale"
[790] "Maritime" "Morrisville" NA
[793] "Adirondack" "Broome" "Cayuga County"
[796] "Clinton" "Columbia-Greene" "Corning"
[799] "Dutchess" "Erie" "Fashion Institute"
[802] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[805] "Herkimer County" "Hudson Valley" "Jamestown"
[808] "Jefferson" "Mohawk Valley" "Monroe"
[811] "Nassau" "Niagara County" "North Country"
[814] "Onondaga" "Orange County" "Rockland"
[817] "Schenectady County" "Suffolk County" "Sullivan County"
[820] "Tompkins Cortland" "Ulster County" "Westchester"
[823] NA "Brockport" "Buffalo State"
[826] "Cortland" "Empire State" "Fredonia"
[829] "Geneseo" "New Paltz" "Old Westbury"
[832] "Oneonta" "Oswego" "Plattsburgh"
[835] "Potsdam" "Purchase" NA
[838] "Albany" "Alfred-Ceramics" "Binghamton"
[841] "Buffalo University" "Cornell Stat" "Downstate Medical"
[844] "ESF" "Optometry" "Stony Brook"
[847] "SUNY Poly" "Upstate Medical" NA
[850] NA "Alfred State" "Canton"
[853] "Cobleskill" "Delhi" "Farmingdale"
[856] "Maritime" "Morrisville" NA
[859] "Adirondack" "Broome" "Cayuga County"
[862] "Clinton" "Columbia-Greene" "Corning"
[865] "Dutchess" "Erie" "Fashion Institute"
[868] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[871] "Herkimer County" "Hudson Valley" "Jamestown"
[874] "Jefferson" "Mohawk Valley" "Monroe"
[877] "Nassau" "Niagara County" "North Country"
[880] "Onondaga" "Orange County" "Rockland"
[883] "Schenectady County" "Suffolk County" "Sullivan County"
[886] "Tompkins Cortland" "Ulster County" "Westchester"
[889] NA "Brockport" "Buffalo State"
[892] "Cortland" "Empire State" "Fredonia"
[895] "Geneseo" "New Paltz" "Old Westbury"
[898] "Oneonta" "Oswego" "Plattsburgh"
[901] "Potsdam" "Purchase" NA
[904] "Albany" "Alfred-Ceramics" "Binghamton"
[907] "Buffalo University" "Cornell Stat" "Downstate Medical"
[910] "ESF" "Optometry" "Stony Brook"
[913] "SUNY Poly" "Upstate Medical" NA
[916] NA "Alfred State" "Canton"
[919] "Cobleskill" "Delhi" "Farmingdale"
[922] "Maritime" "Morrisville" NA
[925] "Adirondack" "Broome" "Cayuga County"
[928] "Clinton" "Columbia-Greene" "Corning"
[931] "Dutchess" "Erie" "Fashion Institute"
[934] "Finger Lakes" "Fulton-Montgomery" "Genesee"
[937] "Herkimer County" "Hudson Valley" "Jamestown"
[940] "Jefferson" "Mohawk Valley" "Monroe"
[943] "Nassau" "Niagara County" "North Country"
[946] "Onondaga" "Orange County" "Rockland"
[949] "Schenectady County" "Suffolk County" "Sullivan County"
[952] "Tompkins Cortland" "Ulster County" "Westchester"
[955] NA "Brockport" "Buffalo State"
[958] "Cortland" "Empire State" "Fredonia"
[961] "Geneseo" "New Paltz" "Old Westbury"
[964] "Oneonta" "Oswego" "Plattsburgh"
[967] "Potsdam" "Purchase" NA
[970] "Albany" "Alfred-Ceramics" "Binghamton"
[973] "Buffalo University" "Cornell Stat" "Downstate Medical"
[976] "ESF" "Optometry" "Stony Brook"
[979] "SUNY Poly" "Upstate Medical" NA
[982] NA "Alfred State" "Canton"
[985] "Cobleskill" "Delhi" "Farmingdale"
[988] "Maritime" "Morrisville" NA
This is the same as typing suny$college_or_institution_name, but many people find it tidier and easier to read when using the pipe.
Data Moves
In this class, we’re using the Data Moves framework (Erickson et al., 2019) to understand data wrangling. A “data move” is “an action that alters a dataset’s contents, structure, or values.” When we download data, it often isn’t in the structure that we need to conduct analyses. Data moves allow us to get it into that structure, and to derive basic insights that will provide context for our analyses.
Filtering
One of the first things we often want to do with our data is filter it, because our data may contain way more data points than we’re actually interested in. To filter our data, we use the filter() function.
There are certain rows of suny that are “summary” rows (totals for each type of college, and for SUNY as a whole). Those rows have no value for the college name column. Get rid of those rows and save the output as a new data frame. (Use Gemini and the SPARCK framework to help you complete this task and the following.)
Why is it important to get rid of these rows before we do anything else?
Answer: The summary rows are totals of the other rows. If we kept them, any sums or averages we calculate would count the same students more than once.
Create a new data frame that only contains data from after 2020 (i.e., 2021 onward).
suny_post2020 <- suny_clean |>filter(year >2020)
Create a new data frame that only shows comprehensive colleges with more than 4,000 full time undergraduate students.
Data frames contain many rows that represent individual data points, but often, we want to reduce our data down into a summary statistic. We can do this with the summarize() function. Summaries are useful not just to get a sense of our data, but because summaries can often be used for further analysis.
Get the minimum and maximum values for full time undergraduates and full time graduate students across the entire dataset. Make sure you’re using your data frame where you excluded the summary rows with blank values for college name. (Hint: Some colleges have no graduate students, so those values are missing, or NA. In order to get a real number returned, we need to tell R to ignore NA values.)
Sometimes we want to calculate summaries for specific subgroups all at the same time. To do this, we can use the group_by() function in combination with summarize().
Calculate the average number of full time graduate students for each year.
The mutate() function should be your best friend. It allows you to create new data frame columns based on calculations done on existing columns. These calculations might combine several columns, or describe some attribute of a single column.
Create a new column that represents the total number of full time students (undergraduate + graduate) for each row. (Hint: What happens when you add a number to NA?)
Let’s say we consider a school to be “big” if it has over 4,000 full time undergraduate students. Create a new variable that indicates, for each row, whether or not a school is “big.”
The answers for the rest of this tutorial (Merging/joining, Pivoting, and Practice) will be posted after we finish them in lab.
Sometimes, we want to merge two datasets together based on a common characteristic. Often, the data that we want to use to do analyses is spread across many different sources.
Download the dataset on this page, which has information on the location of each SUNY campus. Place it in your project folder and load it into R. Clean the column names.
# Write your code here
What is the name of the column from each data frame that contains information that will allow us to join them?
Answer:
Now go ahead and merge them together using one of the _join() functions (e.g., left_join()).
# Write your code here
Pivoting
While not one of the core Data Moves, knowing how to “pivot” your data is very important because certain visualization and analysis tools expect our data to be in a particular structure. There are two functions in the tidyverse that help us pivot: pivot_longer() and pivot_wider(). These functions allow us to switch our data from “long” format to “wide” format and vice versa.
First, calculate the total number of full time undergraduates at each type of institution, in each year. Then use pivot_wider() so that each type of institution has its own column.
# Write your code here
Make sure you compare the two data frames so you can understand what has changed!
Use pivot_longer() on suny_clean so that there is a “student type” column, and all of the enrollment numbers for the different student types are combined into one column.
# Write your code here
Practice
Let’s imagine you have a research question: How has enrollment for each kind of student changed at doctoral degree granting institutions before vs. after Covid? How would you investigate this? Let’s use SPARCK to figure it out. (For this question, count 2011–2019 as “pre-Covid” and 2020–2025 as “post-Covid.”)
S:
P:
A:
R:
# Write your code here
C:
K:
Quarto
As a reminder, this file, which has a .qmd extension, is called a Quarto file. Quarto files are special files that we can use in RStudio because they allow us to combine plain text - like this - and code chunks, which are in the gray boxes. Quarto documents also have a header at the top that specifies the document’s settings.
Quarto and homework
For your homework, you’re going to need to submit an .html file, which is a rendered version of your Quarto file. I have provided you with a Quarto file for your homework. You will need to fill it in, render it to html, and submit that html file.
You must submit a fully rendered html file in order to receive full credit on the homework. To get your file to render, you simply click the Render button above once you’ve completed the homework. If you are having trouble getting your file to render, you may ask me for help, or ask AI to help you troubleshoot your errors. Some of the most common reasons rendering fails:
An object was created in the console but not in a code chunk.
A package was loaded in the console but never in the document.
The code uses an object before the code chunk that creates it.
Install packages in the console only. install.packages() in a Quarto document can cause it to fail to render.
Think of your homework as a cohesive document that guides us through your data analysis. A cohesive document will be easy to read and will contain a clear narrative.
The header
You’ll want to make basic edits to the Quarto file’s header so that it includes your name. You don’t need to worry about other aspects of the header for now.
The plain text
In your homework, you will need to explain what you are doing in plain text. Quarto files use a special type of syntax called Markdown to control what text looks like in the file. In RStudio, most of the tools you need to format text can be accessed in the tool bar at the top of the Editor pane.
But you don’t need to click these tools to format with Markdown. Some examples of Markdown formatting:
The # sign in front of text (with a space between the sign and the text) turns the text into a heading. More hashtag signs create smaller headings.
You can bold text by wrapping it in two asterisks (**) and italicize it by wrapping it in one asterisk (*).
Code chunks
This is where all of your code goes. At baseline, your code chunk runs the code inside of it and displays the output. But you can tinker with some of the settings of the code chunk by putting a chunk option inside of it.
#| eval: false means the code in that chunk will not be run when rendering
#| echo: false displays the code output but not the code itself
#| error: true will allow the document to render even if that code chunk contains an error
#| results: hide prevents the output from being shown, but still runs the code
#| message: false prevents messages from showing in the rendered file
References
Erickson, T., Wilkerson, M., Finzer, W., & Reichsman, F. (2019). Data moves. Technology Innovations in Statistics Education, 12(1). https://doi.org/10.5070/T5121038001
Ismay, C., Kim, A. Y., & Valdivia, A. (2025). The pipe operator: |>. In Statistical inference via data science: A ModernDive into R and the tidyverse (2nd ed., Section 3.1). https://moderndive.com/v2/03-wrangling.html