print("This is a code chunk!")[1] "This is a code chunk!"
Coding Basics
Programming and coding are used interchangeably.
What do we actually mean when we talk about coding/programming?
Programming is telling a computer to do some action to some input info to get output info that you want. When it comes to working with data, programming is how we tell a coding program to load data, process it, plot it, analyze it, and so on (everything!).
We use programming languages (such as R, Python, Java, etc.) to communicate with computer programs and have them do the things we want them to do.
Some of you may have varying degrees of experience with using AI. Many of you may have used it before for uses other than coding. Just to make sure we’re all on the same page, let’s go to Gemini, which is Google’s AI bot. All Columbia students get a higher tier of access to Gemini. For this class, you may use any AI bot that you’d like.
In this class, we will learn coding and data analysis concepts, and you will use AI to help you apply them. On your homework, you can use AI to help you code, but you will also need to indicate that you have some level of understanding of what your code is doing.
Our use of AI will be minimal—but not zero—today because we’re learning the basics.
Coding is hard! It’s learning an entirely new language. You won’t understand everything at first. That is okay. Over time you will begin to see patterns of use that at first seemed completely random and arbitrary. Coding really does get easier with practice. I promise. Trust the process…
R is a programming language, just like Python, Java, HTML, etc. It’s literally just the name of the language—just like Spanish and English are the names of languages humans speak.
RStudio is a program that has been developed to make it easy to code in R.
You may have used something called Posit Cloud in your statistics class. Posit is the name of the company that releases RStudio. Posit Cloud is another product that Posit releases—basically, it’s RStudio “in the cloud.”
The benefit of using Posit Cloud in an educational context is that you can jump right into the coding and don’t have to worry about installing any programs. The drawback is that there’s a level of artificiality to your learning, since when the semester ends you won’t be able to use the platform.
My goal in this class is that using AI can expand the scope of what we can learn. Installing R and RStudio (which you need to do separately) on your computer, rather than using Posit Cloud, is one example of this expansion.
There will probably be issues today since we all have different computers and everyone installs things differently. That’s okay—that’s what today is for. Help each other!
I am going to provide simple instructions for installing R below. You may follow them. OR you may ask Gemini “Give me instructions to download R onto my [Mac/Windows] computer” and then follow the instructions it gives you.
Go to the R Project website.
Click Download R for Windows. Then click base. Then follow installation instructions.
Click Download R for macOS. Then figure out if you have a Mac chip or an Intel chip by going to About this Mac on your computer.
Then click the appropriate .pkg link and follow installation instructions.
Go to the Posit website.
Scroll all the way to the bottom and click RStudio IDE.
Click Download RStudio and follow the installation instructions.
Open RStudio like you would any other app on your computer. Be sure you’re opening RStudio, not R. Let’s get oriented.
In the automatic configuration of RStudio’s panes/mini-windows, the console is likely in the bottom left. In general, you can type R commands in the console, and press Enter to execute those commands. This is where you can execute R code!
Note: the console is a great place to try out new stuff, but it doesn’t save your code as one piece the way a script does – for the most part it is good practice to be writing scripts.
In everyday RStudio life, though, you’ll likely spend most of your time in the editor window (default position should be upper left pane). This is where you edit R script files. R scripts are files that contain R code typed out that you can write once, and then run as many times as you want. You’ll do the vast majority of your code writing in script files, so that you can have records of the data processing and analyses you’ve written.
In this window (default position: upper right pane), you can see some useful stuff pertaining to your current R session:
In this window (default position: lower right pane), you can see even more useful stuff:
R can interact with files on your computer, such as datasets and scripts. We’ll talk more about file paths and locations next class. One way to simplify this aspect of R is to use RStudio’s Projects feature. At its simplest, an RStudio Project is just a folder. Anything you need you can put in that folder, and R will know where it is.
Let’s create a project for this tutorial. Go to File > New Project. Create the project in a new directory, then click R Project. Then give the directory (folder) a name, such as “Tutorial 1”. You should put it somewhere where you’ll easily be able to find it later on (i.e. where it says “Create project as a subdirectory of…”)
Once your new project is created, you should see a new blank RStudio window with the name of the project at the top.
Up until now, you’ve been looking at this file on the web. This file exists in two forms: an html file, which allows you to view it in your browser, and as a .qmd file, which is the file extension for a special type of file you can use within RStudio called Quarto. So just like you have word docs, and jpgs, you also have Quarto files.
Download the .qmd version of this file from the syllabus website appendix, and put it in the folder you created for this project. Then, open it. It should appear in the top left pane.
Quarto files allow you to combine regular text, like this, with code chunks that run code, like the below:
print("This is a code chunk!")[1] "This is a code chunk!"
If you run that (press the Play button on the right side), you’ll see the output of the code. The ability to combine text and code in one document makes .qmd files perfect for coding tutorials such as this one. You can also switch above between viewing the tutorial in “Source” mode and “Visual” mode, which looks a little nicer.
For the rest of this tutorial, you should follow along within this Quarto file in RStudio.
In R, you can install software packages. A software package is a collection of functions that are related in some way. Installing these packages allows you to expand/enhance the things you are capable of in R.
Think of it like installing an app on a smartphone. When you get your smartphone, it can on its own do a bunch of cool things: set alarms, access the internet, and take notes. But you can also install apps (social media, airline apps) to make it more powerful/versatile.
One of the most important packages to install is called tidyverse. The tidyverse package is really a collection of packages. These packages all interact with each other and greatly enhance the coding experience and our capabilities.
Once a package is installed, you again have to tell R to load the packages into your active R session. This allows you to quickly and easily call the functions in those packages. While you only have to install packages once, you have to load them in every R session that you want to use them.
We’ll talk more about the tidyverse in our next lab. For now, we will just install the tidyverse package, and load it for brief use later in the tutorial.
How do you install and load the tidyverse package? Well, now is a great time to use AI to help us figure that out.
First, let’s make sure we’re all on the same page about what exactly it is that we want to do. Below, write in plain language the coding “steps” that we need to take. (Hint: I just gave you the answer two lines above. This isn’t a trick question.)
Answer:
Next, let’s ask AI how to do it. Remember, you get Gemini through Columbia, but you can use any AI bot you’d like. Paste the code below.
# Write your code hereFinally, let’s make sure we understand what happened. Explain what your code is doing.
Answer:
In R, everything you deal with has a name. You refer to things by their names as you examine, use, or modify them. Named entities include variables (like x, or y), data that you have loaded (like my_data), and functions that you use. (More about functions momentarily.) You will spend a lot of time talking about, creating, referring to, and modifying things with names.
Names in R are case-sensitive. The name my_data is not the same as the name My_Data. When choosing names for things, be concise, consistent, and informative. Follow the style of the tidyverse and name things in lower case, separating words with the underscore character, _, as needed. Do not use spaces when naming things, including variables in your data.
Variables and loaded data have names that you create. Functions have names that are set and cannot be changed.
Some objects are built in to R, some are added via packages, and some are created by the user. But almost everything is some kind of object. The code you write will create, manipulate, and use named objects as a matter of course. We can start immediately. Let’s create a vector of numbers. The command c() is a function. It’s short for “combine” or “concatenate.” It will take a sequence of comma-separated things inside its parentheses and join them together into a vector where each element is still individually accessible.
# Write your code hereInstead of sending the result to the console, we can instead assign it to a name. You can type the arrow using < and then -.
# Write your code hereTo see what you made, type the name and hit return:
# Write your code hereEach of our numbers is still there, and can be accessed directly if we want. They are now just part of a new object, a vector, called my_numbers.
This is the most basic action in R. You create an object by assigning a thing to a name. This is done using the assignment operator, or <-. Think of assignment as the verb “gets”, reading left to right. So the bit of code above can be read as “The object my_numbers gets the result of concatenating the following numbers: 1, 2, …”
When you create objects by assigning things to names, they come into existence in R’s environment, which you can view in the top right pane.
Using AI, create a vector of numbers that is 10 numbers long using the c() function. Save it to an object called ai_numbers. Display the output.
# Write your code hereWhat prompt did you send to AI to do the above?
Prompt:
Explain what your code is doing, and how you know it worked correctly.
Answer:
You do almost everything in R by using functions. Think of a function as a special kind of object that can perform actions for you. It produces output based on the input that it receives. Like a good dog, when we want a function to do something for us, we call it. It will reliably do what we tell it. We give the function some information, it acts on that information, and some results come out the other side. Functions can be recognized by the parentheses at the end of their names. This distinguishes them from other objects, such as single numbers, named vectors, tables of data, and so on.
The parentheses are what allow you to send information to the function. Most functions accept one or more named arguments. A function’s arguments are the things it needs to know in order to do something. They can be some bit of your data (data = my_numbers), or a specific value (title = "GDP per Capita"), or an option you want to choose (smoothing = "splines", show = FALSE). For example, the object my_numbers is a numeric vector:
# Write your code hereBut the thing we used to create it, c(), is a function. It concatenates items into a vector composed of the series of comma-separated elements you give it. Similarly, mean() is a function that calculates a simple average for a vector of numbers.
# Write your code hereWhile the function arguments have names that are used internally, (here, x), you don’t strictly need to specify the name for the function to work:
# Write your code hereIf you omit the name of the argument, R will just assume you are giving the function what it needs, and in a default order. The documentation for a function will tell you what the order of required arguments is for any particular function. For simple functions that only require one or two arguments, omitting their names is usually not confusing. For more complex functions, you will typically want to use the names of the arguments rather than try to remember what the ordering is.
To view a function’s documentation, type ?[name of function] in the Console. You can of course always ask AI for how a function works as well.
Functions can return far more than single numbers. The output returned by functions can be a table of data, or a complex object such as the results of a linear model, or the instructions needed to draw a plot on the screen (as we shall see). For example, the summary() function performs a series of calculations on a vector and produces what is in effect a little table with named elements.
As we have already seen with c() and mean(), you can assign the result of a function to a name, thereby creating an object:
# Write your code hereWhen you do this, there’s no output to the console. R just puts the results into the new object, as you instructed. To look inside the object you can type its name and hit return:
# Write your code hereHere are a few more examples of some things you can do with your vector of numbers:
# Write your code hereThe first two functions here gave us the largest value in my_numbers and calculated its standard deviation. It’s worth noticing what R did in the last case: We multiplied my_numbers by five. R interprets that as you asking it to take each element of my_numbers one at a time and multiply it by five. The single value is “recycled” down the length of the vector.
Let’s level up the difficulty here.
Use the seq() function—which stands for “sequence” and is another way of creating a vector—to create a vector with a length of 15, where each number in the sequence increases by 3.
First, type your AI prompt.
Prompt:
Then, run the code:
# Write your code hereFinally, explain what your code is doing. Be sure to explain the different arguments.
Explanation:
Explain what the following code is doing. Don’t use AI to write your explanation.
seq2 <- seq(from = 10, by = 5, length.out = 4)
mean(seq2)[1] 17.5
Answer:
Every data analysis is different, depending on your research question, the structure of your data, and how you want to display and explain your findings.
But broadly, all data analysis follows the same 4 steps.
Once you have your research question, you need to find a dataset (or datasets) that is/are equipped to answer that question. Next week (9/23) we’ll talk more about sources of data and loading data files from your computer.
Almost always, real-world datasets are not in the exact format that you need them to be in to accurately visualize and analyze them. “Cleaning” data or “wrangling” data refers to the process of restructuring and working with your data to prepare it for visualization and analysis. We’ll discuss how to do this in class on 9/23 and 9/30.
Visualization is incredibly important for making sense of data. It’s useful for you as a researcher to understand what the relationships between your variables are, and it’s useful for those who are reading about your research to easily understand what you’ve found. We’ll first focus on data visualization in our 10/7 class.
Finally, you need to quantify what you’ve found from your visuals and exploration and prove that the relationships between your variables are meaningful. This is where statistics come in—statistics gives us a way to test whether patterns in the data are “significant.” We’ll learn the basics of statistical modeling in our 11/4 class.
Note that this is not a statistics class, and our exploration of statistical models to analyze our data will be surface-level.
For this class, I’m going to encourage you to use a particular workflow when coding. You will often have to follow this workflow in class when practicing what we’ve learned, and it will form the structure of your homeworks. I also encourage you to use this workflow implicitly whenever you’re using AI to help you code.
I’m calling this workflow SPARCK. I’m open to other ideas for creative acronyms.
Before you dive in, figure out what it is that you actually want to do. What is the task at hand? What is your end goal? This might just be a simple sentence or two.
Pseudocode is a term that basically just means: saying the steps your code needs to take in plain language. I had you do this earlier when installing and loading tidyverse. In that case, there were essentially two pseudocode steps: installing the package, and loading it. No code needed. It’s a way of building a map for yourself, and breaking down a complex problem into discrete chunks.
What are the prompts you are giving to AI to write the code that you need? Often, this will be very similar to the pseudocode, but you may need additional detail or prompts based on how the AI responds.
Paste the code from AI into your code and run it!
Did your code work as expected? You need to check. There are two things that may go wrong that you need to check for:
Errors: If your code doesn’t work, it will return an error. Learning to read errors in R is very important. They look confusing at first glance, but they are saying something sensible, if you know how to make sense of them. If you call on a variable that you haven’t previously saved, you’ll get an error. If you forget to close parentheses, you’ll get an error. You can also always paste your error into AI and it will explain what it means. If you get an error, you’ll need to make changes to your code to fix the issue.
Incorrect output: Your code might work and still be wrong! Imagine that you want to multiply your my_numbers object by 3, and you accidentally type my_numbers * 4. Well—there’s nothing wrong with that. It’s going to work. But it’s not correct, in that it’s not what you’re trying to do. So even if you don’t have any errors, it’s still very important that you check your code’s output (variables that have been created, visualizations, etc.) to make sure that what your code did is aligned with what you want to do.
If your code doesn’t work - either because of an error or incorrect output - you need to go back to the “A” step and figure out what went wrong. Then run and check it again.
Explain what your code is doing. Look at specific functions and specific arguments that your code is using. Do you understand what the input and output are? What exactly is going on. Being able to explain—especially linking your pseudocode steps to specific lines of code—is very important for your learning and for making coding easier the more you practice.
Speaking of practice, let’s do a final example that brings a lot of things together and that exemplifies a lot of what we’ll be doing in tutorials and homework.
We’re going to do a lot here that we haven’t explicitly learned yet - the point of this practice is to get used to using AI to help you code and to see what’s possible with a dataset.
Let’s imagine we have a very simple research question: How have incarceration rates changed over time?
We’re going to use a dataset called USincarcerations_df from a package called crimedatasets, which includes a ton of datasets related to crime and justice. Let’s save the data to an object. How do we do that? Use SPARCK to figure it out!
S:
P:
A:
R:
# Write your code hereC:
# Write your code hereK:
And that’s how you SPARCK, folks!!
I won’t do SPARCK for the remaining steps, but feel free to follow along as much as you’d like.
Let’s say we want to look at how incarceration rates have changed from 1989 to 2019 (the last year in the dataset), and sort our data into two groups: 1989-2004 and 2005-2019. We need to do a bit of data cleaning to get our data ready for analysis.
my_data_new <- my_data |>
filter(year > 1988) |>
mutate(era = as.factor(if_else(year > 2004, 1,0)))Let’s graph the incarceration rate by year and era!
ggplot(my_data_new, aes(x = year, y = stateFedIncarcerationRate, color = era)) +
geom_line() +
geom_point()Now we’re going to run a linear model to statistically test if the relationship between time and incarceration rates differs by era.
m1 <- lm(stateFedIncarcerationRate ~ year * era, data = my_data_new)
summary(m1)We’ll talk later in the semester about what this output means, but it shows us that incarceration rates are higher in 2005-2019 than they are 1989-2004, but that the trend over time is significantly different between these two eras.
Remember: Practice makes perfect. I am here to help, and your classmates are here to help, and even AI is here to help. Don’t fret. My goal is for you to do well on your lab assignments. If that is also your goal, then I will make as much time as I am able to help you. Onward!
Some of this material is adapted from Kieran Healy’s Data Visualization: A Practical Introduction, which is freely available online.
Some is also adapted from a coding boot camp run by graduate students in the Psychology department.