Tools and Technology

SPEA V706 — Statistics for Research in Public Affairs

Coady Wing

Tools and technology are a big part of how we do science.

  • Learning the tools of a social scientist is part of how you learn to think like a social scientist.

  • Tools can raise productivity and reduce drudgery. But that is only part of the story.

  • They shape the way we think and communicate, the things we measure and notice, and the kinds of questions we decide to ask.

  • Downsides too. Sometimes tools make us lazy or sloppy, blind to key facts or problems.

Some important tools

  • Not an exhaustive list and not quite an endorsement, really. (Although I do like these things.)

  • Not really a course in how to use them; although I expect you to be very good with these things by the end of the semester.

The Toolkit

Tool Job
R all purpose data management and analysis; statistical computing.
RStudio / VS Code Integrated development environments for R and other languages.
LaTeX and Markdown Plain-text ways to write papers and documents.
Overleaf Google docs for LaTeX. Works in your browser.
Git and GitHub Version control and cloud based storage for your code and plaintext files.
Claude Code, Codex Agentic AI tools.
The command line Direct your computer using text based commands.
Quartz IU’s research computing cluster.

R

What is R?

  • A programming language for statistical computing and graphics
  • Free, open source
  • Base installation + thousands of user-created add-on packages for specialized tasks

Integrated Development Environments

What is an IDE?

One application window that manages a whole bunch of things you use while writing code.

  • text editor for your scripts, often with with syntax highlighting that makes it easier to read and spot errors.

  • console where the code runs

  • view of the environment: objects, files, plots, help pages

  • many other features

RStudio

  • Purpose-built IDE for R — probably the friendliest place to start
  • Runs R code, manages your files, and helps you write documents in R Markdown or LaTeX

VS Code

  • A more general-purpose IDE that works with essentially any language
  • R, Python, Julia, Stata, LaTeX, shell scripts
  • Connects to a remote machine and edits files there as if they were local.
  • Agentic AI tools can run inside VS Code, which is pretty helpful sometimes.

LaTeX and Markdown

The Problem with Microsoft Word and Google Docs

  • Equations are painful, and they break when you move the file
  • Citations and cross-references drift out of sync as the paper changes
  • You spend a lot of time fussing with formatting and it often looks bad anyway
  • AI tools and git based version control work best with plain text files

LaTeX

  • A janky programming language for writing and typesetting documents.

  • You write plain text and a compiler produces a PDF.

  • Totally absurd and rickety.

  • And yet…many advantages and you grow to like it a bit.

  • Makes math writing easy:$\hat{\beta} = (X'X)^{-1}X'y$

  • Bibliographies and citations are super easy to manage.

\begin{equation}
  \hat{\beta} = (X'X)^{-1} X'y
\end{equation}

Markdown

  • A much lighter markup language: **bold**, # Heading, - bullet
  • Readable as plain text before it is rendered into anything
  • Could be the wave of the future for academic writing because its so simple. But…LaTeX is pretty grizzled and tenecious.

Overleaf

  • A gift that made LaTeX more accessible and friendly.
  • LaTeX in a browser: nothing to install, and it compiles for you
  • Google-Docs-style collaboration

Git and GitHub

File naming in the state of nature

analysis.R
analysis_final.R
analysis_final_kosali.R
analysis_final_kosali_cw.R
analysis_final_kosali_cw_ks_final.R
analysis_submission_kosali_cw_ks_final_cw.R
analysis_please_god_let_this_be_the_one.R

What is Git? What is GitHub?

A formal version control system developed by Linus Torvalds – the same guy who created Linux. (So…it’s good, right?)

  • Repository — a folder that contains your project, including all the files and a hidden folder that tracks changes
  • Remote repository — a copy of your repository on a server somewhere, usually GitHub
  • History — every saved version, with a note on what changed and why
  • Pull — get the latest version from a remote repository
  • Commit — save a version of your project, with a note on what changed
  • Push — send your saved version to a remote repository
  • Diff — compare changes between two versions

GitHub is a website that hosts git repositories.

This is how software developers manage their code.

And now us too.

Agentic AI

What “Agentic” Means

  • A chatbot answers a question and hands you text to paste somewhere

  • An agentic tool has access to your project: it reads your files, edits them, runs code, and reads the errors

  • Claude Code and Codex run in a terminal or inside your IDE

The Command Line

What the Shell Is

  • A text interface to your computer: you type a command, it does the thing
  • Same machine, same files, no icons in the way
  • macOS has Terminal, Windows has PowerShell
  • Linux has Terminal as well

Enough Commands to Survive

pwd                  # where am I?
ls                   # what's here?
cd code              # go into the code folder
cd ..                # go back up one
mkdir raw_data       # make a folder
cp a.csv b.csv       # copy
mv old.R new.R       # move or rename
rm junk.csv          # delete, with no undo

rm does not use a trash can. Look twice before you press return.

Running R from the Shell

Rscript code/main.r
  • Runs the script start to finish in a fresh session, with no RStudio involved

High Performance Computing

When Your Laptop Is Not Enough

  • Restricted use data that cannot leave a secure server
  • Huge data set that will not fit in RAM
  • A job that would run for three days, during which you would like to use your laptop

What is a super computer?

  • Many machines, sitting in a room, sharing storage.
  • Lots of capacity and strange terminology.
  • Weird and old school but easier than it sounds.

Recap

  • R is a statistical programming environment.
  • RStudio or VS Code are IDE that make it easier to write code and documents.
  • LaTeX and Markdown are tools for technical writing. Overleaf is Google Docs for LaTeX.
  • Git and GitHub is a version control system.