6 Code Workflow for Reproducible Research
Coding is difficult! There are many ways to code something that will achieve the same goal. Our general philosophy at PRIDE is based on a general set of coding principles and workflows that are broadly applicable across coding languages and integrated development environments. We recognize the need for transparent and reproducible science while ensuring that we protect the data privacy of LGBTQIA+ participants at The PRIDE Study. We also note that this is a living document, which we will update as the team and coding philosophy change over time.
We focus on providing general recommendations and scripts that allow individual members at PRIDE to code how they prefer while maintaining reproducibility, ease of collaboration, and data privacy of participants. We do not tell our team members which language to use, or for the most part, which tools to use within a given language. For those interested in specific recommendations, we suggest reading the works of Van Lissa et al. (2021)1 or the accompanying vignette.
Currently, our team can support workflows in R with more limited support for Python. Thus, we offer guidance specifically in R. We will add relevant recommendations if other languages become more prominent in the team
We use R as our primary programming language. It has excellent tools for statistical analysis, visualization, and reproducible research. This chapter provides a brief summary of our approach.
6.1 Code Style and Standard
Consistency of code style is essential for collaboration. It speeds up the reading of code and reduces debates on what code should look like. We prefer the Tidyverse style for coding. This can be automatically implemented using the styler package. Despite the name, this guide also applies to non-Tidyverse code, including base R and data.table.
# style all R-related files in the current directory
styler::style_dir()Note that you should run styler interactively in the console, not in one of your R scripts.
Styler also has a helpful add-in for RStudio to let you style the active file, a code selection, etc.
styler automatically reformats code without changing its behavior. Here are some common examples:
# Inconsistent spacing and indentation
fit_model<-function(outcome,exposure,data){
predictors<-c(exposure)
formula<-as.formula(paste(outcome,"~",exposure))
model<-lm(formula,data=data)
return(model)
}
# Messy pipe and argument alignment
result <- data|>dplyr::filter(age>18)|>
dplyr::select(id,age,sex)|>
dplyr::mutate(adult=TRUE)# Consistent 2-space indentation and spacing around operators
fit_model <- function(outcome, exposure, data) {
predictors <- c(exposure)
formula <- as.formula(paste(outcome, "~", exposure))
model <- lm(formula, data = data)
return(model)
}
# Clean pipe alignment and spacing
result <- data |>
dplyr::filter(age > 18) |>
dplyr::select(id, age, sex) |>
dplyr::mutate(adult = TRUE)6.2 Naming Things
Naming things is an aspect of code style, but it merits a little extra attention because it is important for readability. In general, strive to make your code self-documenting; someone else should be able to read your code and understand what it’s doing without talking to you about it. Self-documenting is easier said than done, so code clearness is an essential type of feedback in code review.
Use descriptive names for objects in code, e.g., model_results rather than out or pride_data rather than d. However, we suggest not to use overly long names (e.g., three words is better than four or five). It is OK to use terse names where the meaning is generally understood, like n for counts or i and j for loop counters.
Similarly, for functions, describe what the function is doing, preferably in verb format, e.g., simulate_data() is better than f() or data_simulator().
We generally prefer snake_case over camelCase and other naming style conventions unless otherwise appropriate. In snake_case, words are written in lowercase and separated by underscores (e.g., participant_id, birth_date), whereas in camelCase, words are joined together with the first word in lowercase and each subsequent word capitalized (e.g., participantId, birthDate).
6.3 Project Organization
In general, we recommend using descriptive folder and file names, e.g., R/ or scripts/ to organize source code and output or reports to organize research outputs. You should also include guidance on the organization of your project in a README file (see Section 6.3.3 for more details about README files).
For example, an R project might look like this:
code/
data/
output/
README.md
my_project.Rproj
The Kiang Lab also has a template for setting new research projects, which includes an appendix that explains the reasoning behind the template.
6.3.1 File Paths
We recommend to not hardcode direct file paths that only exist on your computer:
raw_df = readr::read_csv("/users/ntran/Downloads/data.csv")Our preference is to use file paths that are relative to the root directory of your project. In R/RStudio, a root directory is automatically established when you create a new R project (.Rproj file); you can also create a new .Rproj file by linking it to an existing file directory. This project-based workflow organizes analyses into a self-contained folder. This enables seamless collaboration by allowing team members to access all project data using relative file paths, eliminating the need for hardcoded absolute paths that vary across different users’ systems.
For example, data.csv should be in your project directory (my_project.Rproj), which you can access with relative paths. In addition, we recommend using the here package help specify the relative paths:
raw_df = readr::read_csv(here::here("data", "data.csv"))The here package can let you refer to files and folders from the root directory of your project no matter where the code exists. For instance, here can be helpful when you have a report in a reports/ folder but want to refer to a data file in the data/ folder. Instead of backtracking ../data/data.csv, you can write here("data", "data.csv").
6.3.2 Documentation
Good documentation practice is essential to ensure that anyone who reviews or inherits your code can quickly understand its purpose and functionality. The primary mode of documenting code is using comments to explain why and not what the code is doing.
The “what” should be evident from well-written, clear code itself; if not, your code may be overly complex. The “why” is often more critical because it provides the reasoning for your analytical decisions, methodological choices, and problem-solving approach. This context cannot always be inferred from the code alone.
# Group together trans and gender diverse identities to avoid small cell sizes
clean_df <- raw_df |>
dplyr::mutate(
gi_bin = dplyr::if_else(gender == "cisgender", 0, 1)
)In R, the # is used to provide inline comments. We generally recommend putting inline comments on their own line above the code they describe, not at the end of a line.
6.3.3 README
A README file serves as the primary documentation for your project, explaining its purpose, structure, and how to use it. We recommend including a README file for all research projects to provide collaborators and future users with essential context and guidance. In general, recommended README content should include at least:
- Project overview and objectives
- Data requirements and sources
- Details on installations and setup of any package or version dependencies
6.3.4 Git and GitHub
Git is a version control system that tracks changes to files over time, allowing multiple people to collaborate on a project while maintaining a complete history of edits. GitHub is a web-based platform that hosts Git repositories online, providing tools for collaboration, code review, and project management. Our team uses Git and GitHub for managing and hosting version-controlled repositories for our research projects. This effort is led and managed by our biostatistician, Nguyen Tran. For external collaborators whose analyses were conducted by our internal team and are interested in making the code publicly available, please contact Nguyen Tran to initiate this process as outlined in Section 3.2.2.
Our team does not require new members or trainees to have prior experience with Git and GitHub. We recognize that these tools may be unfamiliar to many researchers. We encourage developing these skills over time as version control is essential for reproducible research and effective collaboration. For trainees who are interested in learning more about Git and GitHub, Nguyen Tran is available to guide you through the process. This includes helping with creating your personal GitHub account, setting up repositories, learning essential git commands, understanding our project documentation standards, and eventually transferring your work to The PRIDE Study GitHub.
There are also several other labs at Stanford provide introductory guidance to Git and GitHub such as the Kiang Lab and Stanford Health Policy and Data Science. We also recommend the paper by Van Lissa et al. (2021)1 for a coding workflow for reproducible science.
In general, we require trainees and collaborators to never upload any data from The PRIDE Study onto GitHub as part of their workflow, whether original data or data generated during analysis. Making data available through a public GitHub repository is prohibited under our data use agreement and violates our data privacy policies. Even if the repository is private and .gitignore is used to prevent uploading sensitive data to GitHub, this is not foolproof (see UK BioBank data leak).
All data shared by The PRIDE Study must be stored securely (see Section 4.1 for examples). If you wish to share data, we recommend either generating a synthetic dataset with similar characteristics to the original data using worcs::synthetic() or using the recommended data availability statement in Section 3.2.2 for your paper submission.
6.3.5 Code Review
Code review is a collaborative practice where team members examine each other’s code before it is finalized or published. This process can be helpful for ensuring the quality and reproducibility of our research.
Trainees or external collaborators working with PRIDE Study data who would like code review should contact Nguyen Tran in advance to arrange this. Our preference is that code reviews are done through GitHub using a pull request. However, for those not familiar with GitHub, please share your code and documentation through a secure file sharing platform (Box, Dropbox, Google Drive) along with a brief description of your analysis objectives and any specific areas where you would like feedback. Please ensure that no participant identifiers are shared during this process. Data sharing can be helpful but is not necessary for code review.
6.4 Script Templates for Reproducible Analysis
To support reproducible research practices, we have developed a collection of script templates in R for handling common and repetitive tasks that are prone to human error.
To view scripts aimed at supporting reproducible workflows in R, please check out our repository on GitHub. Please note that these templates may be updated over time based on feedback from team members.
In addition, we recommend the Epidemiologist R Handbook as an R code reference manual that adresses common data, epidemiological, and statistical challenges.
6.5 Resources
- Van Lissa CJ, Brandmaier AM, Brinkman L, et al. WORCS: A workflow for open reproducible code in science. Data Science. 2021;4(1):29-49. doi:10.3233/DS-210031