Read about courses in a range of fields at a variety of institutions where students have learned TIER-like methods of reproducible research.  Course syllabi, exercises, project instructions and other course documents are available for download.

Quantitative Methods of Social Research, Willamette University

Sociology 401, Spring 2017
Instructor: Michael Niño
  • Course description

    In this course students will apply theory and methodological skills learned in previous course work to their own original quantitative research project. The objective of this course is to train sociology majors to plan and carryout research using: systematic procedures; quantitative data collection methods, strategies, and instruments used in the social sciences (and sociology, in particular); and quantitative data analysis and modeling techniques. Students will learn the core elements of the research process: the scientific method; variables, relationships, and hypotheses; definition and measurement of variables; data validity and reliability; determination of where data will come from (sampling); and issues related to ethics, perspectives, and strategic approaches to social science research, among others. The course will start with a recap of fundamental elements of quantitative methods. Students will then review contemporary sociological research to gain a better understanding of the structure of empirical papers and the art of writing. The course will then shift to an overview of various techniques for analyzing and interpreting quantitative data, and data management using SAS. This course is designed to fulfill the Willamette University writing-centered course designation. Toward this end, students will engage in regular writing, peer-review, and revising exercises.

Econometrics, Dickinson College

Economics 374, Spring 2017
Instructor: Tony Underwood
  • Course description

    This course is a rigorous introduction to econometrics in which the tools of economic theory, mathematics, and statistical inference are applied to the analysis of economic data. We will study and apply multiple regression analysis to both cross-sectional and longitudinal (panel) data in order to familiarize students with the concepts of econometric modeling, estimation, prediction, and hypothesis testing. Students will conduct empirical research projects using Stata to enable students to understand and apply the conventions of empirical research in economics, including: technical writing, reviewing existing literature, data collection and organization, and file management for complete transparency and reproducibility. Throughout the semester students will:

    • Learn the econometric methodology, including how to construct econometric models, estimate relevant economic parameters, predict economic outcomes, and test economic hypotheses using quantitative data.
    • Do regression analysis. They will be able to choose a topic conducive to regression analysis, specify a regression equation, collect data, run descriptive statistics, run regressions, interpret and evaluate the results, and demonstrate the necessary components of a well-written empirical research paper and the economics discipline formatting and style conventions.
    • Learn the basic assumptions of the classical linear regression model and identify and correct (if possible) any violations of these assumptions, such as serial correlation and heteroskedasticity.
    • Evaluate regression results, determining whether the regression coefficients have the expected sign and magnitude, whether the regression coefficients are statistically significant, whether the equation includes irrelevant variables or omits theoretically relevant variables, and whether the goodness of fit of the equation appears adequate.
    • Demonstrate an understanding of Stata syntax, data management skills, and best coding and documentation practices for reproducibility.
    • Demonstrate the ability to place a research question in the context of existing scholarly discourse through an effective literature review.

Honors Thesis, Clark University

Economics 297, Spring 2017
Instructor: Wayne Gray
  • Course description

    An important part of a successful research project is documenting the process well enough that others can reproduce your results (it also makes it much easier for you to extend your own research at some future date).  Students will follow the TIER (Teaching Integrity in Empirical Research) Protocol developed by Richard Ball and others at Haverford College (see for more background information).  The documentation students prepare covers three stages of the research process: the original data source(s) accessed, the steps needed to develop the final analysis dataset(s), and the statistical analyses presented in the paper.  

The Political Economy of Development in Africa, Smith College

Economics 311, Fall 2016
Instructor: Simon Halliday
  • Course description

    There are many approaches to development economics. First, there are many attempts to explain international differences in economic development – the ‘macro’ development problem. Second, there are many differences between the ways in which individuals, firms and states can respond to the challenges of development – the ‘micro’ development problem. The microeconomics of development informs the macroeconomics of development. From micro work we may gain insight into the specific individual-, family-, village- and firm-level institutions and constraints that shape decisions. As we try to understand how micro-level results permeate through an economy, that is, we try to understand the general equilibrium effects of a policy, so may we gain insight into macrolevel differences. Micro and macro tie together closely.

Economic Development, Smith College

Economics 211, Fall 2016
Instructor: Simon Halliday
  • Course description

    A majority of the world’s population lives on less than $2.50/day. Health status, educational attainment, government quality, and many other indicators are substantially lower in developing countries than in developed countries. Why are so many people so poor and what stop them from catching up? What strategies help alleviate poverty? What policies or institutions cause economies to grow and develop? We begin the class by understanding how to define and measure development, studying income, poverty, inequality, human development and health. We proceed to look at economic theories of growth as a tool to understand how agriculture, institutions and transformation have affected and conTnue to affect developing countries. Toward the end of the course we complicate our understanding of development by looking at the role of market failures and externalities, globalization and world trade, and credit and risk.

Replication and Transparency in Empirical Research, Federal University of Pernambuco

Political Science 015, Summer 2016
Instructors: Dalson Figueiredo & Enivaldo Rocha
  • Course description

    This course is an introduction to replication and transparency in Political Science empirical research. The course is organized in three sections. The first one provides an overview of basic mathematical notation and statistical reasoning. The second part will introduce the main features of reproducible research using Stata and other tools. The third section will cover linear regression applications and diagnostics using both simulation and replication.

Senior Thesis in Economics, Union College

Economics 498/499, 2015-16
Instructor: Tomas Dvorak
  • Course description

    All economics majors at Union College must write a senior thesis. Most theses use data and regression analysis, though theses that are more narrative or include a business plan are not uncommon. It has always been common practice for faculty to require submission of data and programs with thesis. More and more students use Stata or R to manipulate and analyze their data. However, some students use the software interactively (point and click), and even if they write programs they are rarely organized in a fashion that promotes reproducibility. At the beginning of this year, Tomas held a workshop for senior thesis students to explain the importance of reproducibility and organization of documents and programs along the the lines of TIER protocol.

    The first crop of thesis students that were exposed to the TIER protocol just finished. It remains to be seen how many theses are reproducible. Tomas' own students followed the protocol but ran into a number of challenges. For example, one student used confidential data and thus is not able to post the data; another student used very large data and is still working on organizing all of his programs and data. Tomas plans to survey his colleagues on the extent to which their advisees' theses are reproducible. The first year of using the TIER protocol for senior thesis was harder than anticipated. This is largely because the students' priority is to get results rather than documenting and organizing the way they got to the results.

Business Analytics, Union College

Economics 364, Winter 2016
Instructor: Tomas Dvorak
  • Course description

    This is the first year Tomas has taught a new course entitled Business Analytics. The key learning objective is for students to be able to manipulate and analyze business data. The course is very hands-on with students programming in R Markdown from the first day. The emphasis is on data manipulation: students need to load in data, summarize, reshape, merge and append the data in multiple ways to get insights. In this course students realize that data preparation and manipulation is 90% of the work of an empirical researcher or an analyst. Running a regression or a sophisticated algorithm is the last and fairly straightforward step. The value of any analysis lies mostly in the quality and organization of the data. Using R Markdown forces students to integrate data manipulation and analysis with text. Their final project is always reproducible because it is a knitted R Markdown document. Even if they don't end up using R Markdown in the future (e.g. their thesis), the course gives them programming skills that are essential to reproducible research.

Behavioral Economics, Smith College

Economics 254, Fall 2015
Instructor: Simon Halliday
  • Course description

    I separate learning goals into goals with different verbs: know, understand, comprehend, analyze, synthesize, do, etc.

    • Know the virtues and limitations of the rational actor model and its application to choice theory and the behavioral sciences
    • Understand the role of economics as a constituent of the behavioral sciences
    • Ask meaningful questions with important potential answers
    • Analyze data from experiments and surveys to answer questions relevant to the behavioral sciences
    • Synthesize different ideas, theories and empirics within the behavioral sciences
    • Design well conceived experiments and surveys to answer important questions
    • Do rigorous data analysis to comprehend relevant data

Economic Statistics with Calculus, Haverford College

Economics 204, Spring 2015
Instructor: Richard Ball
  • Course description

    This is an introductory course in applied statistics that serves as a pre-requisite for an econometrics course that all economics majors are required to take.  The course consists of three hours or class per week, plus a weekly two-hour computer lab devoted to an original research project. Working in small groups, students choose a topic; review the relevant literature; find appropriate statistical data; clean and organize the data; and conduct analyses to see what they can learn about the questions they set out to investigate.  At the end of the semester they present their study and findings in a complete research paper, accompanied by comprehensive replication documentation that meets the specifications of the TIER Protocol.  

    This research project is carried out in several installments: a pre-proposal, a full proposal, a data report, and the final paper. Instructions for each of these installments, including detailed instructions for the replication documentation, are provided in the course materials available for download below.

    A course syllabus is also included in the materials available for download.

Seminar in Environmental and Resource Economics, Colgate University

Economics 428, Spring 2015
Instructor: Michael O'Hara
  • Course description

    Course Objectives and Structure:

    This course is your capstone course in the economics curriculum. The focus is on producing a research paper of publication quality.  To accomplish this, you should be able to:

    • •   state and motivate a cogent research question
    • •   produce a credible literature review
    • •   exhibit knowledge of basic economic theory as it relates to environmental and natural resource issues
    • •   exhibit competence of statistical and econometric techniques if the work is of the empirical/statistical type
    • •   produce clear, well reasoned arguments  with a high standard of writing skill.
    • •   provide all documentation as required by the TIER protocol of Ball and Medeiros (2012)

    A seminar course is very different in tone and structure from other instructional courses in the curriculum.  A seminar is a collaborative learning effort by all those involved. It is not the professor teaching the students, but rather the professor and students all learning together from each other. This means that there is far more responsibility on the part of the students to contribute to each other’s learning. 

    Research project:

    • •   You are required to write an independent research paper examining an aspect of a topic in environmental, resource, or ecological economics. Specific details will be forthcoming, but you should begin examining topics immediately!
    • •   The research project could be (1) applied, empirical and statistical; (2) theoretical. It must be of academic interest, and ideally should be of practical interest (at least to someone).
    • •   I expect the paper to be professionally written and in an economics journal style. Details of this will be discussed.
    • •   We will adhere to the criterion that the paper must be fully documented so that its results can be fully replicated by a reader. This means that all data, code, and metadata must be supplied following the TIER protocol described in Ball and Medeiros (2012) which we will discuss in class.
    • •   In the last few weeks of class, each of you will present your findings to the class.

Senior Thesis, Williams College

Senior Thesis, Fall 2014
Instructor: Tara Watson
  • Course description

    For this project, students use the Project TIER Research Protocol.  This protocol, developed at Haverford College, improves transparency, replicability, and self-discipline in the research process.  The final paper is submitted with the raw data files (in the original file format and converted to Stata or other format) and all do files necessary to implement the analysis in the project.  The students' do files are archived and made available to anyone who wishes to replicate the results.

Econometrics, Williams College

Economics 255, Spring 2014
Instructor: Tara Watson
  • Course description

    The purpose of the group empirical project is to allow students to practice the tools they learn in class, and to expose them to the challenges and joys of empirical research.  In groups of 1-3 people, they write a 15-page empirical paper on a topic of their choice.  They give a 10-15 minute presentation to the class at the end of the semester explaining their project and highlighting their findings.  They then submit their paper, raw data, and Stata programs, as described by a research protocol.

Applied Econometrics, Colgate University

Economics 375, Winter 2014
Instructor: Michael O'Hara
  • Course description

    This applied econometrics course is a requirement for all economics majors at Colgate University.  Students in this section of the course were required to write an original empirical research paper, with complete replication documentation constructed according to the guidelines of the TIER Protocol.  

    Detailed instructions for the research paper can be found in the document available for download below.