- The best custom writing site
- +1 (315) 215-1164
- info@goldentechessays.com

The problem set comprises two parts that are weighted equally. The first part includes conceptual questions, and

the second part asks you to conduct experimental data analysis in Stata or R, report and interpret your results.

The research article mentioned in part I, question 6, and the replication dataset for part II, are available on KEATS.

Please submit your answers to the questions, along with an Appendix including your complete Stata or R code on

KEATS in docx or PDF file format. Your file must include:

• Your written answers to the questions (answers need to be numbered).

• Appendix including the entire Stata or R code required to fully reproduce your answers to part II.

Part I: Conceptual Questions

Question 1 – 8 marks

Politicians and educational researchers often claim that smaller class sizes positively affect learning outcomes among

high school students. A researcher decides to test this hypothesis by randomly assigning 40 6th form students to

one of two small classes with 10 students (treatment), or to a large class with 20 students (control). She finds

a relatively large difference in learning outcomes between students who attend the larger class and students who

attend the smaller classes (the students in smaller classes appear to do better) , but the effect is not statistically

significant. The researcher concludes that class size has no influence on learning outcomes.

a Is this a randomized experiment, a natural experiment, a quasi-experiment, or none of the above (an observational study)? Explain your answer. (1 mark)

b How confident are you in the findings of this study? Explain your answer. (3 marks)

c Are there any issues with this research design? How could this research design be improved? (4 marks)

Question 2 – 10 marks

a What is the excludability assumption? Give an example of a published randomized field experiment in which

the excludability assumption could plausibly be violated. (4 marks)

1

b What could researchers in your example have done in order to minimize the risk of violating the excludability

assumption? (6 marks)

Question 3 – 2 marks

a What are potential outcomes? How do they differ from realised outcomes? (1 mark)

b Explain the following notations: Yi(1), Yi(1) | Di = 0, Yi(0) | Di = 1, and Yi(1) | di = 1. (1 mark)

Question 4 – 10 marks

a Give two examples of published randomized field experiments, one facing one-sided non-compliance, and the

other facing two-sided non-compliance. How do the researchers measure compliance in these two experiments?

(5 marks)

b Under two-sided non-compliance, which types of subjects will reveal which potential outcomes in the randomly

assigned control group? Which types of subjects will reveal which potential outcomes in the randomly assigned

treatment group? (5 marks)

Question 5 – 8 marks

Consider the schedule of outcomes in the table below. If treatment A is administered, the potential outcome is

Yi(A), and if treatment B is administered, the potential outcome is Yi(B). If no treatment is administered, the

potential outcome is Yi(0). The treatment effects are defined as Yi(A) − Yi(0) or Yi(B) − Yi(0).

subject Yi(0) Yi(A) Yi(B)

Miriam 1 2 3

Benjamin 2 3 3

Helen 3 4 3

Eva 4 5 3

Billie 5 6 3

Suppose a researcher plans to assign two observations to the control group and the remaining three observations to

just one of the two treatment conditions. The researcher is unsure which treatment to use.

a Applying equation 3.4 in Gerber and Green (2012, page 57) determine which treatment, A or B, will generate

a sampling distribution with a smaller standard error. (4 marks)

b What does the result in part (a) imply about the feasibility of studying interventions that attempt to close

an existing “achievement gap”? (4 marks)

2

Question 6 – 12 marks

Answer the following questions based on your reading of Arceneaux and Nickerson’s (2010) article “Comparing

Negative and Positive Campaign Messages: Evidence From Two Field Experiments”:

a What are the research question, and the research hypotheses? Summarize the research question, hypotheses

and results of the study in no more than four sentences. (2 marks)

b How many experimental conditions are there in study 1 and study 2? Briefly describe the treatments in both

experiments. (2 marks)

c What are the outcome variables in both field experiments? How are they measured and collected? (2 marks)

d Describe the random assignment procedures employed in studies 1 and 2. What is the respective unit of

randomization? Why did the researchers choose a different strategy for study 2 than for study 1? (3 marks)

e “The experimental design of Study 2 offers a few complications not encountered in Study 1” (p. 64). What

are these complications, and how do the researchers address them? Elaborate. (3 marks)

Part II: Experimental analysis

Researchers conducted a randomized field experiment in order to identify the effect of campaign phone calls on

turnout (Foos and de Rooij 2017). The political campaign they were cooperating with managed to reach around

45% of the treatment groups by telephone, and contacted 0% of the randomly assigned control group. The campaign

then collected outcome data about electoral participation from the public voter register and matched it to subjects

assigned to treatment and control groups. The treatment assignment variable “treatment” is binary (0 control, 1

phone call), and the outcome variable of interest is called turnout, coded 0 if a subject did not vote in the election,

and 1 if the subject voted. Using the replication data file “canvassing.dta”:

Exercise 1: ITT estimation – 8 marks

a Estimate the Intent-to-Treat Effect using the difference-in-proportions estimator. Report your ITT estimate,

and interpret it. (3 marks)

b Estimate the ITT using linear regression. Compare your ITT estimate to the one you obtain in exercise 1a.

Explain what you observe. (3 marks)

c Why are you asked to estimate the ITT, and not the ATE? (2 marks)

3

Exercise 2: Randomization inference – 14 marks

a What is the sharp null hypothesis in this experiment? (2 marks)

b Using randomization-inference, conduct a one-tailed and a two-tailed test of the sharp null hypothesis. Report

the one-tailed and two-tailed p-values. Can you reject the sharp null hypothesis in both instances? When is

it appropriate to choose a one-tailed instead of a two-tailed test? (6 marks)

c Estimate the standard error and 95% confidence-interval using linear regression with robust standard errors.

Report the standard error and 95% confidence interval. What is the interpretation of the confidence interval?

(6 marks)

Exercise 3: CACE estimation – 14 marks

a Estimate and report the Complier Average Causal Effect (CACE), based on an instrumental variable regression

with contact as the endogenous variable (D), and treatment assignment as the exogeneous variable (Z). (4

marks)

b Estimate and report the robust standard error. (2 marks)

c Report the one-tailed and two-tailed p-values. Is your CACE estimate significantly different from 0? (3 marks)

d True or false? “The smaller the ITTD , the larger the CACE.” Explain your answer. (5 marks)

Exercise 4: Blocking – 14 marks

a The actual random assignment in this experiment was blocked on pre-treatment party-support. Re-estimate

the ITT, accounting for random assignment within the three exprimental blocks. The block variable is called

“pid”. Report your ITT estimate, and interpret it. (6 marks)

b Using randomization-inference, conduct one-tailed and a two-tailed tests of the sharp null hypothesis, accounting for random assignment within experimental blocks. Report and discuss the result. Does blocking

narrow the sampling distribution of the ITT? Why/why not? (4 marks)

c Accounting for experimental blocks, report the CACE, the associated standard error, and two-tailed p-value.

Interpret your results.