# Stakes and Signals: An Empirical Investigation of Muddled Information in Standardized Testing

*Germán Reyes, Evan Riehl, Ruqing Xu*  
*May 2026*

**Abstract.** We examine a natural experiment in Brazil in which similar students took the same standardized test as either a low-stakes school accountability exam or a high-stakes admission exam for the country's top universities. Using administrative data and a difference-in-differences design, we find that test score gaps between high- and low-income students expanded on the high-stakes exam, consistent with wealthy students engaging in test prep. Yet the increase in stakes made scores more informative for students' college outcomes. Thus the "muddling" of information on natural ability and test prep improved the quality of the score signal, although it also exacerbated inequality.

# Introduction

Selecting the right talent is crucial to the success of organizations. A fundamental challenge faced by recruiters is incomplete information on candidate quality. To address this challenge, recruiters often rely on signals of candidate quality from mechanisms like standardized tests or case interviews. Yet candidates have an incentive to manipulate their signals through preparation or even outright cheating, particularly when the positions they are applying for are highly desirable.[^1]

An important question for the design of talent selection mechanisms is whether the potential for manipulation degrades the quality of the signal. Social scientists have long hypothesized that strategic manipulation erodes the reliability of signals, an idea that is known as Goodhart's Law (Goodhart, 1975) or Campbell's Law (Campbell, 1979).[^2] Theoretical models of "muddled information" (Frankel and Kartik, 2019) show that, as the stakes of a mechanism increase, signals become more informative about an individual's *gaming ability* and less informative about the *natural action* individuals would take absent signaling concerns.[^3] But it is theoretically ambiguous whether recruiters would prefer to observe signals of candidate quality with or without gaming. Gaming ability may reflect a candidate's knowledge of the recruiting mechanism or manipulation skills that are unrelated to productivity, but it could also reflect work ethic, interest, or other desirable attributes.

This paper conducts an empirical evaluation of the relationship between stakes and signal quality in the context of admission exams for elite universities. We exploit a unique natural experiment in Brazil in which a low-stakes test that measured high school quality was repurposed into a high-stakes admission exam for the country's most selective universities. Our empirical strategy holds the structure of the exam and the composition of exam takers fixed and asks how the increase in exam stakes impacted two important outcomes: 1) test score gaps between advantaged and disadvantaged students; and 2) the predictive power of test scores for individuals' academic potential.

It is *ex ante* unclear how test score inequality and informativeness change as the stakes of an exam increase. Absent signaling concerns, the students who perform better on exams may be those with high intrinsic motivation, conscientiousness, and aptitude (Kreps, 1997; Bénabou and Tirole, 2003). This is consistent with a common belief among education researchers that low-stakes test scores are a better measure of student learning than high-stakes scores because there is less of an incentive to manipulate performance (e.g., Amrein and Berliner, 2002). Further, critics often argue that the use of high-stakes college admission exams helps wealthy students game the system through expensive test prep and other score-boosting strategies (Buchmann, Condron, and Roscigno, 2010; Soares, 2015).[^4] Thus an increase in exam stakes could widen demographic test score gaps and also degrade the quality of the test score signal. On the other hand, "gaming ability" could reflect skills that are important for college success like work ethic, grit, or the capacity to learn new material. The distribution of these skills may be less related to family background than the distribution of low-stakes test scores. In this case, higher stakes could reduce socioeconomic test score gaps while also making exam scores more informative about college potential.

To provide empirical evidence on these relationships, we examine the rollout of a national standardized admission exam for elite Brazilian universities. From 2009–2017, Brazil's system of highly selective federal universities transitioned from their own university-specific admission exams to a common test called the ENEM (*Exame Nacional do Ensino Médio*). Federal universities in different states varied in the timing at which they adopted the ENEM in admissions (Machado and Szerman, 2021; Mello, 2022). Importantly, the ENEM was also used to measure school quality, so many high school seniors took the exam regardless of its role in college admissions. Since most Brazilian students attend college close to home, this meant that some ENEM participants took a low-stakes (from their perspective) school quality exam, while others took a high-stakes test that governed admission to the most selective universities in their state. We define a sample of high school seniors who were likely to take the ENEM regardless of its role in college admissions, and exploit the variation in exam stakes across states and cohorts in a difference-in-differences design.

To implement our design, we link administrative records from the ENEM exam to nationwide college and labor market data. The ENEM data include individuals' responses to each exam question, which allows us to ask how the increase in exam stakes affected students' overall scores in each subject as well as their performance on different types of exam questions. We measure college enrollment, persistence, and graduation outcomes using the 2010–2019 waves of Brazil's higher education census. Lastly, we measure labor market wages using Brazil's national employer-employee data for the years 2016–2018. Using these data, we ask how the increase in the stakes of the ENEM impacted test score gaps between advantaged and disadvantaged students and the informativeness of scores as measured by the correlation coefficient between test scores and outcomes.

We have two main findings. First, test score gaps between advantaged and disadvantaged students widened on the higher stakes exam. Gaps in average ENEM scores between private and public high school students expanded by roughly 10 percent when federal universities adopted the ENEM in admissions (relative to the mean gaps in pre-adoption cohorts). This increase was driven by private school students earning higher scores on the high-stakes exam. Racial and other socioeconomic test score gaps expanded by a similar percentage. The magnitude of our estimates implies that there was a significant increase in the selectivity of the university programs that private school students could gain access to.

Second, the increase in exam stakes caused ENEM scores to become *more* informative for students' academic potential. The adoption of the ENEM exam by federal universities increased the correlation coefficients between ENEM scores and students' college persistence and graduation outcomes by roughly 10–30 percent, depending on the outcome measure. The predictive power of scores increased both overall and measured among students who attended the same college programs, which shows that our findings are driven by an increase in the informativeness of scores rather than by a causal impact of the scores on the programs students attended. Our results on the informativeness of ENEM scores for earnings are inconclusive because our labor market data is measured too early in students' careers.

To shed light on mechanisms, we show that the higher-stakes test led to an improvement in private school students' performance across a wide range of exam skills. The adoption of the ENEM by federal universities led to especially large increases in the scores of students who attended private high schools with test-prep-oriented curriculum, suggesting that our results are partly driven by exam preparation. But our question-level analysis shows that private students performed better across a wide range of exam skills that are aligned with high school and college curricula. Further, exam skills in which we observe larger improvements in private students' performance also tend to be *more* predictive of college outcomes relative to other skills. This suggests that test prep for the higher-stakes ENEM was not confined to narrowly-targeted skills that merely raise exam scores; rather, private students' score gains reflected a broad set of skills that are informative for academic potential.

Our paper provides empirical evidence that informs theoretical work on how gaming impacts signals of performance. Many signaling models assume that the principal's goal is to minimize the agent's ability to manipulate their signal (e.g., Holmstrom and Milgrom, 1991; Ederer, Holden, and Meyer, 2018; Perez-Richet and Skreta, 2022; Ball, 2024). Models of "muddled information" (Fischer and Verrecchia, 2000; Frankel and Kartik, 2019; Frankel and Kartik, 2022) show that signals become relatively more informative about an agent's *gaming ability* when the stakes of the interaction increase. But as Frankel and Kartik (2019) acknowledge, it is unclear whether gaming makes signals more or less informative for outcomes of interest to observers. We show that the influence of gaming ability on standardized tests makes the resulting scores more informative for academic potential. This challenges the common assumption that the gaming of college admission exams necessarily distorts or adds noise to the signal (Krishna et al., 2022; Lee and Suen, 2023). Our findings suggest that gaming ability can instead reflect beneficial characteristic like learning capacity or non-cognitive skills.

Relatedly, our paper is unique within research in the economics of education in showing that higher stakes can increase the informativeness of exams. A large empirical literature finds that educators strategically respond to high-stakes testing by teaching narrowly-defined skills (Jones et al., 1999; Jacob, 2005; Cohodes, 2016), manipulating the test-taking population (Figlio and Getzler, 2006; Cullen and Reback, 2006), prioritizing specific subjects or students (Neal and Schanzenbach, 2010; Reback, Rockoff, and Schwartz, 2014), or even resorting to outright cheating (Jacob and Levitt, 2003). Related work finds that gains in high-stakes test scores from accountability policies do not always translate into improved performance on low-stakes tests (Klein et al., 2000; Jacob, 2005; Corcoran, Jennings, and Beveridge, 2011). For this reason, many education researchers have argued that low-stakes exam scores are a better measure of student learning (e.g., Koretz and Barron, 1998; Amrein and Berliner, 2002). Yet ours is the first paper in this literature to use data on longer-run outcomes to directly test how the informativeness of scores varies between high- and low-stakes exams. We show that, in the context of college admissions, higher stakes tests can provide a better signal of students' academic potential.

Lastly, our paper shows that there is a tradeoff between equality and informativeness in the use of high-stakes college admission exams. There is an ongoing and high-profile debate on whether selective U.S. colleges should use standardized exams in college admissions (Belasco, Rosinger, and Hearn, 2015; Bennett, 2022; Dessein, Frankel, and Kartik, 2023; Dynarski et al., 2023). A central question in this debate is whether admission scores are "biased" in favor of high-income and white students because they have greater access to test prep.[^5] A vast literature shows that there are large gaps in college admission exam scores by race and socioeconomic status (e.g., Bowen and Bok, 1998; Kane, 1998; Card and Rothstein, 2007; Goodman, Gurantz, and Smith, 2020), while other work shows that admission scores are predictive of longer-run outcomes (Rothstein, 2004; Bettinger, Evans, and Pope, 2013; Chetty, Deming, and Friedman, 2023; Riehl, 2023; Friedman, Sacerdote, and Tine, 2024).[^6] But a limitation with papers in both of these literatures is that they focus on a static admissions environment, so it is unclear how test score gaps or informativeness would change if universities used alternative admission criteria. Consistent with the common criticism of admission tests, we find that racial and socioeconomic gaps in performance expand on higher stakes assessments, and we show that test prep is a likely mechanism for this result. Yet we do not find that test prep creates bias in scores as a measure of college performance. Thus our findings suggest that universities face a fundamental tradeoff between equality and informativeness in choosing whether or not to use admission tests.

The paper proceeds as follows. Section 2 provides background on Brazilian higher education and the ENEM exam. Section 3 describes our data and empirical strategy. Sections 4 and 5 present our results on test score inequality and informativeness. Section sec:con concludes.

# Institutional background
## Colleges and high schools in Brazil

The higher education system in Brazil is heavily privatized, but the most prestigious institutions tend to be in its system of *federal universities*. In 2009, there were 59 federal universities, with a presence in all of Brazil's 27 states. Together, federal universities account for about 11 percent of total tertiary enrollment. Brazil also has a system of 40 *state universities* managed by the governments of each state. Federal and state universities are tuition-free, highly selective, and consistently at the top of national college rankings. The Brazilian higher education system additionally includes over 2,000 private universities and technical colleges that enroll roughly 80 percent of all college students. While a handful of these private institutions are elite and selective, the majority are moderately selective or have essentially open enrollment.

The situation is reversed at the secondary level, where private high schools represent a small but socioeconomically-advantaged share of enrollment. In 2009, 14 percent of secondary students attended a private high school, while 85 percent attended a public school managed by the state government.[^7] But students from private high schools are vastly overrepresented in higher education; in 2009, they accounted for 40 percent of all incoming college students and 47 percent of federal university enrollees (Appendix Table 7).

## Federal university admissions and the ENEM exam
Admission to federal universities is highly competitive and is based exclusively on test scores from entrance exams.[^8] Before 2009, each federal university designed and administered its own admission tests, which are called *vestibular* exams. This made it burdensome for students to apply to more than one university as they had to prepare for multiple tests and travel to each school on a specific date to sit for the exam.

To centralize federal university admissions, the Ministry of Education developed a national standardized college admission exam called the ENEM (*Exame Nacional do Ensino Médio*). The ENEM exam was initially created in 1998 for the purpose of providing public information on high school performance. Between 1998 and 2008, the ENEM exam consisted of 63 multidisciplinary questions and an essay. In 2009, the Ministry redesigned and expanded the exam so that it could serve as a tool for college admissions. The post-2009 ENEM exam resembles the ACT exam in the United States; it contains 180 questions across four subject areas (math, language arts, natural sciences, and social sciences) along with a written essay. The exam spans two days of testing and is taken by over five million students each November, making it the second-largest admission test globally. As part of the centralization effort, the Ministry also created a unified admission platform called SISU (*Sistema de Seleção Unificada*) that matches students to colleges based on their preferences and ENEM scores.

Although the college admission version of the ENEM exam began in 2009, federal universities varied in the timing at which they switched from their institution-specific tests to the ENEM. The Ministry of Education provided financial incentives to adopt the ENEM, but universities had unilateral control over their admission methods, and some were initially uncertain about the content of the new ENEM (Machado and Szerman, 2021). Thus, some federal universities began using the ENEM immediately in 2009, while others adopted it five or more years later.[^9] As we describe Section 3, our empirical strategy exploits this variation in the timing of ENEM adoption by federal universities.

## The market for test prep
Preparation for college entrance exams is a central part of the lives of many Brazilian students as they approach the end of high school. Many students who hope to gain admission to elite universities choose to attend private high schools that use test-oriented curricula designed by for-profit companies (e.g., *Sistema Anglo* and *Sistema pH*). These schools offer multiple courses throughout the day that focus specifically on exam subjects and preparation strategies. Private schools frequently tout the successful exam performance of prior cohorts to attract new students, and newspapers publish annual rankings of school-mean scores. In addition, many Brazilian students study for the exams outside of school hours or after completing high school by taking for-profit prep courses known as *cursinhos* (Mitrulis and Penin, 2006; Fernandes, 2015).

The existence of this for-profit market for test prep raises a concern about inequality in access to Brazil's selective universities. Although there are a growing number of non-profit and free online services, test prep remains heavily concentrated in the private sector. Lower-income students often cannot afford to enroll in private high schools or *cursinhos*, and there is typically less emphasis on test prep in public school curricula (Silva, 2014). Our first empirical analysis asks whether these disparities in access to test prep contribute to inequality in access to selective colleges.

# Data and identification
## Data

Our base dataset includes administrative records on all individuals who took the ENEM exam in 2007–2017 (INEP, 2019). This dataset is compiled by the National Institute of Educational Studies, or INEP (*Instituto Nacional de Estudos e Pesquisas Educacionais*). The data contains scores on each exam subject, demographic characteristics, and information on individuals' high schools. We also observe individuals' responses to each exam question, which allows us to measure the questions that individuals got right and wrong. The data for each question includes the learning objectives, the Item Response Theory (IRT) parameters, and the question text.

To measure longer-run outcomes, we link the ENEM data to two other administrative datasets at the individual level (see Appendix 9.2 for details on the merge.). First, we measure college outcomes by linking to INEP's higher education census (*Censo da Educação Superior*) for the years 2010–2019 (INEP, 2022). This dataset contains information on the universe of Brazilian college students, including each student's university, major, admission method, enrollment year, and graduation/drop-out outcome. Second, we measure labor market outcomes by linking to Brazil's employee-employer dataset, the RAIS (*Relação Anual de Informações Sociais*), for the years 2016–2018 (RAIS, 2022). The RAIS is maintained by the Ministry of Labor and covers the entire population of formal-sector workers in Brazil.

## Sample
We begin by defining a sample with a consistent composition of ENEM exam takers over time. The total number of ENEM exam takers increased significantly after the exam was converted into a college admissions test in 2009, as illustrated by the black bars in Panel A of Figure 1. Since our goal is to examine how the increase in the exam's stakes impacted the distribution of scores, we define a sample in which the number of test takers remained relatively constant over these years. For this, we take advantage of the fact that many high school students took the ENEM in their senior year regardless of its stakes because of the exam's role as a measure of high school performance.

Our analysis sample includes high school seniors at schools that met the criteria to be included in the government's performance reports in each year from 2005–2015. To define our sample, we use a dataset that contains school-level mean ENEM scores from 2005–2015, which were computed by INEP and distributed to federal and municipal agencies for publication (INEP, 2019).[^10] Schools were included in the performance reports if a large fraction of their students participated in the exam.[^11] Our analysis sample includes only ENEM exam takers who: 1) are in their last year of high school; and 2) attended a school that appears in the INEP school-level dataset in *each* year from 2005 to 2015. The red bars in Panel A of Figure 1 show that our analysis sample contains a small subset of all ENEM participants, but the number of exam takers in our sample remains relatively constant between 2007 and 2017. Section 3.5 presents tests for balance in our sample.

Table 1 shows that our analysis sample is positively selected on socioeconomic status and academic performance relative to other ENEM test takers. This table reports mean demographic characteristics (Panel A), ENEM scores (Panel B), and college and labor market outcomes (Panel C) for 2009–2017 ENEM participants. Columns (A)–(C) show statistics for all ENEM exam takers, all high school seniors, and high school seniors in our analysis sample, respectively. Our sample contains roughly 2.5 million high school seniors, which is six percent of all ENEM test takers and 22 percent of all high school seniors. On average, students in our sample are four years younger than the typical ENEM participant, and they are roughly ten percentage points (pp) more likely to be white and to have a college-educated parent. Relative to the average test taker and the average high school senior, students in our sample score about 0.2–0.3 standard deviations (SD) higher on each ENEM subject.[^12]

Despite this positive selection, there is substantial inequality between private and public high school students in our sample. Columns (D)–(F) of Table 1 report statistics for private school students, public school students, and the private/public gap. 32 percent of students in our sample attended a private high school. Relative to public school students, private school students were 26pp more likely to be white, 44pp more likely to have a college-educated mother, and 52pp more likely to come from a high-income family. Mean ENEM score gaps are on the order of 1 SD; the test score gap is largest in math, with private school students scoring 1.4 SDs higher than public school students on average.[^13] There is also substantial inequality in college and labor market outcomes. Private school students were 27pp more likely to go to college, 15pp more likely to attend a federal university, and had hourly wages 68 percent higher than public school students.

## ENEM exam stakes
Our identification strategy exploits the gradual adoption of the ENEM exam by federal universities. The solid red line in Panel B of Figure 1 plots the proportion of all federal university enrollees in each year who were admitted using the ENEM exam. Although the college admissions version of the ENEM exam was first administered in December 2009, only 28 percent of federal university students nationwide were admitted using the ENEM in the following year.[^14] The proportion of federal university seats that were allocated using the ENEM grew over subsequent years as more institutions switched from their own tests to the ENEM, reaching a peak of 72 percent in 2016.

This gradual adoption created geographic variation in the exam's stakes because Brazilian students typically attend college in their home state. The black dashed line in Panel B of Figure 1 plots the proportion of federal university enrollees who attended college in the state where they were born. On average, 81 percent of federal university students are from the state where the university is located. Although there is evidence that the ENEM exam increased geographic mobility (Machado and Szerman, 2021), these effects were modest; the proportion of in-state students at federal universities remained above 80 percent throughout 2010–2018. Thus, the stakes of the ENEM exam varied across states and cohorts for students who wished to attend a federal university close to home.

We use this variation to define two measures of ENEM stakes at the state $\times$ year level. Our benchmark measure, which we denote by $\text{ProportionENEM}_{st}$, is a continuous variable that equals the proportion of federal university enrollees in state $s$ and year $t$ who were admitted using the ENEM exam. This continuous treatment variable exploits all variation in ENEM adoption timing. In particular, $\text{ProportionENEM}_{st}$ reflects variation in ENEM adoption across federal universities within the same state as well as variation in the use of the ENEM across programs within the same university.

Second, we define a binary treatment variable that equals one in years after each state "adopted" the ENEM exam. For this, we follow research on tipping points (e.g., Card, Mas, and Rothstein, 2008) in identifying structural breaks in the time series of federal universities' use of the ENEM. For each state $s$, we regress an annual time series of the proportion of federal university enrollees who were admitted using the ENEM on a linear trend break function for each possible candidate adoption year $\tau_s$. We define the state's *ENEM adoption year* as the value $\tau^*_s$ that yields the highest $R^2$ across these regressions.[^15] Our binary measure, which we denote by $\text{HighStakes}_{st}$, is an indicator for years equal to or after the state's ENEM adoption year, $\tau^*_s$.[^16] Our binary treatment variable allows us to present our results using event study graphs, and it helps to address potential concerns about two-way fixed effects models with treatment effect heterogeneity (discussed below).

Figure 2 shows the relationship between our continuous and binary measures of ENEM stakes. In Panel A, we categorize Brazil's 27 states into ten groups based on their year of ENEM adoption, $\tau^*_s$. The graph plots the mean of $\text{ProportionENEM}_{st}$ in these groups ($y$-axis) for each ENEM exam year ($x$-axis). In each group, the proportion of federal university students who were admitted through the ENEM increases sharply in the state's ENEM adoption year. Panel B presents an event-study version of Panel A, in which the $x$-axis denotes years relative to the state's ENEM adoption year. On average, the share of a state's federal university admission slots allocated using the ENEM increased by over 60 percent in the adoption year, and this share remains high in subsequent years. Appendix Table 8 shows the values of $\text{ProportionENEM}_{st}$ and $\text{HighStakes}_{st}$ in each state and exam year.

## Regression models
Our benchmark regression model is a two-way fixed effects specification estimated at the high school $\times$ year level:

$$\begin{equation}
Y_{ht} = \gamma_{s(h)} + \gamma_{t} + \beta \text{ProportionENEM}_{s(h)t}  + \epsilon_{ht}. \end{equation}$$

$Y_{ht}$ is an average outcome for students who attended high school $h$ and took the ENEM in year $t$. We include fixed effects for years, $\gamma_{t}$, and for the states in which each high school is located, $\gamma_{s(h)}$. The variable of interest is our continuous treatment variable, $\text{ProportionENEM}_{s(h)t}$, which measures the stakes of the ENEM exam in state $s(h)$ and cohort $t$. In alternate specifications, we replace $\text{ProportionENEM}_{s(h)t}$ with our binary treatment variable, $\text{HighStakes}_{s(h)t}$. We weight our regressions by the number of individuals in each $ht$ cell to recover population estimates within our sample. Our benchmark regressions include high school seniors who took the college admissions version of the ENEM exam in 2009–2017, which holds the structure of the ENEM exam fixed over time. We cluster standard errors at the state level.

The coefficient of interest, $\beta$, measures how outcomes changed in a school when the stakes of the ENEM exam increased. We estimate equation dd separately for public and private high school students to examine how the increase in exam stakes affected scores in these two populations. In addition, we estimate regressions that fully interact the covariates in equation dd with an indicator for private high schools, $\text{Private}_h$:

$$\begin{equation}
Y_{ht} = \gamma_{s(h)} + \gamma_{t} + \beta \text{ProportionENEM}_{s(h)t} + \big[\tilde{\gamma}_{s(h)} + \tilde{\gamma}_{t} + \beta^{\text{gap}} \text{ProportionENEM}_{s(h)t}\big]\text{Private}_h + \nu_{ht}. \end{equation}$$

The $\beta^{\text{gap}}$ coefficient in equation dd_gap shows how the increase in exam stakes impacted the private/public gap in ENEM scores.

To address potential concerns about treatment effect heterogeneity in two-way fixed effects models (De Chaisemartin and d'Haultfoeuille, 2020), we present robustness results that restrict identification to clean comparisons based on states' ENEM adoption years. Our approach follows Callaway and Sant'Anna (2021) in estimating treatment effects for each pair of ENEM adoption years, $\tau^*_s$ and $\tau^*_{s'}$, and then averaging the pairwise treatment effects to recover a single point estimate. For example, one of our pairs contains states that adopted the ENEM in 2010 and 2011, and we restrict the sample to students who took the exam in 2009–2010. In this pair, the 2010 adopters are our treated group since ENEM adoption "switches on" in 2010, and the 2011 adopters are our control ("not yet treated") group since these states had not yet adopted the exam. We define groups for all pairwise combinations of ENEM adoption years and, within each pair, we restrict the sample to exam cohorts prior to the control group's adoption year. We create a stacked dataset of these pairwise samples and estimate a version of equation dd_gap that uses our binary treatment variable, $\text{HighStakes}_{st}$, and includes interactions with dummies for the pairwise groups. The resulting $\beta^{\text{gap}}$ coefficients are regression-weighted averages of the pairwise treatment effects.[^17]

## Identification assumptions and balance tests
Our identification relies on a parallel trends assumption, which requires that the timing of federal universities' switch to the ENEM exam is unrelated to state-level trends in potential test score outcomes. This assumption could be violated if the adoption of the ENEM exam induced students to enroll in different high schools or impacted the types of students who took the exam.

Table 2 tests this assumption by examining whether the adoption of the ENEM was related to trends in exam taking and school enrollment (Panel A) or the demographic characteristics of students in our analysis sample (Panels B–C). Column (A) shows the mean of each dependent variable in cohorts prior to the state's ENEM adoption year. Columns (B)–(D) present $\beta$ coefficients from equation dd, which we estimate separately for all schools, private schools, and public schools. Column (E) reports $\beta^{\text{gap}}$ coefficients from equation dd_gap, which are equivalent to the differences between the $\beta$ coefficients in columns (C) and (D).

We find no evidence that the adoption of the ENEM exam caused students to attend different high schools or impacted the number of exam participants in our sample. Panel A of Table 2 shows regression results using three outcome variables: 1) the log number of ENEM participants per school/cohort at the high schools in our sample; 2) the log number of seniors per school/cohort at these high schools; and 3) the number of distinct schools that these seniors had attended in the past three years. We find that the increase in ENEM stakes did not significantly affect the number of exam takers or the number of seniors at either public or private schools in our sample. It also did not induce students to switch schools at a higher rate. This evidence is consistent with our prior that the incentives to attend a prep-oriented high school did not change significantly when federal universities switched from their own admission exams to the ENEM; rather, it mainly affected prep schools' incentives on *which* exam to focus on in their curriculum.

Panels B–C of Table 2 show that the increase in ENEM stakes also did not significantly impact the composition of students in our analysis sample. In Panel B, we find no systematic relationship between ENEM stakes and the age, race, parental education, or family income of exam takers in our sample. We do find that a 100 percentage point increase in federal universities' ENEM adoption is associated with a 1.4 percentage point decrease in the fraction of ENEM participants who were female, but this effect is small and it is similar in both public and private high schools. We cannot reject the hypothesis that the coefficients on all demographic characteristics are jointly equal to zero in any subsample (last row of Panel B). In Panel C, we also find no impacts on an index of *predicted* ENEM scores based on all of these demographic characteristics.

Finally, we find no systematic differences in the size, selectivity, or student body characteristics of federal universities that were early- vs. late-adopters of the ENEM exam (Appendix Table 9). For example, universities in the most populous state, São Paulo, adopted the ENEM immediately in 2009, while federal universities in the next two largest states, Minas Gerais and Rio de Janeiro, did not adopt it until 2013. In net, the evidence supports the assumption of parallel trends in potential exam score outcomes within our analysis sample.

# Exam stakes and the distribution of scores
## Effects on test scores

Table 3 presents our main results on how the stakes of the ENEM impacted mean test scores. Column (A) displays the mean private/public school gap in test scores in cohorts prior to the state's ENEM adoption year. Columns (B)–(D) present $\beta$ coefficients from equation dd estimated separately by high school type. Column (E) displays $\beta^{\text{gap}}$ coefficients from equation dd_gap. Our dependent variables are individuals' test scores in SD units averaged at the high school $\times$ cohort level. We examine scores on each of the four multiple-choice tests (math, language arts, natural science, social science), average scores across these four core subjects, and scores on the writing component.

The increase in the stakes of the ENEM led to a widening of private/public test score gaps. Private students' scores increased on the higher-stakes exam in each of the four core subjects (column C), with the largest effect in math (0.143 SDs). Public students' scores did not change significantly on the core subjects (column D), and test score gaps between private and public school students increased (column E). Our point estimate for the average core subject scores implies that a 100 percentage point increase in the adoption of the ENEM by federal universities is associated with a 0.11 SD increase in the private/public test score gap. This effect is nine percent of the mean test score gap in lower-stakes cohorts (column A). We also find that the private/public gap in writing scores expanded by 0.10 SDs.

Figure 3 shows that test score gaps typically widened in the first ENEM exam cohort after its adoption by federal universities. This figure presents estimates from an event study version of equation dd_gap using our binary treatment variable, $\text{HighStakes}_{st}$, and our stacked dataset of pairwise ENEM adoption years. This yields coefficients $\beta^{\text{gap}}_l$ that show how the private/public score gap changed in each year $l$ relative to the state's ENEM adoption year, $\tau^*_s$.[^18] In most subjects, we do not see significant pre-trends in the private/public score gap prior to the ENEM adoption year. In all subjects, we find that the private/public score gap increased by roughly 0.05 to 0.10 SDs in the first cohort after ENEM adoption. These wider gaps decline only slightly in subsequent cohorts. For example, the private/public gap in average core subject scores increased by 0.08 SDs in the year of ENEM adoption, and it was still 0.05 SDs higher measured four years later (Panel E).

The magnitudes of our estimates represent meaningful increases in private students' chances of gaining admission to federal university programs. The effect of higher stakes on the private/public gap in average core subject scores ($\beta^{\text{gap}} =  0.11$) is 21 percent of a standard deviation in the distribution of cutoff scores for admission to federal university programs.[^19] To put this magnitude in perspective, consider a private school student whose low-stakes ENEM score would have put them exactly at the cutoff for a program at the 50th percentile of their state's distribution of federal university programs. Our estimate of $\beta^{\text{gap}}$ implies that this student's high-stakes ENEM score would instead make them eligible for a program at the 58th percentile.

Panel A of Figure 4 shows that racial and socioeconomic test score gaps also expanded on the higher stakes ENEM exam (see also Appendix Table 12). The white bars represent mean gaps in average core subject scores in low-stakes cohorts for different demographic groups. The grey bars represent estimates of $\beta^{\text{gap}}$ from a version of equation dd_gap that replaces $\text{Private}_h$ with a dummy for the more advantaged group. We find that the gap in average scores between white/non-white students expanded by 0.06 SDs on the higher-stakes test. Similarly, the average score gap between students with college/non-college educated mothers expanded by 0.08 SDs, and the gap by family income expanded by 0.09 SDs. These point estimates are smaller than our estimate for the private/public high school gap, but they are similar as a percentage of the mean gap in low-stakes cohorts. We do not find a significant effect on the male/female test score gap.[^20]

## Robustness to model specification

Table 4 examines whether our results on private/public test score gaps are sensitive to model specification. Column (A) reproduces our benchmark estimates of $\beta^{\text{gap}}$ from column (E) of Table 3. Columns (B)–(F) present estimates of $\beta^{\text{gap}}$ from alternative specifications.

Our results are robust to including demographic controls and to using a binary measure of exam stakes. In column (B) of Table 4, we estimate equation dd_gap including high school $\times$ year averages of age, gender, and dummies for race, parental education, and family income bins. These demographic controls do not significantly alter our point estimates, which is consistent with the findings of our balance tests in Table 2. In column (C), we replace our continuous treatment variable, $\text{ProportionENEM}_{st}$, with our binary measure of ENEM stakes, $\text{HighStakes}_{st}$. This specification reduces the magnitudes of $\beta^{\text{gap}}$ by about 50 percent in each subject, which is expected since $\text{ProportionENEM}_{st}$ increases by roughly 50 percent following a state's adoption of the ENEM (Figure 2, Panel B). Yet we continue to find that the increase in exam stakes widened private/public test score gaps in each subject, and the coefficient for the average core subject score remains statistically significant at $p<0.05$.

Next, we examine the robustness of our results to potential concerns about two-way fixed effects models with treatment effect heterogeneity. For this, we use three different samples from the stacked dataset described in Section 3.3 (see also Appendix Table 10). Column (D) of Table 4 includes all pairwise combinations of ENEM adoption years that we can estimate using 2009–2017 exam takers. Column (E) focuses on a single pairwise comparison between the two most common ENEM adoption years—2009 and 2013—which together account for 13 states (see Appendix Table 9). This sample includes 2007–2012 test takers because we require a pre-period to estimate a treatment effect for 2009 adopters. In column (F), we include all 2007–2017 test takers and all pairwise combinations in our stacked dataset.[^21]

Our benchmark estimates are robust to each of these alternative specifications. The point estimates in column (D) are similar to those in column (C), which shows that our results are not impacted by restricting identification to clean pairwise comparisons. We continue to find positive and significant estimates of $\beta^{\text{gap}}$ when we restrict to the simple model that compares only 2009 and 2013 adopters (column E). Lastly, our results are similar in the full stacked dataset with 2007–2017 test takers (column F). The consistency of estimates across specifications shows that our results are not the result of averaging oppositely-signed treatment effects with negative weights.

## Other robustness tests

Our results are robust to alternate definitions of our analysis sample. Our benchmark sample includes seniors at high schools that met the criteria to be included in the government's ENEM performance reports in each year from 2005–2015 (see Section 3.2). Appendix Table 11 includes samples defined by both laxer criteria (e.g., schools that appear in *any* year) and stricter criteria (e.g., at least a 50% ENEM participation rate in all years). In all samples, we find positive and statistically significant estimates of $\beta^{\text{gap}}$ for average core subject scores. With the exception of writing scores, the magnitudes of our estimates for each subject are relatively stable across samples.

Our estimates in Table 3 are mostly robust to an alternative method of statistical inference for settings with a relatively small number of clusters. Our benchmark estimates allow for correlated errors within each of Brazil's 27 states, which is below the rule of thumb for potential few-cluster issues in Angrist and Pischke (2010). Appendix Table 13 shows that our estimate of $\beta^{\text{gap}}$ for average core subject scores remains significant at $p<0.05$ using $p$-values from the wild $t$ bootstrap procedure recommended by Cameron, Gelbach, and Miller (2008). The score gap estimates for other subjects remain significant at $p<0.10$ except for natural science and writing.

Lastly, our results are not driven by federal universities' adoption of affirmative action or changes in the size of their admission quotas. Many federal universities implemented reserved quotas for disadvantaged students during the late 2000s and early 2010s (Mello, 2022), which could have impacted the achievement of high school seniors through a motivational channel (Akhtari, Bau, and Laliberté, 2020). To examine this possibility, we use Brazil's higher education census to compute the fraction of new university students in each state $\times$ year who enrolled through reserved quotas and then add this variable as a control in our regressions. Appendix Table 14 shows that private/public test score gaps are not significantly related to the rollout of affirmative action and that our estimates of $\beta^{\text{gap}}$ remain positive and significant with these controls. Appendix Table 15 shows that our treatment variable is unrelated to the total number of federal university enrollees, which suggests that federal universities did not alter the number of admission slots when they switched to the ENEM exam.

## Mechanisms

Our finding that private students earned higher scores on the high-stakes ENEM exam may be driven by several mechanisms. One possibility is that the increase in staked induced students to exert more effort while taking the exam. The typical private school student had a better chance of gaining admission to federal universities than the typical public student, so private students had a stronger incentive to increase effort when the exam stakes increased. There is significant overlap between the distribution of private school ENEM scores and the distribution of admission cutoff scores for federal university programs, while the public school score distribution is shifted well to the left (Appendix Figure 8).[^22] Thus, moderate increases in ENEM scores were unlikely to significantly affect the admission chances of many public students. Although we cannot observe student effort in our data, this may partly explain why we only find increases in private students' scores.

The increase in private students' scores could also reflect an increase in ENEM-specific test preparation. Students who wished to attend selective college programs may have switched their preparation efforts from the *vestibular* exams to the ENEM exam when federal universities adopted the ENEM. Anecdotally, many private schools and test-prep companies altered their curriculum to focus more on the skills and content covered by the ENEM. By contrast, test prep is not a major focus of the curriculum at most public high schools, in part because the ENEM is a very difficult test for the typical public student. Thus the teaching practices at public high schools were less likely to change with the ENEM rollout, which may explain the expansion of the private/public ENEM score gap.[^23]

We empirically examine the role of exam preparation using two measures of students' test prep activity. First, we obtained lists of schools that use test-oriented curricula from the websites of four prominent test prep companies (*Sistema Anglo*, *Sistema pH*, *Elite Rede de Ensino*, and *Curso Objetivo*) and merged these lists to our sample of high schools using geocoded addresses. This allows us to define a set of "prep schools" whose curriculum is specifically focused on preparation for college admission exams.[^24] Second, we use a variable from the ENEM questionnaire that indicates whether individuals took an entrance exam preparation course. This question does not distinguish between courses that focused on the ENEM exam and courses that focused on other *vestibular* exams, and, likely for this reason, we do not find evidence that ENEM adoption impacted the proportion of students who took a test prep course. Yet, if these courses were more likely to focus on the ENEM exam after its adoption by federal universities, this could raise the average ENEM scores of students who took preparation courses. Appendix Table 16 provides details on these measures of test prep as well as the associated regression results.

Panel B of Figure 4 shows that the increase in ENEM stakes led to larger test score gaps between students who did/did not engage in test prep as defined by these measures. ENEM adoption led to a 0.18 SD increase in the gap in mean ENEM scores between prep-focused private schools and public schools (second row in Panel B), which is roughly 60 percent larger than our point estimate for the overall private/public gap (0.11 SDs, first row). The third row of Panel B shows that prep schools had slightly lower average scores than other private schools in our sample in the low-stakes cohorts, and this gap closed by 0.08 SDs ($p<0.05$) following ENEM adoption. Lastly, we find that the increase in ENEM stakes led to a large increase in the score gap between students who did/did not take a test prep course (fourth row), with a point estimate of 0.23 SDs. These heterogeneity results suggest that test prep is an important mechanism for the increase in the private/public test score gap.

In sum, this section showed that test score gaps between advantaged and disadvantaged students expanded when the stakes of the ENEM exam increased. Critics of high-stakes tests often argue that they give wealthy students a leg up in the college admission process. Our findings are consistent with this claim. Yet is it unclear whether higher stakes cause exams to be "biased" in favor of high-income students, as the question of bias depends on what the exam is intended to measure. To shed light on the relationship between exam stakes and informativeness, we now turn to our second empirical analysis.

# Exam stakes and the informativeness of scores
## Potential channels
Are high-stakes exam scores more or less informative for students' academic potential than low-stakes scores? The answer, according to both economic literature and public debates on standardized testing, remains unclear.[^25]

On the one hand, an increase in exam stakes may reduce the informativeness of scores by distorting effort toward activities that improve test performance rather than activities that promote beneficial learning. In their seminal paper on incentive contracts, Holmstrom and Milgrom (1991) highlight the possibility that teachers who are rewarded for student test performance may focus on "the narrowly defined basic skills that are tested on standardized exams." Frankel and Kartik (2019) show theoretically that an increase in the stakes of an exam causes test scores to become relatively more informative about individuals' *gaming ability* and relatively less informative about their *natural action* that would arise in the absence of signaling concerns. Similarly, critics of high-stakes testing often argue that the ability to "game the system" through test prep may be unrelated—or even negatively related—to an individual's potential for academic success (e.g., Harris et al., 2011).

Yet it is also possible that high-stakes testing encourages students to reveal or develop skills that are beneficial for their academic careers. As Frankel and Kartik (2019) note, an individual's gaming ability for standardized tests could reflect work ethic or the capacity to learn new material. Moreover, if high-stakes scores are more correlated with family income, they may be more predictive of college outcomes because family resources help students succeed in school. High-stakes testing may also compel students to accumulate new skills that benefit them beyond the exam. Students may learn important academic material if the exams are well-aligned with school curricula (Lazear, 2006). They may also develop non-cognitive skills like cognitive endurance (Brown et al., 2022; Reyes, 2023) or a growth mindset (Dweck, 2006) while preparing for exams.

## Outcome variables and regression model

To shed light on the relative strength of these mechanisms, we ask how the increase in the stakes of the ENEM impacted the informativeness of ENEM scores for students' college and labor market outcomes. Our sample for this analysis includes the subset of students in our high school senior sample who took the ENEM exam in 2009–2014, excluding the 2011 cohort because of a data issue.[^26] We linked this sample to Brazil's higher education census to measure enrollment, persistence, and graduation outcomes at all Brazilian colleges in the years 2010–2019. We measure earnings outcomes using Brazil's national employer-employee data for the years 2016–2018.

We define three types of outcome variables based on the sample for which we can measure each outcome (see Appendix app:var_def for details). Our first set of variables include outcomes that we can define for all ENEM participants in our sample. These include an indicator for enrolling in any college, an indicator for completing a college degree during our data period, and an indicator for appearing in the RAIS dataset (a measure of formal employment). Second, we define measures of college persistence and graduation within the subsample of individuals who enrolled in college. These outcomes include indicators for persisting in college several years after enrolling and an indicator for completing the program within five years. Lastly, for individuals who appear in the RAIS dataset, we measure labor market earnings using an individual's mean hourly wage from 2016–2018 (measured in both logs and levels). Many of the ENEM participants in our sample were still in college during 2016–2018, and even those who had left college were early in their careers. Thus it is important to stress that our earnings outcomes do not reflect the long-run returns to individuals' college investments.

We modify our regression model to examine how federal universities' adoption of the ENEM impacted the *correlation* between students' ENEM scores and longer-run outcomes. Our regression model for exam informativeness is a state $\times$ year version of equation dd:

$$\begin{equation}
  Y_{st} = \gamma_{s} + \gamma_{t} + \beta \text{ProportionENEM}_{st}  + \epsilon_{st}.
\end{equation}$$ The dependent variable, $Y_{st}$, is the correlation coefficient between ENEM scores and a longer-run outcome among students who attended high school in state $s$ and took the ENEM in year $t$. This specification follows testing agencies' standard practice of measuring exam validity using correlations coefficients (e.g., Kobrin et al., 2008).[^27] As above, the variable of interest, $\text{ProportionENEM}_{st}$, is the proportion of federal university enrollees in state $s$ and year $t$ who were admitted using the ENEM exam. Thus $\beta$ measures the impact of a 100 percentage point increase in ENEM adoption on the correlation between ENEM scores and outcomes.

In complementary regressions, we also follow exam validity research in restricting comparisons to students who attended the same college programs. In addition to the potential mechanisms discussed in Section 5.1, a student's ENEM scores may be correlated with their longer-run outcomes through their direct impact on which college and/or major they gained admission to. Further, many federal universities that adopted the ENEM simultaneously joined the SISU centralized college admission system, which likely impacted the matching of students to college programs (see Section 2.2). To reduce the influence of these direct impacts on student/college matches and isolate the predictive power of scores, we residualize both ENEM scores and outcomes on dummies for college $\times$ major pairs and then compute correlation coefficients using these residuals. These residual correlations reflect variation in the informativeness of scores among students who attended the same college programs.[^28]

## Effects on informativeness

Table 5 shows our main results on how federal university adoption of the ENEM exam impacted the informativeness of ENEM scores. Column (A) shows the mean correlation coefficient between average core subject scores and each outcome variable in cohorts prior to ENEM adoption. Columns (B)–(C) present $\beta$ coefficients from equation dd_styr using our continuous treatment variable, $\text{ProportionENEM}_{st}$. Columns (D)–(E) present $\beta$ coefficients using our binary treatment variable, $\text{HighStakes}_{st}$. In columns (B) and (D), the dependent variables, $Y_{st}$, are raw correlation coefficients. In columns (C) and (E), the dependent variables are correlation coefficients measured after residualizing ENEM scores and outcomes on college $\times$ major dummies.

We find that scores on the higher-stakes ENEM exam were *more* informative for students' college enrollment, persistence, and graduation outcomes. Panel A of Table 5 shows that the informativeness of average ENEM scores for both college enrollment and college degree attainment increased when federal universities adopted the ENEM in admissions. The point estimates in column (B) imply that a $100$ percentage point increase in the adoption of the ENEM is associated with a $0.036$ increase in the correlation between average ENEM scores and an indicator for college enrollment, and a $0.033$ increase in the correlation between average ENEM scores and an indicator for completing a college degree by 2019. We find similar (but less precise) estimates using the binary treatment variable (column D). The results are also similar we when compare degree attainment outcomes for students who attended the same college programs (columns C and E). This suggests that our findings are not driven by direct impacts of ENEM performance or the SISU system on the program that students attended.

The increase in the informativeness of ENEM scores is even more pronounced for college persistence outcomes measured within the population of college enrollees. In Panel B of Table 5, the outcome variables include indicators for persisting in college one and three years after enrolling as well as an indicator for completing the program within five years. The mean correlations between these outcomes and ENEM scores tend to be lower than for the outcomes in Panel A, but the estimated $\beta$ coefficients in columns (B)–(E) are broadly similar in the two panels. Thus as a percentage of the mean correlation coefficients in lower-stakes cohorts, the impact of the higher-stakes exam on the informativeness of scores was larger for college persistence outcomes. For example, the estimates in column (B) imply that a 100 percentage point increase in ENEM adoption is associated with a $23$ percent increase in the correlation between ENEM scores and 3-year persistence rates, and a $62$ percent increase in the correlation between scores and 5-year graduation rates. Appendix Table 17 shows that the results are mostly robust to the wild t bootstrap procedure.

We do not find conclusive evidence on the relationship between exam stakes and the informativeness of scores for hourly wages. Panel C of Table 5 shows that scores on the higher-stakes exam also became more correlated with hourly wages measured in levels, but these results are not robust to using log wages. Further, the coefficients for both wage levels and log wages lose statistical significance when we use a wild t cluster bootstrap for inference (Appendix Table 17). These inconclusive results are likely due to the fact that we observe wages for only one-quarter of students in our sample because many individuals were still in college during our data period.

Figure 5 shows that informativeness increased in all four core subjects. The translucent areas depict the mean correlations in lower-stakes cohorts (analogous to column A of Table 5), and the darker areas depict the $\beta$ coefficients from our benchmark regression model (analogous to column B of Table 5). The increase in exam stakes caused ENEM scores to become more correlated with degree completion, 3-year college persistence, and 5-year program completion in each of the four core subjects, and the magnitudes are relatively similar in each subject. We also find increases in the informativeness of writing scores, but these are smaller in magnitude and imprecisely estimated.

## Correlation with demographics

Why were high-stakes ENEM scores more predictive of college success? As noted above, one possibility is that high-stakes scores may have been more correlated with other individual characteristics that help students succeed in college, such as family income or parental education. To assess this potential channel, we show how demographic controls impact the results on exam informativeness.

Figure 6 plots $\beta$ coefficients from different specifications of equation dd_styr. The white bars reproduce the results from our benchmark specification (Table 5, column B). The darker-colored bars show $\beta$ coefficients including four sets of demographic controls: 1) a dummy for private high schools; 2) dummies for family income, parental education, and race; 3) gender and age; and 4) high school dummies. In each case we include the new control variables plus the controls from the previous specification. Thus, these $\beta$ coefficients show how ENEM adoption impacted the exam's capacity to identify academic potential among students from similar demographic groups. Appendix Table 18 shows the corresponding regression results.

Demographic variables explain some, but not all, of the increase in the informativeness of ENEM scores. For example, the $\beta$ coefficient for 3-year college persistence rates falls by 37 percent when we control for private high school and socioeconomic variables, and the coefficient for 5-year program completion rates falls by 60 percent. The impact of private high school and socioeconomic controls is similar or more modest for most other outcomes. Gender and age do not explain much of the increase in informativeness for most outcomes. The inclusion of high school dummies reduces the $\beta$ coefficients by more than 80 percent for college persistence and degree completion, but these dummies do not have additional explanatory power for college enrollment or unconditional college graduation.

Importantly, these findings show that the higher-stakes ENEM scores provided new information on students' academic potential above and beyond easily observed demographic characteristics. By the end of our sample period, most federal universities had reserved quotas for public high school, low-income, and/or Black applicants, but otherwise admissions were based solely on ENEM performance. Thus our specification with private school, family income, and race controls shows that the higher-stakes ENEM exam helped federal universities identify applicants who were more likely to succeed within the set of demographic characteristics that they considered in admissions.

## Narrow vs. broad-based learning
To further explore mechanisms, we use question-level ENEM data to examine whether the gains in private students' scores were driven by narrowly-targeted or broad-based learning. ENEM questions are based on a reference matrix of skills that educators regard as important for students to learn in high school. For example, math questions cover *topic areas* such as algebra, geometry, and statistics. Within each topic area, questions are designed to test specific *competencies* such as identifying concepts, solving problems, and constructing arguments. Our data includes individuals' responses to each question, which allows us to estimate our regression model dd_gap separately for questions in each topic area or competency.[^29] In these regressions, the dataset is at the high school ($h$) $\times$ year ($t$) $\times$ question ($q$) level, and the dependent variable is the proportion of correct answers in each $htq$ cell. This would allow us to measure the impact of ENEM adoption on students' question-level performance. We focus on math performance in the main text because it is the subject with the largest increase in ENEM score gaps (Table 3) and because math exams are often thought to be more amenable to test prep (Riehl and Welch, 2023). Appendix Table 19 shows results for questions in language arts, natural science, and social science.

Panel A of Table 6 begins by showing results that pool across all 405 math questions in our sample (9 years $\times$ 45 questions per exam). Column (C) reports the mean proportion of correct answers for public students in cohorts prior to the state's ENEM adoption year, and column (E) reports the mean private/public gap in these cohorts. The average public school student answered 29.1 percent of the questions correctly, and the private/public gap in correct answers was 17.6pp. Columns (D) and (F) report the $\beta$ and $\beta^\text{gap}$ coefficients from equation dd_gap. The estimate of $\beta^\text{gap}$ in Panel A implies that a 100 percentage point increase in ENEM adoption by federal universities is associated with a 2.4pp increase in the private/public gap in correct responses averaged across all questions. This is consistent with our finding for scale scores in Table 3.

Panels B–C of Table 6 show that the private/public gap in correct responses increased across a wide range of math topic areas and competencies. These panels display results from estimating equation dd_gap separately for each topic area (Panel B) and competency (Panel C). We find positive and statistically significant estimates of $\beta^\text{gap}$ in all seven math topic areas, with estimates ranging from 1.3pp in algebra to 3.6pp for questions on proportions. The estimates at the competency level are less powered since these regressions typically include only 10–15 questions across all years, but the $\beta^\text{gap}$ coefficients are positive and greater than 0.8pp in 29 out of the 30 competencies. Appendix Table 19 shows that the private/public gap in correct responses also increased across a wide range of topic areas in language arts, natural science, and social science.

Although the estimates of $\beta^\text{gap}$ in Table 6 are uniformly positive, there is substantial variation in these coefficients across exam skills. In Figure 7, we ask whether the exam skills with the largest gains in private students' performance are more or less informative for college persistence. The $x$-axes depict the $\beta^\text{gap}$ coefficients from column (F) of Table 6 estimated at the topic area (Panel A) and competency (Panel B) levels. The $y$-axes display the informativeness of each topic area or competency for college persistence rates. We define informativeness as the average difference in 3-year persistence rates between individuals who answered each question correctly and those who did not, calculated across all questions within the same topic area or competency.[^30] For example, our measure of informativeness for the algebra topic area is 0.046, which means that students who correctly answered a typical algebra question were 4.6pp more likely to persist in their college program for three years than students with an incorrect answer. Figure 7 includes exam skills from each of the four core subjects, as illustrated by the marker colors and symbols. The dashed line shows the linear relationship between exam skill informativeness and the $\beta^\text{gap}$ coefficients.

Exam skills that experienced larger increases in the private/public gap in correct responses tend to be *more* informative for college persistence. Correct answers on the ENEM are highly informative for individuals' academic potential; the $y$-axes of Figure 7 show that students with a correct answer to an average question were roughly 5pp more likely to persist in college for three years than students with a wrong answer. Exam skills that contributed more to the expansion of the private/public test score gap (larger $\beta^{\text{gap}}$ coefficients) also tend to be more informative for college persistence. For example, at the topic area level, math questions on proportions and interpreting data have the largest $\beta^{\text{gap}}$ coefficients, and they are also among the most informative exam skills.

There is also a positive relationship between $\beta^{\text{gap}}$ and skill informativeness for most other college and labor market outcomes. Appendix Table 20 shows competency-level OLS regressions of informativeness on $\beta^{\text{gap}}$ for each of the outcomes in Table 5. We estimate regressions that pool across all four core subjects as well as separate regressions for each subject. In the pooled regressions, we find positive and mostly significant OLS coefficients for every outcome, with a particularly strong relationship between the $\beta^{\text{gap}}$ coefficients and informativeness for wages. The positive relationship between informativeness and $\beta^{\text{gap}}$ also arises within competencies on the math and language arts exams; we find no significant relationship in natural science or social science. This positive relationship is robust to including controls for question difficulty and other IRT parameters (Appendix Table 21). Although the magnitudes of the OLS coefficients are slightly reduced, we find no evidence that skills with larger $\beta^{\text{gap}}$ coefficients are less informative for student outcomes.

## Discussion

The results in this section run counter to the common criticism that high-stakes exams cause students to prepare for narrow exam skills that do not benefit them outside of the test. The adoption of the ENEM by federal universities increased the overall predictive power of scores for college enrollment, persistence, and graduation. Further, the increase in ENEM stakes caused private school students to perform better across a wide range of exam skills, and these performance gains were driven by skills that are more informative for longer-run outcomes.

Our mechanism analysis indicates that the increase in informativeness is partly due to an increase in the correlation between test scores and socioeconomic factors that influence both access to test prep and success in college. Yet our findings suggest that the higher stakes exam also helped students reveal or develop other harder-to-measure dimensions of academic potential. One of the government's objectives in redesigning the ENEM exam was to create a test that is better aligned with high school and college curriculum than many of the university-specific *vestibular* exams. Thus the adoption of the ENEM by federal universities may have redirected students' preparation efforts toward material that benefited them in college and/or rewarded students who had learned this material in high school.

# Conclusion

Every year, admission committees at elite universities allocate scarce slots among many applicants on whom they have limited information. These committees seek to bolster their schools' reputations by admitting talented students (MacLeod et al., 2017), and so they consider signals of candidate quality such as standardized test scores, high school grades, and personal essays. Yet candidates have a strong incentive to manipulate these signals due to the perceived value of attending an elite university. This process gives an advantage to individuals who have the know-how and resources to improve their application credentials.

This paper examined how incentives to manipulate performance affect the distribution and informativeness of university admission scores. On the one hand, we found that test score gaps between private and public high school students increased when elite Brazilian universities began using the exam in admissions. In the language of theoretical work on muddled information (Frankel and Kartik, 2019), private school students had relatively higher "gaming ability" for standardized admission tests. Gaming ability may partially reflect access to test preparation resources; indeed, many wealthy Brazilian students take expensive prep courses and attend private high schools that focus on exam preparation. In this sense, our findings corroborate the concern that high-stakes admission exams give a leg up to wealthy students.

On the other hand, our paper showed that incentives to manipulate exam performance can actually *improve* the quality of the test score signal from the standpoint of admission committees. We found that the predictive power of exam scores for college enrollment, persistence, and graduation increased when top Brazilian universities adopted the exam in admissions. This suggests that gaming ability may also reflect characteristics that help students succeed in college such as a willingness to exert effort, the capacity to learn new material, or family resources. This finding runs counter to the common narrative that test prep causes exam scores to be biased as a measure of academic potential.

Our findings highlight the challenge that universities face in seeking to admit both academically-prepared and diverse student bodies. Our paper shows that this problem is *not* solely due to demographic gaps in pre-college achievement; rather, the high-stakes nature of university admissions exacerbates the tension between diversity and informativeness. Brazilian universities balance these objectives by reserving admission slots for disadvantaged applicants and then admitting students who have the highest test scores within each pool. U.S. colleges are more constrained in their ability to consider demographic characteristics, particularly in the wake of the Supreme Court's 2023 ban on race-based affirmative action. Improved access to low-cost test prep through organizations like *Descomplica* in Brazil and the Khan Academy in the United States may help, but there is limited evidence on their effectiveness.

More broadly, the tradeoff between diversity and informativeness is likely to arise in other high-stakes settings such as recruiting at prestigious firms. We hope future research will shed light on the consequences of muddled information in other education and labor markets.

<div class="singlespace">

# References
<div class="thebibliography">

Akhtari, M., N. Bau, and J.-W. P. Laliberté (2020). Affirmative action and pre-college human capital. NBER Working Paper No. 27779.

Amrein, A. L. and D. C. Berliner (2002). High-stakes testing & student learning. *Education Policy Analysis archives* *10*, 18–18.

Angrist, J. D. and J.-S. Pischke (2010). The credibility revolution in empirical economics: How better research design is taking the con out of econometrics. *Journal of Economic Perspectives* *24*(2), 3–30.

Azmat, G., C. Calsamiglia, and N. Iriberri (2016). Gender differences in response to big stakes. *Journal of the European Economic Association* *14*(6), 1372–1400.

Ball, I. (2024). Scoring strategic agents. *American Economic Journal: Microeconomics (Accepted)*.

Belasco, A. S., K. O. Rosinger, and J. C. Hearn (2015). The test-optional movement at America's selective liberal arts colleges: A boon for equity or something else? *Educational Evaluation and Policy Analysis* *37*(2), 206–223.

Bénabou, R. and J. Tirole (2003). Intrinsic and extrinsic motivation. *The review of economic studies* *70*(3), 489–520.

Bennett, C. T. (2022). Untested admissions: Examining changes in application behaviors and student demographics under test-optional policies. *American Educational Research Journal* *59*(1), 180–216.

Bettinger, E. P., B. J. Evans, and D. G. Pope (2013). Improving college performance and retention the easy way: Unpacking the ACT exam. *American Economic Journal: Economic Policy* *5*(2), 26–52.

Bhattacharya, D., S. Kanaya, and M. Stevens (2017). Are university admissions academically fair? *Review of Economics and Statistics* *99*(3), 449–464.

Bowen, W. G. and D. Bok (1998). *The Shape of the River: Long-term Consequences of Considering Race in College and University Admissions*. Princeton University Press.

Brown, C. L., S. Kaur, G. Kingdon, and H. Schofield (2022). Cognitive endurance as human capital. NBER Working Paper No. 30133.

Buchmann, C., D. J. Condron, and V. J. Roscigno (2010). Shadow education, american style: Test preparation, the sat and college enrollment. *Social forces* *89*(2), 435–461.

Bulman, G. (2015). The effect of access to college assessments on enrollment and attainment. *American Economic Journal: Applied Economics* *7*(4), 1–36.

Cai, X., Y. Lu, J. Pan, and S. Zhong (2019). Gender gap under pressure: Evidence from China's national college entrance examination. *Review of Economics and Statistics* *101*(2), 249–263.

Callaway, B. and P. H. Sant'Anna (2021). Difference-in-differences with multiple time periods. *Journal of Econometrics* *225*(2), 200–230.

Cameron, A. C., J. B. Gelbach, and D. L. Miller (2008). Bootstrap-based improvements for inference with clustered errors. *The Review of Economics and Statistics* *90*(3), 414–427.

Campbell, D. T. (1979). Assessing the impact of planned social change. *Evaluation and program planning* *2*(1), 67–90.

Card, D., A. Mas, and J. Rothstein (2008). Tipping and the dynamics of segregation. *The Quarterly Journal of Economics* *123*(1), 177–218.

Card, D. and J. Rothstein (2007). Racial segregation and the black–white test score gap. *Journal of Public Economics* *91*(11-12), 2158–2184.

Cawley, J., J. Heckman, and E. Vytlacil (1999). On policies to reward the value added by educators. *Review of Economics and Statistics* *81*(4), 720–727.

Chetty, R., D. J. Deming, and J. N. Friedman (2023). Diversifying society's leaders? the causal effects of admission to highly selective private colleges. NBER Working Paper No. 31492.

Cohodes, S. R. (2016). Teaching to the student: Charter school effectiveness in spite of perverse incentives. *Education Finance and Policy* *11*(1), 1–42.

College Board (2017). Suite of Assessments Annual Report. <https://reports.collegeboard.org/media/pdf/2017-total-group-sat-suite-assessments-annual-report.pdf> (accessed July 2023).

Corcoran, S. P., J. L. Jennings, and A. A. Beveridge (2011). Teacher effectiveness on high-and low-stakes tests. Society for Research on Educational Effectiveness.

Cullen, J. B. and R. Reback (2006). Tinkering toward accolades: School gaming under a performance accountability system. In *Improving school accountability*, Volume 14, pp. 1–34. Emerald Group Publishing Limited.

De Chaisemartin, C. and X. d'Haultfoeuille (2020). Two-way fixed effects estimators with heterogeneous treatment effects. *American Economic Review* *110*(9), 2964–96.

Dessein, W., A. Frankel, and N. Kartik (2023). Test-optional admissions. arXiv preprint arXiv:2304.07551.

Dweck, C. S. (2006). *Mindset: The new psychology of success*. Random house.

Dynarski, S., A. Nurshatayeva, L. C. Page, and J. Scott-Clayton (2023). Addressing nonfinancial barriers to college access and success: Evidence and policy implications. In *Handbook of the Economics of Education*, Volume 6, pp. 319–403. Elsevier.

Ederer, F., R. Holden, and M. Meyer (2018). Gaming and strategic opacity in incentive provision. *The RAND Journal of Economics* *49*(4), 819–854.

Fernandes, S. (2015). Neoliberalization of education in Brazil: the impact of cursinhos and the private sector on pedagogical practices and access to university. *Canadian Journal of Latin American and Caribbean Studies/Revue canadienne des études latino-américaines et caraı̈bes* *40*(3), 351–367.

Figlio, D. N. and L. S. Getzler (2006). Accountability, ability and disability: Gaming the system? In *Improving school accountability*, Volume 14, pp. 35–49. Emerald Group Publishing Limited.

Fischer, P. E. and R. E. Verrecchia (2000). Reporting bias. *The Accounting Review* *75*(2), 229–245.

Frankel, A. and N. Kartik (2019). Muddled information. *Journal of Political Economy* *127*(4), 1739–1776.

Frankel, A. and N. Kartik (2022). Improving information from manipulable data. *Journal of the European Economic Association* *20*(1), 79–115.

Friedman, J., B. Sacerdote, and M. Tine (2024). Standardized test scores and academic performanceat ivy-plus colleges. Technical report, Opportunity Insights.

Goodhart, C. (1975). Problems of monetary management: the uk experience in papers in monetary economics. *Monetary Economics* *1*.

Goodman, J., O. Gurantz, and J. Smith (2020). Take two! SAT retaking and college enrollment gaps. *American Economic Journal: Economic Policy* *12*(2), 115–58.

Goodman, S. (2016). Learning from the test: Raising selective college enrollment by providing information. *Review of Economics and Statistics* *98*(4), 671–684.

Harris, P., B. M. Smith, J. Harris, et al. (2011). *The myths of standardized tests: Why they don't tell you what you think they do*. Rowman & Littlefield Publishers.

Holmstrom, B. and P. Milgrom (1991). Multitask principal-agent analyses: Incentive contracts, asset ownership, and job design. *Journal of Law, Economics and Organization* *7*(1), 24–52.

Hoxby, C. and S. Turner (2013). Expanding college opportunities for high-achieving, low income students. Stanford Institute for Economic Policy Research Discussion Paper.

INEP (2019a). Microdata for ENEM exam (Exame Nacional do Ensino Médio). Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, Brasília, Brazil. <https://www.gov.br/inep/pt-br/acesso-a-informacao/dados-abertos/microdados/enem> (accessed September 2019).

INEP (2019b). School-level ENEM scores (Exame Nacional do Ensino Médio). Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, Brasília, Brazil. <https://www.gov.br/inep/pt-br/acesso-a-informacao/dados-abertos/microdados/enem-por-escola> (accessed September 2019).

INEP (2022a). Microdata for Higher Education Census (Censo da Educação Superior). Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, Brasília, Brazil. <https://www.gov.br/inep/pt-br/acesso-a-informacao/dados-abertos/microdados/censo-da-educacao-superior> (accessed April 2022).

INEP (2022b). Microdata for School Census (Censo Escolar). Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, Brasília, Brazil. <https://www.gov.br/inep/pt-br/acesso-a-informacao/dados-abertos/microdados/censo-da-educacao-superior> (accessed April 2022).

Instituto Nacional de Estudos e Pesquisas Educacionais An�sio Teixeira (Inep) (2009, junho). Relat�rio pedag�gico: Enem 2008.

Jacob, B. and J. Rothstein (2016). The measurement of student ability in modern assessment systems. *Journal of Economic Perspectives* *30*(3), 85–108.

Jacob, B. A. (2005). Accountability, incentives and behavior: The impact of high-stakes testing in the Chicago Public Schools. *Journal of Public Economics* *89*(5-6), 761–796.

Jacob, B. A. and S. Levitt (2003). Rotten apples: An investigation of the prevalence and predictors of teacher cheating. *The Quarterly Journal of Economics* *118*(3), 843–877.

Jones, M. G., B. D. Jones, B. Hardin, L. Chapman, T. Yarbrough, and M. Davis (1999). The impact of high-stakes testing on teachers and students in north carolina. *The Phi Delta Kappan* *81*(3), 199–203.

Kane, T. J. (1998). Racial and ethnic preferences in college admissions. *Ohio St. LJ* *59*, 971.

Klein, S. P., L. Hamilton, D. F. McCaffrey, B. Stecher, et al. (2000). What do test scores in texas tell us? *Education policy analysis archives* *8*, 49–49.

Kobrin, J. L., B. F. Patterson, E. J. Shaw, K. D. Mattern, and S. M. Barbuti (2008). Validity of the SAT® for predicting first-year college grade point average. research report no. 2008-5. Technical report, College Board.

Koretz, D. M. and S. I. Barron (1998). *The validity of gains in scores on the Kentucky Instructional Results Information System (KIRIS).* ERIC.

Kreps, D. M. (1997). Intrinsic motivation and extrinsic incentives. *The American economic review* *87*(2), 359–364.

Krishna, K., S. Lychagin, W. Olszewski, R. Siegel, and C. Tergiman (2022). Pareto improvements in the contest for college admissions. NBER Working Paper No. 30220.

Lazear, E. P. (2006). Speeding, terrorism, and teaching to the test. *The Quarterly Journal of Economics* *121*(3), 1029–1061.

Lee, F. X. and W. Suen (2023). Gaming a selective admissions system. *International Economic Review* *64*(1), 413–443.

Machado, C. and C. Szerman (2021). Centralized college admissions and student composition. *Economics of Education Review* *85*, 102184.

MacLeod, W. B., E. Riehl, J. E. Saavedra, and M. Urquiola (2017). The big sort: College reputation and labor market outcomes. *American Economic Journal: Applied Economics* *9*(3), 223–261.

Mello, U. (2022). Centralized admissions, affirmative action, and access of low-income students to higher education. *American Economic Journal: Economic Policy* *14*(3), 166–97.

Mitrulis, E. and S. T. d. S. Penin (2006). Pré-vestibulares alternativos: da igualdade à eqüidade. *Cadernos de Pesquisa* *36*, 269–298.

Muller, J. (2018). *The tyranny of metrics*. Princeton University Press.

Neal, D. and D. W. Schanzenbach (2010). Left behind by design: Proficiency counts and test-based accountability. *The Review of Economics and Statistics* *92*(2), 263–283.

Nielsen, E. (2023). Test questions, economic outcomes, and inequality. Working Paper.

OECD (2021). *Education in Brazil: An International Perspective*. OECD Publishing, Paris.

Ors, E., F. Palomino, and E. Peyrache (2013). Performance gender gap: does competition matter? *Journal of Labor Economics* *31*(3), 443–499.

Pallais, A. (2015). Small differences that matter: Mistakes in applying to college. *Journal of Labor Economics* *33*(2), 493–520.

Perez-Richet, E. and V. Skreta (2022). Test design under falsification. *Econometrica* *90*(3), 1109–1142.

RAIS (2022). elação Anual de Informações Sociais. Ministério do Trabalho e Emprego, Brasília, Brazil. <http://www.rais.gov.br/> (accessed April 2022).

Reback, R., J. Rockoff, and H. L. Schwartz (2014). Under pressure: Job security, resource allocation, and productivity in schools under No Child Left Behind. *American Economic Journal: Economic Policy* *6*(3), 207–41.

Reyes, G. (2023). Cognitive endurance, talent selection, and the labor market returns to human capital. arXiv preprint arXiv:2301.02575.

Riehl, E. (2023). Do less informative college admission exams reduce earnings inequality? evidence from Colombia. *Journal of Labor Economics*. Forthcoming.

Riehl, E. and M. Welch (2023). Accountability, test prep incentives, and the design of math and English exams. *Journal of Policy Analysis and Management* *42*(1), 60–96.

Rosenbaum, M., A. Savage, G. A. Ellis, K. B. Farkas, and M. L. Lavetter (2020). Kawika Smith et al. vs. Regents of the University of California, Superior Court of the State of California, County of Alameda, case no. RG19046222. <https://publiccounsel.org/wp-content/uploads/2022/02/2020.06.15-Amended-Complaint.pdf>.

Rothstein, J. M. (2004). College performance predictions and the SAT. *Journal of Econometrics* *121*(1), 297–317.

Silva, R. (2014, Dec). Escolas privadas e técnicas dominam o Enem. *Jornal Cruzeiro do Sul*. <https://www2.jornalcruzeiro.com.br/materia/587165/escolas-privadas-e-tecnicas-dominam-o-enem> (Accessed June 2023).

Soares, J. A. (2015). *SAT wars: The case for test-optional college admissions*. Teachers College Press.

</div>

</div>

# Figures and tables
<figure id="fig:enrol" data-latex-placement="ht">
<div class="centering">
<p><embed src="results/enem_takers.pdf" /> <embed src="results/enem_adoption.pdf" /></p>
</div>
<p><em>Notes</em>: This figure illustrates time variation in the number of ENEM exam takers and the proportion of federal university enrollees who were admitted using the ENEM.</p>
<p>Panel A shows the total number of individuals who took the ENEM each year from 2007 to 2017. Each bar displays the overall number of ENEM exam takers (black bars) and the number of exam takers in our analysis sample (red bars). Panel B shows the proportion of new enrollees in federal universities admitted through the ENEM exam in each year from 2009 to 2018 (red solid line) and the proportion of new enrollees in federal universities who attended a university in their birth state from 2010 to 2018 (black dashed line).</p>
<figcaption>Adoption of ENEM exam by federal universities</figcaption>
</figure>

<figure id="fig:event_adoption" data-latex-placement="t">
<div class="centering">
<p><embed src="results/adoption_share.pdf" /> <embed src="results/adoption_share_legend.pdf" /> <embed src="results/event_enem_pct.pdf" /></p>
</div>
<p><em>Notes</em>: This figure illustrates the staggered adoption of the ENEM exam by federal universities. The outcome in both panels is the proportion of new enrollees in federal universities in state <span class="math inline"><em>s</em></span> who were admitted using the ENEM exam administered in year <span class="math inline"><em>t</em></span> (the calendar year prior to enrollment), denoted as <span class="math inline">ProportionENEM<sub><em>s</em><em>t</em></sub></span>.</p>
<p>Panel A plots the mean of <span class="math inline">ProportionENEM<sub><em>s</em><em>t</em></sub></span> for groups of states based on their ENEM adoption year, <span class="math inline"><em>τ</em><sub><em>s</em></sub><sup>*</sup></span>, as indicated in the legend. See Section <a href="#sub:stakes" data-reference-type="ref" data-reference="sub:stakes">3.3</a> for the definition of ENEM adoption years, <span class="math inline"><em>τ</em><sub><em>s</em></sub><sup>*</sup></span>.</p>
<p>Panel B plots event-study coefficients, <span class="math inline"><em>β</em><sub><em>l</em></sub></span>, from the state (<span class="math inline"><em>s</em></span>) <span class="math inline">×</span> year (<span class="math inline"><em>t</em></span>) <span class="math inline">×</span> pairwise group (<span class="math inline"><em>g</em></span>) level regression:</p>
<p><span class="math display">$$\begin{equation*}
\text{ProportionENEM}_{stg} = \gamma_{sg} + \gamma_{tg} + \sum_{l = -7}^{7} \beta_l \mathbbm{1}\{t-\tau^*_{s}=l\} + \epsilon_{stg},
\end{equation*}$$</span></p>
<p>where <span class="math inline"><em>l</em></span> denotes years relative to <span class="math inline"><em>τ</em><sub><em>s</em></sub><sup>*</sup></span>. Pairwise groups (<span class="math inline"><em>g</em></span>) are defined described in Section <a href="#sub:regressions" data-reference-type="ref" data-reference="sub:regressions">3.4</a>. The regression includes state <span class="math inline">×</span> group dummies (<span class="math inline"><em>γ</em><sub><em>s</em><em>g</em></sub></span>), year <span class="math inline">×</span> group dummies (<span class="math inline"><em>γ</em><sub><em>t</em><em>g</em></sub></span>), and dummies for years <span class="math inline"><em>l</em></span> (<span class="math inline">$\mathbbm{1}\{t-\tau^*_{s}=l\}$</span>), with <span class="math inline"><em>l</em> = −1</span> omitted as the reference year. The graph plots the <span class="math inline"><em>β</em><sub><em>l</em></sub></span> coefficients from <span class="math inline"><em>l</em> = −4</span> to <span class="math inline"><em>l</em> = 4</span>. Dashed lines depict 95% confidence intervals using standard errors clustered at the state level.</p>
<figcaption>Variation in ENEM adoption by federal universities across states and years</figcaption>
</figure>

<div class="landscape">

<figure id="fig:event_scores" data-latex-placement="t">
<div class="centering">
<div class="minipage">
<embed src="results/event_math.pdf" />
</div>
<div class="minipage">
<embed src="results/event_lang.pdf" />
</div>
<div class="minipage">
<embed src="results/event_ssci.pdf" />
</div>
<div class="minipage">
<embed src="results/event_nsci.pdf" />
</div>
<div class="minipage">
<embed src="results/event_overall.pdf" />
</div>
<div class="minipage">
<embed src="results/event_writ.pdf" />
</div>
</div>
<p><em>Notes</em>: This figure presents event study estimates of the impact of ENEM adoption on test score gaps between private and public school students. The sample includes all pairwise combinations of ENEM adoption years for which treatment effects can be estimated using 2009–2017 exam takers (the boxed cells in Appendix Table <a href="#tab:stacked" data-reference-type="ref" data-reference="tab:stacked">10</a>). Pairwise groups are defined described in Section <a href="#sub:regressions" data-reference-type="ref" data-reference="sub:regressions">3.4</a>.</p>
<p>Each panel plots the <span class="math inline"><em>β</em><sub><em>l</em></sub><sup>gap</sup></span> coefficients (<span class="math inline"><em>y</em></span>-axis) from equation <a href="#event_study" data-reference-type="eqref" data-reference="event_study">[event_study]</a> for years <span class="math inline"><em>l</em> = −4</span> to <span class="math inline"><em>l</em> = 4</span> relative to the state’s ENEM adoption year, <span class="math inline"><em>τ</em><sub><em>s</em>(<em>i</em>)</sub><sup>*</sup></span> (<span class="math inline"><em>x</em></span>-axis). The dependent variables are ENEM subject scores in SD units, as listed in the title of each panel. “Average (core subjects)” is the average score across math, language arts, natural science, and social science. Dashed lines depict 95% confidence intervals using standard errors clustered at the state level.</p>
<figcaption>Event studies for effects of ENEM adoption on private/public test score gaps</figcaption>
</figure>

</div>

<figure id="fig:dd_oth_gaps" data-latex-placement="ht">
<div class="centering">
<p><embed src="results/dd_oth_gaps.pdf" /> <embed src="results/dd_prep_gaps.pdf" /></p>
</div>
<p><em>Notes</em>: This figure shows the impact of ENEM adoption on various gaps in average ENEM scores.</p>
<p>Panel A shows impacts on demographic test score gaps. “High-income” individuals are defined as those with a family income greater than or equal to twice the minimum wage. Panel B shows impacts on test score gaps between students who did and did not engage in test prep activities, as defined by two different measures. We define “prep schools” as private schools whose curriculum is specifically focused on preparation for college admission exams. To define this measure, we obtained lists of schools that use test-oriented curricula from the websites of four prominent test prep companies and matched them to our sample of high schools using geocoded addresses. For the last bar in Panel B, we use a variable from the ENEM questionnaire that indicates whether the student took an entrance exam preparation course. See Appendix Table <a href="#tab:prep_gaps" data-reference-type="ref" data-reference="tab:prep_gaps">16</a> for details on these measures of test prep activity.</p>
<p>White bars represent mean gaps in average (core subjects) ENEM scores in low-stakes cohorts for each demographic/test prep group. Gray bars show estimates of <span class="math inline"><em>β</em><sup>gap</sup></span> from a specification of equation <a href="#dd_gap" data-reference-type="eqref" data-reference="dd_gap">[dd_gap]</a> that replaces <span class="math inline">Private<sub><em>h</em></sub></span> with a dummy variable for the first group listed in the heading (e.g., high-income, prep school, etc.). For this figure, we estimate equation <a href="#dd_gap" data-reference-type="eqref" data-reference="dd_gap">[dd_gap]</a> at the individual level rather than at the high school <span class="math inline">×</span> year level. Black bars represent 95% confidence intervals using standard errors clustered at the state level.</p>
<figcaption>Effects of ENEM adoption on gaps in average (core subjects) scores</figcaption>
</figure>

<figure id="fig:dd_correlation_subj" data-latex-placement="ht">
<div class="centering">
<embed src="results/dd_correlation_subjs.pdf" />
</div>
<p><em>Notes</em>: This figure shows the impacts of ENEM adoption on the informativeness of subject-specific ENEM scores for longer-run outcomes. Lighter-shaded areas depict the average correlation coefficients between subject scores and outcomes in low-stakes cohorts (i.e., cohorts where <span class="math inline">HighStakes<sub><em>s</em><em>t</em></sub> = 0</span>). Darker-shaded areas depict <span class="math inline"><em>β</em></span> estimates from equation <a href="#dd_styr" data-reference-type="eqref" data-reference="dd_styr">[dd_styr]</a>, where the outcome variables are state <span class="math inline">×</span> year correlation coefficients between subject scores and outcomes. The black bars represent <span class="math inline">95%</span> confidence intervals using standard errors clustered at the state level.</p>
<figcaption>Effects of ENEM adoption on the informativeness of subject scores for longer-run outcomes</figcaption>
</figure>

<figure id="fig:dd_correlation_demo" data-latex-placement="ht">
<div class="centering">
<embed src="results/dd_correlation_demo_legend.pdf" />
<embed src="results/dd_correlation_demo.pdf" />
</div>
<p><em>Notes</em>: This figure shows the impacts of ENEM adoption on the informativeness of average (core subjects) scores for longer-run outcomes after controlling for demographic characteristics.</p>
<p>The bars depict <span class="math inline"><em>β</em></span> coefficients from equation <a href="#dd_styr" data-reference-type="eqref" data-reference="dd_styr">[dd_styr]</a>. The dependent variable is correlation coefficient between average (core subjects) ENEM scores and the longer-run outcome in state <span class="math inline"><em>s</em></span> and ENEM cohort <span class="math inline"><em>t</em></span>. The white bars reproduce the estimates from column (B) of Table <a href="#tab:dd_correlation" data-reference-type="ref" data-reference="tab:dd_correlation">5</a>. The darker-colored bars show the <span class="math inline"><em>β</em></span> coefficients in specifications that control for four sets of demographic controls: 1) a dummy for private high schools; 2) dummies for family income categories, mother’s education, father’s education, and race; 3) a gender dummy and age; and 4) high school dummies. Each darker-colored bar includes the new controls along with all controls from the previous bars. Percentages indicate the ratio between the <span class="math inline"><em>β</em></span> coefficient for the given bar and the <span class="math inline"><em>β</em></span> coefficient for the specification without demographic controls (white bars).</p>
<figcaption>Effects of ENEM adoption on the correlation between outcomes and test scores controlling for demographics</figcaption>
</figure>

<figure id="fig:return_beta" data-latex-placement="ht">
<div class="centering">
<embed src="results/return_beta_legend.pdf" />
<p><embed src="results/return_beta_persist_3yr_area.pdf" /> <embed src="results/return_beta_persist_3yr_skill.pdf" /></p>
</div>
<p><em>Notes</em>: This figure shows the relationship between the informativeness of ENEM exam skills for college persistence (<span class="math inline"><em>y</em></span>-axis) and the effect of ENEM stakes on the private/public gap (<span class="math inline"><em>x</em></span>-axis). To define the informativeness of each exam skill for 3-year college persistence (<span class="math inline"><em>y</em></span>-axis), we compute the mean difference in 3-year persistence rates between individuals who got each question correct and incorrect using 2009–2014 (excluding 2011) ENEM participants in our analysis sample, and then average these differences across all questions in the same topic area (Panel A) or competency (Panel B). Our measure of the effect of ENEM stakes on the private/public gap (<span class="math inline"><em>x</em></span>-axis) is the <span class="math inline"><em>β</em><sup>gap</sup></span> coefficient from estimating equation <a href="#dd_gap" data-reference-type="eqref" data-reference="dd_gap">[dd_gap]</a> separately for groups of ENEM questions in each topic area (Panel A) and competency (Panel B). The dependent variable in these regressions is an indicator for a correct answer, and the sample includes 2009–2017 ENEM participants in our analysis sample. Marker colors and shapes depict exam subjects, as described in the legend. The dashed line shows the linear relationship between informativeness and the <span class="math inline"><em>β</em><sup>gap</sup></span> coefficients.</p>
<figcaption>Informativeness for college persistence vs. impact of exam stakes on private/public gap by exam skill</figcaption>
</figure>

<div class="center">

<div id="tab:summ">

+:--------------------------------------+----------------:+----------------:+----------------:+-----------:+----------------:+----------------:+
|                                       | \(A\)           | \(B\)           | \(C\)           | \(D\)      | \(E\)           | \(F\)           |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
|                                       |                 |                 | Analysis sample (high school seniors)                            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| 4-7                                   | All exam        | All HS          | All             | Private    | Public          | Private/        |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
|                                       | takers          | seniors         | schools         | schools    | schools         | public gap      |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
|                                       |                 |                 |                 |            |                 |                 |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Age at exam                           | 22.14           | 18.55           | 17.91           | 17.44      | 18.13           | $-$0.70 |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Female                                | 0.58            | 0.59            | 0.58            | 0.55       | 0.60            | $-$0.05 |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| White                                 | 0.40            | 0.44            | 0.51            | 0.69       | 0.43            | 0.26            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Black                                 | 0.12            | 0.11            | 0.08            | 0.04       | 0.10            | $-$0.06 |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Brown                                 | 0.44            | 0.42            | 0.37            | 0.23       | 0.44            | $-$0.21 |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Mother completed college              | 0.15            | 0.18            | 0.27            | 0.56       | 0.13            | 0.44            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Father completed college              | 0.11            | 0.13            | 0.21            | 0.49       | 0.07            | 0.41            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Family income \> 2x min. wage         | 0.35            | 0.38            | 0.49            | 0.85       | 0.32            | 0.52            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Private high school                   | 0.24            | 0.24            | 0.32            | 1.00       | 0.00            | 1.00            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
|                                       |                 |                 |                 |            |                 |                 |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Math score                            | $-$0.03 | $-$0.01 | 0.32            | 1.28       | $-$0.13 | 1.42            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Language arts score                   | 0.08            | 0.04            | 0.24            | 0.78       | $-$0.01 | 0.79            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Natural science score                 | $-$0.17 | $-$0.18 | 0.05            | 0.75       | $-$0.28 | 1.03            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Social science score                  | 0.30            | 0.22            | 0.43            | 1.07       | 0.14            | 0.93            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Average score (core subjects)         | 0.05            | 0.02            | 0.30            | 1.12       | $-$0.09 | 1.20            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Writing score                         | $-$0.41 | $-$0.38 | $-$0.13 | 0.50       | $-$0.43 | 0.93            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
|                                       |                 |                 |                 |            |                 |                 |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Ever enrolled in college              |                 |                 | 0.76            | 0.95       | 0.67            | 0.27            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Enrolled in a federal university      |                 |                 | 0.16            | 0.26       | 0.11            | 0.15            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Graduated college within 5 years      |                 |                 | 0.17            | 0.23       | 0.15            | 0.08            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Ever graduated college                |                 |                 | 0.31            | 0.43       | 0.25            | 0.18            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Persisted in college for 3 years      |                 |                 | 0.66            | 0.73       | 0.61            | 0.12            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Fraction of college credits completed |                 |                 | 0.69            | 0.75       | 0.64            | 0.11            |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Appears in RAIS in 2016–2018         |                 |                 | 0.26            | 0.20       | 0.29            | $-$0.08 |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Hourly wage (BRL)                     |                 |                 | 48.89           | 70.03      | 41.59           | 28.44           |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Number of exam takers                 | 40,391,604      | 11,626,416      | 2,512,214       | 807,293    | 1,704,921       | 2,512,214       |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+
| Number of high schools                | 46,584          | 45,867          | 3,276           | 1,437      | 1,839           | 3,276           |
+---------------------------------------+-----------------+-----------------+-----------------+------------+-----------------+-----------------+

: Summary statistics for 2009–2017 ENEM exam takers

</div>

</div>

*Notes*: This table reports summary statistics on ENEM exam takers. Column (A) includes all individuals who took the ENEM exam in 2009–2017 who have a non-zero, non-missing test score in all four core subjects. Column (B) includes all exam takers in these years who were high school seniors at the time of the exam. Column (C) includes exam takers in our analysis sample. Columns (D) and (E) include present results for private an public high school students without our analysis sample, respectively. Column (F) displays the difference between columns (D) and (E).

Panel (A) presents demographic characteristics of the exam takers, including age, gender, race, parental education, family income, and whether they attended a private high school. Panel (B) reports average ENEM scores (in SD units). "Average score (core subjects)" is the average score across math, language arts, natural science, and social science. Panel (C) displays college and labor market outcomes for the exam takers in our analysis sample. The last two rows report the number of exam takers and high schools. See Appendix app:var_def for details on variable definitions.

<div class="center">

<div id="tab:balance">

+:------------------------------------+:--+-----------:+:--+-----------------:+:--------+-----------------:+:--------+-----------------:+:-------+-----------------:+:--+--:+:--+--:+:--+--:+:--+--:+:--+
|                                     |   | \(A\)      |   | \(B\)            |         | \(C\)            |         | \(D\)            |        | \(E\)            |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   | Low-stakes |   |                  |         |                  |         |                  |        |                  |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   | mean       |   | DD coefficients                                                                                        |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| 3-3                                 |   | All        |   | All              |         | Private          |         | Public           |        | Private/         |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Dependent variable                  |   | schools    |   | schools          |         | schools          |         | schools          |        | public gap       |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   |                  |         |                  |         |                  |        |                  |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Log \# exam takers in school        |   | 4.713      |   | 0.076            |         | $-$0.004 |         | 0.089            |        | $-$0.093 |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.065           | )       | (0.118           | )       | (0.056           | )      | (0.107           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Log \# HS seniors in school         |   | 5.118      |   | $-$0.033 |         | $-$0.072 |         | $-$0.033 |        | $-$0.039 |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.022           | )       | (0.053           | )       | (0.032           | )      | (0.070           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| \# schools attended in past 3 years |   | 1.220      |   | $-$0.010 |         | $-$0.023 |         | $-$0.010 |        | $-$0.013 |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.012           | )       | (0.015           | )       | (0.016           | )      | (0.017           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   |                  |         |                  |         |                  |        |                  |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Age at exam                         |   | 18.190     |   | 0.030            |         | 0.010            |         | 0.073            |        | $-$0.063 |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.054           | )       | (0.015           | )       | (0.083           | )      | (0.075           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Female                              |   | 0.599      |   | $-$0.014 | $^{**}$ | $-$0.013 | $^{**}$ | $-$0.012 | $^{*}$ | $-$0.000 |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.006           | )       | (0.005           | )       | (0.006           | )      | (0.006           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| White                               |   | 0.469      |   | $-$0.007 |         | $-$0.009 |         | 0.001            |        | $-$0.010 |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.010           | )       | (0.008           | )       | (0.010           | )      | (0.012           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Mother completed college            |   | 0.258      |   | 0.001            |         | 0.009            |         | 0.000            |        | 0.009            |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.008           | )       | (0.010           | )       | (0.005           | )      | (0.010           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Father completed college            |   | 0.195      |   | 0.006            |         | 0.013            |         | 0.005            |        | 0.008            |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.006           | )       | (0.011           | )       | (0.004           | )      | (0.009           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Family income \> 2x min. wage       |   | 0.476      |   | 0.003            |         | 0.016            |         | $-$0.002 |        | 0.018            |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.022           | )       | (0.010           | )       | (0.028           | )      | (0.031           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Joint balance test ($p$ value)      |   |            |   | 0.159            |         | 0.308            |         | 0.206            |        | 0.708            |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   |                  |         |                  |         |                  |        |                  |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| Predicted ENEM score                |   | 0.181      |   | 0.004            |         | 0.017            |         | $-$0.001 |        | 0.018            |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
|                                     |   |            |   | (0.011           | )       | (0.012           | )       | (0.011           | )      | (0.014           | ) |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| $N$ (# exam takers)                 |   | 492,436    |   | 2,512,214        |         | 807,293          |         | 1,704,921        |        | 2,512,214        |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+
| $N$ (# HS seniors)                  |   | 707,255    |   | 3,283,616        |         | 913,767          |         | 2,369,849        |        | 3,283,616        |   |   |   |   |   |   |   |   |   |
+-------------------------------------+---+------------+---+------------------+---------+------------------+---------+------------------+--------+------------------+---+---+---+---+---+---+---+---+---+

: Balance tests for analysis sample

</div>

</div>

*Notes*: This table presents balance tests for 2009–2017 ENEM test takers in our analysis sample (column C of Table 1). In Panel A, the dependent variables are: 1) the log number of exam takers per school; 2) the log number of high school seniors per school; 3) the average number of schools each senior attended in the past three years. In Panel B, the dependent variables are exam taker demographic characteristics. In Panel C, the dependent variable is the predicted value from a regression of average ENEM score (core subjects) on age, gender, and dummies for race, mother's education, father's education, and family income bins. The dependent variables are high school $\times$ year totals (log counts) and averages (all other variables).

Column (A) shows the mean of each dependent variable in exam cohorts prior to each state's ENEM adoption year (i.e., cohorts with $\text{HighStakes}_{st}=0$). Columns (B)–(D) display $\beta$ coefficients from equation dd estimated using all students, private students, and public students, respectively. Column (E) displays $\beta^{\text{gap}}$ coefficients from equation dd_gap estimated using all students. The last row of Panel B shows the $p$ value from an $F$ test that the coefficients in Panel B are jointly equal to zero. Parentheses contain standard errors clustered at the state level.

\* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:scores">

+:------------------------------+:--+-----------:+:--+----------:+:----+--------:+:---------+-----------------:+:-------+-----------:+:---------+--:+:--+--:+:--+--:+:--+--:+:--+
|                               |   | \(A\)      |   | \(B\)     |     | \(C\)   |          | \(D\)            |        | \(E\)      |          |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
|                               |   | Low-stakes |   |           |     |         |          |                  |        |            |          |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
|                               |   | mean       |   | DD coefficients                                                               |          |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
| 3-3                           |   | Private/   |   | All       |     | Private |          | Public           |        | Private/   |          |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
| Dependent variable            |   | public gap |   | schools   |     | schools |          | schools          |        | public gap |          |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
| Math score                    |   | 1.358      |   | 0.022     |     | 0.143   | $^{**}$  | $-$0.015 |        | 0.158      | $^{*}$   |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
|                               |   |            |   | (0.055    | )   | (0.058  | )        | (0.070           | )      | (0.079     | )        |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
| Language arts score           |   | 0.837      |   | 0.035     |     | 0.068   | $^{***}$ | $-$0.008 |        | 0.076      | $^{***}$ |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
|                               |   |            |   | (0.035    | )   | (0.020  | )        | (0.034           | )      | (0.026     | )        |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
| Natural science score         |   | 1.059      |   | 0.026     |     | 0.062   | $^{*}$   | $-$0.003 |        | 0.065      | $^{*}$   |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
|                               |   |            |   | (0.040    | )   | (0.031  | )        | (0.042           | )      | (0.034     | )        |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
| Social science score          |   | 1.010      |   | 0.019     |     | 0.056   | $^{*}$   | $-$0.024 |        | 0.081      | $^{***}$ |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
|                               |   |            |   | (0.034    | )   | (0.029  | )        | (0.029           | )      | (0.023     | )        |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
| Average score (core subjects) |   | 1.229      |   | 0.029     |     | 0.095   | $^{**}$  | $-$0.014 |        | 0.110      | $^{**}$  |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
|                               |   |            |   | (0.043    | )   | (0.036  | )        | (0.045           | )      | (0.040     | )        |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
| Writing score                 |   | 0.784      |   | 0.049     |     | 0.165   | $^{**}$  | 0.063            | $^{*}$ | 0.102      | $^{*}$   |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
|                               |   |            |   | (0.035    | )   | (0.072  | )        | (0.033           | )      | (0.058     | )        |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+
| $N$ (# exam takers)           |   | 492,436    |   | 2,512,214 |     | 807,293 |          | 1,704,921        |        | 2,512,214  |          |   |   |   |   |   |   |   |   |
+-------------------------------+---+------------+---+-----------+-----+---------+----------+------------------+--------+------------+----------+---+---+---+---+---+---+---+---+

: Effects of ENEM adoption on test scores in public and private high schools

</div>

</div>

*Notes*: This table shows the effect of ENEM adoption on the test scores of private and public high school students. The sample includes 2009–2017 ENEM exam takers in our analysis sample (column C of Table 1). The dependent variables are ENEM subject scores in SD units. "Average score (core subjects)" is the average score across math, language arts, natural science, and social science.

Column (A) shows the mean private/public score gap in exam cohorts prior to each state's ENEM adoption year (i.e., cohorts with $\text{HighStakes}_{st}=0$). Columns (B)–(D) display $\beta$ coefficients from equation dd estimated using all students, private school students, and public school students, respectively. Column (E) displays $\beta^{\text{gap}}$ coefficients from equation dd_gap estimated using all students in the analysis sample. Parentheses contain standard errors clustered at the state level.

\* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="landscape">

<div class="center">

<div id="tab:robust">

  ------------------------------- -- ------------------ ---------- ------------------ ---------- ------------------ --------- ------------ ---------- ------------------ ---------- -------------- ---------- -- -- -- -- -- --
                                                  \(A\)                         \(B\)                         \(C\)                  \(D\)                         \(E\)                     \(F\)                           
                                                                                Demo-                                                                       2009 vs 2013                   Stacked                           
                                              Benchmark                       graphic                        Binary                Stacked                      adopters                regression                           
  Dependent variable                              model                      controls                     treatment             regression                  (2007–2012)              (2007–2017)                           
  Math score                                      0.158 $^{*}$                  0.128 $^{*}$                  0.105 $^{**}$          0.116 $^{**}$                 0.086 $^{***}$            0.061 $^{**}$                   
                                                 (0.079 )                      (0.067 )                      (0.050 )               (0.055 )                      (0.019 )                  (0.024 )                         
  Language arts score                             0.076 $^{***}$                0.072 $^{***}$                0.042 $^{**}$          0.055 $^{***}$                0.106 $^{***}$            0.062 $^{***}$                  
                                                 (0.026 )                      (0.023 )                      (0.017 )               (0.014 )                      (0.031 )                  (0.022 )                         
  Natural science score                           0.065 $^{*}$                  0.074 $^{**}$                 0.032                  0.025                         0.059 $^{***}$            0.022                           
                                                 (0.034 )                      (0.036 )                      (0.030 )               (0.016 )                      (0.017 )                  (0.014 )                         
  Social science score                            0.081 $^{***}$                0.053 $^{*}$                  0.042 $^{**}$          0.053 $^{**}$                 0.046                     0.042                           
                                                 (0.023 )                      (0.028 )                      (0.020 )               (0.020 )                      (0.035 )                  (0.025 )                         
  Average score (core subjects)                   0.110 $^{**}$                 0.094 $^{**}$                 0.064 $^{**}$          0.072 $^{***}$                0.088 $^{**}$             0.049 $^{**}$                   
                                                 (0.040 )                      (0.040 )                      (0.030 )               (0.023 )                      (0.029 )                  (0.024 )                         
  Writing score                                   0.102 $^{*}$                  0.144 $^{**}$                 0.035                  0.023                         0.058                     0.064 $^{***}$                  
                                                 (0.058 )                      (0.058 )                      (0.044 )               (0.028 )                      (0.047 )                  (0.021 )                         
  $N$ (# exam takers)                         2,512,214                     2,512,214                     2,512,214              5,858,862                     1,099,500                15,738,474                           
  Treatment variable:                        Continuous                    Continuous                        Binary                 Binary                        Binary                    Binary                           
  Demographic controls:                                                           Yes                                                                                                                                        
  Level of dataset:                    HS $\times$ year              HS $\times$ year              HS $\times$ year                Stacked              HS $\times$ year                   Stacked                           
  Included exam cohorts:                     2009–2017                    2009–2017                    2009–2017             2009–2017                    2007–2012                2007–2017                           
  ------------------------------- -- ------------------ ---------- ------------------ ---------- ------------------ --------- ------------ ---------- ------------------ ---------- -------------- ---------- -- -- -- -- -- --

  : Robustness checks on the effects of ENEM adoption on private/public school test score gaps

</div>

</div>

*Notes*: This table examines the robustness of our estimates of the effects of ENEM adoption on private/public test score gaps.

In columns (A)–(D), the sample includes 2009–2017 ENEM exam takers in our analysis sample (column C of Table 1). In columns (E)–(F), the sample also includes 2007–2008 ENEM exam takers from the same set of high schools. The dependent variables are ENEM subject scores in SD units. "Average score (core subjects)" is the average score across math, language arts, natural science, and social science. Columns (E)–(F) include scores from the 2007–2008 ENEM tests; in these columns, we standardize scores to have a mean of 0 and SD of 1 within each year of our sample. For the 2007–2008 exams, "average score" is the reported core component score, and we compute math, language arts, natural science, and social science scores by categorizing the multiple-choice questions into these four subjects and then estimating a scale scores using the IRT parameters. See Appendix app:var_def for details.

Column (A) reproduces the estimates from column (E) of Table 3, which are the $\beta^{\text{gap}}$ coefficients from equation dd_gap. Column (B) shows estimates of equation dd_gap including high school $\times$ year averages of age, gender, and dummies for race, mother's education, father's education, and family income bins. Column (C) shows estimates of equation dd_gap replacing the continuous treatment variable, $\text{ProportionENEM}_{st}$, with the binary treatment variable, $\text{HighStakes}_{st}$. Columns (D)–(F) show estimates of $\beta^{\text{gap}}$ from equation dd_gap_stacked using the stacked dataset, which contains pairwise combinations of ENEM adoption years (as described in Section 3.4). Column (D) includes all pairwise combinations for which we can estimate treatment effects using 2009–2017 exam takers (the boxed cells in Appendix Table 10). Column (E) includes 2007–2012 exam takers and a single pair of ENEM adoptions years, 2009 and 2013 (the bolded cells in Appendix Table 10). Column (F) includes all 2007–2017 exam takers and all pairwise combinations (all cells in Appendix Table 10).

Parentheses contain standard errors clustered at the state level. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

</div>

<div class="center">

<div id="tab:dd_correlation">

+:----------------------------------------+:-------------+-----------------:+:-------------+-----------------:+:-------------+-----------------:+:-------------+-----------------:+:---------+-----------------:+:---------+--:+:--+--:+:--+--:+:--+--:+:--+
|                                         |              | \(A\)            |              | \(B\)            |              | \(C\)            |              | \(D\)            |          | \(E\)            |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | Low-stakes       |              | Benchmark model                                    |              | Binary treatment                               |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+----------------------------------------------------+--------------+------------------------------------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | mean             |              | DD coefficients                                    |              | DD coefficients                                |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| 3-3 Dependent variable: Correlation     |              | Raw              |              | Raw              |              | Within-          |              | Raw              |          | Within           |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| between average ENEM scores and\...     |              | corr.            |              | corr.            |              | program          |              | corr.            |          | program          |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              |                  |              |                  |              |                  |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| **Panel A. Outcomes for all exam takers**                                                                                                                                       |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Enrolled in any college by 2019         |              | 0.372            |              | 0.036            | $^{***}$     |                  |              | 0.022            | $^{***}$ |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.006           | )            |                  |              | (0.004           | )        |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Finished college within 5 years of ENEM |              | 0.121            |              | 0.014            | $^{*}$       | 0.026            | $^{***}$     | 0.007            |          | 0.015            | $^{***}$ |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.008           | )            | (0.007           | )            | (0.004           | )        | (0.005           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Earned a college degree by 2019         |              | 0.257            |              | 0.033            | $^{**}$      | 0.038            | $^{***}$     | 0.016            |          | 0.023            | $^{***}$ |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.015           | )            | (0.010           | )            | (0.011           | )        | (0.007           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Appears in RAIS in 2016–2018           |              | $-$0.112 |              | 0.056            |              | 0.020            | $^{*}$       | 0.014            |          | 0.008            |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.044           | )            | (0.011           | )            | (0.025           | )        | (0.007           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| $N$ (# exam takers)                     |              | 336,175          |              | 1,266,412        |              | 1,266,412        |              | 1,266,412        |          | 1,266,412        |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              |                  |              |                  |              |                  |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| **Panel B. Outcomes for college enrollees**                                                                                                                                     |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Persisted in college for 1 year         |              | 0.064            |              | 0.008            |              | 0.024            | $^{***}$     | 0.007            |          | 0.011            | $^{*}$   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.014           | )            | (0.007           | )            | (0.011           | )        | (0.006           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Persisted in college for 3 years        |              | 0.142            |              | 0.033            | $^{**}$      | 0.043            | $^{***}$     | 0.018            | $^{**}$  | 0.024            | $^{***}$ |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.013           | )            | (0.008           | )            | (0.009           | )        | (0.006           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Completed program within 5 years        |              | 0.071            |              | 0.044            | $^{**}$      | 0.035            | $^{***}$     | 0.022            | $^{*}$   | 0.018            | $^{**}$  |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.018           | )            | (0.011           | )            | (0.011           | )        | (0.007           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Fraction of college credits completed   |              | 0.214            |              | 0.003            |              | 0.014            |              | $-$0.011 |          | 0.013            | $^{*}$   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.014           | )            | (0.013           | )            | (0.011           | )        | (0.007           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| $N$ (# in higher ed.)                   |              | 274,022          |              | 966,649          |              | 966,649          |              | 966,649          |          | 966,649          |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              |                  |              |                  |              |                  |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| **Panel C. Outcome for individuals in RAIS**                                                                                                                                    |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Hourly wage (BRL)                       |              | 0.200            |              | 0.046            | $^{**}$      | 0.027            | $^{**}$      | 0.027            | $^{***}$ | 0.012            | $^{*}$   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.018           | )            | (0.010           | )            | (0.007           | )        | (0.006           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Log hourly wage                         |              | 0.362            |              | $-$0.029 | $^{**}$      | $-$0.001 |              | $-$0.017 | $^{*}$   | $-$0.001 |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              | (0.014           | )            | (0.010           | )            | (0.008           | )        | (0.006           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| $N$ (# in RAIS)                         |              | 80,382           |              | 328,773          |              | 328,773          |              | 328,773          |          | 328,773          |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+

: Effects of ENEM adoption on the informativeness of ENEM scores for longer-run outcomes

</div>

</div>

*Notes*: This table shows the impacts of ENEM adoption on the informativeness of ENEM scores for longer-run student outcomes. Our measure of informativeness is the correlation coefficient between the outcome in the column header and average (core subject) ENEM scores using data from our analysis sample for each state $\times$ year pair in 2009–2014 (excluding 2011). We estimate equation dd_styr using these correlation coefficients as dependent variables, weighting each state $\times$ year observation by the number of ENEM test takers for whom the outcome is defined.

Column (A) shows the mean correlation coefficients in exam cohorts prior to each state's ENEM adoption year. Columns (B)–(E) display $\beta$ coefficients from equation dd_styr. Columns (B) and (D) use raw correlation coefficients as dependent variables. In columns (C) and (E), the dependent variables are correlation coefficients computed after demeaning all variables within college $\times$ program cells. Regressions in columns (B)–(C) use the continuous treatment variable, $\text{ProportionENEM}_{st}$. Columns (D)–(E) use the binary treatment variable, $\text{HighStakes}_{st}$.

Parentheses contain standard errors clustered at the state level.

\* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:dd_het_areas">

+:------------------------------------------------+:-------------:+--------------:+-------------------------:+:--------------+--------------:+-------------------------:+:--------:+
| \(A\)                                           | \(B\)         | \(C\)         | \(D\)                    |               | \(E\)         | \(F\)                    |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
|                                                 |               | Outcome: Proportion of correct answers                                                                         |
+-------------------------------------------------+---------------+------------------------------------------+---------------+------------------------------------------+----------+
| 3-8                                             |               | Public students                          |               | Private/public gap                       |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| 3-4 Question group                              | $N_q$         | Mean          | $\beta$ (SE)             |               | Mean          | $\beta^\text{gap}$ (SE)  |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
|                                                 |               |               |                          |               |               |                          |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| All questions                                   | 405           | 0.291         | $-$0.005 (0.010) |               | 0.176         | 0.024 (0.009)            | \*\*     |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
|                                                 |               |               |                          |               |               |                          |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| **Panel B. Topic area (and competency reference numbers)**                                                                                 |                          |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Numbers (1–5)                                  | 67            | 0.307         | $-$0.010 (0.011) |               | 0.159         | 0.021 (0.011)            | \*       |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Geometry (6–9)                                 | 57            | 0.317         | 0.003 (0.007)            |               | 0.160         | 0.016 (0.008)            | \*       |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Measurements (10–14)                           | 62            | 0.257         | $-$0.006 (0.010) |               | 0.193         | 0.029 (0.009)            | \*\*\*   |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Proportions (15–18)                            | 51            | 0.336         | $-$0.008 (0.019) |               | 0.225         | 0.036 (0.015)            | \*\*     |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Algebra (19–23)                                | 66            | 0.264         | $-$0.003 (0.005) |               | 0.172         | 0.013 (0.007)            | \*       |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Interpreting data (24–26)                      | 47            | 0.325         | $-$0.014 (0.017) |               | 0.193         | 0.034 (0.015)            | \*\*     |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Statistics (27–30)                             | 55            | 0.241         | $-$0.000 (0.007) |               | 0.137         | 0.022 (0.009)            | \*\*     |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| All coefficients equal ($p$ value)              |               |               | 0.299                    |               |               | 0.015                    |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
|                                                 |               |               |                          |               |               |                          |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| **Panel C. Competencies (top 5 and bottom 5 by $\beta^\text{gap}$/mean)**                                                                  |                          |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Evaluate interventions using proportions (18)   | 12            | 0.293         | $-$0.012 (0.023) |               | 0.219         | 0.060 (0.017)            | \*\*\*   |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Use tables/graphs to construct arguments (26)   | 14            | 0.363         | 0.001 (0.023)            |               | 0.233         | 0.042 (0.019)            | \*\*     |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Identify units of measurement (10)              | 10            | 0.375         | $-$0.001 (0.020) |               | 0.313         | 0.042 (0.018)            | \*\*     |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Calculate statistical quantities from data (27) | 15            | 0.220         | 0.005 (0.010)            |               | 0.140         | 0.039 (0.022)            | \*       |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Identify proportional relationships (15)        | 12            | 0.395         | $-$0.013 (0.025) |               | 0.274         | 0.035 (0.023)            |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| $\dots$                                         |               |               |                          |               |               |                          |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Use numbers to construct arguments (4)          | 15            | 0.266         | $-$0.007 (0.010) |               | 0.161         | 0.011 (0.012)            |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Use algebra to construct arguments (22)         | 9             | 0.211         | 0.006 (0.008)            |               | 0.107         | 0.011 (0.009)            |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Solve problems using geometry (8)               | 18            | 0.236         | 0.013 (0.003)            | \*\*\*        | 0.147         | 0.009 (0.010)            |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Interpret Cartesian graphs (20)                 | 11            | 0.541         | $-$0.018 (0.027) |               | 0.209         | 0.008 (0.039)            |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| Evaluate interventions using statistics (30)    | 10            | 0.253         | 0.001 (0.011)            |               | 0.072         | $-$0.011 (0.016) |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+
| 21 coefficients equal ($p$ value)               |               |               | 0.000                    |               |               | 0.000                    |          |
+-------------------------------------------------+---------------+---------------+--------------------------+---------------+---------------+--------------------------+----------+

: Effects of ENEM adoption on math exam performance by topic area and competency

</div>

</div>

*Notes*: This table shows the impacts of ENEM adoption on performance in different topic areas and competencies on the math subject test.

The sample includes 2009–2017 ENEM exam takers in our analysis sample (column C of Table 1). Regressions are at the high school ($h$) $\times$ year ($t$) $\times$ exam question ($q$) level. The dependent variable is the proportion of correct answers in each $htq$ cell for questions on the math subject test. Panel A presents results from a regression that includes all math questions. Panels B and C present results from separate regressions for each of 7 math topic area and 30 math competencies, as defined by ENEM test designers. Panel C displays only the top 5 and bottom 5 competencies based on the values in Column (F). See Appendix 9.4 for details on math topic areas and compentencies.

Column (A) specifies the group of questions contained in each regression. Column (B) indicates the number of questions in each group. Column (C) shows the mean proportion of correct answers for public school students in cohorts prior to each state's ENEM adoption year (i.e., cohorts with $\text{HighStakes}_{st}=0$). Column (E) shows the mean private/public gap in the proportion of correct answers in these cohorts. Columns (D) and (F) display the $\beta$ and $\beta^{\text{gap}}$ coefficients from equation dd_gap estimated for each group of questions. In Panel B, the last row reports $p$ values from $F$ tests that the 7 topic area coefficients in columns (D) or (F) are equal. In Panel C, the last row reports $p$ values from $F$ tests that 21 competency coefficients (the first 3 in each topic area) are jointly equal.

Parentheses contain standard errors clustered at the state level.

\* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

</div>

# Appendix figures and tables

<figure id="fig:dist_scores_cutoffs" data-latex-placement="htp">
<div class="centering">
<embed src="results/dist_scores_cutoffs.pdf" />
</div>
<p><em>Notes</em>: This figure displays the distribution of ENEM test scores and federal university cutoff scores. The solid red line shows the distribution of average ENEM scores for private school students in the analysis sample who took the ENEM exam in cohorts prior to each state’s ENEM adoption year (i.e., cohorts with <span class="math inline">HighStakes<sub><em>s</em><em>t</em></sub> = 0</span>). The dashed black line represents the corresponding distribution for public school students. In both distributions, average ENEM scores are calculated as the average score across math, language arts, natural science, and social science.</p>
<p>The green short-dashed line shows the distribution of cutoff scores for unreserved admissions to all federal university programs in 2016. The blue long-dashed line shows the corresponding distribution for reserved quotas at federal university programs, which include quotas for public high school students, low-SES students, and/or underrepresented minority students.</p>
<figcaption>Distributions of average ENEM scores and federal university admission cutoffs</figcaption>
</figure>

<div class="landscape">

<figure id="fig:event_scores_callaway" data-latex-placement="t">
<div class="centering">
<div class="minipage">
<embed src="results/callaway_math.pdf" />
</div>
<div class="minipage">
<embed src="results/callaway_lang.pdf" />
</div>
<div class="minipage">
<embed src="results/callaway_ssci.pdf" />
</div>
<div class="minipage">
<embed src="results/callaway_nsci.pdf" />
</div>
<div class="minipage">
<embed src="results/callaway_overall.pdf" />
</div>
<div class="minipage">
<embed src="results/callaway_writ.pdf" />
</div>
</div>
<p><em>Notes</em>: This figure plots event study estimates of the impact of ENEM adoption on private/public test score gaps using the method of Callaway and Sant’Anna (2021). We estimate the regressions using data from 2009–2017 ENEM exam takers in our analysis sample (column C of Table <a href="#tab:summ" data-reference-type="ref" data-reference="tab:summ">1</a>). We collapse the data at the state <span class="math inline">×</span> year (<span class="math inline"><em>s</em><em>t</em></span>) level and define the dependent variables to be the <em>difference</em> in mean ENEM subject scores (in SD units) between private and public schools in each <span class="math inline"><em>s</em><em>t</em></span> cell. “Average (core subjects)” is the average score across math, language arts, natural science, and social science.</p>
<p>We estimate the event study coefficients with Callaway and Sant’Anna (2021)’s <code>csdid</code> package for Stata using the binary treatment variable, <span class="math inline">HighStakes<sub><em>s</em><em>t</em></sub></span>, and not-yet-treated states as the control group. We then plot the results using the authors’ <code>csdid_plot</code> command. Coefficients in the post-treatment period (red circles) represent the change in the private/public test score gap between treated and not-yet-treated states in event time <span class="math inline"><em>t</em></span> relative to one year before the treated group’s ENEM adoption year (event time <span class="math inline">−1</span>). Coefficients in the pre-treatment period (red circles) represent the change in the private/public test score gap between treated and not-yet-treated states in event year <span class="math inline"><em>t</em></span> relative to event year <span class="math inline"><em>t</em> − 1</span>. Light-colored rectangles depict 95% confidence intervals using standard errors clustered at the state level.</p>
<figcaption>Callaway and Sant’Anna (2021) event studies for the impact of ENEM adoption on private/public test score gaps</figcaption>
</figure>

</div>

<div class="center">

<div id="tab:summ_hs_college">

+:---------------------------+--------:+---------:+----------:+---------:+------------:+-----------:+
| \(A\)                      | \(B\)   | \(C\)    | \(D\)     | \(E\)    | \(F\)       | \(G\)      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
|                            | \# of   | Prop. of | \# of     | Prop. of | \# students | Attended a |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| Institution type           | schools | schools  | students  | students | per school  | private HS |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
|                            |         |          |           |          |             |            |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| Federal high schools       | 100     | 0.004    | 9,772     | 0.005    | 98          | 1.000      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| State high schools         | 16,583  | 0.702    | 1,823,524 | 0.849    | 110         | 0.000      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| Municipal high schools     | 373     | 0.016    | 23,156    | 0.011    | 62          | 0.000      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| Private high schools       | 6,567   | 0.278    | 290,366   | 0.135    | 44          | 1.000      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| All high schools           | 23,623  | 1.000    | 2,146,818 | 1.000    | 91          | 0.140      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
|                            |         |          |           |          |             |            |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| **Panel B. New college enrollees in 2009**                                                        |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| Federal universities       | 59      | 0.025    | 225,112   | 0.108    | 3,815       | 0.471      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| State universities         | 40      | 0.017    | 119,489   | 0.057    | 2,987       | 0.370      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| Municipal universities     | 9       | 0.004    | 22,453    | 0.011    | 2,495       | 0.319      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| Private universities       | 225     | 0.094    | 1,018,698 | 0.489    | 4,528       | 0.458      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| Public technical colleges  | 168     | 0.070    | 55,609    | 0.027    | 331         | 0.259      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| Private technical colleges | 1,888   | 0.790    | 640,021   | 0.307    | 339         | 0.331      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
| All colleges               | 2,389   | 1.000    | 2,081,382 | 1.000    | 871         | 0.401      |
+----------------------------+---------+----------+-----------+----------+-------------+------------+
|                            |         |          |           |          |             |            |
+----------------------------+---------+----------+-----------+----------+-------------+------------+

: Summary of Brazilian high school and college systems

</div>

</div>

*Notes*: This table presents summary statistics on the Brazilian high school and college systems. Panel A presents statistics for students who were high school seniors in 2009 using data from the National Primary and Secondary School Census (*Censo Escolar*). Panel B presents statistics for students who were new college enrollees in 2009 using data from the National Higher Education Census (*Censo da Educação Superior*).

Column (A) categorizes high schools by ownership (federal, state, municipal, or private), and colleges by both ownership and institution type (university or technical college). "University" includes both *Universidade* and *Centro Universitário* institutions. "Technical colleges" include *Faculdade*, *Instituto Federal de Educação Ciência e Tecnologia*, and *Centro Federal de Educação Tecnológica* institutions. Columns (B) and (C) show the number and proportion of schools in each category. Columns (D) and (E) show the number and proportion of students who attended schools in each category. Column (F) shows the number of students per school (column D divided by column B). Column (G) shows the proportion of students at each school type who attended a private high school. Throughout the paper, we include the small number of federal high schools in the group of private high schools since both tend to enroll wealthier and high-achieving students.

<div class="center">

<div id="tab:enem_pct">

+:-------------------------+----------:+---------------:+---------------:+---------------:+---------------:+---------------:+---------------:+---------------:+---------------:+---------------:+---------------:+---------------:+
| \(A\)                    | \(B\)     | \(C\)          | \(D\)          | \(E\)          | \(F\)          | \(G\)          | \(H\)          | \(I\)          | \(J\)          | \(K\)          | \(L\)          | \(M\)          |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
|                          |           | Proportion admitted using ENEM by exam year ($\text{ProportionENEM}_{st}$)                                                                                                               |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| 3-13                     | \# 2009   |                |                |                |                |                |                |                |                |                |                |                |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| State                    | enrollees | 2007           | 2008           | 2009           | 2010           | 2011           | 2012           | 2013           | 2014           | 2015           | 2016           | 2017           |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Pernambuco (PE)          | 7,375     | 0.00           | **0.98**       | **0.93**       | **0.94**       | **0.93**       | **0.94**       | **0.92**       | **0.91**       | **0.92**       | **0.92**       | **0.95**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Amazonas (AM)            | 2,821     | 0.00           | 0.00           | **0.32**       | **0.36**       | **0.38**       | **0.42**       | **0.43**       | **0.42**       | **0.50**       | **0.47**       | **0.47**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Espirito Santo (ES)      | 3,302     | 0.00           | 0.00           | **0.99**       | **0.92**       | **1.00**       | **0.90**       | **0.92**       | **0.89**       | **0.87**       | **0.94**       | **0.92**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Maranhão (MA)            | 2,359     | 0.00           | 0.00           | **0.98**       | **0.88**       | **0.95**       | **0.94**       | **0.91**       | **0.87**       | **0.89**       | **0.78**       | **0.92**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Mato Grosso (MT)         | 3,582     | 0.00           | 0.00           | **0.64**       | **0.93**       | **0.90**       | **0.86**       | **0.83**       | **0.94**       | **0.95**       | **0.89**       | **0.97**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Paraná (PR)              | 6,820     | 0.00           | 0.00           | **0.53**       | **0.51**       | **0.51**       | **0.47**       | **0.58**       | **0.60**       | **0.63**       | **0.57**       | **0.57**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Rio Grande do Sul (RS)   | 12,723    | 0.00           | 0.00           | **0.42**       | **0.37**       | **0.42**       | **0.41**       | **0.39**       | **0.41**       | **0.62**       | **0.63**       | **0.62**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| São Paulo (SP)           | 5,774     | 0.00           | 0.00           | **0.87**       | **0.95**       | **0.89**       | **0.80**       | **0.74**       | **0.70**       | **0.74**       | **0.77**       | **0.71**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Ceará (CE)               | 3,704     | 0.00           | 0.00           | 0.00           | **0.85**       | **0.85**       | **0.82**       | **0.85**       | **0.83**       | **0.87**       | **0.77**       | **0.81**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Mato Grosso do Sul (MS)  | 2,710     | 0.00           | 0.00           | 0.00           | **0.66**       | **0.63**       | **0.54**       | **0.65**       | **0.72**       | **0.71**       | **0.62**       | **0.56**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Paraíba (PB)             | 7,160     | 0.00           | 0.00           | 0.02           | **0.40**       | **0.45**       | **0.55**       | **0.94**       | **0.90**       | **0.92**       | **0.93**       | **0.90**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Acre (AC)                | 955       | 0.00           | 0.00           | 0.00           | 0.00           | **0.96**       | **0.76**       | **0.94**       | **0.89**       | **0.82**       | **0.86**       | **0.83**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Alagoas (AL)             | 3,008     | 0.00           | 0.00           | 0.00           | 0.00           | **0.90**       | **0.83**       | **0.87**       | **0.88**       | **0.94**       | **0.95**       | **0.93**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Piauí (PI)               | 3,592     | 0.01           | 0.00           | 0.55           | 0.29           | **0.82**       | **1.00**       | **0.71**       | **0.97**       | **0.96**       | **0.58**       | **0.98**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Rio Grande do Norte (RN) | 6,443     | 0.00           | 0.00           | 0.34           | 0.15           | 0.06           | **0.61**       | **0.70**       | **0.82**       | **0.82**       | **0.79**       | **0.82**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Rondônia (RO)            | 1,106     | 0.00           | 0.00           | 0.08           | 0.02           | 0.31           | **0.77**       | **0.69**       | **0.79**       | **0.81**       | **0.95**       | **0.96**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Bahia (BA)               | 6,251     | 0.00           | 0.00           | 0.16           | 0.16           | 0.31           | 0.36           | **0.90**       | **0.83**       | **0.82**       | **0.81**       | **0.77**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Distrito Federal (DF)    | 5,296     | 0.00           | 0.00           | 0.00           | 0.01           | 0.10           | 0.04           | **0.23**       | **0.24**       | **0.20**       | **0.21**       | **0.21**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Minas Gerais (MG)        | 20,918    | 0.00           | 0.08           | 0.19           | 0.20           | 0.29           | 0.36           | **0.72**       | **0.74**       | **0.75**       | **0.72**       | **0.73**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Pará (PA)                | 2,937     | 0.00           | 0.00           | 0.00           | 0.00           | 0.03           | 0.09           | **0.89**       | **0.82**       | **0.78**       | **0.76**       | **0.81**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Rio de Janeiro (RJ)      | 16,871    | 0.02           | 0.02           | 0.22           | 0.37           | 0.49           | 0.45           | **0.77**       | **0.77**       | **0.78**       | **0.50**       | **0.48**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Roraima (RR)             | 842       | 0.00           | 0.00           | 0.00           | 0.07           | 0.11           | 0.17           | **0.48**       | **0.46**       | **0.38**       | **0.41**       | **0.39**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Goiás (GO)               | 3,636     | 0.00           | 0.00           | 0.00           | 0.01           | 0.14           | 0.19           | 0.36           | **0.85**       | **0.83**       | **0.77**       | **0.84**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Tocantins (TO)           | 1,602     | 0.00           | 0.00           | 0.14           | 0.15           | 0.20           | 0.08           | 0.18           | **0.83**       | **0.88**       | **0.78**       | **0.84**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Santa Catarina (SC)      | 4,632     | 0.00           | 0.00           | 0.15           | 0.14           | 0.15           | 0.12           | 0.12           | 0.15           | **0.35**       | **0.34**       | **0.34**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Amapá (AP)               | 309       | 0.00           | 0.00           | 0.00           | 0.00           | 0.49           | 0.43           | 0.44           | 0.40           | 0.38           | **0.98**       | **0.61**       |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| Sergipe (SE)             | 3,309     | 0.00           | 0.00           | 0.01           | 0.02           | 0.03           | 0.02           | 0.01           | 0.01           | 0.06           | 0.02           | 0.02           |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+
| All states               | 140,037   | 0.00           | 0.07           | 0.30           | 0.37           | 0.45           | 0.48           | 0.66           | 0.69           | 0.72           | 0.67           | 0.68           |
+--------------------------+-----------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+

: Proportion of federal university enrollees admitted using the ENEM by state and year

</div>

</div>

*Notes*: This table shows the proportion of federal university enrollees who were admitted using the ENEM by state and year. Column (A) lists the 27 states of Brazil. Column (B) shows the number of new federal university enrollees in each state in the 2009 calendar year. Columns (C)–(M) show the proportion of new federal university enrollees who were admitted using the ENEM based on the year students took the ENEM exam (the calendar year prior to the enrollment year). The sample for these statistics consists of new enrollees in bachelor's programs at federal universities, using data from the Brazilian Higher Education Census.

The figures in columns (C)–(M) are the values we use for the continuous treatment variable, $\text{ProportionENEM}_{st}$, where $s$ denotes states and $t$ denotes years. Bolded figures represent state $\times$ years that we classify as high stakes using the binary treatment variable, $\text{HighStakes}_{st}$. See Section 3.3 for details on the definition of $\text{ProportionENEM}_{st}$ and $\text{HighStakes}_{st}$.

<div class="center">

<div id="tab:adopt">

+:-------------------------+-----------:+-----------:+-----------:+-----------:+-----------:+-----------:+-------:+-------:+-------:+-------:+--------:+
| \(A\)                    | \(B\)      | \(C\)      | \(D\)      | \(E\)      | \(F\)      | \(G\)      | \(H\)  | \(I\)  | \(J\)  | \(K\)  | \(L\)   |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
|                          | Year of state's ENEM adoption ($\tau^*_s$)                                                                      |         |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| 2-11 Characteristic      | 2008       | 2009       | 2010       | 2011       | 2012       | 2013       | 2014   | 2015   | 2016   | Never  | All     |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
|                          |            |            |            |            |            |            |        |        |        |        |         |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| \# states                | 1          | 7          | 3          | 3          | 2          | 6          | 2      | 1      | 1      | 1      | 27      |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| \# federal universities  | 3          | 15         | 6          | 3          | 3          | 24         | 2      | 1      | 1      | 1      | 59      |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| \# 2009 enrollees        | 7,375      | 37,381     | 13,574     | 7,555      | 7,549      | 53,115     | 5,238  | 4,632  | 309    | 3,309  | 140,037 |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| Mean university size     | 2,458      | 2,492      | 2,262      | 2,518      | 2,516      | 2,213      | 2,619  | 4,632  | 309    | 3,309  | 2,373   |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| Mean cutoff score (2016) | 670        | 682        | 669        | 656        | 657        | 711        | 660    | 707    | 670    | 655    | 688     |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
|                          |            |            |            |            |            |            |        |        |        |        |         |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| **Panel B. Characteristics of 2009 enrollees**                                                         |        |        |        |        |         |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| Age at enrollment        | 23.64      | 24.53      | 23.80      | 24.31      | 24.17      | 24.15      | 23.60  | 24.38  | 26.81  | 24.76  | 24.21   |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| Female                   | 0.49       | 0.46       | 0.48       | 0.50       | 0.45       | 0.49       | 0.48   | 0.44   | 0.51   | 0.48   | 0.48    |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| White                    | 0.42       | 0.67       | 0.33       | 0.46       | 0.40       | 0.58       | 0.46   | 0.85   | 0.40   | 0.30   | 0.56    |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| Black                    | 0.07       | 0.07       | 0.40       | 0.08       | 0.07       | 0.09       | 0.08   | 0.04   | 0.15   | 0.10   | 0.12    |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| Brown                    | 0.46       | 0.19       | 0.24       | 0.44       | 0.51       | 0.31       | 0.43   | 0.09   | 0.41   | 0.57   | 0.29    |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
| Private high school      | 0.40       | 0.43       | 0.60       | 0.26       | 0.74       | 0.52       | 0.58   | 0.79   | 0.87   | 0.62   | 0.52    |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+
|                          |            |            |            |            |            |            |        |        |        |        |         |
+--------------------------+------------+------------+------------+------------+------------+------------+--------+--------+--------+--------+---------+

: Characteristics of federal universities by state's ENEM adoption year

</div>

</div>

*Notes*: This table shows characteristics of federal universities and their student bodies by their state's ENEM adoption year. Columns (B)–(K) categorize the federal universities by the year in which their state adopted the ENEM exam, $\tau^*_s$, as defined in Section 3.3. Column (L) includes all federal universities. The ENEM adoption years for each state are:

- 2008: Pernambuco.

- 2009: Amazonas, Espirito Santo, Maranhão, Mato Grosso, Paraná, Rio Grande do Sul, São Paulo.

- 2010: Ceará, Mato Grosso do Sul, Paraíba.

- 2011: Acre, Alagoas, Piauí.

- 2012: Rio Grande do Norte, Rondônia.

- 2013: Bahia, Distrito Federal, Minas Gerais, Pará, Rio de Janeiro, Roraima.

- 2014: Goiás, Tocantins.

- 2015: Santa Catarina.

- 2016: Amapá.

- Never: Sergipe.

Data on enrollment size and student characteristics are from the Brazilian Higher Education Census. In Panel A, the number of universities, the number of enrollees, and the mean university size are defined using new 2009 enrollees in bachelor's programs at federal universities. In Panel B, some demographic variables are missing in the 2009 census year, so we compute student characteristics using students who enrolled in 2009 but appear in any census year between 2009 and 2018.

The mean cutoff score (2016) is from a public data request from the centralized admission platform SISU (*Sistema de Seleção Unificada*). These averages correspond to non-reserved quotas for bachelor's degree programs at federal universities in the year 2016. The cutoff scores are typically weighted averages of ENEM scores in up to five subjects (math, language arts, natural science, social science, and writing) and are presented in ENEM scale score units.

We obtained the SISU data in March 2020 from public data request number [#23480008751201932](https://buscalai.cgu.gov.br/PedidosLai/DetalhePedido?id=1718596).

<div class="center">

<div id="tab:stacked">

+:---------------------------:+:---------------------------:+----------------------------:+----------------:+----------------:+----------------:+----------------:+----------------:+----------------:+----------------:+----------------:+----------------:+----------------:+----------------:+-------:+--:+--:+
| \(A\)                       | \(B\)                       |                             | \(C\)           | \(D\)           | \(E\)           | \(F\)           | \(G\)           | \(H\)           | \(I\)           | \(J\)           | \(K\)           | \(L\)           | \(M\)           | \(N\)  |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
|                             |                             | Number of test takers in sample (in thousands) by exam year                                                                                                                                                                       |        |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 1-2 Treated                 | Control                     |                             |                 |                 |                 |                 |                 |                 |                 |                 |                 |                 |                 |        |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| group                       | group                       |                             | 2007            | 2008            | 2009            | 2010            | 2011            | 2012            | 2013            | 2014            | 2015            | 2016            | 2017            | Total  |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2008                        | 2009                        |                             | 114             | 123             |                 |                 |                 |                 |                 |                 |                 |                 |                 | 237    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2008                        | 2010                        |                             | 29              | 30              | 31              |                 |                 |                 |                 |                 |                 |                 |                 | 90     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2008                        | 2011                        |                             | 11              | 12              | 13              | 14              |                 |                 |                 |                 |                 |                 |                 | 50     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2008                        | 2012                        |                             | 13              | 15              | 15              | 16              | 18              |                 |                 |                 |                 |                 |                 | 78     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2008                        | 2013                        |                             | 68              | 78              | 70              | 81              | 88              | 89              |                 |                 |                 |                 |                 | 474    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2008                        | 2014                        |                             | 18              | 21              | 20              | 21              | 25              | 25              | 25              |                 |                 |                 |                 | 154    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2008                        | 2015                        |                             | 12              | 13              | 14              | 15              | 16              | 17              | 17              | 17              |                 |                 |                 | 120    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2008                        | 2016                        |                             | 7               | 8               | 9               | 9               | 11              | 11              | 11              | 11              | 10              |                 |                 | 86     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2008                        | Never                       |                             | 8               | 9               | 10              | 10              | 12              | 12              | 12              | 11              | 11              | 11              | 10              | 115    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2009                        | 2010                        |                             | 131             | 139             | 121             |                 |                 |                 |                 |                 |                 |                 |                 | 391    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2009                        | 2011                        |                             | 113             | 121             | 104             | 120             |                 |                 |                 |                 |                 |                 |                 | 456    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2009                        | 2012                        |                             | 115             | 124             | 105             | 122             | 128             |                 |                 |                 |                 |                 |                 | 594    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2009                        | 2013                        |                             | **170**         | **186**         | **161**         | **187**         | **198**         | **198**         |                 |                 |                 |                 |                 | 1,100  |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2009                        | 2014                        |                             | 120             | 130             | 110             | 126             | 135             | 134             | 136             |                 |                 |                 |                 | 890    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2009                        | 2015                        |                             | 114             | 122             | 104             | 121             | 126             | 126             | 127             | 129             |                 |                 |                 | 969    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2009                        | 2016                        |                             | 109             | 116             | 99              | 114             | 121             | 120             | 121             | 123             | 124             |                 |                 | 1,048  |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2009                        | Never                       |                             | 110             | 117             | 100             | 115             | 121             | 121             | 122             | 124             | 125             | 127             | 117             | 1,299  |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 6-14 2010                   | 2011                        |                             | 28              | 28              | 28              | 41              |                 |                 |                 |                 |                 |                 |                 | 125    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2010                        | 2012                        |                             | 31              | 31              | 29              | 43              | 46              |                 |                 |                 |                 |                 |                 | 180    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2010                        | 2013                        |                             | 86              | 94              | 84              | 108             | 115             | 118             |                 |                 |                 |                 |                 | 606    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2010                        | 2014                        |                             | 35              | 37              | 34              | 48              | 52              | 55              | 56              |                 |                 |                 |                 | 318    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2010                        | 2015                        |                             | 29              | 29              | 28              | 42              | 44              | 46              | 48              | 47              |                 |                 |                 | 314    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2010                        | 2016                        |                             | 24              | 24              | 23              | 36              | 38              | 40              | 42              | 41              | 40              |                 |                 | 309    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2010                        | Never                       |                             | 25              | 25              | 24              | 37              | 39              | 41              | 43              | 42              | 41              | 40              | 36              | 393    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2011                        | 2012                        |                             | 12              | 13              | 12              | 14              | 16              |                 |                 |                 |                 |                 |                 | 67     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2011                        | 2013                        |                             | 67              | 76              | 67              | 79              | 85              | 86              |                 |                 |                 |                 |                 | 460    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2011                        | 2014                        |                             | 16              | 19              | 17              | 19              | 22              | 22              | 22              |                 |                 |                 |                 | 138    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2011                        | 2015                        |                             | 10              | 11              | 11              | 13              | 14              | 14              | 14              | 14              |                 |                 |                 | 101    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2011                        | 2016                        |                             | 5               | 6               | 6               | 7               | 8               | 8               | 8               | 8               | 8               |                 |                 | 63     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2011                        | Never                       |                             | 6               | 7               | 7               | 8               | 9               | 9               | 9               | 9               | 8               | 9               | 8               | 89     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2012                        | 2013                        |                             | 70              | 79              | 69              | 81              | 87              | 88              |                 |                 |                 |                 |                 | 474    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2012                        | 2014                        |                             | 19              | 22              | 19              | 21              | 24              | 25              | 24              |                 |                 |                 |                 | 154    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2012                        | 2015                        |                             | 13              | 14              | 13              | 15              | 16              | 16              | 16              | 16              |                 |                 |                 | 119    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2012                        | 2016                        |                             | 8               | 9               | 7               | 9               | 10              | 10              | 10              | 10              | 10              |                 |                 | 84     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2012                        | Never                       |                             | 9               | 10              | 8               | 10              | 11              | 11              | 11              | 11              | 11              | 11              | 10              | 112    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2013                        | 2014                        |                             | 74              | 85              | 74              | 86              | 94              | 95              | 97              |                 |                 |                 |                 | 605    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2013                        | 2015                        |                             | 68              | 77              | 68              | 80              | 85              | 87              | 89              | 90              |                 |                 |                 | 644    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2013                        | 2016                        |                             | 63              | 72              | 63              | 74              | 80              | 81              | 83              | 84              | 83              |                 |                 | 682    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2013                        | Never                       |                             | 64              | 73              | 63              | 75              | 80              | 82              | 84              | 84              | 84              | 85              | 80              | 855    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2014                        | 2015                        |                             | 17              | 20              | 17              | 20              | 22              | 23              | 23              | 23              |                 |                 |                 | 167    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2014                        | 2016                        |                             | 12              | 15              | 12              | 14              | 17              | 17              | 17              | 17              | 18              |                 |                 | 140    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2014                        | Never                       |                             | 13              | 16              | 13              | 15              | 17              | 18              | 18              | 18              | 19              | 19              | 18              | 184    |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2015                        | 2016                        |                             | 6               | 7               | 6               | 8               | 8               | 9               | 9               | 9               | 9               |                 |                 | 72     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2015                        | Never                       |                             | 7               | 8               | 7               | 9               | 9               | 10              | 10              | 10              | 10              | 11              | 7               | 97     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 2016                        | Never                       |                             | 2               | 3               | 2               | 3               | 3               | 4               | 4               | 4               | 4               | 4               | 4               | 37     |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| 6-14                        |                             |                             | 813             | 1,016           | 1,054           | 1,014           | 740             | 535             | 345             | 181             | 162             | 5,859           |                 |        |   |   |
+-----------------------------+-----------------------------+-----------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| Tab 4, Col E sample         | 170             | 186             | 161             | 187             | 198             | 198             |                 |                 |                 |                 |                 | 1,100  |   |   |
+-----------------------------------------------------------------------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+
| Tab 4, Col F sample         | 2,079           | 2,274           | 1,898           | 2,087           | 2,052           | 1,864           | 1,311           | 950             | 615             | 318             | 289             | 15,738 |   |   |
+-----------------------------------------------------------------------------------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+-----------------+--------+---+---+

: Visualization of stacked dataset for event studies and robustness tests

</div>

</div>

*Notes*: This table presents the stacked dataset for our event studies (Figure 3) and robustness tests (Table 4). Columns (A) and (B) show the pairwise combinations of ENEM adoption years, $\tau^*_s$ and $\tau^*_{s'}$. Columns (C) through (N) indicate the number of observations (in thousands) or each pair by exam year. The boxed cells show the sample for both Figure 3 and column (D) of Table 4, defined by the pairwise treatment effects that we can estimate using 2009–2017 exam takers. The bold cells show the sample for column (E) of Table 4, which includes 2007–2012 exam takers in the 2009 vs. 2013 pair of ENEM adoptions years. The sample for column (F) of Table 4 includes all cells in this table. The bottom three rows summarize the totals for each sample by exam year.

<div class="center">

<div id="tab:hsgap_alt_sample">

  ------------------------------- -- ----------- ---------- ------------ ---------- --------------------- ---------- -------------------------- ---------- ------------------------- ---------- -- -- -- -- -- -- -- --
                                           \(A\)                   \(B\)                            \(C\)                                 \(D\)                                \(E\)                                 
                                                                                                                                       Partici-                             Partici-                                 
                                            Main                  Appear                           Appear                                pation                               pation                                 
  Dependent variable                      sample                any year                         pre-ENEM                              pre-ENEM                            all years                                 
  Math score                               0.158 $^{*}$            0.097                            0.112 $^{*}$                          0.158 $^{**}$                        0.127 $^{*}$                          
                                          (0.079 )                (0.058 )                         (0.060 )                              (0.070 )                             (0.069 )                               
  Language arts score                      0.076 $^{***}$          0.048 $^{***}$                   0.054 $^{***}$                        0.088 $^{***}$                       0.072 $^{***}$                        
                                          (0.026 )                (0.017 )                         (0.013 )                              (0.028 )                             (0.024 )                               
  Natural science score                    0.065 $^{*}$            0.047                            0.046                                 0.102 $^{**}$                        0.080 $^{**}$                         
                                          (0.034 )                (0.032 )                         (0.030 )                              (0.036 )                             (0.035 )                               
  Social science score                     0.081 $^{***}$          0.066 $^{**}$                    0.069 $^{***}$                        0.119 $^{**}$                        0.103 $^{**}$                         
                                          (0.023 )                (0.026 )                         (0.025 )                              (0.044 )                             (0.043 )                               
  Average score (core subjects)            0.110 $^{**}$           0.074 $^{**}$                    0.081 $^{**}$                         0.135 $^{***}$                       0.110 $^{**}$                         
                                          (0.040 )                (0.035 )                         (0.032 )                              (0.043 )                             (0.040 )                               
  Writing score                            0.102 $^{*}$            0.124 $^{**}$                    0.129 $^{**}$                         0.037                                0.006                                 
                                          (0.058 )                (0.048 )                         (0.049 )                              (0.105 )                             (0.119 )                               
  $N$ (# exam takers)                  2,512,214              10,991,098                        6,774,892                               856,165                              718,748                                 
  Appear in INEP:                      All years                Any year              At least 2005–2008                   At least 2005–2008                            All years                                 
  Participation rate:                                                                                                  $\geq$ 50% in 2005–2008              $\geq$ 50% in all years                                 
  ------------------------------- -- ----------- ---------- ------------ ---------- --------------------- ---------- -------------------------- ---------- ------------------------- ---------- -- -- -- -- -- -- -- --

  : The impact of ENEM adoption on private/public test score gaps using alternative samples

</div>

</div>

*Notes*: This table examines the robustness of our estimates of the effects of ENEM adoption on the private/public school test score gaps to using alternative samples.

The dependent variables are ENEM subject scores in SD units. "Average score (core subjects)" is the average score across math, language arts, natural science, and social science.

Each column shows $\beta^{\text{gap}}$ coefficients from equation dd_gap estimated on different samples of high schools. Column (A) reproduces the estimates in the main sample, which consists of high schools that appeared in the INEP report in each year from 2005 to 2015. Column (B) relaxes the selection criterion and includes high schools that appeared at least once in the INEP report from 2005 to 2015. Column (C) includes high schools that appeared in the INEP report every year in the pre-ENEM period, i.e., from 2005 to 2008. In addition to the requirements in column (C), column (D) requires a participation rate of over 50% every year in the pre-ENEM period, meaning that at least 50% of the seniors in the high school took the ENEM in those years. Finally, column (E) includes high schools listed in the INEP report every year from 2005 to 2015 and maintained a participation rate of over 50% in all years.

Note that in certain years, INEP required a minimum participation rate for high schools to be included in the report. As a result, columns (A) to (C) inherently satisfy this participation requirement. The criteria in columns (D) and (E) impose additional participation requirements beyond the inherent INEP requirements. Details on the INEP requirement on participation rate can be found in Appendix 9.3.

Parentheses contain standard errors clustered at the state level. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:oth_gaps">

  ------------------------------- -- ----------- ---------- ----------- ---------- ------------------ -------- ------------ --------- ------------------ --- ----------- ---------- -- -- -- -- -- --
                                           \(A\)                  \(B\)                         \(C\)                 \(D\)                        \(E\)           \(F\)                           
                                        Private/                                                                     Mother                       Father                                           
                                          public                 White/                         Male/              college/                     college/        High/low                           
  Dependent variable                      school              non-white                        female            no college                   no college          income                           
  Math score                               0.158 $^{*}$           0.069 $^{*}$       $-$0.017                 0.111 $^{*}$                 0.082           0.125 $^{*}$                    
                                          (0.079 )               (0.038 )                      (0.016 )              (0.058 )                     (0.060 )        (0.067 )                         
  Language arts score                      0.076 $^{***}$         0.027              $-$0.007                 0.047                        0.011           0.050 $^{*}$                    
                                          (0.026 )               (0.021 )                      (0.012 )              (0.034 )                     (0.034 )        (0.025 )                         
  Natural science score                    0.065 $^{*}$           0.041 $^{*}$       $-$0.019 $^{*}$          0.039             $-$0.001           0.055 $^{**}$                   
                                          (0.034 )               (0.023 )                      (0.011 )              (0.031 )                     (0.033 )        (0.025 )                         
  Social science score                     0.081 $^{***}$         0.060 $^{***}$     $-$0.017                 0.067 $^{**}$                0.022           0.073 $^{***}$                  
                                          (0.023 )               (0.020 )                      (0.011 )              (0.030 )                     (0.028 )        (0.024 )                         
  Average score (core subjects)            0.110 $^{**}$          0.057 $^{**}$      $-$0.017                 0.076 $^{*}$                 0.033           0.088 $^{**}$                   
                                          (0.040 )               (0.024 )                      (0.012 )              (0.041 )                     (0.039 )        (0.038 )                         
  Writing score                            0.102 $^{*}$           0.005              $-$0.014                 0.052                        0.066           0.061 $^{*}$                    
                                          (0.058 )               (0.033 )                      (0.014 )              (0.039 )                     (0.045 )        (0.032 )                         
  $N$ (# exam takers)                  2,512,214              2,387,052                     2,512,214             2,489,743                    2,489,191       2,487,270                           
  Low-stakes average score gap             1.229                  0.477                         0.271                 0.882                        1.027           0.892                           
                                                                                                                                                                                                   
  ------------------------------- -- ----------- ---------- ----------- ---------- ------------------ -------- ------------ --------- ------------------ --- ----------- ---------- -- -- -- -- -- --

  : The impact of ENEM adoption on demographic test score gaps

</div>

</div>

*Notes*: This table shows estimates of the impact of ENEM adoption on test score gaps between students from different demographic groups, as listed in the column headers.

The table displays $\beta^{\text{gap}}$ coefficients from versions of equation dd_gap estimated at the individual level. The dependent variables are ENEM subject scores in SD units. "Average score (core subjects)" is the average score across math, language arts, natural science, and social science. We estimate the regressions using data from 2009–2017 ENEM exam takers in our analysis sample (column C of Table 1). Some columns have slightly smaller sample sizes due to missing values of demographic variables.

Column (A) reproduces our benchmark $\beta^{\text{gap}}$ coefficients from column (E) of Table 3. Columns (B)–(F) display $\beta^{\text{gap}}$ coefficients from a specification of equation dd_gap that replaces the $\text{Private}_h$ dummy with an indicator for being in the more advantaged demographic group. Column (B) defines "non-white" as students who self-identify as Black, Brown, or Indigenous. Columns (D) and (E) define "college" as having a college or post-graduate degree. Column (F) defines a student as "high-income" if his/her reported family income was greater than or equal to twice the minimum wage in the exam year.

The bottom row shows the gap in the average score (core subjects) in exam cohorts prior to each state's ENEM adoption year (i.e., cohorts with $\text{HighStakes}_{st}=0$). Parentheses contain standard errors clustered at the state level. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:wildt_scores">

+:------------------------------+:--+--------:+:----+--------:+:----+-----------------:+:----+-----------:+:---+--:+:--+--:+:--+--:+:--+--:+:--+--:+:--+
|                               |   | \(A\)   |     | \(B\)   |     | \(C\)            |     | \(D\)      |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | DD coefficients                                                     |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
| 3-9                           |   | All     |     | Private |     | Public           |     | Private/   |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
| Dependent variable            |   | schools |     | schools |     | schools          |     | public gap |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
| Math score                    |   | 0.022   |     | 0.143   |     | $-$0.015 |     | 0.158      |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | (0.690  | )   | (0.021  | )   | (0.829           | )   | (0.056     | )  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | 0.781 |   | 0.090 |   | 0.857          |   | 0.096    |  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
| Language arts score           |   | 0.035   |     | 0.068   |     | $-$0.008 |     | 0.076      |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | (0.334  | )   | (0.002  | )   | (0.820           | )   | (0.007     | )  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | 0.438 |   | 0.016 |   | 0.881          |   | 0.002    |  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
| Natural science score         |   | 0.026   |     | 0.062   |     | $-$0.003 |     | 0.065      |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | (0.520  | )   | (0.052  | )   | (0.951           | )   | (0.071     | )  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | 0.613 |   | 0.264 |   | 0.885          |   | 0.134    |  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
| Social science score          |   | 0.019   |     | 0.056   |     | $-$0.024 |     | 0.081      |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | (0.581  | )   | (0.064  | )   | (0.402           | )   | (0.002     | )  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | 0.677 |   | 0.356 |   | 0.450          |   | 0.070    |  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
| Average score (core subjects) |   | 0.029   |     | 0.095   |     | $-$0.014 |     | 0.110      |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | (0.497  | )   | (0.013  | )   | (0.752           | )   | (0.012     | )  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | 0.633 |   | 0.082 |   | 0.799          |   | 0.036    |  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
| Writing score                 |   | 0.049   |     | 0.165   |     | 0.063            |     | 0.102      |    |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | (0.168  | )   | (0.031  | )   | (0.065           | )   | (0.090     | )  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+
|                               |   | 0.128 |   | 0.052 |   | 0.118          |   | 0.138    |  |   |   |   |   |   |   |   |   |   |   |
+-------------------------------+---+---------+-----+---------+-----+------------------+-----+------------+----+---+---+---+---+---+---+---+---+---+---+

: Clustered and wild $t$ bootstrap $p$ values for effects of ENEM adoption on test scores

</div>

</div>

*Notes*: This table examines the robustness of our estimates of the effects of ENEM adoption on test scores to the inference method. The point estimates reproduce the regression coefficients shown in columns (B) through (E) of Table 3. Parentheses contain $p$ values derived from standard errors clustered at the state level, as in Table 3. Brackets show $p$ values from a wild $t$ bootstrap with 1000 replications that imposes the null hypothesis, as recommended by Cameron, Gelbach, and Miller (2008).

<div class="center">

<div id="tab:aa_adoption">

+:--------------------------------------------------------+:--+----------:+:---------+-----------------:+:---------+-----------------:+:---------+-----------------:+:---------+--:+:--+--:+:--+--:+:--+--:+:--+--:+:--+
|                                                         |   | \(A\)     |          | \(B\)            |          | \(C\)            |          | \(D\)            |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          | Controls for affirmative action adoption                                     |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| 5-9                                                     |   | Benchmark |          | Federal          |          | Federal &        |          | All              |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| Covariates                                              |   | model     |          | universities     |          | state univ.      |          | universities     |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          |                  |          |                  |          |                  |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionENEM}_{s(h)t} \times \text{Private}_h$ |   | 0.158     | $^{*}$   | 0.151            | $^{*}$   | 0.157            | $^{*}$   | 0.158            | $^{*}$   |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   | (0.079    | )        | (0.078           | )        | (0.079           | )        | (0.080           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionAA}_{s(h)t} \times \text{Private}_h$   |   |           |          | 0.005            |          | 0.039            |          | 0.029            |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          | (0.088           | )        | (0.122           | )        | (0.116           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          |                  |          |                  |          |                  |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionENEM}_{s(h)t} \times \text{Private}_h$ |   | 0.076     | $^{***}$ | 0.080            | $^{***}$ | 0.074            | $^{***}$ | 0.084            | $^{***}$ |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   | (0.026    | )        | (0.023           | )        | (0.024           | )        | (0.026           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionAA}_{s(h)t} \times \text{Private}_h$   |   |           |          | $-$0.109 | $^{**}$  | $-$0.105 |          | $-$0.126 |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          | (0.051           | )        | (0.071           | )        | (0.076           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          |                  |          |                  |          |                  |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionENEM}_{s(h)t} \times \text{Private}_h$ |   | 0.065     | $^{*}$   | 0.062            | $^{*}$   | 0.063            | $^{*}$   | 0.066            | $^{*}$   |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   | (0.034    | )        | (0.033           | )        | (0.034           | )        | (0.035           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionAA}_{s(h)t} \times \text{Private}_h$   |   |           |          | $-$0.024 |          | $-$0.043 |          | $-$0.016 |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          | (0.061           | )        | (0.085           | )        | (0.072           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          |                  |          |                  |          |                  |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionENEM}_{s(h)t} \times \text{Private}_h$ |   | 0.081     | $^{***}$ | 0.075            | $^{***}$ | 0.080            | $^{***}$ | 0.076            | $^{***}$ |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   | (0.023    | )        | (0.024           | )        | (0.022           | )        | (0.025           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionAA}_{s(h)t} \times \text{Private}_h$   |   |           |          | 0.029            |          | 0.036            |          | 0.091            |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          | (0.063           | )        | (0.089           | )        | (0.080           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          |                  |          |                  |          |                  |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionENEM}_{s(h)t} \times \text{Private}_h$ |   | 0.110     | $^{**}$  | 0.106            | $^{**}$  | 0.108            | $^{**}$  | 0.111            | $^{**}$  |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   | (0.040    | )        | (0.040           | )        | (0.040           | )        | (0.042           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionAA}_{s(h)t} \times \text{Private}_h$   |   |           |          | $-$0.029 |          | $-$0.021 |          | $-$0.006 |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          | (0.067           | )        | (0.096           | )        | (0.085           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          |                  |          |                  |          |                  |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionENEM}_{s(h)t} \times \text{Private}_h$ |   | 0.102     | $^{*}$   | 0.083            |          | 0.103            | $^{*}$   | 0.086            |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   | (0.058    | )        | (0.052           | )        | (0.054           | )        | (0.054           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $\text{ProportionAA}_{s(h)t} \times \text{Private}_h$   |   |           |          | 0.199            | $^{**}$  | 0.206            | $^{*}$   | 0.285            | $^{**}$  |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
|                                                         |   |           |          | (0.078           | )        | (0.106           | )        | (0.133           | )        |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+
| $N$ (# exam takers)                                     |   | 2,512,214 |          | 2,512,214        |          | 2,512,214        |          | 2,512,214        |          |   |   |   |   |   |   |   |   |   |   |
+---------------------------------------------------------+---+-----------+----------+------------------+----------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+---+---+

: Robustness to controls for affirmative action adoption

</div>

</div>

*Notes*: This table examines the robustness of our results on private/public school test score gaps to controls for affirmative action adoption. The sample includes ENEM test takers in our high school senior sample (column C of Table 1). The dependent variables are the ENEM subject scores listed in the panel titles (in SD units). Column (A) replicates our benchmark results from column (E) of Table 3, which are the $\beta^{\text{gap}}$ coefficients on the interaction between $\text{ProportionENEM}_{s(h)t}$ and a dummy for private high schools, $\text{Private}_h$ from equation dd_gap. In columns (B)–(D) we add in a measure of the adoption of affirmative action at the state $\times$ year level, $\text{ProportionAA}_{s(h)t}$, and its interaction with $\text{Private}_h$. We compute $\text{ProportionAA}_{s(h)t}$ as the proportion of all new enrollees in state $s(h)$ and year $t$ who were admitting through reserved quotas using higher education census data. Columns (B)–(D) define $\text{ProportionAA}_{s(h)t}$ using only federal universities, federal and state universities, and all universities, respectively.

Parentheses contain standard errors clustered at the state level.

\* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:dd_slots">

+:------------------------------------------------------+:--+-----------:+:--+-----------------:+:------+-----------------:+:--+--:+:--+--:+:--+--:+:--+--:+:--+--:+:--+--:+:--+
|                                                       |   | \(A\)      |   | \(B\)            |       | \(C\)            |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   | Low-stakes |   |                  |       |                  |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   | mean       |   | DD coefficients                             |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| 3-3                                                   |   |            |   | Continous        |       | Binary           |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| Dependent variable                                    |   |            |   | treatment        |       | treatment        |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   |            |   |                  |       |                  |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| Log \# admitted students                              |   | 4.416      |   | 0.046            |       | 0.007            |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   |            |   | (0.093           | )     | (0.045           | ) |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| Prop. admitted through reserved quotas                |   | 0.155      |   | 0.032            |       | 0.004            |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   |            |   | (0.047           | )     | (0.028           | ) |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| $N$ (# institutions $\times$ programs $\times$ years) |   | 10,544     |   | 39,027           |       | 39,027           |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   |            |   |                  |       |                  |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| Log \# admitted students                              |   | 8.693      |   | $-$0.111 |       | $-$0.015 |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   |            |   | (0.081           | )     | (0.032           | ) |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| Prop. admitted through reserved quotas                |   | 0.155      |   | 0.037            |       | 0.008            |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   |            |   | (0.051           | )     | (0.031           | ) |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| $N$ (# institutions $\times$ years)                   |   | 148        |   | 553              |       | 553              |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   |            |   |                  |       |                  |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| Log \# admitted students                              |   | 9.527      |   | $-$0.083 |       | $-$0.007 |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   |            |   | (0.089           | )     | (0.035           | ) |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| Prop. admitted through reserved quotas                |   | 0.155      |   | 0.031            |       | 0.007            |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
|                                                       |   |            |   | (0.047           | )     | (0.030           | ) |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+
| $N$ (# states $\times$ years)                         |   | 71         |   | 243              |       | 243              |   |   |   |   |   |   |   |   |   |   |   |   |   |
+-------------------------------------------------------+---+------------+---+------------------+-------+------------------+---+---+---+---+---+---+---+---+---+---+---+---+---+

: The impact of ENEM adoption on the number of slots offered by federal universities

</div>

</div>

*Notes*: This table presents estimates of the impact of ENEM adoption on the number of admission slots offered by federal universities.

The dependent variables are: (1) the logarithm of the number of admitted students (a proxy for the number of slots); and (2) the proportion of students admitted through quotas reserved for low-income or minority students. We compute each variable at three levels: institution $\times$ program $\times$ year (Panel A), institution $\times$ year (Panel B), and state $\times$ year (Panel C).

Column (A) shows the mean of each variable for admission cohorts prior to each state's ENEM adoption year (i.e., cohorts with $\text{HighStakes}_{st}=0$). Columns (B) and (C) display $\beta$ coefficients from a specification of equation dd estimated at the level corresponding to the one the dependent variable is defined. Column (B) uses the continuous treatment variable, $\text{ProportionENEM}_{st}$, while column (C) uses the binary treatment variable, $\text{HighStakes}_{st}$.

Regressions for the log number of admitted students use fixed weights over time, based on the number of admitted students in 2009. Regressions for the proportion of students admitted through quotas reserved for low-income or minority students are weighted by the number of admitted students in each respective year.

Parentheses contain standard errors clustered at the state level. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:prep_gaps">

  ------------------------------- -- ----------- ---------- ----------- ---------- ------------------ ---------- ------------- --------- -- -- -- -- -- -- -- -- -- --
                                           \(A\)                  \(B\)                         \(C\)                    \(D\)                                      
                                        Private/                  Prep/                         Prep/              Took/didn't                                      
                                          public                 public                 other private                   take a                                      
  Dependent variable                      school                 school                        school              prep course                                      
  Math score                               0.158 $^{*}$           0.202 $^{**}$                 0.052                    0.215 $^{**}$                              
                                          (0.079 )               (0.094 )                      (0.052 )                 (0.080 )                                    
  Language arts score                      0.076 $^{***}$         0.115 $^{**}$                 0.040                    0.194 $^{**}$                              
                                          (0.026 )               (0.045 )                      (0.031 )                 (0.083 )                                    
  Natural science score                    0.065 $^{*}$           0.134 $^{***}$                0.075 $^{**}$            0.185 $^{**}$                              
                                          (0.034 )               (0.043 )                      (0.028 )                 (0.076 )                                    
  Social science score                     0.081 $^{***}$         0.172 $^{***}$                0.097 $^{***}$           0.212 $^{**}$                              
                                          (0.023 )               (0.030 )                      (0.030 )                 (0.089 )                                    
  Average score (core subjects)            0.110 $^{**}$          0.180 $^{***}$                0.076 $^{**}$            0.232 $^{**}$                              
                                          (0.040 )               (0.051 )                      (0.034 )                 (0.093 )                                    
  Writing score                            0.102 $^{*}$           0.244                         0.154                    0.028                                      
                                          (0.058 )               (0.153 )                      (0.124 )                 (0.045 )                                    
  $N$ (# exam takers)                  2,512,214              1,779,119                       807,293                  500,938                                      
  Low-stakes average score gap             1.229                  1.147              $-$0.087                    0.371                                      
                                                                                                                                                                    
  ------------------------------- -- ----------- ---------- ----------- ---------- ------------------ ---------- ------------- --------- -- -- -- -- -- -- -- -- -- --

  : The impact of ENEM adoption on test score gaps by test prep activity

</div>

</div>

*Notes*: This table presents estimates of the impact of ENEM adoption on test score gaps between students who did/did not engage in test prep activities.

We define two different measures of students' test prep activity. Our first measure of test prep activity defines a set of "prep schools" with curricula specifically focused on preparation for college admission exams. We obtained lists of schools that use test-oriented curricula from the websites of four prominent test prep companies: [*Sistema Anglo*](https://www.sistemaanglo.com.br/encontre_uma_escola.php), [*Sistema pH*](https://web.archive.org/web/20190729023359/http://www.sistemadeensinoph.com.br/encontre-uma-escola), [*Elite Rede de Ensino*](https://ensinoelite.com.br/unidades/), and [*Curso Objetivo*](https://www.curso-objetivo.br/curso/unidades.aspx). These lists provided the names and street addresses of each test-oriented school. We then obtained names and addresses for each school in our sample from INEP's administrative [](https://www.gov.br/inep/pt-br/acesso-a-informacao/dados-abertos/inep-data/catalogo-de-escolas) data. We geocoded the addresses in both datasets using the [`ggmap`](https://cran.r-project.org/web/packages/ggmap/index.html) package for $R$. Finally, we matched schools whose addresses were within 100 meters of each other and manually checked all matches using school names. We define schools that matched using this procedure as "prep schools" (all of the matched schools are private high schools).

Our second measure of test prep activity is based on a variable from the ENEM questionnaire that indicates whether individuals took an entrance exam preparation course. The question does not distinguish between courses focused on the ENEM exam and those focused on other *vestibular* exams. This question is only available in the 2009–2011 and 2013–2014 cohorts, and there are missing values for many students in these cohorts.

The table displays $\beta^{\text{gap}}$ coefficients from specifications of equation dd_gap estimated at the individual level. Column (A) reproduces our benchmark $\beta^{\text{gap}}$ coefficients from column (E) of Table 3. Columns (B) to (D) present $\beta^{\text{gap}}$ coefficients from specifications of equation dd_gap where we replace the $\text{Private}_h$ dummy with an indicator for prep schools (columns B and C) or with an indicator for individuals who took a prep course (column D). The bottom row shows the gap in the average score (core subjects) for exam cohorts prior to each state's ENEM adoption year (i.e., cohorts with $\text{HighStakes}_{st}=0$). The dependent variables are ENEM subject scores in SD units. "Average score (core subjects)" is the average score across math, language arts, natural science, and social science. We estimate the regressions using data from 2009–2017 ENEM exam takers in our analysis sample (column C of Table 1)

Parentheses contain standard errors clustered at the state level. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:wildt_informativeness">

+:----------------------------------------+:--+-----------------:+:------+-----------------:+:---+-----------------:+:-------+-----------------:+:---+--:+:--+--:+:--+--:+:--+--:+:--+--:+:--+
|                                         |   | \(A\)            |       | \(B\)            |    | \(C\)            |        | \(D\)            |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | Benchmark model                             |    | Binary treatment                             |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+---------------------------------------------+----+----------------------------------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | DD coefficients                             |    | DD coefficients                              |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| 3-5                                     |   | Raw              |       | Within-          |    | Raw              |        | Within           |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Dependent variable                      |   | corr.            |       | program          |    | corr.            |        | program          |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Enrolled in any college by 2019         |   | 0.036            |       |                  |    | 0.022            |        |                  |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.000           | )     |                  |    | (0.000           | )      |                  |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.000          |     |                  |    | 0.000          |      |                  |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Finished college within 5 years of ENEM |   | 0.014            |       | 0.026            |    | 0.007            |        | 0.015            |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.091           | )     | (0.001           | )  | (0.116           | )      | (0.010           | )  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.096          |     | 0.028          |  | 0.152          |      | 0.044          |  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Earned a college degree by 2019         |   | 0.033            |       | 0.038            |    | 0.016            |        | 0.023            |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.037           | )     | (0.001           | )  | (0.179           | )      | (0.005           | )  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.144          |     | 0.016          |  | 0.244          |      | 0.032          |  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Appears in RAIS in 2016–2018           |   | 0.056            |       | 0.020            |    | 0.014            |        | 0.008            |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.209           | )     | (0.095           | )  | (0.567           | )      | (0.310           | )  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.501          |     | 0.238          |  | 0.675          |      | 0.352          |  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Persisted in college for 1 year         |   | 0.008            |       | 0.024            |    | 0.007            |        | 0.011            |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.581           | )     | (0.004           | )  | (0.564           | )      | (0.097           | )  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.563          |     | 0.014          |  | 0.595          |      | 0.092          |  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Persisted in college for 3 years        |   | 0.033            |       | 0.043            |    | 0.018            |        | 0.024            |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.017           | )     | (0.000           | )  | (0.047           | )      | (0.000           | )  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.056          |     | 0.000          |  | 0.064          |      | 0.008          |  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Completed program within 5 years        |   | 0.044            |       | 0.035            |    | 0.022            |        | 0.018            |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.025           | )     | (0.003           | )  | (0.063           | )      | (0.019           | )  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.140          |     | 0.030          |  | 0.092          |      | 0.042          |  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Fraction of college credits completed   |   | 0.003            |       | 0.014            |    | $-$0.011 |        | 0.013            |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.812           | )     | (0.292           | )  | (0.330           | )      | (0.064           | )  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.785          |     | 0.382          |  | 0.342          |      | 0.126          |  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Hourly wage (BRL)                       |   | 0.046            |       | 0.027            |    | 0.027            |        | 0.012            |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.017           | )     | (0.013           | )  | (0.001           | )      | (0.061           | )  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.176          |     | 0.180          |  | 0.036          |      | 0.128          |  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
| Log hourly wage                         |   | $-$0.029 |       | $-$0.001 |    | $-$0.017 |        | $-$0.001 |    |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | (0.050           | )     | (0.900           | )  | (0.053           | )      | (0.866           | )  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+
|                                         |   | 0.284          |     | 0.905          |  | 0.136          |      | 0.889          |  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+---+------------------+-------+------------------+----+------------------+--------+------------------+----+---+---+---+---+---+---+---+---+---+---+

: Clustered and wild $t$ bootstrap $p$ values for effects of ENEM adoption on the informativeness of test scores for longer-run outcomes

</div>

</div>

*Notes*: This table examines the robustness of our estimates of the effects of ENEM adoption on the informativeness of test scores for student longer-run outcomes to the inference method. The point estimates reproduce the regression coefficients shown in columns (B) through (E) of Table 5. Parentheses contain $p$ values derived from standard errors clustered at the state level, as in Table 5. Brackets show $p$ values from a wild $t$ bootstrap with 1000 replications that imposes the null hypothesis, as recommended by Cameron, Gelbach, and Miller (2008).

<div class="center">

<div id="tab:dd_correlation_demo">

+:----------------------------------------+:-------------+-----------------:+:-------------+-----------------:+:-------------+-----------------:+:-------------+-----------------:+:---------+-----------------:+:---------+--:+:--+--:+:--+--:+:--+--:+:--+
|                                         |              | \(A\)            |              | \(B\)            |              | \(C\)            |              | \(D\)            |          | \(E\)            |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | Control variables (cumulative)                                                                                                                       |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| 3-11 Dependent variable: Correlation    |              | No               |              | Private          |              | \+ SES           |              | \+ Age           |          | HS               |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| between average ENEM scores and\...     |              | controls         |              | HS               |              | & race           |              | & gender         |          | dummies          |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              |                  |              |                  |              |                  |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| **Panel A. Outcomes for all exam takers**                                                                                                                                       |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Enrolled in any college by 2019         |              | 0.036            | $^{***}$     | 0.028            | $^{***}$     | 0.036            | $^{***}$     | 0.031            | $^{***}$ | 0.028            | $^{***}$ |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.006           | )            | (0.008           | )            | (0.007           | )            | (0.011           | )        | (0.009           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Finished college within 5 years of ENEM |              | 0.014            | $^{*}$       | 0.016            | $^{**}$      | 0.012            |              | 0.005            |          | 0.008            |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.008           | )            | (0.006           | )            | (0.008           | )            | (0.011           | )        | (0.008           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Earned a college degree by 2019         |              | 0.033            | $^{**}$      | 0.017            |              | 0.014            |              | 0.009            |          | 0.012            |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.015           | )            | (0.013           | )            | (0.017           | )            | (0.019           | )        | (0.014           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Appears in RAIS in 2016–2018           |              | 0.056            |              | 0.009            |              | 0.009            |              | 0.009            |          | 0.012            |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.044           | )            | (0.024           | )            | (0.022           | )            | (0.021           | )        | (0.019           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| $N$ (# exam takers)                     |              | 1,257,079        |              | 1,257,079        |              | 1,257,079        |              | 1,257,079        |          | 1,257,079        |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              |                  |              |                  |              |                  |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| **Panel B. Outcomes for college enrollees**                                                                                                                                     |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Persisted in college for 1 year         |              | 0.008            |              | 0.013            |              | 0.012            |              | 0.013            |          | $-$0.000 |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.014           | )            | (0.010           | )            | (0.009           | )            | (0.009           | )        | (0.010           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Persisted in college for 3 years        |              | 0.033            | $^{**}$      | 0.023            | $^{**}$      | 0.021            | $^{**}$      | 0.022            | $^{**}$  | 0.006            |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.013           | )            | (0.009           | )            | (0.008           | )            | (0.008           | )        | (0.011           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Completed program within 5 years        |              | 0.044            | $^{**}$      | 0.026            |              | 0.018            |              | 0.020            |          | 0.008            |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.018           | )            | (0.018           | )            | (0.020           | )            | (0.021           | )        | (0.015           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Fraction of college credits completed   |              | 0.004            |              | $-$0.008 |              | $-$0.007 |              | $-$0.006 |          | $-$0.002 |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.014           | )            | (0.013           | )            | (0.013           | )            | (0.013           | )        | (0.017           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| $N$ (# in higher ed.)                   |              | 959,339          |              | 959,339          |              | 959,339          |              | 959,339          |          | 959,339          |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              |                  |              |                  |              |                  |              |                  |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| **Panel C. Outcome for individuals in RAIS**                                                                                                                                    |          |                  |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Hourly wage (BRL)                       |              | 0.047            | $^{**}$      | 0.049            | $^{***}$     | 0.040            | $^{**}$      | 0.046            | $^{***}$ | 0.053            | $^{***}$ |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.018           | )            | (0.016           | )            | (0.015           | )            | (0.016           | )        | (0.016           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| Log hourly wage                         |              | $-$0.029 | $^{**}$      | $-$0.037 | $^{**}$      | $-$0.026 | $^{**}$      | $-$0.013 |          | 0.007            |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
|                                         |              | (0.014           | )            | (0.015           | )            | (0.012           | )            | (0.015           | )        | (0.010           | )        |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+
| $N$ (# in RAIS)                         |              | 326,827          |              | 326,827          |              | 326,827          |              | 326,827          |          | 326,703          |          |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+------------------+--------------+------------------+----------+------------------+----------+---+---+---+---+---+---+---+---+

: Effects of ENEM adoption on the correlation of average ENEM scores and outcomes controlling for demographics

</div>

</div>

*Notes*: This table present estimates of the impacts of ENEM adoption on the correlation between average ENEM scores and student outcomes, controlling for demographic characteristics.

The table shows $\beta$ coefficients from equation dd_styr estimated on the analysis sample. Column (A) reproduces the coefficients from column (B) of Table 5, where the dependent variable, $Y_{st}$, is the raw correlation coefficient between average (core subjects) ENEM scores and the longer-run outcomes of individuals who attended high school in state $s$ and took the ENEM in year $t$. In columns (B)–(E), the dependent variables are state $\times$ year correlation coefficients between ENEM scores and outcomes after residualizing all variables on demographic controls. These columns incrementally include four sets of demographic controls: a dummy for private high schools (column B); dummies for family income categories, mother's education, father's education, and race (column C); a gender dummy and a linear term in age (column D); and high school dummies (column E). Each column includes the new controls plus all controls from the preceding columns.

Parentheses contain standard errors clustered at the state level. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:dd_het_subjs">

+:-----------------------------------+:--------:+---------:+-------------------------:+:---------+----------:+-------------------------:+:--------:+
| \(A\)                              | \(B\)    | \(C\)    | \(D\)                    |          | \(E\)     | \(F\)                    |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
|                                    |          | Outcome: Proportion of correct answers                                                           |
+------------------------------------+----------+-------------------------------------+----------+--------------------------------------+----------+
| 3-8                                |          | Public students                     |          | Private/public gap                   |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| 3-4 Question group                 | $N_q$    | Mean     | $\beta$ (SE)             |          | Mean      | $\beta^\text{gap}$ (SE)  |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
|                                    |          |          |                          |          |           |                          |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Communication (1–4)               | 54       | 0.468    | $-$0.003 (0.009) |          | 0.193     | 0.019 (0.009)            | \*\*     |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Foreign language (5–8)            | 80       | 0.424    | $-$0.009 (0.013) |          | 0.234     | $-$0.005 (0.008) |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Body language (9–11)              | 29       | 0.521    | $-$0.018 (0.010) | \*       | 0.139     | 0.016 (0.013)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Art (12–14)                       | 42       | 0.425    | 0.005 (0.012)            |          | 0.188     | 0.019 (0.007)            | \*\*     |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Literary text (15–17)             | 63       | 0.342    | $-$0.004 (0.006) |          | 0.157     | 0.022 (0.006)            | \*\*\*   |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Linguistics (18–20)               | 41       | 0.449    | $-$0.003 (0.015) |          | 0.193     | 0.020 (0.012)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Argumentation (21–24)             | 63       | 0.450    | 0.000 (0.005)            |          | 0.171     | 0.013 (0.006)            | \*\*     |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Portuguese (25–27)                | 39       | 0.410    | $-$0.011 (0.008) |          | 0.205     | 0.017 (0.010)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Social communication (28–30)      | 33       | 0.455    | 0.001 (0.011)            |          | 0.186     | 0.024 (0.007)            | \*\*\*   |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| All coefficients equal ($p$ value) |          |          | 0.034                    |          |           | 0.021                    |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
|                                    |          |          |                          |          |           |                          |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| **Panel B. Natural science**                                                                               |                          |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Human constructions (1–4)         | 57       | 0.274    | 0.002 (0.009)            |          | 0.166     | 0.013 (0.006)            | \*\*     |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Technology (5–7)                  | 34       | 0.255    | 0.005 (0.005)            |          | 0.102     | 0.010 (0.004)            | \*\*     |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Environmental conservation (8–12) | 58       | 0.360    | 0.005 (0.006)            |          | 0.178     | 0.004 (0.005)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Ecosystems (13–16)                | 54       | 0.316    | $-$0.004 (0.008) |          | 0.204     | 0.018 (0.011)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Scientific methods (17–19)        | 49       | 0.297    | $-$0.006 (0.010) |          | 0.189     | 0.006 (0.008)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Physics (20–23)                   | 57       | 0.269    | $-$0.010 (0.008) |          | 0.158     | 0.009 (0.004)            | \*       |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Chemistry (24–27)                 | 65       | 0.246    | $-$0.001 (0.002) |          | 0.127     | 0.006 (0.007)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Biology (28–30)                   | 31       | 0.416    | $-$0.012 (0.014) |          | 0.192     | 0.018 (0.014)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| All coefficients equal ($p$ value) |          |          | 0.023                    |          |           | 0.045                    |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
|                                    |          |          |                          |          |           |                          |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| **Panel C. Social science**                                                                                |                          |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Culture (1–5)                     | 73       | 0.385    | $-$0.011 (0.006) | \*       | 0.184     | 0.022 (0.007)            | \*\*\*   |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Geography (6–10)                  | 66       | 0.353    | 0.004 (0.006)            |          | 0.214     | 0.006 (0.006)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Social institutions (11–15)       | 74       | 0.376    | $-$0.006 (0.008) |          | 0.173     | 0.021 (0.005)            | \*\*\*   |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Technology (16–20)                | 61       | 0.376    | $-$0.010 (0.005) | \*       | 0.186     | 0.009 (0.007)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Citizenship (21–25)               | 62       | 0.432    | 0.002 (0.007)            |          | 0.192     | 0.015 (0.009)            |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| Society and nature (26–30)        | 69       | 0.405    | $-$0.001 (0.008) |          | 0.168     | 0.013 (0.007)            | \*       |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+
| All coefficients equal ($p$ value) |          |          | 0.000                    |          |           | 0.000                    |          |
+------------------------------------+----------+----------+--------------------------+----------+-----------+--------------------------+----------+

: Effects of ENEM adoption on exam performance by topic area in language arts, natural science, and social science

</div>

</div>

*Notes*: This table presents estimates of the impacts of ENEM adoption on performance on different topic areas of the language arts (Panel A), natural science (Panel B), and social science (Panel C) tests.

This table is analogous to the math test results in Panel B of Table 6. The sample includes 2009–2017 ENEM exam takers in our analysis sample (column C of Table 1). Regressions are at the high school ($h$) $\times$ year ($t$) $\times$ exam question ($q$) level. The dependent variable is the proportion of correct answers in each $htq$ cell. Each row displays results from a separate regression for questions in the same topic area, as defined by ENEM test designers.

Column (A) specifies the group of questions contained in each regression. Column (B) indicates the number of questions in each group. Column (C) shows the mean proportion of correct answers for public school students in cohorts prior to each state's ENEM adoption year (i.e., cohorts with $\text{HighStakes}_{st}=0$). Column (E) shows the mean private/public gap in the proportion of correct answers in those cohorts. Columns (D) and (F) display the $\beta$ and $\beta^{\text{gap}}$ coefficients from equation dd_gap estimated for each group of questions. The last row of each panel reports $p$ values from $F$ tests that the topic area coefficients in columns (D) or (F) are jointly equal.

Parentheses contain standard errors clustered at the state level. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:return_beta">

+:----------------------------------------+:-------------+-----------------:+:-------------+-----------------:+:-------------+-------------:+:-------------+-------------:+:---------+-----------------:+:---------+---------:+:--------+--:+:--+--:+:--+--:+:--+
|                                         |              | \(A\)            |              | \(B\)            |              | \(C\)        |              | \(D\)        |          | \(E\)            |          | \(F\)    |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              | Mean inform-     |              | Change in return to a correct answer                                                                                             |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+----------------------------------------------------------------------------------------------------------------------------------+---------+---+---+---+---+---+---+
|                                         |              | ativeness        |              | from 1 SD increase in $\beta^\text{gap}$                                                                                         |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| 3-3 Dependent variable:                 |              | All              |              |                  |              | Lang.        |              | Nat.         |          | Soc.             |          | All      |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Informativeness for\...                 |              | subjs            |              | Math             |              | arts         |              | science      |          | science          |          | subjs    |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              |                  |              |              |              |              |          |                  |          |          |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| **Panel A. Outcomes for all exam takers**                                                                                                                               |          |                  |          |          |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Enrolled in any college by 2019         |              | 0.101            |              | 0.015            | $^{**}$      | 0.012        | $^{**}$      | 0.000        |          | $-$0.000 |          | 0.006    | $^{**}$ |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.007           | )            | (0.005       | )            | (0.006       | )        | (0.003           | )        | (0.003   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Finished college within 5 years of ENEM |              | 0.029            |              | 0.004            | $^{*}$       | 0.005        | $^{***}$     | 0.001        |          | 0.000            |          | 0.002    |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.002           | )            | (0.002       | )            | (0.002       | )        | (0.001           | )        | (0.001   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Earned a college degree by 2019         |              | 0.071            |              | 0.011            | $^{**}$      | 0.010        | $^{***}$     | 0.002        |          | $-$0.001 |          | 0.004    | $^{*}$  |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.005           | )            | (0.003       | )            | (0.005       | )        | (0.002           | )        | (0.002   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Appears in RAIS in 2016–2018           |              | $-$0.016 |              | $-$0.000 |              | 0.008        | $^{***}$     | 0.000        |          | $-$0.001 |          | 0.002    | $^{*}$  |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.002           | )            | (0.002       | )            | (0.001       | )        | (0.001           | )        | (0.001   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| $N$ (# competencies)                    |              | 120              |              | 30               |              | 30           |              | 30           |          | 30               |          | 120      |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              |                  |              |              |              |              |          |                  |          |          |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| **Panel B. Outcomes for college enrollees**                                                                                                                             |          |                  |          |          |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Persisted in college for 1 year         |              | 0.012            |              | 0.002            | $^{**}$      | 0.001        |              | 0.000        |          | $-$0.000 |          | 0.001    | $^{*}$  |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.001           | )            | (0.001       | )            | (0.001       | )        | (0.000           | )        | (0.000   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Persisted in college for 3 years        |              | 0.047            |              | 0.007            | $^{***}$     | 0.003        |              | 0.000        |          | $-$0.001 |          | 0.002    | $^{**}$ |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.002           | )            | (0.002       | )            | (0.002       | )        | (0.001           | )        | (0.001   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Completed program within 5 years        |              | 0.023            |              | 0.003            |              | 0.005        | $^{***}$     | 0.001        |          | 0.001            |          | 0.002    |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.002           | )            | (0.002       | )            | (0.002       | )        | (0.001           | )        | (0.001   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Fraction of college credits completed   |              | 0.048            |              | 0.007            | $^{***}$     | 0.006        | $^{***}$     | 0.001        |          | $-$0.001 |          | 0.003    | $^{**}$ |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.003           | )            | (0.002       | )            | (0.002       | )        | (0.001           | )        | (0.001   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| $N$ (# competencies)                    |              | 120              |              | 30               |              | 30           |              | 30           |          | 30               |          | 120      |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              |                  |              |              |              |              |          |                  |          |          |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| **Panel C. Outcome for individuals in RAIS**                                                                                                                            |          |                  |          |          |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Hourly wage (BRL)                       |              | 13.285           |              | 3.052            | $^{***}$     | 1.766        | $^{***}$     | 0.039        |          | $-$0.893 |          | 0.831    | $^{**}$ |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.726           | )            | (0.604       | )            | (0.730       | )        | (0.538           | )        | (0.413   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| Log hourly wage                         |              | 0.153            |              | 0.034            | $^{***}$     | 0.021        | $^{***}$     | 0.002        |          | $-$0.008 |          | 0.011    | $^{**}$ |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
|                                         |              |                  |              | (0.008           | )            | (0.007       | )            | (0.008       | )        | (0.006           | )        | (0.004   | )       |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+
| $N$ (# competencies)                    |              | 120              |              | 30               |              | 30           |              | 30           |          | 30               |          | 120      |         |   |   |   |   |   |   |
+-----------------------------------------+--------------+------------------+--------------+------------------+--------------+--------------+--------------+--------------+----------+------------------+----------+----------+---------+---+---+---+---+---+---+

: The relationship between skill informativeness for longer-run outcomes and the impact of ENEM stakes on private/public skill gap

</div>

</div>

*Notes*: This table presents OLS estimates of the relationship between the informativeness of ENEM competencies for longer-run outcomes and the change in the private/public gap in performance on the higher-stakes exam.

We measure the informativeness of each exam competency for a given outcome as the mean difference in a given outcome between individuals who got each question correct and incorrect using 2009–2014 (excluding 2011) ENEM participants in our analysis sample, and then average these differences across all questions in the same competency. For outcomes in Panel C, the sample excludes 2013–2014 ENEM participants. To measure the effect of ENEM stakes on the private/public gap, $\beta^{\text{gap}}$, we estimate equation dd_gap separately for groups of ENEM questions in each competency (as in Table 6, Panel C, column F).These coefficients represent the change in the private/public gap in the proportion of correct answers resulting from an increase in ENEM stakes.

Column (A) shows the mean informativeness averaged across all subjects and competencies. Columns (B)–(F) show OLS coefficients from bivariate regressions of informativeness on the $\beta^{\text{gap}}$ coefficients, estimated by subject and pooling across all subjects. We normalize the OLS coefficients to represent a 1 SD increase in $\beta^{\text{gap}}$.

Parentheses contain robust standard errors. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:return_beta_diff">

+:----------------------------------------+:-------------+------------------:+:-------------+-------------:+:-------------+-------------:+:-------------+-------------:+:--------+--:+:--+--:+:--+--:+:--+--:+:--+--:+:--+
|                                         |              | \(A\)             |              | \(B\)        |              | \(C\)        |              | \(D\)        |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | Change in return to a correct answer                                     |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------------------------------------------------------------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | from 1 SD increase in $\beta^\text{gap}$                                 |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| 5-9 Dependent variable:                 |              | Mean              |              | Bivariate    |              | Controls for |              | IRT          |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Informativeness for\...                 |              | informativeness   |              | regression   |              | \% correct   |              | controls     |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              |              |              |              |              |              |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| **Panel A. Outcomes for all exam takers**                                                                                                                            |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Enrolled in any college by 2019         |              | 0.1011            |              | 0.0063       | $^{**}$      | 0.0011       |              | 0.0028       |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.0031      | )            | (0.0010      | )            | (0.0020      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Finished college within 5 years of ENEM |              | 0.0294            |              | 0.0020       |              | 0.0009       | $^{**}$      | 0.0011       | $^{*}$  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.0013      | )            | (0.0004      | )            | (0.0006      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Earned a college degree by 2019         |              | 0.0707            |              | 0.0045       | $^{*}$       | 0.0026       | $^{**}$      | 0.0029       | $^{*}$  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.0023      | )            | (0.0011      | )            | (0.0015      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Appears in RAIS in 2016–2018           |              | $-$0.0162 |              | 0.0018       | $^{*}$       | 0.0017       |              | 0.0001       |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.0011      | )            | (0.0010      | )            | (0.0010      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| $N$ (# competencies)                    |              | 120               |              | 120          |              | 120          |              | 120          |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              |              |              |              |              |              |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| **Panel B. Outcomes for college enrollees**                                                                                                                          |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Persisted in college for 1 year         |              | 0.0123            |              | 0.0007       | $^{*}$       | 0.0000       |              | 0.0003       |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.0003      | )            | (0.0001      | )            | (0.0003      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Persisted in college for 3 years        |              | 0.0472            |              | 0.0023       | $^{**}$      | 0.0006       |              | 0.0014       | $^{*}$  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.0011      | )            | (0.0004      | )            | (0.0008      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Completed program within 5 years        |              | 0.0225            |              | 0.0017       |              | 0.0012       | $^{**}$      | 0.0013       | $^{*}$  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.0012      | )            | (0.0005      | )            | (0.0008      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Fraction of college credits completed   |              | 0.0481            |              | 0.0027       | $^{**}$      | 0.0017       | $^{***}$     | 0.0019       | $^{**}$ |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.0012      | )            | (0.0006      | )            | (0.0009      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| $N$ (# competencies)                    |              | 120               |              | 120          |              | 120          |              | 120          |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              |              |              |              |              |              |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| **Panel C. Outcome for individuals in RAIS**                                                                                                                         |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Hourly wage (BRL)                       |              | 13.2852           |              | 0.8314       | $^{**}$      | 0.8954       | $^{**}$      | 0.7313       | $^{*}$  |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.4131      | )            | (0.4140      | )            | (0.4185      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| Log hourly wage                         |              | 0.1534            |              | 0.0110       | $^{**}$      | 0.0098       | $^{**}$      | 0.0085       | $^{**}$ |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
|                                         |              |                   |              | (0.0043      | )            | (0.0040      | )            | (0.0040      | )       |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+
| $N$ (# competencies)                    |              | 120               |              | 120          |              | 120          |              | 120          |         |   |   |   |   |   |   |   |   |   |   |
+-----------------------------------------+--------------+-------------------+--------------+--------------+--------------+--------------+--------------+--------------+---------+---+---+---+---+---+---+---+---+---+---+

: The relationship between skill informativeness for longer-run outcomes and the impact of ENEM stakes on private/public skill gap, controlling for question characteristics

</div>

</div>

*Notes*: This table presents OLS estimates of the relationship between the informativeness of ENEM competencies for longer-run outcomes and the change in the private/public performance gap on the higher-stakes exam, controlling for question characteristics.

This table is analogous to Table 20, but includes specifications that control for question characteristics. Column (B) reproduces the OLS estimates shown in column (F) of Table 20. Column (C) adds a quadratic term for the mean proportion of correct answers in each competency, interacted with subject dummies. Column (D) includes a quadratic term for the mean IRT parameters in each competency, interacted with subject dummies. We normalize the OLS coefficients to represent a 1 SD increase in $\beta^{\text{gap}}$.

Parentheses contain robust standard errors. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

<div class="center">

<div id="tab:dd_het_words">

+:--------------------------------------+:---------:+----------:+-------------------------:+:----------+----------:+------------------------:+:--------:+
| \(A\)                                 | \(B\)     | \(C\)     | \(D\)                    |           | \(E\)     | \(F\)                   |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
|                                       |           | Outcome: Proportion of correct answers                                                            |
+---------------------------------------+-----------+--------------------------------------+-----------+-------------------------------------+----------+
| 3-8                                   |           | Public students                      |           | Private/public gap                  |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| 3-4 Question group                    | $N_q$     | Mean      | $\beta$ (SE)             |           | Mean      | $\beta^\text{gap}$ (SE) |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
|                                       |           |           |                          |           |           |                         |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Not in study guide                    | 262       | 0.306     | $-$0.007 (0.011) |           | 0.186     | 0.026 (0.010)           | \*\*     |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| In study guide                        | 143       | 0.267     | $-$0.003 (0.008) |           | 0.161     | 0.021 (0.008)           | \*\*     |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| cone (*cone*)                         | 8         | 0.471     | $-$0.010 (0.026) |           | 0.210     | 0.024 (0.030)           |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| cube (*cubo*)                         | 8         | 0.179     | $-$0.000 (0.008) |           | 0.182     | 0.029 (0.010)           | \*\*\*   |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| cylinder (*cilindro*)                 | 10        | 0.374     | $-$0.011 (0.013) |           | 0.157     | 0.021 (0.022)           |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| directly (*diretamente*)              | 8         | 0.284     | 0.009 (0.011)            |           | 0.218     | 0.030 (0.010)           | \*\*\*   |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| median (*mediana*)                    | 12        | 0.226     | 0.009 (0.010)            |           | 0.130     | 0.055 (0.021)           | \*\*     |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| parallelepiped (*paralelepípedo*)     | 9         | 0.314     | 0.001 (0.034)            |           | 0.342     | 0.043 (0.028)           |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| possibilities (*possibilidade*)       | 8         | 0.275     | $-$0.015 (0.006) | \*\*      | 0.126     | 0.021 (0.017)           |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| possible (*possíveis*)                | 10        | 0.310     | $-$0.015 (0.023) |           | 0.168     | 0.011 (0.022)           |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| prism (*prisma*)                      | 8         | 0.368     | $-$0.002 (0.015) |           | 0.136     | 0.014 (0.025)           |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| probability (*probabilidade*)         | 25        | 0.234     | $-$0.010 (0.008) |           | 0.131     | 0.028 (0.010)           | \*\*     |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| pyramid (*pirâmide*)                  | 10        | 0.337     | $-$0.035 (0.018) | \*        | 0.138     | 0.042 (0.017)           | \*\*     |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| rectangle (*retângulo*)               | 10        | 0.277     | 0.006 (0.011)            |           | 0.192     | 0.024 (0.014)           | \*       |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| square (*quadrado*)                   | 35        | 0.256     | $-$0.001 (0.005) |           | 0.197     | 0.009 (0.006)           |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| triangle (*triângulo*)                | 8         | 0.231     | $-$0.002 (0.006) |           | 0.120     | 0.038 (0.010)           | \*\*\*   |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Fewer than 8 occurrences              | 40        | 0.268     | $-$0.005 (0.007) |           | 0.146     | 0.016 (0.008)           | \*       |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| In vs. not in study guide ($p$ value) |           |           | 0.310                    |           |           | 0.157                   |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
|                                       |           |           |                          |           |           |                         |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| **Panel B. Study guide concepts matched to solutions**                                                           |                         |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Not in study guide                    | 231       | 0.317     | $-$0.005 (0.010) |           | 0.193     | 0.023 (0.011)           | \*\*     |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| In study guide                        | 173       | 0.259     | $-$0.004 (0.009) |           | 0.155     | 0.025 (0.008)           | \*\*\*   |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Geometric formulas                    | 64        | 0.261     | $-$0.004 (0.009) |           | 0.174     | 0.026 (0.007)           | \*\*\*   |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Proportions (''Rule of 3'')           | 12        | 0.330     | $-$0.011 (0.018) |           | 0.205     | 0.058 (0.010)           | \*\*\*   |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Manipulating fractions                | 58        | 0.248     | $-$0.004 (0.008) |           | 0.161     | 0.029 (0.009)           | \*\*\*   |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Radicals                              | 13        | 0.195     | 0.009 (0.006)            | \*        | 0.139     | 0.024 (0.014)           | \*       |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Combinatory and statistical analysis  | 25        | 0.224     | $-$0.003 (0.004) |           | 0.103     | 0.024 (0.015)           |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Probability                           | 36        | 0.255     | $-$0.010 (0.010) |           | 0.138     | 0.025 (0.007)           | \*\*\*   |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| Trigonometric formulas                | 8         | 0.231     | 0.006 (0.006)            |           | 0.104     | 0.017 (0.007)           | \*\*     |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+
| In vs. not in study guide ($p$ value) |           |           | 0.791                    |           |           | 0.591                   |          |
+---------------------------------------+-----------+-----------+--------------------------+-----------+-----------+-------------------------+----------+

: Heterogeneity by topics in the *Me Salva!* study guide (math test)

</div>

</div>

*Notes*: This table presents estimates of the impact of ENEM adoption on student performance on math questions covered in a study guide by the test prep company *Me Salva!*. The sample includes 2009–2017 ENEM exam takers in our analysis sample (column C of Table 1). Regressions are at the high school ($h$) $\times$ year ($t$) $\times$ exam question ($q$) level. The dependent variable is the proportion of correct answers in each $htq$ cell. Panel A groups question based on whether the question text contains keywords from the *Me Salva!* study guide, restricting to words that appear in eight or more questions. Panel B groups questions based on whether the solution text (prepared by another test prep company, *Descomplica*) requires concepts from the *Me Salva!* study guide. See Appendix 9.4 for details on these data sources and our match between *Me Salva!* topics and ENEM questions.

Column (A) specifies the group of questions for each regression. Column (B) indicates the number of questions in each group. Column (C) shows the mean proportion of correct answers for public school students in cohorts prior to each state's ENEM adoption year (i.e., cohorts with $\text{HighStakes}_{st}=0$). Column (E) shows the mean private/public gap in the proportion of correct answers in those cohorts. Columns (D) and (F) display the $\beta$ and $\beta^{\text{gap}}$ coefficients from equation dd_gap estimated for each group of questions. The last row of each panel reports $p$ values from an $F$ test that the coefficients in the first and second rows are equal.

Parentheses contain standard errors clustered at the state level. \* $p<0.10$, \*\* $p<0.05$, \*\*\* $p<0.01$.

# Theoretical appendix

## Impacts of exam stakes on test score gaps and informativeness
This section presents a simple framework that illustrates how exam stakes can impact inequality in test scores and their informativeness for longer-run outcomes. This framework helps to interpret our main empirical results and shed light on the potential mechanisms.

### Model setup

We consider a population of exam takers that are characterized by their socioeconomic status (SES) and their abilities. Let $X_i$ denote an observable measure of individual $i$'s SES, such as an indicator for attending a private high school. We let $a_i$ denote individual $i$'s ability to perform well on a college admission exam, which is not directly observable in data. We refer to $a_i$ as *test ability* to emphasize that it may be distinct from other abilities that help the individual perform well in college and the labor market.

We assume that the stakes of the college admission exam vary across cohorts, and that test takers in high-stakes cohorts engage in more test prep. For simplicity, we assume that individuals are randomly assigned to either a low-stakes or a high-stakes exam cohort, and we let $H_i$ be a binary indicator for the high-stakes cohort.[^31] Individuals have a stronger incentive to perform well in the high-stakes exam cohort, and so we assume that these individuals engage in additional test prep. This additional prep increases their test ability by an amount that we denote by $e_i$. We interpret $e_i$ broadly; for example, it may include studying test prep books, taking preparatory courses, focusing more intently during the school year, or exerting more effort on the exam.

A test taker's preparedness for the exam, as a function of their cohort, is given by: $$\begin{equation}
\theta_i=a_i+H_ie_i.\end{equation}$$ where we refer to $\theta_i$ as *test skill*. Individuals in low-stakes cohorts have test skill $\theta_i=a_i$, while individuals in high-stakes cohorts have test skill $\theta_i=a_i+e_i$.

An individual's score on the college admission exam is a noisy measure of their test skill. We let $T_i$ denote individual $i$'s test score, which is observable in the data. We assume test scores are given by: $$\begin{equation}
T_i=\theta_i+\epsilon^T_i,\end{equation}$$ where $\epsilon^T_i$ is random noise that reflects variation in test performance due to factors like guessing and health on exam day.

We let $Y_i$ denote an observable measure of college success, such as persisting in college after enrolling or completing a college degree. We model $Y_i$ as a function of test ability, $a_i$, test prep, $e_i$, and SES, $X_i$: $$\begin{equation}
Y_i=\alpha a_i + \beta e_i + \gamma X_i + v_i + \epsilon^Y_i. \end{equation}$$

The parameter $\alpha$ represents the effect of test ability on college success; it is natural to assume $\alpha>0$ since individuals with high test scores tend to perform better in college. Similarly, an individual's SES may help them succeed in college above and beyond their test ability, and so we assume $\gamma>0$. The parameter $\beta$ allows for the possibility that test prep directly affects college performance, although, as we discuss below, the sign of this parameter is less clear. In addition to these three factors, we allow college success to depend on other abilities that are unrelated to test skill, which we denote by $v_i$, and a random noise term, $\epsilon^Y_i$.

### Effects of exam stakes on test score gaps

In Section 4, we find that increasing the stakes of a college admission exam increases test score gaps between high- and low-SES students. If $X_i$ is a binary indicator for high SES, then this result can be written as: $$\begin{equation}
\mathrm{E}[T_i | X_i = 1, H_i = 1] - \mathrm{E}[T_i | X_i = 0, H_i = 1] > \mathrm{E}[T_i | X_i = 1, H_i = 0] - \mathrm{E}[T_i | X_i = 0, H_i = 0]. \end{equation}$$ In other words, the mean difference in test scores, $T_i$, between high- and low-SES exam takers is larger in the high-stakes cohort ($H_i=1$) than in the low-stakes cohort ($H_i=0$).

Plugging in equations test_skill–score and using the factor that $H_i$ is independent of $a_i$, expression score_effect becomes: $$\begin{equation}
\mathrm{E}[e_i | X_i = 1] - \mathrm{E}[e_i | X_i = 0] > 0.\end{equation}$$ Expression prep_effect states that higher stakes exams lead to larger socioeconomic test score gaps increase if high SES students engage in more test prep than low SES students.

### Effects of exam stakes on informativeness

In Section 5, we find that high-stakes exam scores are more informative for various measures of college success than lower-stakes exam scores. Our measure of informativeness is the correlation between exam scores, $T_i$, and a college outcome, $Y_i$, or: $$\begin{equation}
\mathrm{Corr}(T_i,Y_i) = \frac{\mathrm{Cov}(T_i,Y_i)}{\sigma_T\sigma_Y},
\end{equation}$$ where $\sigma_T$ and $\sigma_Y$ are the standard deviations of test scores and outcomes. Ignoring these standard deviations for simplicity, our main result can be written as: $$\begin{equation}
\mathrm{Cov}(T_i,Y_i|H_i=1) > \mathrm{Cov}(T_i,Y_i|H_i=0).\end{equation}$$ Using equations test_skill–success, expression validity_effect simplifies to: $$\begin{equation}
\gamma \mathrm{Cov}(X_i,e_i)+(\alpha+\beta)\mathrm{Cov}(a_i,e_i) + \mathrm{Cov}(v_i,e_i) + \beta\text{var}(e_i) > 0.\end{equation}$$

Expression validity_mechanisms shows that there are two broad channels through which additional incentives for test prep could increase the predictive validity of exam scores:

1.  **Correlational channels:** $\gamma \mathrm{Cov}(X_i,e_i) + (\alpha+\beta)\mathrm{Cov}(a_i,e_i) + \mathrm{Cov}(v_i,e_i) > 0$.

    First, high-stakes exam scores may be more predictive of college outcomes because they are more correlated with predetermined characteristics that also benefit students in college. These predetermined characteristics could include SES, $X_i$, test ability, $a_i$, or other unobserved abilities that impact college success, $v_i$. If the amount of test prep, $e_i$, is sufficiently correlated with any of these characteristics, this would lead test scores to be more informative for college outcomes.

    Our results in Section 4 suggest that $\mathrm{Cov}(X_i,e_i)>0$, so all else equal, one would expect a correlation between test prep and SES to increase the informativeness of test scores. For example, wealthy students may have greater access to test prep services, and family wealth also helps students succeed in college.

    The signs of the covariances between test prep, $e_i$, and the ability terms, $a_i$ and $v_i$, are *a priori* less clear. On the one hand, students who know that they are likely to succeed in college may have the strongest incentives to prep for the higher-stakes exam, which would imply positive covariances between $e_i$ and $a_i$ or $v_i$. On the other hand, students with low test ability may have the greatest benefits from engaging in test prep, which would imply negative covariances. Thus we do not have a strong prior on the signs of the covariances between test prep and abilities.

2.  **Skill accumulation channel:** $\beta\text{var}(e_i) > 0$.

    A second potential mechanism is that test prep may be directly beneficial for college success. This channel is operative if the test skills that individuals accumulate from prepping, $e_i$, directly improve their college outcomes, $Y_i$. In our framework, this skill accumulation channel exists if $\beta>0$. Brazil's ENEM exam was designed to be closely aligned with high school and college curricula, so test designers certainly hoped that students would learn useful material through their exam preparation. On the other hand, high-stakes exams are often criticized for creating incentives to learn test-taking strategies that are not useful outside the exam. This perverse incentive could imply that $\beta=0$ or even $\beta<0$ if test prep crowds out other useful learning.[^32]

Distinguishing between the correlational and skill accumulation channels is challenging because of unobserved abilities that impact both test scores and college success ($a_i$ and $v_i$). Figure 6 presents evidence that the increase in informativeness is likely to be partly driven by a correlational channel, as the correlation between test scores and outcomes decreases when we control for demographic characteristics. Yet we also find that the increase in exam stakes caused students to perform better across a wide range of exam skills that are related to high school and college curricula (Table 6). While this result may be due to either skill accumulation or a correlation with unobserved abilities, it shows that higher-stakes exams help colleges identify students who are more academically prepared for their programs.

## Relationship between correlation coefficients and Blackwell informativeness
In our empirical analysis, we measure the informativeness of test scores using the correlation coefficient between student outcomes and test scores, which is different from the theoretical concept of Blackwell informativeness used in Frankel and Kartik (2019). This subsection explores the relationship between these two measures of informativeness. Specifically, we prove that with binary states and signals, an increase in a signal's Blackwell informativeness leads to a higher correlation between states and signal realizations.

Suppose that both the state $\Theta$ and the signal $S$ take one of two values, high or low, coded as $1$ and $0$. We use $\theta_1$ and $\theta_0$ interchangeably with $\Theta=1$ and $\Theta=0$, and similarly for $s_1$ and $s_0$. The signal realizations are generated according to the following conditional probabilities:

<div class="minipage">

$\setlength{\extrarowheight}{2pt}
    \begin{array}{rr|c c}
        &  \multicolumn{1}{c}{}      & \multicolumn{2}{c}{\textup{Signal $S$}} \\
        &       & 0 & 1 \ \cline{2-4}
        \multirow{2}{*}{\textup{State $\Theta$}} & 0             &  p_0 & 1-p_0 \ 
         & 1            & 1-p_1 &  p_1 \ 
    \end{array}$

</div>

Table appendix:table:signal shows $\Pr(s\mid\theta)$, the conditional probability of observing a signal $s$ given a state $\theta$. A signal is completely characterized by $p_0$ and $p_1$. We require that the signal realizations be appropriately ordered: $p_1 > 1-p_0$ and $p_0 > 1-p_1$, meaning the high signal realization occurs more frequently when the state is high compared to when the state is low, and vice versa for the low signal. This is likely true in the context of test scores as signals for abilities.

Following Frankel and Kartik (2019), we say that signal $S$ is (Blackwell) more informative than signal $S'$ if and only if for any prior, the distribution of posterior beliefs after observing $S$ is a mean-preserving spread of the distribution of posterior beliefs after observing $S'$. In the context of binary signals, it is equivalent to saying that the interval between the two posteriors after observing the high and low realizations under $S$ contains the interval between the two posteriors under $S'$.

We are interested in the relationship between Blackwell informativeness of a signal and the linear correlation between the signal realizations and the states. We show that when a signal becomes Blackwell more informative, the linear correlation increases. The following proposition formalizes this statement,

<div id="prop:blackwell" class="prop">

**1**. *Consider a binary state $\Theta$ and two binary signals $S$ and $S'$. If $S$ is more informative Blackwell than $S'$, then $$\begin{align*}
    \mathrm{Corr}(\Theta,S) > \mathrm{Corr}(\Theta,S').
\end{align*}$$*

</div>

To prove Proposition 1, we express the correlation in terms of quantities that directly relate to a signal's Blackwell informativeness. We first note that the correlation coefficient can be written as

$$\begin{align}
    \mathrm{Corr}(\Theta,S) &= \frac{\mathrm{Cov}(\Theta,S)}{\sqrt{\mathrm{Var}(\Theta)\mathrm{Var}(S)}} \ &= \frac{\sum_i(\theta_i-\mathrm{E}(\Theta))(s_i-\mathrm{E}(S))}{\sqrt{\sum_i(\theta_i-\mathrm{E}(\Theta))^2(s_i-\mathrm{E}(S))^2}}.
\end{align}$$

For any prior $\mu = \Pr(\theta = 1) = \Pr(\theta_1)$ and any binary signal $S$, denote the posterior after observing $s_0$ as $\mu_0 = \Pr(\theta_1 \mid s_0)$ and the posterior after observing $s_1$ as $\mu_1 =\Pr(\theta_1 \mid s_1)$.

We want to express $p_0$ and $p_1$ in terms of $\mu$, $\mu_0$, and $\mu_1$. Using Bayes' theorem:

$$\begin{align}
    p_1 &= \Pr(s_1\mid \theta_1) 
    &= \frac{\Pr(\theta_1\mid s_1) \Pr(s_1)}{\Pr(\theta_1)}  
    &= \frac{\mu_1 \big(p_1\mu + (1-p_0)(1-\mu)\big)}{\mu}.
    \end{align}$$

Similarly we can rewrite $p_0$:

$$\begin{align}
    p_0 &= \frac{\Pr(\theta_0\mid s_0) \Pr(s_0)}{\Pr(\theta_0)} 
    &= \frac{(1-\mu_0) (p_0(1-\mu) + (1-p_1)\mu)}{1-\mu}.
    \end{align}$$

Rewriting appendix:eq:p1 and appendix:eq:p0, we obtain:

$$\begin{align*}
    \mu_0(1-\mu)p_0 &= (1-\mu_0)\mu (1-p_1) \\
    (1-\mu_1)\mu p_1 &= \mu_1(1-\mu)(1-p_0),
\end{align*}$$

or equivalently,

$$\begin{align}
    p_0 =\frac{1-\mu_0}{\mu_0}\frac{\mu}{1-\mu}(1-p_1) \\
    p_1 =\frac{\mu_1}{1-\mu_1}\frac{1-\mu}{\mu}(1-p_0). \end{align}$$

Substituting appendix:eq:p1solved into appendix:eq:p0solved and simplifying, we solve for $p_0$ and $1-p_0$:

$$\begin{align}
    p_0 =\frac{1-\mu_0}{1-\mu}\frac{\mu_1-\mu}{\mu_1-\mu_0}, \quad 1-p_0 =  \frac{1-\mu_1}{1-\mu}\frac{\mu-\mu_0}{\mu_1-\mu_0}.
\end{align}$$

Similarly, we solve for $p_1$ and $1-p_1$:

$$\begin{align}
    p_1 =\frac{\mu_1}{\mu}\frac{\mu-\mu_0}{\mu_1-\mu_0}, \quad 1-p_1 = \frac{\mu_0}{\mu}\frac{\mu_1-\mu}{\mu_1-\mu_0}.
\end{align}$$

We can also express the variances and covariance in terms of $\mu$, $\mu_0$, and $\mu_1$. Since $\Theta$ is a Bernoulli random variable with success probability $\mu$, its variance is

$$\begin{align}
    \mathrm{Var}(\Theta) = \mu(1-\mu).
\end{align}$$

Signal $S$ is also a Bernoulli random variable but with success probability $\Pr(s_1) = p_1\mu + (1-p_0)(1-\mu)$. Therefore, its variance can be written as

$$\begin{align}
    \mathrm{Var}(S) &= [p_1\mu + (1-p_0)(1-\mu)][1-p_1\mu -(1-p_0)(1-\mu)] 

    &=\frac{(\mu_1-\mu)(\mu-\mu_0)}{(\mu_1-\mu_0)^2}.
\end{align}$$

The covariance between the state and the signal can be written as

$$\begin{align}
    \mathrm{Cov}(\Theta,S) &= \mathrm{E}[\Theta S] - \mathrm{E}[\Theta]\mathrm{E}[S] 
    &= \Pr(\Theta=1, S=1) - \Pr[\Theta=1]\Pr[S=1] 

    &= \frac{(\mu_1-\mu)(\mu-\mu_0)}{\mu_1-\mu_0}.
\end{align}$$

Now we are ready to derive the correlation coefficient: $$\begin{align}
 \mathrm{Corr}(\Theta,S) &= \frac{\mathrm{Cov}(\Theta,S)}{\sqrt{\mathrm{Var}(\Theta)\mathrm{Var}(S)}} 

    &=\frac{1}{\sqrt{\mu(1-\mu)}}\sqrt{(\mu_1-\mu)(\mu-\mu_0)}
\end{align}$$

Expression eq:corr_signal shows that the correlation coefficient can be represented in terms of the distance between the posteriors and the prior under the signal.

We can rewrite the correlation coefficient as a function of quantities directly related to Blackwell informativeness. Let $x \equiv \mu_1-\mu$ and $y \equiv \mu - \mu_0$. Then,

$$\begin{align*}
    \mathrm{Corr}(\Theta, S) = \frac{1}{\sqrt{\mu(1-\mu)}}\sqrt{xy}.
\end{align*}$$

For a fixed prior $\mu$, if signal $S$ is Blackwell more informative than signal $S'$, by Blackwell's theorem, $x$ and $y$ must be weakly greater than $x'$ and $y'$ (with at least one strict inequality). As a result, $\mathrm{Corr}(\Theta, S) > \mathrm{Corr}(\Theta, S')$. Therefore, we conclude that when a binary signal becomes Blackwell more informative about the state, the correlation between the state and the signal realizations will also increase.

We have proved this claim for the case of binary signals and binary states to illustrate our point. We conjecture that similar results should hold for multiple and continuous states and signals, provided the signal realizations and states are appropriately ordered. However, the proof of this general result is beyond the scope of this paper.

# Empirical appendix

## Variable definitions

This section provides details on the variables that we use in our paper. These variables come from the following datasets (see Section 3):

- Data on ENEM exam takers in the years 2007–2017 (INEP, 2019). This dataset contains student-level test scores and demographic variables collected from a survey that test-takers complete when they enroll to take the exam.

- INEP's Higher Education Census (*Censo da Educação Superior*) for the years 2010–2019 (INEP, 2022). This dataset contains information on a student's enrollment in higher education institutions, enrollment year, graduation year, credits completed in his/her program, and total credits required in the program curriculum.

- Brazil's employee-employer dataset, the RAIS (*Relação Anual de Informações Sociais*), for the years 2016–2018 (RAIS, 2022). This dataset contains information on an individual's employment, average monthly earnings, and average monthly contracted hours.

- INEP's School Census (*Censo Escolar*) for the years 2007–2017 (INEP, 2022). This dataset contains annual student-level data on enrollment and grade level in all Brazilian primary and secondary schools.

### Test scores (ENEM data)

- **Subject scores.** The post-2009 ENEM scores, as reported to the public, are scaled to have a mean of 500 and an SD of 100 in the population of 2009 high school seniors who took the exam. Throughout the paper, we report ENEM scores in SD units relative to this population. For ENEM scores in math, language arts, natural science, and social science, our transformation is:

  $$\text{Transformed subject score} = \frac{\text{Raw subject score} - 500}{100}.$$

  After the transformation, a score of zero in our paper is equivalent to the performance of the average high school senior who took the ENEM in 2009, and a score of one is 1 SD higher within this population. These transformations preserve the comparability of test scores across cohorts.

  The 2007–2008 ENEM reported only a single core component score plus a writing score. To define scores for each subject, we first categorize the multiple choice questions into math, language arts, natural science, and social science, and then compute a separate score for each subject using the IRT parameters.

  Since the reference populations differ for the 2007–2008 and 2009–2017 exams, in regressions where we use scores from both periods, we standardize the scores to have mean 0 and SD 1 within each year of our sample.

- **Average score (core subjects).** The post-2009 average scores are calculated by taking the average of the four subject scores, and then standardizing relative to the reference population. Our transformation is:

  $$\text{Transformed average score} = \frac{\text{Raw average score} - 500}{86.7},$$

  where 86.7 is the SD of the average score in the reference population. After transformation, a score of zero in our paper is equivalent to the performance of the average high school senior who took the ENEM in 2009, and a score of one is 1 SD higher within this population.

  For the 2007–2008 ENEM, the average score is defined as the core component score. In the regressions where we use scores from both the 2007–2008 and 2009–2017 exams, we standardize the scores to have mean 0 and SD 1 within each year of our sample.

- **Writing scores.** The post-2009 writing score is also standardized relative to the reference population. Our transformation is:

  $$\text{Transformed writing score} = \frac{\text{Raw writing score} - 597}{137},$$ where 137 is the SD of the writing score in the reference population.

  For the 2007–2008 ENEM, we use the writing score as included in the data. In regressions where we use scores from both the 2007–2008 and 2009–2017 exams, we standardize the writing scores to have mean 0 and SD 1 within each year of our sample.

### Exam-taker characteristics (ENEM data).

- **Private high school.** We define "private high schools" to include both private and federal high schools since their students are comparable in terms of socioeconomic status and achievement. Only 0.5 percent of all Brazilian students attended a federal high school in 2009 (see Appendix Table 7). "Public high schools" include both state and municipality schools.

- **Race.** In Brazil, race is commonly classified in five groups: *branco* (white), *pardo* (brown), *preto* (Black), *amarelo* (yellow), and indigenous. Since Asian and indigenous people represent a small proportion of the population in Brazil (less than three percent in our sample), we use indicators for three major racial groups: *branco* (white), *pardo* (brown), and *preto* (Black). We set the indicator variables to missing if the student declined to declare his/her racial identity (about two percent of the students in our sample).

- **Parental education.** Parental education is measured based on the highest educational completed by each parent, categorized into eight levels ranging from "none" to "post-graduate." Using these categorical variables, we define two indicator variables, "Mother completed college" and "Father completed college," which equal one if the respective parent has completed a "college" or "post-graduate" degree.

- **Family income.** Family income is measured in multiples of the minimum wage. We define an indicator variable "Family income \> 2x min. wage" (or "High-income"), which equals one if the reported income exceeds twice the minimum wage.

### College outcomes (*Censo da Educação Superior*)

- **Completed program within 5 years.** An indicator variable that equals one if the student completed their program in INEP's higher education census within 5 years of their enrollment year. This variable is defined for ENEM exam takers who appear in the higher education census using the first college program they enrolled in.

- **Earned a college degree by 2019.** An indicator variable that equals one if the student is completed any degree from an institution in INEP's higher education census by 2019. This variable is defined for all ENEM exam takers.

- **Enrolled in any college by 2019.** An indicator variable that equals one if the student is ever observed in the INEP's higher education census for the years 2010–2019. This variable is defined for all ENEM exam takers.

- **Finished college with 5 years of ENEM.** An indicator variable that equals one if the student is completed any degree from an institution in INEP's higher education census within 5 years of taking the ENEM exam. This variable is defined for all ENEM exam takers.

- **Fraction of college credits completed.** The number credits completed on the student's last record in INEP's higher census census divided by the total credits required to graduate from the program. This variable is defined for ENEM exam takers who appear in the higher education census using the first college program they enrolled in.

- **Persisted in college for 1 year (or 3 years).** An indicator variable that equals one if the student's last year on record in INEP's higher education census is greater than or equal to 1 year (or 3 years) after their enrollment year. This variable is defined for ENEM exam takers who appear in the higher education census using the first college program they enrolled in.

### Labor market outcomes (RAIS data)

- **Appears in RAIS in 2016–2018.** An indicator variable that equals one if a student appears in the 2016–2018 RAIS, indicating that the student has engaged in formal employment. This variable is defined for all ENEM exam takers.

- **Hourly wage (BRL).** Average monthly earnings divided by average monthly contracted hours computed using the 2016–2018 RAIS. If the individual is observed with earnings in multiple years in 2016–2018, we take the average across the multiple years. Wages are expressed in terms of nominal Brazilian Reals.

- **Log hourly wage.** The natural log of Hourly wage (BRL), computed as described above.

### High-school enrollment (ENEM data and *Censo Escolar*)

- **Log number of exam takers in school.** The natural log of the number of ENEM test-takers per school/cohort as measured in the ENEM data.

- **Log number of high school seniors in school.** The natural log of the number of enrolled senior students per school/cohort as measured in the *Censo Escolar*.

- **Number of schools attended in past 3 years.** The number of distinct high schools that a senior student attended in the three years before taking the ENEM exam as measured in the *Censo Escolar*. This variable is computed by appending the *Censo Escolar* data for the years 2007–2017 and counting the number of different high schools that each senior student attended in the three years preceding their senior year, using a unique student identifier across years.

## Data and merging
Our base dataset contains all individuals who took the ENEM exam from 2007 to 2017. This dataset includes student-level and question-level information. The student-level data includes self-reported demographic and socioeconomic status (SES) measures, such as sex, race, high-school type (public/private), parental education, and family income. The question-level data includes student responses to each exam question, the question subject, and the skill tested. From this dataset, we only keep high school seniors with a non-zero, non-missing score on each subject test. These restrictions exclude, for example, individuals who took the exam after graduating from high school or who missed one of the testing days.

To measure long-run outcomes, we combine the 2009–2014 ENEM records with two individual-level administrative datasets using individuals' national ID numbers (*Cadastro de Pessoas Físicas*).[^33] The linkage was conducted in the secured data room at the INEP facilities in Brasília, Brazil. We exclude students with missing national IDs (0.04 percent) and those who took the exam more than three times in our data (0.10 percent). We also exclude the 2011 cohort because of a data issue: the crosswalk variable that INEP created to match individuals across their different datasets is not correctly defined for this cohort.

We measure college outcomes using Brazil's higher-education census from 2010 to 2019. This dataset offers comprehensive information about all college enrollees, including their university of enrollment, major, the academic year when they enrolled, and their year of graduation. 65.6 percent of high-school seniors taking the ENEM during 2009–2014 appear in the census data.

We measure labor-market outcomes using an administrative employee-employer matched dataset called RAIS (*Relação Anual de Informações Sociais*) from 2016 to 2018. The RAIS contains data on workers employed in the formal sector. It does not include data on individuals working within the informal sector, those who are self-employed, or individuals who are currently unemployed. This dataset includes worker-level and firm-level information. The worker-level data includes educational attainment, occupation, and earnings. The firm-level data includes total employee count, the industry they operate within, and their geographical location.

## Sample definition
This section describes the criteria for high schools to be included in the INEP annual performance reports, which we use to define our main analysis sample.

As noted in the main text, our analysis sample consists of the set of high schools that were included in the annual ENEM performance reports created by INEP in *each* year in 2005–2015. High schools were included in the performance reports if they met two conditions. First, the high school needed to have at least 10 test-takers who declared that they would graduate that year. Before 2011, all individuals taking the ENEM were recorded as test-takers for the calculation of the participation rate, regardless of whether they completed the test or not. Since 2011, to be considered a test-taker, an individual has to complete the four subject tests plus the writing essay, and obtain a non-zero score in all subjects.[^34] Second, starting in 2009, the INEP required a minimum *participation rate* to be included in the report. The participation rate is the total number of ENEM test-takers in the high school divided by the number of enrolled students in the final years of high school, based on the records of the High School Census. Between 2009 and 2010, the minimum participation rate to be included in the report was 2 percent. In 2011, INEP increased the minimum participation rate to 50 percent.

At the schools in our sample, the average ENEM participation rate over the 2005–2015 period was 70 percent. Table 2 presents balance tests that show that the number of exam takers in our analysis sample and the characteristics of these exam takers did not change significantly when the stakes of the ENEM increased. Appendix Table 11 shows the robustness of our main results to alternative sample definitions.

## Categorization of math questions
This subsection provides details on the categorization of math questions that we use for the heterogeneity analyses in Table 6 and Appendix Table 22.

In Table 6, we categorize math questions into 7 *topic areas* (Panel B) and 30 *competencies* (Panel C) defined by ENEM test designers. These topic areas and competencies are in the `ITENS_PROVA_****.TXT` files of the microdata. The labels for each topic area and competency are defined in `Matriz_Referencia_Enem.pdf`, which is included with the microdata and is also available online at the link in this footnote.[^35] Table 23 shows the translated and shortened labels that we use for Table 6. We also present results by topic area for language arts, natural science, and social science in Appendix Table 19.

<div class="center">

<div id="tab:area_labels">

  -------------- ------------------------------------------------ ------------------- --------------------------------------------------
  \(A\)          \(B\)                                            \(C\)               \(D\)
  Topic area     Competency (and reference number)                Topic area          Competency (and reference number)
  Numbers        Recognize numbers (1)                            Proportions         Solve problems using proportions (16)
  Numbers        Identify numerical patterns (2)                  Proportions         Use proportions to construct arguments (17)
  Numbers        Solve problems using numbers (3)                 Proportions         Evaluate interventions using proportions (18)
  Numbers        Use numbers to construct arguments (4)           Algebra             Identify algebraic relationships (19)
  Numbers        Evaluate interventions using numbers (5)         Algebra             Interpret Cartesian graphs (20)
  Geometry       Project 3D objects into 2D space (6)             Algebra             Solve problems using algebra (21)
  Geometry       Identify geometric shapes (7)                    Algebra             Use algebra to construct arguments (22)
  Geometry       Solve problems using geometry (8)                Algebra             Evaluate interventions using algebra (23)
  Geometry       Use geometry to construct arguments (9)          Interpreting data   Make inferences using data in tables/graphs (24)
  Measurements   Identify units of measurement (10)               Interpreting data   Solve problems using data in tables/graphs (25)
  Measurements   Use scales in everyday situations (11)           Interpreting data   Use tables/graphs to construct arguments (26)
  Measurements   Solve problems using magnitudes (12)             Statistics          Calculate statistical quantities from data (27)
  Measurements   Use measurements to construct arguments (13)     Statistics          Solve problems using statistics (28)
  Measurements   Evaluate interventions using measurements (14)   Statistics          Use statistics to construct arguments (29)
  Proportions    Identify proportional relationships (15)         Statistics          Evaluate interventions using statistics (30)
  -------------- ------------------------------------------------ ------------------- --------------------------------------------------

  : Topic areas and competencies for ENEM math exam

</div>

</div>

*Notes*: This table shows the 7 topic areas (columns A and C) and 30 competencies (columns B and D) for the ENEM math exam. Labels are translated and shortened by the authors from the descriptions in `Matriz_Referencia_Enem.pdf`, which is included with the microdata.

In Appendix Table 22, we define groups of questions based on whether the questions are related to topics covered in a study guide created by *Me Salva!*, which is a well-known Brazilian test prep company. The study guide is called *The Approved Book: One topic per day to pass the ENEM*.[^36] In Panel A of Appendix Table 22, we match keywords from the *Me Salva!* study guide to the text of each question. In Panel B of Appendix Table 22, we match concepts from the *Me Salva!* study guide to solutions to each question that were created by another well-known test prep company called *Descomplica*. *Descomplica* creates solutions to each question from previous ENEM exams and makes them publicly-available online to help students prepare.[^37]

We match keywords and concepts in the *Me Salva!* study guide to ENEM questions using text analysis. The *Me Salva!* study guide contains tips for solving questions in seven different content areas, and in each content area, there are keywords that appear in bold in the text. For Panel A of Appendix Table 22, we search the question text for the keywords from the study guide. For Panel B of Appendix Table 22, we search the *Descomplica* solutions for both the keywords and for regular expressions that indicate questions in which the solution is likely to depend on concepts from the study guide. The content areas and search terms that we use are as follows; the keywords that we use in both Panels A and B appear in italics, and the regular expression searches that we use in Panel B appear in plain text:

- **Geometric formulas.** *cilindro, cone, cubo, equilatero, esfera, hexagonal, hexagono, losango, paralelepipedo, piramide, prisma, quadrado, quadrangular, retangulo, trapezio, triangular, triangulo*.

- **Proportions ("Rule of 3").** *diretamente, grandezas, inversamente, proporcionais, proporção, regra de três*.

- **Manipulating fractions.** Solutions that contain at least two fractions (0-9+/0-9+) and also an equals sign ($=$).

- **Radicals.** Solutions that contain a square or cube root sign ($\sqrt{}$ or $\sqrt[3]{}$).

- **Combinatory and statistical analysis.** *arranjo, combinação, mediana, moda, permutação, possibilidades*. Solutions that contain arrangement or combination notation (A0-9,0-9 or C0-9,0-9).

- **Probability.** *combinação, possiveis, probabilidade*. Solutions that contain combination notation (C0-9,0-9).

- **Trigonometric formulas.** *cos, cossec, cosseno, cotg, sec, sen, seno, tangente, tg*.

In all cases we trim plural, adjective, and masculine/feminine endings to words before matching. In Panel A, we display results for keywords that appear in eight or more questions, and we group all other words into the "Fewer than 8 occurrences" category. In Panel B, we display results separately for each of the seven *Me Salva!* content areas. In both panels, we also show results pooling across all questions that do/don't match any search term in the study guide.

## ENEM participation prior to federal university adoption

This appendix discusses factors that contributed to ENEM participation before federal universities began using the exam for admissions. Although federal universities did not adopt the ENEM until 2009 or later, many students—particularly those from public high schools—took the exam in earlier years. We highlight three key factors that drove participation: 1) early adoption by other higher education institutions; 2) the creation of scholarship programs that used ENEM scores; and 3) policies that reduced barriers to exam access.

First, many non-federal higher education institutions had already incorporated ENEM scores into their admission processes by the late 2000s. According to a 2008 INEP Report, the number of institutions using ENEM scores grew from 93 in 1999 to approximately 470 in 2005, reaching over 700 institutions by 2008 (Instituto Nacional de Estudos e Pesquisas Educacionais An�sio Teixeira (Inep), 2009). These institutions included private universities, municipal universities, and some state universities, which together enrolled roughly 89 percent of all college students in 2009 (Section 2). Many of these institutions used ENEM scores as a supplement to their own entrance exams, typically by adding bonus points to applicants' vestibular scores based on their ENEM performance. Some institutions also offered direct admission to high-scoring ENEM participants.

This institutional adoption is reflected in students' self-reported reasons for taking the exam. In survey data collected during ENEM registration, "college admission" was consistently among the top reported reasons for taking the exam among both public and private school students between 2001 and 2008 (Appendix Figure 10). Roughly 50–60 percent of private school students reported taking the exam for college admission purposes. Among public school students, the proportion taking the exam for college admissions grew from almost 40 percent in 2001 to almost 60 percent in 2008.

<figure id="fig:enem-reasons" data-latex-placement="H">
<div class="centering">
<p><embed src="results/reason_priv.pdf" /> <embed src="results/reason_pub.pdf" /></p>
</div>
<p><em>Notes</em>: This figure shows the evolution of students’ reported reasons for taking the ENEM exam between 2001 and 2008, separately for private and public school students. The responses are grouped into four categories: to test knowledge/reasoning ability, for college admission purposes, for career preparation/employment prospects, and don’t know/missing. The original response options in Portuguese were: “Para testar seus conhecimentos / capacidade de raciocínio” (to test knowledge/reasoning), “Para entrar na faculdade / conseguir pontos para o vestibular” (for college admission), and “Para ter um bom emprego / saber se está preparado(a) para o futuro profissional” (for career preparation/jobs).</p>
<figcaption>Reasons for taking ENEM exam by school type, 2001–2008</figcaption>
</figure>

Second, the creation of ProUni (*Programa Universidade para Todos*) in 2004 significantly increased ENEM participation among low-income students. ProUni is a need-based scholarship program for private universities that uses ENEM scores to determine eligibility. The program established score cutoffs for each university program based on the number of available seats. Appendix Figure 11 shows a sharp increase in ENEM participation following ProUni's implementation, particularly among public school students.

<figure id="fig:enem-participation" data-latex-placement="H">
<div class="centering">
<embed src="results/testtakers.pdf" />
</div>
<p><em>Notes</em>: This figure shows the number of high school students taking the ENEM exam from 1998 to 2008, separately for public and private schools. The vertical dashed line marks the elimination of registration fees for public school students and low-income families in 2001.</p>
<figcaption>Number of high school students taking ENEM exam, 1998–2008</figcaption>
</figure>

Third, policy changes in the early 2000s reduced barriers to exam access. In 2001, the Ministry of Education eliminated registration fees for public school students and low-income families. Appendix Figure 11 shows that this policy change was followed by a substantial increase in participation among public school students.

[^1]: Influential books such as *The Tyranny of Metrics* (Muller, 2018) have popularized the notion that agents strategically manipulate signals of their quality in various high-stakes settings.

[^2]: Goodhart's Law is named after economist Charles Goodhart, who wrote: "When a measure becomes a target, it ceases to be a good measure" (Goodhart, 1975). Campbell's Law is named after psychologist Donald Campbell, who noted that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor" (Campbell, 1979).

[^3]: In this literature, gaming ability refers to the marginal cost of improving the signal, while natural action is the signal individuals send without stakes.

[^4]: Goodman, Gurantz, and Smith (2020) show that higher-income students are more likely to retake the SAT, which raises their admission-relevant scores. Affluent students are also more likely to hire tutors and procure special test accommodations like extended time. See: ["SAT/ACT tutoring: $1500 for 90 minutes. And 14 sessions are required. Really,"](https://www.washingtonpost.com/news/answer-sheet/wp/2014/08/31/satact-tutoring-1500-for-90-minutes-and-14-sessions-are-required-really/) Valerie Strauss, *The Wall Street Journal*, August 31, 2014; and ["Many More Students, Especially the Affluent, Get Extra Time to Take the SAT,"](https://www.wsj.com/articles/many-more-students-especially-the-affluent-get-extra-time-to-take-the-sat-11558450347) Douglas Belkin, Jennifer Levitz and Melissa Korn, *The Wall Street Journal*, May 21, 2019.

[^5]: For example, in 2019 a group of students and advocacy groups sued the University of California over its use the SAT and ACT exams. The plaintiffs' complaint quotes UC Regents Chairman John Pérez: "The highest predictive value of an SAT isn't in how well a student will do in school, but how well they were able to avail themselves of prep material. And access to that prep material is still disproportionately tied to family income" (Rosenbaum et al., 2020).

[^6]: Related research shows how the design and implementation of admission exams can either reduce or decrease inequality in college access (Hoxby and Turner, 2013; Bulman, 2015; Pallais, 2015; Goodman, 2016; Bhattacharya, Kanaya, and Stevens, 2017; Reyes, 2023).

[^7]: Roughly 0.5 percent of Brazilian students attend a high school managed by the federal government (Appendix Table 7). Our empirical analysis defines "private high schools" to include both privately- and federally-managed high schools since their students are similar in socioeconomic status and achievement.

[^8]: During our sample period, many federal universities implemented reserved quotas for minority and/or public school students. Within these quota groups, admissions are based solely on entrance exam scores. Below we discuss the implications of the affirmative action quotas for our empirical analysis.

[^9]: Some state universities also adopted the ENEM as their admission test, but to this date, many still design and administer their own *vestibular* exams.

[^10]: INEP published high school ENEM performance measures with the goal of "assisting teachers, principals and other educational managers in identifying deficiencies and good practices" (INEP, 2019). There were no financial incentives tied to school performance.

[^11]: At the schools in our sample, the mean ENEM participation rate from 2005–2015 was 70 percent. The criterion for inclusion in the reports changed over this time period, as we describe in Appendix 9.3.

[^12]: ENEM scores, as reported to the public, are scaled to have a mean of 500 and a SD of 100 in the population of 2009 high school seniors who took the exam. Throughout the paper, we report ENEM scores in SD units relative to this population. For ENEM subject scores, our transformation is: Transformed score = (Scale score $-$ 500)/100. Our transformation is different for writing and overall scores since they are on different scales. In all cases, a score of zero in our paper is equivalent to the performance of the average high school senior who took the ENEM in 2009, and a score of one is 1 SD higher within this population. These transformations preserve the comparability of test scores across cohorts. See Appendix app:var_def for details.

[^13]: For reference, the white/Black gap in the 2017 U.S. SAT math exam was 0.85 SDs (College Board, 2017).

[^14]: The ENEM is administered near the end of each year, and scores are used for admission to university cohorts that begin in the following calendar year.

[^15]: Specifically, we estimate the following regression for each state $s$: $$\text{ProportionENEM}_{st} = \delta^0_s + \delta^1_s\mathbbm{1}\{t \geq \tau_s\} + \delta^2_s \mathbbm{1}\{t \geq \tau_s\}(t-\tau_s) + \delta^3_s\mathbbm{1}\{t < \tau_s\}(t-\tau_s) + \epsilon_{st},$$ where $\text{ProportionENEM}_{st}$ is our continuous treatment variable. We estimate this regression for all candidate adoption years $\tau_s \in \{2008, ..., 2016\}$ and pick the value $\tau^*_s$ that yields the highest $R^2$ value. Lastly, we define our binary treatment variable to be $\text{HighStakes}_{st} = \mathbbm{1}\{t \geq \tau^*_s\}.$ We define one state (Sergipe) as a "never adopter" since the value of $\text{ProportionENEM}_{st}$ never exceeds 0.06.

[^16]: Throughout the paper, we refer to cohorts prior to each state's ENEM adoption year as "low stakes" cohorts for brevity. The ENEM was consequential for students in these cohorts who wished to attend a federal university in other states that had already adopted the ENEM. The ENEM was also used to determine ProUni (*O Programa Universidade Para Todos*) scholarships and eligibility for the federal FIES (*O Fundo de Financiamento Estudantil*) financial aid system (OECD, 2021). These incentives mattered mainly for students who wished to attend private universities, as public universities in Brazil are tuition-free. Despite these other incentives, we argue that the adoption of the ENEM by federal universities significantly increased the exam's stakes for students who wished to attend a selective university close to home.

[^17]: Our stacked specification uses our binary treatment variable, $\text{HighStakes}_{s(h)t}$, in the following regression: $$\begin{equation}
    Y_{htg} = \gamma_{s(h)g} + \gamma_{tg} + \beta \text{HighStakes}_{s(h)t} + \big[\tilde{\gamma}_{s(h)g} + \tilde{\gamma}_{tg} + \beta^{\text{gap}} \text{HighStakes}_{s(h)t}\big]\text{Private}_h + \epsilon_{htg}.\end{equation}$$ In this specification, the dataset is at the high school ($h$) $\times$ year ($t$) $\times$ pairwise group ($g$) level, and we include state $\times$ group dummies, $\gamma_{s(h)g}$ and $\tilde{\gamma}_{s(h)g}$, and year $\times$ group dummies, $\gamma_{tg}$ and $\tilde{\gamma}_{tg}$. Appendix Table 10 shows the pairwise groups and the structure of our stacked dataset.

[^18]: Figure 3 plots $\beta_l^{\text{gap}}$ coefficients from the high school ($h$) $\times$ year ($t$) $\times$ pairwise group ($g$) level regression $$\begin{equation}
    Y_{htg} = \gamma_{s(h)g} + \tilde{\gamma}_{s(h)g}\text{Private}_h + \gamma_{tg} + \tilde{\gamma}_{tg}\text{Private}_h + \sum_{l = -7}^{7} \big[\beta_l + \beta_l^{\text{gap}} \text{Private}_h\big] \mathbbm{1}\{t-\tau^*_{s(h)}=l\} + \epsilon_{htg}, \end{equation}$$ where $l$ denotes years relative to the state's ENEM adoption year, $\tau^*_{s(h)}$. We include state $\times$ group dummies, $\gamma_{s(h)g}$, year $\times$ group dummies, $\gamma_{tg}$, and dummies for years $l$, $\mathbbm{1}\{t-\tau^*_{s(h)}=l\}$, omitting $l=-1$. We interact all covariates with a dummy for private schools, $\text{Private}_h$, and plot the $\beta_l^{\text{gap}}$ coefficients from $l=-4$ to 4. We also see clear evidence that our results are driven by a causal effect of ENEM adoption using the default event study figures from Callaway and Sant'Anna (2021)'s `csdid` Stata package (Appendix Figure 9).

[^19]: In 2016 data from the SISU admission system, the within-state standard deviation of cutoff scores for federal university programs is 0.52 SDs (in the test score units of our sample). Thus 0.11/0.52 $\approx$ 21 percent.

[^20]: Our null result for gender differs from work that finds that male/female test score gaps are larger on high stakes exams (Ors, Palomino, and Peyrache, 2013; Azmat, Calsamiglia, and Iriberri, 2016; Cai et al., 2019). This work interprets their results as evidence of gender differences in responses to competitive pressure. We think our results are more attributable to test preparation (see below), which occurs over a longer time span and is likely to be gender neutral.

[^21]: Note that in columns (E)–(F) of Table 4, the sample includes two cohorts that took the old 63-question version of the ENEM exam (2007–2008), so these estimates may reflect the effects of the ENEM redesign in addition to the impacts of the exam's adoption by federal universities. The 2007–2008 ENEM reported only a single core component score plus a writing score. To define scores for each subject, we categorized the multiple choice questions into math, language arts, natural science, and social science, and then computed a separate score for each subject using the IRT parameters. Since the reference populations differ for the 2007–2008 and 2009–2017 exams, our regressions in columns (E)–(F) of Table 4 standardize scores to have mean 0 and SD 1 within each year of our sample. See Appendix app:var_def for details.

[^22]: Some individuals in our sample would have been eligible for admission through reserved quotas for disadvantaged students, but these quotas were not fully implemented at many federal universities until 2016. Appendix Figure 8 shows that most public students' ENEM scores were also well below the distribution of cutoff scores for these reserved quotas.

[^23]: Another possibility is that public school curricula was slower to respond to the ENEM rollout than private school curricula, which may explain the slight fade-out of some of our test scores results in Figure 3.

[^24]: We focus on these four companies because they list the names and addresses of affiliate schools on their websites. There are other prominent test prep companies in Brazil for which we could not find lists of affiliate schools, so it is likely that some other private schools in our sample also have test-oriented curricula. Thus, the coefficients that we estimate may be attenuated due to under-classification of prep schools.

[^25]: Appendix 8.1 presents a theoretical framework that illustrates the potential channels through which exam stakes can impact test score gaps and exam informativeness. This section briefly describes the intuition.

[^26]: We exclude 2011 ENEM takers from our analysis of exam informativeness because the crosswalk variable that INEP created to match individuals across their different datasets is not correctly defined for this cohort. We also exclude 2015–2017 ENEM takers from this analysis because we do not observe many of the longer-run outcomes in these cohorts given the timing of our data.

[^27]: Another way of measuring the information content of an exam is to "anchor" scores to an economic outcome of interest (e.g., Cawley, Heckman, and Vytlacil, 1999; Jacob and Rothstein, 2016; Nielsen, 2023). We focus on correlation coefficients between scale scores and outcomes because scale scores are what colleges use to admit students. Our measure of informativeness is closely related to the theoretical concept of Blackwell informativeness used in Frankel and Kartik (2019) and subsequent studies on muddled information. Appendix 8.2 proves that with binary states and signals, an increase in a signal's Blackwell informativeness necessarily leads to a higher correlation between states and signal realizations.

[^28]: Rothstein (2004) points out that it is difficult to justify the sample selection assumptions that are implicit in many validity studies, and he offers a new estimator for settings in which the variables that determine admission to selective colleges are observable (e.g., the University of California system). We cannot implement his estimator because we do not observe the *vestibular* scores that federal universities used in admissions prior to ENEM adoption. Our analysis partially addresses Rothstein's concerns by including some outcomes that are defined for our entire sample (e.g., college enrollment and unconditional college graduation). Further, our regression model dd_styr estimates *changes* in exam informativeness across cohorts, which differences out the impact of unobservable determinants of sample selection that do not vary over time. Like Rothstein (2004), our benchmark results rely on the strong assumption that students' decisions about which college program to attend are ignorable, but we also present results that examine within-program validity.

[^29]: This analysis follows Jacob (2005) and Cohodes (2016) in using item-level data to shed light on the mechanisms for test scores gains. Unlike these papers, we are also able to link item-level data to longer-run outcomes to directly measure the informativeness of different exam skills.

[^30]: This measure of informativeness is closely related to the correlation coefficient between ENEM scores and outcomes, which is the measure of informativeness we use in Table 5. The correlation between an indicator for correctly answering question $j$, $C_{ij}$, and an outcome, $Y_i$, can be written as a function of the difference in the mean outcomes of students who got the question correct and incorrect, $\text{corr}(C_{ij}, Y_i) = (\mathbb{E}[Y_i | C_{ij} = 1] - \mathbb{E}[Y_i | C_{ij} = 0]) \sigma_{C_{j}}/\sigma_{Y},$ where $\sigma_{C_{j}}$ and $\sigma_{Y}$ are the standard deviations of $C_{ij}$ and $Y_i$.

[^31]: Our empirical analysis relies on a parallel trends version of this assumption.

[^32]: There is another potential channel that is outside the scope of our framework: high-stakes exam scores may be more informative for college outcomes if they increase the "match quality" between individuals and college programs. This channel could arise if test scores, $T_i$, have a causal effect on outcomes, $Y_i$, through their influence on which college and/or major students attend. In other words, high-stakes exam scores may be more informative for college success if the distribution of these scores leads to better student/college matches (e.g., on the basis of academic preparation). To distinguish between this channel and the correlational/skill accumulation channels, we follow the standard practice that testing agencies use to measure predictive validity. Specifically, we present results that examine the correlation between $T_i$ and $Y_i$ after de-meaning both variables *within* college programs. These results reflect only variation in informativeness among students who enrolled in the same colleges and majors.

[^33]: We do not include 2015–2017 ENEM takers in our analysis of long-run outcomes because we do not observe many of these outcomes in these cohorts given the timing of our data.

[^34]: Only the following grades are considered in the report: 3rd and 4th grade of regular high school (*ensino médio regular 3a e 4a série*), 3rd and 4th grade of teaching track high school *(ensino médio magistério 3a e 4a série*), non-serialized regular and teaching track high school (*ensino médio não-seriado, regular e magistério*), vocational education and high school for youth and adult education (*educação profissionalizante e ensino médio para educação de jovens e adultos*).

[^35]: See: <https://download.inep.gov.br/download/enem/matriz_referencia.pdf> (accessed in June 2023).

[^36]: The Portuguese title is *O Livro do Aprovado: Um conteúdo por dia para passar no ENEM*. See: <https://cdn.mesalva.com/uploads/medium/attachment/MS2018-livro-do-aprovado.pdf> (accessed in June 2023).

[^37]: See: <https://descomplica.com.br/gabarito-enem/questoes/?cor=azul> (accessed in June 2023). One question is behind a paywall (Question 145 in the *Azul* book of the 2015 math exam), and thus we exclude it from our analysis in Panel B of Table 22).