Passing the Test
Pencils Down? Computerized Testing and Student Achievement
John Gordanier, Orgul Ozturk & Crystal Zhan
Education Finance and Policy, forthcoming
Abstract:
Computer-based testing (CBT) is becoming an increasingly popular format of assessment in educational settings. If students face a digital divide in terms of access to computers at school and at home, CBT may exacerbate measured student achievement gaps. In this paper, we use the rollout of CBT in South Carolina starting in 2015 to investigate its effect on measured student performance. We link student-level test scores and poverty measures to the share of students taking CBT in a grade of a school and show that CBT has a significant negative impact on test scores of multiple subjects. The negative impact is not uniform across student subgroups but rather particularly large for students in poor households. There is little evidence that the effect fades as students and schools become more experienced with computerized testing. These results suggest that the testing mode change might have distributional consequences. However, we do find a smaller effect in schools where technology is more readily available, implying that school-level investments could mitigate the effect.
Power in a Pandemic: Teachers’ Unions and Their Responses to School Reopening
Bradley Marianno et al.
AERA Open, January 2022
Abstract:
Drawing on Bachrach and Baratz’s first and second faces of interest group power, we explore the relationship between teachers’ union power and reopening decisions during the fall 2020 semester in 250 large districts around the United States. We leverage a self-collected panel data set of reopening decisions coupled with measures of teachers’ union first face power (drawn from social media postings on teachers’ unions’ Facebook pages) and second face power (operationalized as district size, whether the school district negotiates a collective bargaining agreement with the teachers’ union, the length of the collective bargaining agreement, and the amount of revenue raised by the union). We found that school districts where teachers’ unions exhibit strong second face power (but not first face power) were less likely to start the school year with in-person instruction, were less likely to ever open during fall semester with in-person instruction and spent fewer weeks in in-person learning.
Information and Credible Sanctions in Curbing Online Cheating Among Undergraduates: A Field Experiment
Daniel Dench & Theodore Joyce
NBER Working Paper, February 2022
Abstract:
The rapid increase in online instruction in higher education has heightened concerns about cheating. We use a randomized control design to test whether informing students that we can detect plagiarism reduces cheating. We further test whether informing students they have been caught cheating reduces subsequent cheating. We find informing students about our capability to detect plagiarism has little effect on cheating. Notifying students that they have been caught cheating and are on a watch list reduces subsequent cheating attempts by at least 65 percent depending on the class and sample. We test for peer effects but conclude we cannot credibly identify peer effects distinct from own-cheating propensities.
New Evidence on Teacher Pay
Krishna Regmi
ILR Review, forthcoming
Abstract:
Prior research has shown that teachers receive lower pay compared to people with the same educational level who work in other occupations. This article challenges that literature and shows that by applying novel statistical approaches, the pay differentials are reduced, and even become pay premiums. In particular, these approaches provide unifying estimates that turn an earnings penalty between female teachers and non-teachers of approximately 10% — based on a standard approach in the literature — into an earnings premium of 5 to 10%. Likewise, estimates based on these approaches erase up to two-thirds of the earnings gap between male teachers and non-teachers. Moreover, going beyond the traditional focus on the mean, the author decomposes the pay gap across the entire earnings distribution. Estimates show that although teachers have a substantial earnings premium at the bottom of the distribution, they have a large earnings penalty at the top.
Build It and will They Come?: The Effect of Investing in Cultural Consumption Amenities in Higher Education on Student-Level Outcomes
Siân Mughan, Jessica Sherrod Hale & Joanna Woronkowicz
Research in Higher Education, February 2022, Pages 60–91
Abstract:
Despite a climate of fiscal scarcity, higher education institutions are making big investments in campus consumption amenities while reducing instructional expenditures and growing increasingly reliant on tuition revenue. Few empirical studies exist exploring why universities increasingly invest in these amenities; however, one compelling explanation is that in an increasingly competitive market universities use amenities as a means of attracting students. Using resource dependency theory, this article examines these investments through the lens of marketization of public services. A unique dataset of HEI investments in cultural consumption amenities from 2000 to 2016 is used to estimate the effect of investing in cultural consumption amenities on various student-level outcomes. The evidence suggests that consumption amenities investments are associated with an increase in the yield rate (the proportion of admitted students who choose to enroll in the institution) and a decrease in the percentage of students paying in-state tuition. Also, higher-value investments are associated with a modest increase in out-of-state tuition, a decrease in in-state and out-of-state fees, and an increase in SAT scores. Taken together, the findings suggest that cultural consumption amenities investments may help attract more lucrative students and students who are strongly considering enrolling (as indicated by their application).
Teacher Labor Market Equilibrium and Student Achievement
Michael Bates et al.
NBER Working Paper, February 2022
Abstract:
We study whether reallocating existing teachers across schools within a district can increase student achievement, and what policies would help achieve these gains. Using a model of multi-dimensional value-added, we find meaningful achievement gains from reallocating teachers within a district. Using an estimated equilibrium model of the teacher labor market, we find that achieving most of these gains requires directly affecting teachers' preferences over schools. In contrast, directly affecting principals' selection of teachers can lower student achievement. Our analysis highlights the importance of equilibrium and second-best reasoning in analyzing teacher labor market policies.
Local Implementation of State-Level Discipline Policy: Administrator Perspectives and Contextual Factors Associated With Compliance
Kaitlin Anderson & Sarah McKenzie
AERA Open, February 2022
Abstract:
Many states and school districts are implementing reforms to reduce reliance on exclusionary discipline such as out-of-school suspension and expulsion. This article uses survey and administrative data to study the implementation of a state-level policy limiting elementary school out-of-school suspension and expulsions. While the results are limited in sample size and generalizability, we find that survey respondents from relatively disadvantaged schools reported greater difficulties and challenges with implementation and tended to comply at lower rates. Policy makers seeking more equitable implementation of these types of reforms should work with schools to better understand and attend to local preferences, buy-in, and capacity for implementation.
Unfinished Business? Academic and Labor Market Profile of Adults With Substantial College Credits But No Degree
Kelli Bird et al.
Educational Evaluation and Policy Analysis, forthcoming
Abstract:
Recent state policy efforts have focused on increasing attainment among adults with some college but no degree (SCND). Yet little is actually known about the SCND population. Using data from the Virginia Community College System (VCCS), we provide the first detailed profile on the academic, employment, and earnings trajectories of the SCND population and how these compare with VCCS graduates. We show that the share of SCND students who are academically ready to re-enroll and would benefit from doing so may be substantially lower than policy makers anticipate. Specifically, we estimate that few SCND students (approximately 3%) could fairly easily re-enroll in fields of study from which they could reasonably expect a sizable earnings premium from completing their degree.
Returns to bachelor’s degree completion among stopouts
Amanda Gaulke
Economics of Education Review, February 2022
Abstract:
The recent returns to bachelor’s degree completion for those with interrupted college enrollment (stopouts) is unknown. This information is especially important since re-enrollment programs are being sold as a ‘win-win’ for both schools and students. This paper contributes to the literature by using the National Longitudinal Study of Youth 1997 cohort with a fixed-effects difference-in-differences regression to estimate recent labor market benefits for stopouts. Re-enrolling and completing a bachelor’s degree leads to a significant increase in employment of 9.8 percentage points and a significant increase in real (2014) annual income of $5,392.
School's Out: How Summer Youth Employment Programs Impact Academic Outcomes
Alicia Sasser Modestino & Richard Paulsen
Education Finance and Policy, forthcoming
Abstract:
Recently there has been an emphasis on how time spent outside of the classroom can affect student outcomes, including high school graduation, with the hope of closing academic achievement gaps along socioeconomic and racial lines. This paper provides experimental evidence regarding a particular type of out-of-school activity — early work experience — on high school academic outcomes for low-income inner-city youth. Using randomized admissions lotteries for students who applied to the Boston Summer Youth Employment Program (SYEP), we estimate the effect of being selected to participate on academic outcomes as measured by administrative school records. We find that SYEP lottery winners are 4.4 percentage points more likely to graduate from high school on time and 2.5 percentage points less likely to drop out of high school during the four years after participating in the program relative to the control group. These improvements appear to be driven by better attendance and course performance in the year after being selected for the program, with the program's impact on attendance persisting into the second year. Survey data suggest that the Boston SYEP may affect academic outcomes by increasing aspirations to attend college, gaining basic work habits, and improving social skills during the summer.
Unintended Consequences: Understanding the Relationship Between Dual Enrollment Participation, College Undermatch, and Bachelor’s Degree Attainment
Sanja Jagesic et al.
Research in Higher Education, February 2022, Pages 119–139
Abstract:
Recent research suggests that dual enrollment programs are a useful recruiting tool for colleges and universities as many high school students remain at their dual enrollment institution after high school graduation. Students staying enrolled at their dual enrollment institution for college may be beneficial for institutions, but is it beneficial for participating dual enrollment students? We find that students who participate in dual enrollment in high school are more likely to experience college undermatch than similar students who do not participate in any postsecondary acceleration opportunities. Students who participate in a dual enrollment program at a 2-year institution and stay at the institution after high school graduation are up to 29% points more likely to undermatch than the average dual enrollment student enrolling in a different postsecondary institution after high school graduation. The difference is reduced to 9% points for dual enrollment students who stay at a four-year institution. Most significantly, our research shows that the decision to stay at a 2-year dual enrollment institution has negative consequences for bachelor’s degree attainment. Students who stay at a 2-year dual enrollment institution where they are undermatched have about a 33% point lower probability of completing a bachelor’s degree when compared to similar 2-year dual enrollment students who move to a 4-year institution where they are not undermatched after high school graduation.
Only a matter of time? The role of time in school on four-day school week achievement impacts
Paul Thompson & Jason Ward
Economics of Education Review, February 2022
Abstract:
Previous evidence has shown disparate achievement impacts of the four-day school week within specific states. This paper examines the effects of the four-day school week on achievement across 12 states to contextualize these four-day school week impacts nationally. We estimate these effects using a difference-in-differences design with data from the Stanford Educational Data Archive and a proprietary longitudinal national database of four-day school week use from 2009 to 2018. We find reductions in both math and English/language arts achievement when examining four-day school weeks nationally, but these aggregate effects appear to be masking important heterogeneity due to differences in overall time in school across districts. When stratifying four-day week districts into districts with low, middle, and high levels of time in school, we find statistically significant negative effects on math achievement for four-day school week districts with low time in school, but no statistically significant effects for four-day school week districts with middle or high time in school. Our findings suggest that maintaining sufficient overall time in school should be a key consideration for school districts contemplating four-day school week adoption.
Lateral Reading on the Open Internet: A District-Wide Field Study in High School Government Classes
Sam Wineburg et al.
Journal of Educational Psychology, forthcoming
Abstract:
Young people turn to the Internet to become informed about the issues that concern them and their communities. How can they learn to distinguish credible information from sham? In a study conducted in an urban school district, we tested a classroom-based intervention in which students were taught evaluation strategies derived from research with professional fact checkers. We provided professional development to high school teachers who then implemented six 1-hour lessons across three months in a district-mandated government course. Using a matched/control design, students in treatment classrooms (n = 271) were compared to peers (n = 228) in regular classrooms. A cluster-randomized, repeated-measures analysis showed that students in experimental classrooms grew significantly in their ability to judge the credibility of digital content compared to students in control classrooms. We conclude by addressing how the study’s findings inform efforts to prepare students to make sense of the information that streams across their digital devices.
The impact of pork-barrel capital funding in schools: Evidence from participatory budgeting in NYC
Michah Rothbart, David Schwegman & Iuliia Shybalkina
Public Budgeting & Finance, forthcoming
Abstract:
Pork-barrel spending is a form of public spending controlled by individual legislators and primarily serving a local interest. In this paper, we investigate the impact of a type of pork, council member capital discretionary education spending voted upon in a participatory budgeting (PB) process, on school budgets and performance in New York City. Exploiting plausibly exogenous variation in discretionary spending induced by the PB elections, we find winning a PB election increases school pork appropriations. However, we find no evidence these transfers from council members improve fiscal and performance outcomes. Further, pork may interfere with school budgeting.
Free to Spend? Institutional Autonomy and Expenditures on Executive Compensation, Faculty Salaries, and Research Activities
Taylor Odle
Research in Higher Education, February 2022, Pages 1–32
Abstract:
System governing boards fulfill roles as both regulators and protectors for their multiple constituent campuses. While centralized control provides many benefits to institutional members, such advantages are also be accompanied by limitations upon campuses to pursue expanded missions, fully compete, and spend according to individual rather than collective priorities. In this study, I leverage a natural experiment where one state “freed” six public universities by removing the oversight of a centralized board. Given a novel opportunity to assess how institutions respond to a new intra-state market characterized by deregulation and increased competition for students, faculty, and other scarce resources, I first focus on changes in three institutional expenditure areas closely tied to entrepreneurship, competition, and prestige-seeking: executive compensation, faculty salaries, and spending on research activities. After constructing a novel dataset with administrative records on compensation and public data on expenditures, I employ complementary difference-in-differences and synthetic control approaches which yield robust evidence suggesting that newly independent universities increased the salary of their president/chancellor by approximately 6.2% (or $19,000), increased the average full professor salary by 2.2% (nearly $2,000), and increased research expenditures by an average of 12% (or $2 million). These findings not only advance the nascent literature on how state governance structures influence institutional behaviors but also provide useful evidence for policymakers considering the intended (and potentially unintended) consequences of similar governance reorganizations.
A Bridge to Graduation: Post-Secondary Effects of an Alternative Pathway for Students Who Fail High School Exit Exams
Jane Arnold Lincove, Catherine Mata & Kalena Cortes
NBER Working Paper, February 2022
Abstract:
High school exit exams are meant to standardize the quality of public high schools and to ensure that students graduate with a set of basic skills and knowledge. Evidence suggests that a common perverse effect of exit exams is an increase in dropout for students who have difficulty passing tests, with a larger effect on minority students. To mitigate this, some states offer alternative, non-tested pathways to graduation for students who have failed their exit exams. This study investigates the post-secondary effects of an alternative high school graduation program. Among students who initially fail an exit exam, those who eventually graduate through an alternative project-based pathway have lower college enrollment, but similar employment outcomes to students who graduate by retaking and passing their exit exams. Compared to similar students who fail to complete high school, those students who take the alternative pathway have better post-secondary outcomes in both education and employment.
The effect of breakfast after the bell on student academic achievement
Andres Cuadros-Meñaca, Michael Thomsen & Rodolfo Nayga
Economics of Education Review, February 2022
Abstract:
Many schools are beginning to serve breakfast after the bell (BAB) in order to expand breakfast participation. Improved nutrition may lead to better academic achievement, but BAB may also cut into instruction time. Using a difference-in-differences (DID) estimation with variation in treatment timing, we find modest negative effects of BAB on math and no effects on English, Language, and Arts (ELA) test scores. The negative effects, while small, were driven by more affluent children. The null findings cannot be explained by the lack of an effect on breakfast participation or school attendance increases. Average treatment effects by grade indicate positive effects on math scores for vulnerable children when BAB is adopted in earlier grades and negative but modest effects in later grades for more affluent children. Overall, we conclude that BAB can be incorporated into the school day without adversely impacting academic achievement in a meaningful way.
Fraction ball: Playful and physically active fraction and decimal learning
Andres Bustamante et al.
Journal of Educational Psychology, forthcoming
Abstract:
This study tested a novel approach to capitalizing on the benefits of play for informal math learning. Two experiments evaluated a platform called “Fraction Ball,” that provides an embodied, playful, and physically active learning experience by modifying the lines on a basketball court to support rational number learning. In the Pilot Experiment, 69 fifth–sixth graders were randomly assigned to play a set of four different Fraction Ball games or attend normal physical education (PE) class and completed rational number pretests and posttests. After strategic improvements to expand the intervention, the same protocol was implemented in the Efficacy Experiment with 160 fourth–sixth graders. Playing Fraction Ball for four PE class periods (Pilot Experiment) improved students’ ability to convert fractions to decimals. Playing a revised version of six different Fraction Ball games for six PE class periods (Efficacy Experiment) significantly improved children’s rational number understanding as reflected by higher scores in overall accuracy, with positive impacts on several subtests. Fraction Ball represents a low-cost, highly scalable intervention that promotes math learning in a fun and engaging approach.