Learn something
What Do Test Scores Miss? The Importance of Teacher Effects on Non-Test Score Outcomes
Kirabo Jackson
NBER Working Paper, May 2016
Abstract:
This paper extends the traditional test-score value-added model of teacher quality to allow for the possibility that teachers affect a variety of student outcomes through their effects on both students’ cognitive and noncognitive skill. Results show that teachers have effects on skills not measured by test-scores, but reflected in absences, suspensions, course grades, and on-time grade progression. Teacher effects on these non-test-score outcomes in 9th grade predict effects on high-school completion and predictors of college-going — above and beyond their effects on test scores. Relative to using only test-score measures of teacher quality, including both test-score and non-test-score measures more than doubles the predictable variability of teacher effects on these longer-run outcomes.
---------------------
Matthew Steinberg & Rachel Garrett
Educational Evaluation and Policy Analysis, June 2016, Pages 293-317
Abstract:
As states and districts implement more rigorous teacher evaluation systems, measures of teacher performance are increasingly being used to support instruction and inform retention decisions. Classroom observations take a central role in these systems, accounting for the majority of teacher ratings upon which accountability decisions are based. Using data from the Measures of Effective Teaching study, we explore the extent to which classroom composition influences measured teacher performance based on classroom observation scores. The context in which teachers work — most notably, the incoming academic performance of their students — plays a critical role in determining teachers’ measured performance. Furthermore, the intentional sorting of teachers to students has a significant influence on measured performance. Implications for high-stakes teacher accountability policies are discussed.
---------------------
Signaling in Higher Education: The Effect of Access to Elite Colleges on Choice of Major
Valerie Bostwick
Economic Inquiry, July 2016, Pages 1383–1401
Abstract:
I propose a model of postsecondary education in which major field of study can be used by individuals to signal productivity to employers. Under this signaling model, I show that geographic areas with high access to elite universities result in fewer science, technology, engineering, and mathematics (STEM) majors among lower ability students at nonelite colleges. This is distinct from the prediction of a full information model in which access to elite schools should only affect high ability individuals directly. Using data from the National Center for Education Statistics' Baccalaureate & Beyond survey, I find evidence that is consistent with the signaling model prediction, specifically a 2.3–3.7 percentage point or 16%–25% decrease in the probability of choosing a STEM major among lower ability students in areas with greater access to elite colleges.
---------------------
Giving College Credit Where it is Due: Advanced Placement Exam Scores and College Outcomes
Jonathan Smith, Michael Hurwitz & Christopher Avery
Journal of Labor Economics, forthcoming
Abstract:
We implement a regression discontinuity design using the continuous raw Advanced Placement (AP) exam scores, which are mapped into the observed 1-5 integer scores, for over 4.5 million students. Earning higher AP integer scores positively impacts college completion and subsequent exam taking. Specifically, attaining credit-granting integer scores increases the probability that a student will receive a bachelor’s degree within four years by 1 to 2 percentage points per exam. We also find that receiving a score of 3 over a 2 on junior year AP exams causes students to take between 0.06 and 0.14 more AP exams senior year.
---------------------
Student Loan Information Provision and Academic Choices
Maximilian Schmeiser, Christiana Stoddard & Carly Urban
American Economic Review, May 2016, Pages 324-328
Abstract:
We examine the effect of a student loan information intervention on changes in college major using administrative data from the Montana University System from 2002-2014. Our difference-in-difference-in-differences strategy exploits the relative trends for students at Montana State University above and below the cutoff for receiving a warning letter about their student debt, compared to their counterparts at the University of Montana. We find that students who receive information suggesting they may be unlikely to be able to repay their loans are more likely to switch to higher earning majors, with higher academic performers most likely to choose STEM fields.
---------------------
Returns to Education: The Causal Effects of Education on Earnings, Health and Smoking
James Heckman, John Eric Humphries & Gregory Veramendi
NBER Working Paper, May 2016
Abstract:
This paper estimates returns to education using a dynamic model of educational choice that synthesizes approaches in the structural dynamic discrete choice literature with approaches used in the reduced form treatment effect literature. It is an empirically robust middle ground between the two approaches which estimates economically interpretable and policy-relevant dynamic treatment effects that account for heterogeneity in cognitive and non-cognitive skills and the continuation values of educational choices. Graduating college is not a wise choice for all. Ability bias is a major component of observed educational differentials. For some, there are substantial causal effects of education at all stages of schooling.
---------------------
Why Do We Tenure? Analysis of a Long Standing Risk-Based Explanation
Jonathan Brogaard, Joseph Engelberg & Edward Dickersin Van Wesep
University of Colorado Working Paper, May 2016
Abstract:
Using a sample of all academics that pass through top 50 economics and finance departments between 1996 and 2014, we study whether the granting of tenure leads faculty to pursue riskier ideas. We use the extreme tails of ex-post citations as our measure of risk and find that both the number of publications and the portion that are “home runs” peak at tenure and fall steadily for a decade thereafter. Similar patterns holds for elite (top 10) institutions, for faculty with longer tenure cycles, and for promotion to Full Professorship. We find the opposite pattern among poorly-cited publications: their numbers steadily rise after tenure. The decline in both the quantity and quality of publications points to tenure incentivizing less effort in publishing rather than more risk-taking.
---------------------
Anton Lager et al.
International Journal of Epidemiology, forthcoming
Background: Cognitive and socio-emotional abilities are powerful predictors of death and disease as well as of social and economic outcomes. Education is societies’ main way of promoting these abilities, ideally so that inequalities by socioeconomic background are reduced. However, the extent to which education serves these cognitive, social-emotional and equality objectives is relatively unknown and intensively debated. Drawing on a Swedish school reform that was explicitly designed as a massive quasi-experiment, we assessed differential impact of education on intelligence and emotional control across childhood socioeconomic position. We also assessed initial differences in abilities by childhood socioeconomic position and how well childhood socioeconomic position and abilities predict all-cause mortality.
Methods: The Swedish comprehensive school reform, rolled out during the 1950s, extended compulsory education from 8 to 9 years in some municipalities whereas others were kept as controls for the sake of evaluation. We followed eight full cohorts of Swedish boys born between 1951 and 1958, who lived in 1017 municipalities with known experimental status (344 336 boys) and whose childhood socioeconomic position was known (320 182 boys). At conscription, intelligence was measured by four subtests and emotional control (calm and efficient responses in various situations) was rated by a military psychologist. Both measures were standardized to have a mean of 100 and standard deviation of 15. All-cause mortality was recorded until 49–56 years of age.
Results: The reform had an average positive impact on intelligence of 0.75 IQ units (95% confidence interval (CI): 0.54, 0.97; P < 0.0005). The impact on emotional control was negative; −0.50 units (95% CI: −0.72, −0.28; P < 0.0005). Both effects differed by socioeconomic background so that the average IQ difference between sons of high non-manual and unqualified manual workers was reduced from 16.32 to 15.57 units and the difference in emotional control was reduced from 6.50 to 5.63 units. All-cause mortality was predicted by low childhood socioeconomic position [hazard ratio (HR) = 1.15 [95% CI: 1.11, 1.20], P < 0.0005], low intelligence [HR = 1.39 (95% CI: 1.34, 1.44), P < 0.0005] as well as low emotional control [HR = 1.61 (95% CI: 1.55, 1.67), P < 0.0005] in mutually adjusted models.
Conclusions: Extending compulsory education promoted intelligence but lowered emotional control, and reduced disparities over social background in both. Emotional control was the strongest predictor of all-cause mortality. Our results are in line with the idea that education is important in our efforts to achieve healthy, competent and fair societies, but much more work is needed to understand the links between education and non-cognitive skills.
---------------------
Stephanie Riegg Cellini & Nicholas Turner
NBER Working Paper, May 2016
Abstract:
We draw on population-level administrative data from the U.S. Department of Education and the Internal Revenue Service to quantify the impact of for-profit college attendance on the employment and earnings of over 1.4 million students. We characterize both the within-student earnings effects and joint distributions of earnings effects and increases in student debt. Our descriptive analysis of degree-seeking students suggests that on average associate’s and bachelor’s degree students experience a decline in earnings after attendance, relative to their own earnings in years prior to attendance. Master’s degree students and students who complete their degrees appear to experience better outcomes, with positive earnings effects. Our difference-in-difference analysis of certificate students suggests that despite the much higher costs of attendance, earnings effects are smaller in the for-profit sector relative to the effects for comparable students in public community colleges—a result that holds for all but one of the top ten fields of study. In absolute terms, we find no evidence of improved earnings post-enrollment for students in any of the top ten for-profit fields and we can rule out that average effects are driven by a few low-performing institutions.
---------------------
The impact of education tax benefits on college completion
Mahmoud Elsayed
Economics of Education Review, August 2016, Pages 16–30
Abstract:
This paper uses a nationally representative sample from the 2004-09 Beginning Postsecondary Students (BPS) survey to examine the effect of education tax benefits on college completion. The paper employs a propensity score matching approach to correct for differences between eligible and ineligible students. Results suggest that tax benefits increase the likelihood of completing a college degree by 8 percentage points. The effect of tax benefits is largest for students who attended private and four-year institutions.
---------------------
Student Responsiveness to Earnings Data in the College Scorecard
Michael Hurwitz & Jonathan Smith
College Board Working Paper, April 2016
Abstract:
The release of the U.S. Department of Education’s College Scorecard in September 2015 has the potential to influence millions of students’ college decisions. Each college’s scorecard includes the college’s annual cost and graduation rate in addition to the previously unavailable median earnings. Using the universe of SAT score sends to colleges and the exact date on which these scores are sent, we estimate how students respond to the release of these quality metrics. We find that data on annual cost and graduation rate, both of which were previously available, did not impact the volume of score sends received by colleges. By contrast, we estimate that each 10 percent increase in earnings results in a 2.4 percent increase in score sends. The impact is driven almost entirely by well-resourced high schools and students.
---------------------
Brian King et al.
School Psychology Quarterly, forthcoming
Abstract:
The present study tested the efficacy of the On-Task in a Box program for increasing on-task behavior and academic accuracy of highly off-task students. Six students in 2nd and 3rd grades were identified by their classroom teacher as highly off-task. Following identification, the students participated in the On-Task in a Box intervention. Results of the study found immediate and large effects, which were maintained following discontinuation of the intervention. Collateral improvements in accuracy on math probes completed during independent seatwork were also observed. Teacher and participant responses to intervention acceptability questionnaires indicate the program was viewed positively. Implications for school-based adoption of the program are presented, and limitations and future research are discussed.
---------------------
Jennifer Heissel
Economics of Education Review, forthcoming
Abstract:
Over one million K-12 students pursue virtual education every year, but researchers know very little about the effectiveness of such programs. This paper exploits a district policy change that suddenly shifted advanced eighth graders into a virtual classroom for Algebra I. After the policy, higher-ability eighth graders in the treatment district began taking Algebra I in the virtual classroom at rates similar to the statewide average of their peers in traditional classrooms. The change in course delivery provides a unique opportunity to study effects of a virtual course on academic outcomes. The analysis uses variation in program uptake across performance quintile, district, and year in a difference-in-difference-in-difference approach to estimate the causal effect of the virtual course, finding that eighth grade virtual students tend to underperform relative to eighth graders who took Algebra I in a traditional classroom and relative to pre-policy, same-district students who had to take the course in ninth grade.
---------------------
Adam Looney & Constantine Yannelis
Brookings Papers on Economic Activity, Fall 2015, Pages 1-89
Abstract:
This paper examines the rise in student loan default and delinquency. It draws on a unique set of administrative data on federal student borrowing matched to earnings records from de-identified tax records. Most of the increase in default is associated with borrowers at for-profit schools, 2-year institutions, and certain other nonselective institutions. Historically, students at these institutions have constituted a small share of all student borrowers. These nontraditional borrowers have largely come from lower-income families, attended institutions with relatively weak educational outcomes, faced poor labor market outcomes after leaving school, and defaulted at high rates. In contrast, default rates have remained low among borrowers who attended most 4-year public and nonprofit private institutions and among graduate school borrowers — who collectively represent the vast majority of the federal loan portfolio — despite the severe recession and these borrowers’ relatively high loan balances. The higher earnings, low rates of unemployment, and greater family resources of this latter category of borrowers appear to have helped them avoid adverse loan outcomes even during times of hardship. Decomposition analysis indicates that changes in the characteristics of borrowers and the institutions they attended are associated with much of the doubling in default rates between 2000 and 2011, with changes in the type of schools attended, debt burdens, and labor market outcomes explaining the largest share.
---------------------
Mark Wiederspan
Economics of Education Review, April 2016, Pages 79–96
Abstract:
The degree to which students are able to make adequate repayments on their student loans and avoid default is of special concern for colleges. If too many former students go into default, the college will face sanctions by the federal government and lose eligibility to provide currently enrolled students federal financial aid, such as the Pell grant. To avoid these sanctions, some colleges have chosen not to participate in federal loan programs by excluding loans from students' financial aid packages. In this article, I investigate the student-level impacts associated with the decision of community colleges to opt out of the Stafford loan program. Utilizing administrative records from over 50 community colleges located in a single state, I estimate the within-college differences in outcomes for Pell-eligible students before and after an institution opts out of the federal loan program. I find that Pell-eligible students enrolling when the community college offered federal loans were 7.6 percentage points more likely to borrow than Pell-eligible students who enrolled when the institutions opted out. Overall borrowing also increased by $368 a year. I also find that students borrowing a loan attempted 19 additional credits in their first year of enrollment and were more likely to attempt and complete math and science courses than non-borrowers.
---------------------
Education loans and wealth building among young adults
Min Zhan, Xiaoling Xiang & William Elliott
Children and Youth Services Review, July 2016, Pages 67–75
Abstract:
With the use of education loans growing rapidly as a way to finance college education, it is important to examine how such loans impact the future financial well-being. This study examines the association between education loans and postcollege wealth accumulation among young adults, the group with the greatest share of outstanding education loans. Data come from 15 rounds of data of the 1997 National Longitudinal Survey of Youth, and the analyses control for a number of student characteristics, college experiences, and parental income. Results from a treatment-effects model indicate that having education loans upon leaving college is negatively related to postcollege net worth, financial assets, nonfinancial assets, and value of primary housing. Furthermore, having education loans also has an additional negative link to the value of net worth among Black young adults. The relationship between the amount of education loans and wealth accumulation is not statistically significant among those with outstanding loans. The study findings indicate the importance of developing alternative approaches, instead of additional loans and other credits, to meet the financial needs of college students.
---------------------
Di Xu & Madeline Trimble
Educational Evaluation and Policy Analysis, June 2016, Pages 272-292
Abstract:
The annual number of certificates awarded by community colleges has increased dramatically, but relatively little research has been conducted on the economic benefits of certificates in the labor market. Based on detailed student-level information from matched college transcript and employment data in two states, this article estimates the relationship between earning a certificate and student earnings and employment status after exiting college. Our results indicate that certificates have positive impacts on earnings in both states overall, and in cases where there is no impact on earnings, certificates may nonetheless lead to increased probability of employment. In addition, we find substantial variation in the returns across fields of study and, more importantly, across specific programs within a particular field. Finally, in-depth analysis of the industry of employment before and after college enrollment indicates that many adult learners use certificate programs to switch to a new industry, which may not necessarily boost their earnings, at least in the short run. Our results therefore point to the importance of including multiple measures to evaluate the benefits of a certificate program, rather than merely evaluating its impact on overall earnings.
---------------------
Are College Costs Worth it? How Ability, Major, and Debt Affect the Returns to Schooling
Douglas Webber
Economics of Education Review, forthcoming
Abstract:
This paper examines the financial value over the course of a lifetime of pursuing a college degree under a variety of different settings (e.g. major, student loan debt, individual ability). I account for ability/selection bias and the probability that entering freshmen will not eventually graduate. I find the financial proposition of attending college is a sound investment for most individuals and cost scenarios, although some scenarios do not pay off until late in life, or ever. I estimate the present discounted value of attending college for the median student to vary between $85,000 and $300,000 depending on the student’s major. Most importantly, the results of this paper emphasize the role that risk (e.g. the nontrivial chance that a student will not eventually graduate) plays in the cost-benefit analysis of obtaining a college degree.
---------------------
Determinants of Organizational Failure in the Milwaukee School Voucher Program
Michael Ford & Fredrik Andersson
Policy Studies Journal, forthcoming
Abstract:
In this article, we utilize 25 years of data on the Milwaukee voucher program to test the relationships among organizational liabilities of newness, institutional affiliation, market-share, and regulatory environment on the cumulative risk of school failure. Overall, we find that 41 percent of all private voucher schools operating in Milwaukee between 1991 and 2015 failed. Start-up voucher schools, and those unaffiliated with a larger institution, have comparatively higher failure risk over time. The results of our analysis shed light onto one of the long-term impacts of a public policy reform premised on social entrepreneurship. We conclude that policymakers should consider the consequences of organizational failure when utilizing nongovernment entities in the provision of public goods, and that scholars focus increased attention to understanding the negative externalities created by public entrepreneurship.
---------------------
Anca Gaston, Sherri Moore & Leslie Butler International
Journal of Educational Research, 2016, Pages 136–142
Abstract:
This study used a prospective matched case-control design to examine the effects of sitting on a stability ball on inattention, hyperactivity, oppositional defiant behaviours, and anxious/depressive symptomatology among 23 experimental and 18 control grade 2 students. Classroom teachers completed the NICHQ Vanderbilt Assessment Scale at baseline and 8-weeks (T2) and 5-months (T3) after the experimental group switched to stability balls. Social validity was assessed at year-end. ANCOVAs controlling for baseline scores demonstrated that students in the experimental condition had improved attention at T2 and T3 and reduced anxious/depressive symptoms at T2. All students and the classroom teacher preferred the balls. In conclusion, sitting on stability balls is well received and may represent an effective classroom management strategy for improving attention.
---------------------
A day at the museum: The impact of field trips on middle school science achievement
Emilyn Ruble Whitesell
Journal of Research in Science Teaching, forthcoming
Abstract:
Field trips are an important feature of the United States’ education system, although in the current context of high-stakes tests and school accountability, many schools are shifting resources away from enrichment. It is critical to understand how field trips and other informal learning experiences contribute to student test scores, but little research has explored the impact of field trips on standard measures of academic learning. In this study, I used 6 years of student-level data to estimate the impact of field trips to informal science education institutions on New York City students’ performance on the state's standardized eighth-grade science exam. Using a rigorous identification strategy with school fixed effects to capitalize on variation in field trip participation within schools over time, I found small positive effects of exposure to field trips on students’ science test scores and proficiency. Effects were largest for Hispanic students and those who qualified for free or reduced-price lunch. This evidence that field trips can contribute positively to student achievement is meaningful for policymakers and administrators, as it suggests schools can provide enrichment experiences that families desire without sacrificing student test scores. The study suggests that field trips might be an effective tool for reducing achievement gaps.
---------------------
Paul von Hippel et al.
Economics of Education Review, August 2016, Pages 31–45
Abstract:
Sixteen US states have begun to hold teacher preparation programs (TPPs) accountable for teacher quality, as estimated by teacher value-added to student test scores. Yet it is not easy to identify TPPs whose teachers are substantially better or worse than average. The true differences between TPPs are small; the estimated differences are not very reliable; and when many TPPs are compared, multiple comparisons increase the danger of misclassifying ordinary TPPs as good or bad. Using large and diverse data from Texas, we evaluate statistical methods for estimating teacher quality differences between TPPs. The most convincing estimates come from a value-added model where confidence intervals are widened by the Bonferroni correction and by the inclusion of teacher random effects (or teacher clustering in large TPPs). Using these confidence intervals, it is rarely possible to tell which TPPs, if any, are better or worse than average. The potential benefits of TPP accountability may be too small to balance the risk that a proliferation of noisy TPP estimates will encourage arbitrary and ineffective policy actions.
---------------------
School Reforms and Pupil Performance
Andrew Eyles, Claudia Hupkau & Stephen Machin
Labour Economics, forthcoming
Abstract:
The relationship between school reforms, specifically those involving the introduction of new school types, and pupil performance is studied. The particular context is the introduction of academy schools in England, but related evidence on Swedish free schools and US charter schools is also presented. The empirical evidence shows a causal positive impact of the conversion of disadvantaged schools to academies on end of school pupil performance and on subsequent probability of degree completion at university. There is heterogeneity in this impact, such that more disadvantaged pupils and those attending London academies experience bigger performance improvements.
---------------------
School District and Housing Price Responses to Fiscal Stress Labels: Evidence from Ohio
Paul Thompson
Journal of Urban Economics, July 2016, Pages 54–72
Abstract:
This paper examines the effect of the Ohio fiscal stress labeling system on school district outcomes and housing prices. Under this policy, financially troubled districts are labeled and required to implement financial recovery plans. In response to these plans, districts increase local tax revenues and decrease capital and operating expenditures. Although these recovery plans lead to better long-term financial health for school districts, there appear to be some negative impacts on welfare in these districts during the duration of the label. I find that residential home sale prices fall following fiscal stress label receipt, but rise again once the label is removed. These districts also undergo substantial restructuring, including reductions in enrollments, teachers, and schools, which coincide with a transitory reduction in math proficiency rates following label receipt.
---------------------
The Marginal Effect of K-12 English Language Development Programs: Evidence from Los Angeles Schools
Nolan Pope
Economics of Education Review, forthcoming
Abstract:
The growing number of K-12 non-native English speaking students increases the value of optimizing education policy to meet their academic needs. Using a regression discontinuity in test scores from the Los Angeles Unified School District, I analyze the optimal age and English proficiency level for students to enter and exit English language development (ELD) programs. I find marginal kindergarteners receive small academic gains from entering ELD classes. Marginal 2nd to 4th graders who are reclassified from ELD to mainstream English classes receive large benefits to their English test scores (0.25 SD) and GPA that persist over the next 7 years. Boys receive the majority of the benefits. I find no evidence that students reclassified in later grades receive any benefit. Achievement gains can be obtained by enrolling more students into ELD programs in kindergarten and choosing to transition them into mainstream English classes sooner.