By Aaron Olanie and Kjell Christophersen
In a time of unprecedented scrutiny on the value of higher education, the American Institute for Research (AIR) and Nexus Research and Policy Center released a study estimating the student and taxpayer return on investment (ROI) of an associate’s degree for 579 institutions.
Due to a number of data limitations, the authors make various assumptions that both overestimate the costs of producing associate’s degrees and underestimate the benefits, lowering both student and taxpayer ROIs. Unfortunately, absent these data limitations or caveats, important context is lost. To the authors’ credit, they generally make these limitations explicit in footnotes throughout the main study and appendices. Because of the limitations, however, we fear lay audiences, or even seasoned economists, are likely to draw misleading conclusions about the taxpayer and student ROIs reported in this study. The purpose of these comments is to provide the needed context and make clear where and how the failure of accounting for the limitations will generate far lower- than-expected ROIs.
The AIR study does not estimate a taxpayer ROI for their investment in the college as a whole, i.e., the entire student body regardless of what the students achieve academically during the analysis period. Nor does the study estimate an ROI for the average student. To be sure, some earn their associate’s degree and some earn certificates, yet many more (in fact the large majority of students) attend but don’t earn any degrees or credentials.
Whereas community colleges serve a wide variety of students, the authors limit their focus to associate’s degree-seeking students that complete within three years. Certificate, transfer, non-credit, and workforce students are not considered. The authors make this very clear. However, within the sample of students completing an associate’s degree in three years, there are students that continue pursuing their education and receive a bachelor’s degree. In footnote 41, the authors reveal they are likely underestimating the earnings gained from attending community college by excluding the earnings of these students, which make up approximately 14 percent of their sample.
Due to a number of data limitations, the authors make various assumptions that both overestimate the costs of producing associate’s degrees and underestimate the benefits, lowering both student and taxpayer ROIs. Unfortunately, absent these data limitations or caveats, important context is lost.
The second half of footnote 41 brings us to our next point. Here they assume students who do not earn an associate’s degree continue to earn the high school level wages. However, even students who do not complete degrees have still learned valuable skills they can apply in the workplace, and thus they command higher wages. Moreover, students in technical fields often drop out and take jobs before obtaining their associate’s degree because employers value the skills obtained more than the degrees. Again, the authors acknowledge this creates a significant downward bias on their estimated benefits and subsequently the estimated ROIs.
To estimate the student ROI of an associate’s degree, benefits (higher earnings) and costs must be estimated. Costs consist of tuition and fees, books, and the forgone wages students could be earning if they were not pursuing their education. The forgone wages — generally the most significant cost — represent the opportunity cost of the associate’s degree. The authors’ estimation of the opportunity cost is particularly troublesome.
The first problem stems from the fact that approximately 75 percent of community college students work while attending college. The wages earned while attending college must be accounted for when estimating the opportunity cost. In footnote 34, the authors assert the forgone wages used to estimate the opportunity cost of attaining an associate’s degree should equal the difference between the wages these students earn while attending college and the average high school earnings. This means only part of the high school earnings should be claimed as forgone earnings. The authors, however, did not take this approach. Instead, they argued that because they could not reliably estimate the average earnings of students while attending college, it is better to treat the entire average high school earnings as forgone earnings. In short, they assume community college students don’t work while completing their degree, which is far removed from reality. Again, the authors recognize this shortcoming, and therefore that they overestimate the cost of the associate’s degree, resulting in a lower ROI for students.
A second problem is that the average state-level high school earnings are not regionalized. These earnings are compared to the average earnings of graduates from specific colleges. This is important because the regional economy that community colleges serve can vary drastically from region to region within each state. A community college may be located in a region where average earnings are considerably lower than the state average. The actual earnings potential of students in this region with a high school diploma may be significantly below the state average. This has two effects: (1) It overstates the forgone earnings and cost of an associate’s degree, and (2) it underestimates the gain in earnings potential of an associate’s degree. Both effects create a downward bias on student and taxpayer ROI. The authors recognize this downward bias in footnote 46.
Turning to the taxpayer ROI, we flag three fundamental problems: First, the college financial data used to estimate taxpayer costs does not discount for the costs related to producing certificates. As the authors note in footnote 103, this overestimates the cost of producing associate’s degrees. Second, the costs of students that do not complete their degree are included, but none of their benefits. The goal of the study was to estimate the ROI of an associate’s degree for both students and taxpayers. We question the choice to include these costs given this scope. Had the study set out to estimate a taxpayer ROI of the college as a whole, there would be no question these costs must be included as well as the benefits accrued to all of the other categories of students not receiving a credential. Finally, in footnote 28 the authors recognize they ignore any broader societal benefits of attending college.
To recap, the study has the following biases:
1. Students that receive an associate’s degree and continue their education are not considered. This underestimates the earnings gained from an associate’s degree and reduces student and taxpayer ROIs
2. The authors assume all students in their sample who do not complete an associate’s degree only earn high school level wages. And they assume no students work while attending college, even though they acknowledge around 75 percent of community college students do work while attending college. This overestimates student costs and underestimates student and taxpayer benefits of an associate’s degree, lowering both student and taxpayer ROI.
3. The high school wages used are state averages. If a college is located in a region with high school level wages below the state average, the student cost will be overestimated and the student and taxpayer benefits will be underestimated, lowering both student and taxpayer ROI. The opposite is true for areas with wages above the state average.
4. The taxpayer costs of an associate’s degree include the costs of producing certificates. This overestimates the taxpayer costs and reduces their ROI
5. The taxpayer costs of students that dropout and do not complete an associate’s degree are included. This overestimates the taxpayer cost and reduces their ROI.
6. Social benefits are ignored. This underestimates the taxpayer benefits and reduces their ROI.
While we must give the authors credit for acknowledging the above biases, we hope their future work will make these types of shortcomings explicit in the text rather than relegating them to footnotes. We are confident the study’s authors will agree that stakeholders should take these biases into consideration when drawing conclusions from the study’s results. Without the above biases, the student and taxpayer ROIs would be substantially higher. Policies informed by such results would likely be quite different than policies based on the results presented in the AIR study.
For questions or more information, please feel free to contact us.