Rankings, Money, and BYU

Money magazine has just released a new ranking of U.S. universities that has received a bit of attention. BYU does quite well, landing in ninth place overall, just behind Stanford, Harvard, Harvey Mudd, and Cooper Union. As BYU is a topic that seems to attract more than its share of angst and acrimony, it’s nice to see some evidence that it’s succeeding in its mission.

The criteria and methodology used by Money are those that will warm an accountant’s heart, being based on data, graduate earnings, and value-added outcomes rather than reputation. The authors attempted to factor in that schools serve different types of students, and that students pursue different kinds of careers. Charging wealthy, well-prepared students top dollar for business degrees so that they can go on to high-flying careers in finance is not supposed to be automatically rewarded more than educating low-income students and setting them on the path to middle-class careers without leaving them drowning in debt. The unusual mix of schools that show up in the Money top ten suggest that the authors have only partially succeeded; they are evenly split between elite universities who have long sent grads on to high-paying careers, STEM-focused schools like MIT and Harvey Mudd, and small (or even tiny) schools with a narrow academic focus. And then there’s BYU.

Besides being a Mormon school, BYU stands out in a few ways. It’s one of the least expensive schools, both in terms of net price and its graduates’ average debt. Among the top ten, BYU also stands out for its sheer size: It enrolls as many students as the all of the schools ahead of it in the rankings combined. Larger schools don’t appear until #24 and #28 (Texas A&M and the University of Florida, respectively), suggesting that BYU is something like the flagship state university of Deseret, more comparable to a large public university than to other private religious schools.

Rankings, of course, are much like elections: Pick a desired outcome, and someone can find a rankings system that will return the desired result. BYU does well on the Money rankings because its institutional priorities are a fairly good match for the authors’ criteria, and BYU has been consistent in pursuing those institutional priorities. If you read through the Money authors’ methodology, you can see where BYU likely did better or worse than other schools.

Quality of Education. The Money rankings use several different proxies here. Graduation rates (weighted 50%) are probably a wash for BYU due to many students taking time out for missionary service. (The study did take well-established work-study programs into account, but it’s not clear if students interrupting their education for 18 or 24 months of missionary service were treated the same way. It will be interesting if the newly lowered missionary age will eventually lead to more students starting college after their missions rather than interrupting their educations, leading to higher 4- and 6-year completion rates for BYU.) On the other hand, the average ACT/SAT score of incoming BYU freshmen (15%) is quite respectable. BYU’s student/faculty ratio (10%) isn’t exceptional, but BYU may score well for “quality of professors” (5%, based on non-easiness, non-hotness data supplied by Ratemyprofessors.com; the highest-scoring RMP faculty member in the nation was at BYU until a few years ago). One area where BYU does do quite well is yield (20%), the percentage of accepted students who actually enroll in classes, as many BYU applicants are highly motivated to attend BYU as their first choice. (The study authors admit that these are indirect and imperfect measures. For student/faculty ratios, I would prefer surveys to consider only tenure-track rather than contingent or part-time faculty.) While quality of education is difficult to measure, I would expect that the average BYU academic program is little different from the average program among, say, the USN&WR top 100 national universities.

Affordability. BYU is incredibly inexpensive compared to other universities, and it probably owes much of its overall success to this category. A longer time to degree may increase the net price (weighted 40%) somewhat, depending on how missions are being treated. The figure of $82,000 for four or even five years of tuition plus room and board in Provo strikes me as somewhat high; it would be interesting to see the full data table used by the Money authors. BYU also does well on measures of student or parental debt (40% combined) and debt default rates (20%).

Outcomes. I expect that BYU did similarly to schools with students of similar academic backgrounds in this category, which is based on how well the career services center is staffed (10%) and, for the bulk of the weighting, on various ways of looking at early- and mid-career income data from PayScale.com. One drawback BYU faces is geography. BYU grads have the lowest salaries among the top ten schools, but BYU is also the only school not located in California or on the east coast. As the authors admit, they did not factor in cost-of-living data, so graduates who find high-paying jobs in expensive areas (like the coasts) may not actually being doing as well as lower-earning graduates in cheaper areas (like Santaquin). The Money rankings also ignore students who go on to graduate and professional programs rather than beginning careers immediately after graduation, and the effect of that omission on BYU is hard to estimate.

As a way to sell magazines, rankings are highly effective. As conveyors of useful information, they depend on the soundness of their data and methodology, and one could spend a long time criticizing both for the Money rankings. The future of college rankings, however, will probably look a lot like what appears in Money: data-driven, focused on value added outcomes, worried more about net tuition price and student debt than about markers of academic prestige. A large state university offering a broad palette of academic programs might learn something useful by selecting BYU as a peer and taking a look at what it’s doing well.