A couple of years ago — spurred by the helpful questions of a commentator on this blog — I tried to figure out whether the University of Miami School of Law was reporting starting salary data about its graduates in a full, fair, and honest manner. I thought now would be a good time to revisit that question, using the law school's presentation of new data on last year's class, a group suffering the worst legal hiring recession since the Great Depression.
Two years ago, the short answer to my question about whether the data was presented accurately turned out to be yes, but only if you read the data very, very carefully, more carefully than one would expect of even a reasonably prudent law applicant not trained in statistics.
The slightly longer version of the old answer was that then and now UM complies with external reporting standards set by the ABA and NALP and that these reporting standards tend to mask some realities about starting salaries. UM nonetheless adheres to them because a failure to do so would (1) Mean our numbers were reported with an asterisk, making it look like we have something to hide; (2) Make the UM numbers no longer comparable with other schools' numbers; (3) Put UM at a terrible competitive disadvantage.
The even longer version of the old answer (skip down if you want the new stuff) was this, lifted from More About Starting Salaries:
According to Career Development Office, the reason why the both the $104,500 number [for average starting salary for those employed by firms] and the more detailed but somewhat different pie charts accompanying it [which, based on firm size, suggested a lower number] are accurate has to do with response rates, differing data sets, and national reporting standards.
Not everyone who responded to the law school's survey about what they were doing immediately after graduation chose to disclose their salary. Thus, the charts about firm size, for example, are based on a bigger data pool than the salary number. In 2007 we had 378 JDs. Of that group, 346 had replied to our survey at the time the Viewbook was produced. Of that 346, however, not all worked for firms — and of the group that worked for firms only about 46% gave us salary data. So the average salary number of $104,500 is based on the data provided by that 46%.
Since firm size and starting salary are related, you might reasonably object — as I did — that it would be more reasonable to pro-rate the responses of the people who gave salary data on the assumption that the people who didn't fill in that part of the survey earned similar amounts by comparable firm size. And I still think there's something to that. But I'm told by the Career Office — and I believe them — that the average salary data is presented the way it is because that's how all law schools do it and the goal is to provide prospective students with numbers that can fairly be compared to what is provided by other law schools.
The Career Development Office avers that it collects the data and reports it in accordance with ABA and NALP guidelines, using the same methods that every other accredited law school in the country uses. Were the law school to do something else, the administration notes, it would no longer be reporting to students in the way it reports to the ABA and NALP. That would mean our data would have an asterisk. And even if we were doing it in order to provide better data the inevitable conclusion that most people would draw is that we were trying to hide something. So the Catch-22 is that we have to do it this way, possibly sacrificing some statistical excellence and even accuracy, or else we'll look like we're engaged in some sort of cover-up. And, of course, in addition to having an asterisk, we'd be harming our competitive position since we'd have gone to some trouble to calculate and report a lower number which would harm marketing and recruiting.
Well, here we are now in a very bad legal hiring year, and U.M has again provided some employment data for 2009 graduates that, on first blush, looks somewhat cheery:
In this latest report, the median salary for a UM law grad employed in a firm of two persons or more is $112,500 and the average ($111,696) is almost the same. Two years earlier the average was $104,500. So things are going great, right?
Well. Not so fast.
The critical statistic may be the reporting percentage in the fine print. The salary data in the 2008 edition of the Viewbook was based on 46% participation of the graduates (although the firm size info was based on a 91.5% response rate of those employed by firms). The 2009 salary data has only a 33% response rate. In 2008 the percentage of graduates providing salary info was 39%, and in 2007 it was 40%. Human nature suggests that the people who have good jobs are more likely to respond to questionnaires about their fortunes than those who are struggling. So even though both the median and average in the new data may be higher, the lower response rate sets that off, indeed might more than set that off, depending on how badly the missing 11% are doing. (One might hypothesize from these numbers that some students are still getting high-paying jobs, and that a lot of others are suffering, but without more data it is hard so say with confidence that this is what is actually going on.)
Despite this tendency to over-cheeriness, the law school's presentation of the data in the new graphic is, I think, significantly superior to the old version that I was unhappy with. The fine print at the bottom isn't actually that tiny and is much more explicit about where the numbers come from, it includes the reporting rate, and it also gives a median and average for all respondents in all jobs (median 90K, average 95K). Given the double-humped curve that faces graduates who get jobs (see Starting Salaries For Law Students are BiModal — If Not Bipolar), with one peak well over $110K and other peak somewhere in the $60Ks, both that median and average conceal as much as they reveal, but it is still better information, more clearly presented, than the law school made available two years ago. I believe that a prospective law student should be able to understand what's going on much more clearly than was the case two years ago, where frankly it took far too much effort to work it all out.
If I were asking for the moon, I'd want time series data, especially regarding response rates, and a comparison to some regional and national averages, but I don't know where we'd get that aggregate information, and for balance you'd probably want to have something showing general economic conditions too. Or, maybe, how about a second set of pie charts with 2/3 in light gray showing “we don't know” in order to make clearer the limitations of our data?
Here is what we do know about the class of 2009, as reported recently to the ABA:
The very worst face you can put on this, counting all non-respondents as unemployed, is that at nine months after graduation, 10% of the class 2009 didn't have a job, and 5% only worked part time. (The situation with the 5% of graduates working in non-legal employment is very hard to figure out. Those few who reported their salaries all reported amounts from good to great.)
As regards the 90% of the class (95% of respondents) whom we know have some sort of job, we have the salary distribution data in the charts above, but it is based on what only four out of ten with jobs, 36% of the total graduates, were willing to report. And, again, human nature suggests that those reporting may not be representative of the whole.
I should add that I'm genuinely curious and uncertain about the state of the market for this coming year's graduates, as anecdotally I'm hearing contradictory things. Students are clearly worried. The national news is still quite grim. The law school is sufficiently concerned about its graduates' prospects to create an on-the-job post-graduate training program, the Legal Corps. (See Meet the UM Law Legal Corps Fellowship Program and More on the Legal Corps.) On the other hand, a completely unscientific sample of local partners mostly me tell things are up a bit from last year's trough. I thought more rising 3Ls had summer jobs last summer than I expected. Jobs-in-hand for the 3Ls right now is not particularly great, although I do know a number of strong students with jobs nailed down. Rumor has it that the number of applicants for the Legal Corps program is substantially less than expected (or feared). Complicating matters, one of the oddities about the South Florida jobs market is that even in boom years only the very biggest firms have tended to extend employment offers before graduates pass the bar. In so doing they save themselves the expense of carrying a nonlawyer at lawyer pay rates, they don't have to contribute towards studying for the bar, and they avoid the awkwardness of dealing with someone who fails. So the most meaningful measuring point for the outcomes for this year's graduates, the class of 2011, remains far in the future.
As to the question I posed at the start of this entry — is the law school reporting starting salary data about its graduates to prospective students in a full, fair, and honest manner? — the answer is a less-qualified yes than previously, but still not an unqualified yes. Despite what must be pressure to make things look good when times are bad, the overall quality of the law school administration's presentation of the data and the fullness of the report both have improved. Unfortunately, the recent graduates' response rate on salary has dropped — a fact you could only figure out from the pie charts if you compared them to earlier years' reports — and although this is not the law school's fault, the consequence is that although the employment data look to me to be very reliable, the salary data likely give a rosier picture of the distribution than one would guess to be the actual case. Caveat emptor — and not just at UM.