Bar Pass Rates are Over-Rated As A Measure of Law School Quality

The Florida Bar has released the bar pass rate scores for first-time test takers in the July 2006 bar exam. As you can see from the table below UM did “well” compared to other law schools in the state:

FSU, 88.1
University of Miami, 85.7
University of Florida, 81.2
Stetson, 81
Florida International, 79
Florida Coastal, 75.4
Nova Southeastern, 74.9
Barry, 72
St. Thomas, 63.1
Florida A&M, 56.3

Non-Florida law school graduates, 71.6

Our “good” score this year means that this is an appropriate time to post on why bar pass scores are much less meaningful as indicators of law school quality than they may seem — there’s less chance it will seem like sour grapes.

Let me start by saying that there certainly comes a point where a substantially lower bar pass rate than other schools in the state is a sign of a problem that a law school should work to fix. Most people come to law school in order to become lawyers. If they can’t pass the bar, at least on second try, in most cases they have wasted large amounts of time and money. If this is happening to a substantial fraction of the class, and it isn’t happening nearly as much in other law schools in the same state, then something is wrong either with the teaching, the work ethic, or with the admission policy. Note that the latter may not be the school’s direct fault: as there are more and more law schools it becomes increasingly likely that some schools simply are unable to attract enough students with enough discipline or talent, which puts pressure on the school to either teach to the bar, or to flunk a greater fraction of the entering students.

We’d love everyone to pass; realistically, not everyone studies hard enough for an exam that is three parts memorization, two parts issue spotting, and one part legal reasoning. How low is too low a bar pass rate? Reasonable people could disagree as to exactly where that line is. In part, the answer varies by state, since different bar examiners score in different ways. It’s also important to note that in recent years many states (including Florida) have substantially raised the minimum score needed to pass the bar. Thus today’s scores in Florida are not comparable to scores a decade ago, as people are failing now who would have passed then.

That said, there are two sets of reasons why bar pass scores don’t tell you as much as you might think they do.

First, the scores quoted above — the only ones readily available to us or anyone — reflect only in-state takers. The more you run a national program, the more your students take bar exams in the other 49 states (and the District of Columbia) where they hope to practice. Quite commonly, these out-of-state test takers include many of the best and most motivated students — it takes more work and often better grades to get a job far from where you studied. So, for the better law schools there is an element of selection bias in the in-state data that is relatively absent for the schools whose students have less chance of out-of-state work.

Second, law schools differ enormously in the extent to which they teach to the bar exam. Any law school with even minimal pretensions to grandeur is going to concentrate on teaching legal reasoning, legal theory, legal history, and just about anything other than the rote learning that occupies the bulk of the bar exam. And, the more you run a national program drawing students from all over, the less you want to teach them even predominantly the law of one state where so many of them will not be practicing. (Indeed, I’d argue that even if you knew that all your grads would practice in one state you would still need to teach national principles since most practices will involve issues that cross state boundaries.) A three year bar review course may raise your pass rate, but it will not train nearly as good or as flexible lawyers as a program that stresses critical thinking and analysis.

The downside of a quality program is that it fails in substantial part to teach to the bar — an exam which at best tests a small fraction of the things that it takes to be a good lawyer, and which many would say doesn’t even do that. And that means the students will have to take a cram course after they graduate to memorize all the rote rules they didn’t learn in law school. Better a six-week cram course than a three-year one, I’d say. (Indeed, my law school, Yale, was notorious for teaching nothing relevant to the bar at all, a reputation that only somewhat exaggerated the reality. A consequence was that I actually enjoyed a lot of the bar review course as it contained concepts and material wholly new to me — spendthrift trusts! what a concept! But I digress.)

In summary, and strange as it may seem, a very high bar pass rate is not necessarily the sign of a great law school; it may even be the sign of too much emphasis on cramming, and not enough on learning to think like a lawyer. Conversely, there comes a point where the bar pass rate is so low that it is a sign that the law school is doing something wrong — admitting the wrong people, failing to create the right work ethic, not teaching some fundamental principles of law, or something — but unless one is flirting with that uncertain point, one really shouldn’t make too much of relatively small annual fluctuations in a school’s bar pass rate or of relatively small differences between schools.

5 thoughts on “Bar Pass Rates are Over-Rated As A Measure of Law School Quality

  1. Do you think that the calibur of student that surrounds other students (leading to greater level of ‘discourse’ in and outside of class) is an important factor when ranking lawschools? Meaning, that schools who recruit the top students, the ones that are motivated and work harder, potentially have a better enviroment for learning and thinking?

    If I look at the top-ten schools, they are all near 100% bar passage rates (mid-90′s). And, this is in state bars not of their state (mostly NY bar). What does this tell us?

  2. Do you think that the calibur of student that surrounds other students (leading to greater level of ‘discourse’ in and outside of class) is an important factor when ranking lawschools? Meaning, that schools who recruit the top students, the ones that are motivated and work harder, potentially have a better enviroment for learning and thinking?

    If I look at the top-ten schools, they are all near 100% bar passage rates (mid-90′s). And, this is in state bars not of their state (mostly NY bar). What does this tell us?

  3. What research I’ve seen shows that an individual’s likelihood of passing the bar is highly correlated with entry LSAT and grades. Your observation as to the ‘caliber’ of the students admitted being related to bar passage is consistent with that finding. I think that both data sets reflect work habits, speed, test-taking ability — and, to some extent, legal aptitude. It also suggests that law schools in general do not differ much in their value-added as regards preparing students for the bar — which may be a feature rather than a bug given what I said above.

    Independently of that, I am sure that all other things being equal — on average, at least, as circumstances differ — there is value in going to a law school where one will be surrounded by happy, high-performing students. I suspect that the effect is most pronounced in extra-curricular activities, although it must bleed into the classroom sometimes as well. Of course, if the nature of the student body begins to affect the actual curriculum — e.g. if the law school gets so worried about the bar pass rate that it goes into bar prep mode rather than lawyer-training mode — then the effect on education is much more serious.

  4. Thanks for shooting my hopes out of the sky for relying on numbers to speculate whether I passed the bar. Where I graduated from has a 93% pass rate in the state I took the bar, but I was an “average” student based on our grade curve, and only did moderately well on the LSAT. Guess I should start my application for the February exam.

  5. The last time I looked — which was a long time ago, before our students’ credentials had strengthened to their present state — the bottom 10% or so of our class had a substantially greater risk of failing the bar than did the rest of the class. Quite often this seemed to be due to the same life distractions which had harmed their grades: childcare, sick relatives, demanding jobs. An awful lot of passing the bar is putting in the relentless hours it takes to memorize all that stuff; it’s hard to pass if you can’t throw yourself into it pretty hard.

    But cheer up! If you were “average” in a state where most people pass, that sounds like pretty good odds. And even if you were not, if you studied hard, remember that at the individual-level correlations are not destiny.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Notify me of followup comments via e-mail. You can also subscribe without commenting.