Bar Pass Rates are Over-Rated As A Measure of Law School Quality

The Florida Bar has released the bar pass rate scores for first-time test takers in the July 2006 bar exam. As you can see from the table below UM did “well” compared to other law schools in the state:

FSU, 88.1
University of Miami, 85.7
University of Florida, 81.2
Stetson, 81
Florida International, 79
Florida Coastal, 75.4
Nova Southeastern, 74.9
Barry, 72
St. Thomas, 63.1
Florida A&M, 56.3

Non-Florida law school graduates, 71.6

Our “good” score this year means that this is an appropriate time to post on why bar pass scores are much less meaningful as indicators of law school quality than they may seem — there’s less chance it will seem like sour grapes.

Let me start by saying that there certainly comes a point where a substantially lower bar pass rate than other schools in the state is a sign of a problem that a law school should work to fix. Most people come to law school in order to become lawyers. If they can’t pass the bar, at least on second try, in most cases they have wasted large amounts of time and money. If this is happening to a substantial fraction of the class, and it isn’t happening nearly as much in other law schools in the same state, then something is wrong either with the teaching, the work ethic, or with the admission policy. Note that the latter may not be the school’s direct fault: as there are more and more law schools it becomes increasingly likely that some schools simply are unable to attract enough students with enough discipline or talent, which puts pressure on the school to either teach to the bar, or to flunk a greater fraction of the entering students.

We’d love everyone to pass; realistically, not everyone studies hard enough for an exam that is three parts memorization, two parts issue spotting, and one part legal reasoning. How low is too low a bar pass rate? Reasonable people could disagree as to exactly where that line is. In part, the answer varies by state, since different bar examiners score in different ways. It’s also important to note that in recent years many states (including Florida) have substantially raised the minimum score needed to pass the bar. Thus today’s scores in Florida are not comparable to scores a decade ago, as people are failing now who would have passed then.

That said, there are two sets of reasons why bar pass scores don’t tell you as much as you might think they do.

First, the scores quoted above — the only ones readily available to us or anyone — reflect only in-state takers. The more you run a national program, the more your students take bar exams in the other 49 states (and the District of Columbia) where they hope to practice. Quite commonly, these out-of-state test takers include many of the best and most motivated students — it takes more work and often better grades to get a job far from where you studied. So, for the better law schools there is an element of selection bias in the in-state data that is relatively absent for the schools whose students have less chance of out-of-state work.

Second, law schools differ enormously in the extent to which they teach to the bar exam. Any law school with even minimal pretensions to grandeur is going to concentrate on teaching legal reasoning, legal theory, legal history, and just about anything other than the rote learning that occupies the bulk of the bar exam. And, the more you run a national program drawing students from all over, the less you want to teach them even predominantly the law of one state where so many of them will not be practicing. (Indeed, I’d argue that even if you knew that all your grads would practice in one state you would still need to teach national principles since most practices will involve issues that cross state boundaries.) A three year bar review course may raise your pass rate, but it will not train nearly as good or as flexible lawyers as a program that stresses critical thinking and analysis.

The downside of a quality program is that it fails in substantial part to teach to the bar — an exam which at best tests a small fraction of the things that it takes to be a good lawyer, and which many would say doesn’t even do that. And that means the students will have to take a cram course after they graduate to memorize all the rote rules they didn’t learn in law school. Better a six-week cram course than a three-year one, I’d say. (Indeed, my law school, Yale, was notorious for teaching nothing relevant to the bar at all, a reputation that only somewhat exaggerated the reality. A consequence was that I actually enjoyed a lot of the bar review course as it contained concepts and material wholly new to me — spendthrift trusts! what a concept! But I digress.)

In summary, and strange as it may seem, a very high bar pass rate is not necessarily the sign of a great law school; it may even be the sign of too much emphasis on cramming, and not enough on learning to think like a lawyer. Conversely, there comes a point where the bar pass rate is so low that it is a sign that the law school is doing something wrong — admitting the wrong people, failing to create the right work ethic, not teaching some fundamental principles of law, or something — but unless one is flirting with that uncertain point, one really shouldn’t make too much of relatively small annual fluctuations in a school’s bar pass rate or of relatively small differences between schools.