The US News Law School rankings are leaking out. Some years we get a lower score than the year before, and then I think I shouldn't carp about the whole thing for fear of it looking like sour grapes. Some years we get a higher score than the year before, and then I carp.
The idea of ranking law schools is not ridiculous. The way US News does it is very ridiculous. The survey data relies on the opinions of people who in most cases may be very informed about a few law schools but as a class are not likely to be particularly well informed about many law schools — even though they may be judges, hiring partners, law Deans and professors. And increasingly the survey data is self-referential: people have heard school X has a high/low ranking, so it must be good/bad, right?
At its grossest level, there is no doubt US News captures something real: the top N schools (10? 14? 15? 20? 20+?) really are better than the middle N or lowest N. But are the middle N significantly better than the bottom N? Sometimes, yes, but only sometimes. Here the picture gets very cloudy — not least because “better” ought to be “better for whom”; once you get away from the most elite, best resourced (i.e. high endowment), most prestigious law schools, what is best depends on factors that are personal: urban/rural, North/South, East/Middle/West, large/small, best in town/best town and so on.
The US News systems are designed to churn. Changed numbers sells magazines. Having the numbers stay the same doesn't. Yet it's hard to believe many schools change very much from year to year. Yes, once a while a school suffers a crisis or an epiphany, but those are pretty rare events.
There are inbuilt biases in the US News scoring system that favor small schools, and schools in cities with high starting salaries. Not to mention that in South Florida the market has more medium-sized firms than in other cities our size, and those firms rarely make offers until a candidate has passed the bar, notably depressing the 'employment at graduation' rate.
I sympathize with aspiring students who need a guide to the perplexed when sorting through their options. It's such a shame that the information market's first-mover advantage has allowed such a crummy measure to dominate.
so what number in this terribly flawed system are we?
Think of it like an imperfect but nonetheless stable system. BusinessWeek has gotten into the rankings game (I believe), and so even US News is not all-powerful. I must admit I don’t know how things work today; law tends to be extremely conservative, meaning resistant to change (that’s my wordperfect example). US News was certainly king when I was choosing schools (oh, but I ignored it completely and went with the home-town school).
But still, its fairly stable. Think about it. Each school has to respect the system, but the system itself is always being tinkered with, each ranker is constantly making changes (as you note), and neither can deviate too far away from a system that at least makes some sense. I.e. not arbitrary. This keeps everyone on their toes, and no school, likewise, can afford to diverge significantly from the communal path of constant development.
Well, that’s all I can do to argue the alternative. It wasn’t easy but it is also not too easy to dismiss I think.
In any case the social sciences need a set of clearer, standard methodologies to use and refer to.