Nicholas Kristoff Looks In The Mirror

Nicholas Kristoff in his opinion piece in the New York Times today says unlike Jim Cramer he welcomes accountability in the media and offers up some thoughts on why its needed.

Ever wonder how financial experts could lead the world over the economic cliff?

One explanation is that so-called experts turn out to be, in many situations, a stunningly poor source of expertise. There’s evidence that what matters in making a sound forecast or decision isn’t so much knowledge or experience as good judgment — or, to be more precise, the way a person’s mind works.

snip

The expert on experts is Philip Tetlock, a professor at the University of California, Berkeley. His 2005 book, “Expert Political Judgment,” is based on two decades of tracking some 82,000 predictions by 284 experts. The experts’ forecasts were tracked both on the subjects of their specialties and on subjects that they knew little about.

The result? The predictions of experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board.

“It made virtually no difference whether participants had doctorates, whether they were economists, political scientists, journalists or historians, whether they had policy experience or access to classified information, or whether they had logged many or few years of experience,” Mr. Tetlock wrote.

Indeed, the only consistent predictor was fame — and it was an inverse relationship. The more famous experts did worse than unknown ones. That had to do with a fault in the media. Talent bookers for television shows and reporters tended to call up experts who provided strong, coherent points of view, who saw things in blacks and whites. People who shouted — like, yes, Jim Cramer!

snip

The marketplace of ideas for now doesn’t clear out bad pundits and bad ideas partly because there’s no accountability. We trumpet our successes and ignore failures — or else attempt to explain that the failure doesn’t count because the situation changed or that we were basically right but the timing was off.

For example, I boast about having warned in 2002 and 2003 that Iraq would be a violent mess after we invaded. But I tend to make excuses for my own incorrect forecast in early 2007 that the troop “surge” would fail.

So what about a system to evaluate us prognosticators? Professor Tetlock suggests that various foundations might try to create a “trans-ideological Consumer Reports for punditry,” monitoring and evaluating the records of various experts and pundits as a public service. I agree: Hold us accountable!

Kristoff's call for a "trans-ideological Consumer Reports for punditry" kinda comports with an idea I have been tossing around in my head about a kind of "truth in advertising" for "news" sources. I personally believe that we have come to a point in time were any media outlet that uses the word "news" to describe its content should be held to a high standard. The thing about it for me is that we hold advertisers to account when they lie to consumers. We have truth in advertising laws that say that a business can be fined or worse if they put up a commercial that makes false claims about its product. But the ironic thing is those same commercials that are held to a standard of factual accuracy many times run in between "news" shows or one cable "news" networks or along side columns in print media where the pundits get to stretch, pervert or outright suppress the truth. Now what I am talking about isn't matters of opinion. For instance its ok for a person to believe that the stimulus bill is wrong for America. That is basically their right of free speech. But when they start making up what is in the stimulus bill or taking something in the stimulus bill and use deception to give a false impression about what a certain provision is supposed to do where the facts are verifiable, we should have some kind of independent arbiter of the truth that has the ability to fine these outlets and expose them for putting out bad information. As an alternative these outlets could have the option of not using the word "news" to describe what they do and then they could do and say whatever they like but without the same level of credibility.

Now I understand that this concept would be hard to implement in the real world. Both sides of any argument will claim that the independent commission leans to the other side. And then there would be the question of who would set up the commission and most people wouldn't want it to be the government for fear of subverting the first amendment rights of a free press. Still I believe there can be and has to be a way to make this happen that is satisfactory for all sides. In the end, I believe, that with this added measure of accountability our news sources will step up to the plate and start putting out more accurate data and the populous would be better served by our press because of it.