Let me recommend Greg Marx's critique of the AP's Going Rogue "fact check", which illustrates how the paradigm can go horribly wrong.
As Marx notes, the AP critique frequently targets claims that fall short of the standard we tried to uphold at Spinsanity -- statements that are false or blatantly misleading. In some cases, the claims in question are simply standard political rhetoric -- like almost every politician, Sarah Palin frames herself positively and makes her opponents look bad while omitting arguably relevant context. In others, the AP objects to statements (Alaskans don't want help from government, her political career wasn't driven by ambition) that can't be directly falsified.
In the past, I've noted similarly ill-conceived fact-checks at Factcheck.org (here and here) and the Washington Post. After working on Spinsanity, I understand the temptation to publish an article after investing a lot of time in research and the pressure to produce content on a schedule, but it's important to reserve fact-checks for claims that are truly objectionable.
The best parallel here is Michael Moore's Sicko. As my former Spinsanity co-editor Ben Fritz and I wrote, Moore's previous dissembling led to extensive fact-checking of the film (just as with Palin). Despite finding relatively few mistakes (especially in comparison with Moore's previous work), several outlets published highly nitpicky critiques of the film anyway (as the AP did with Palin):
The mainstream media has started fact-checking Michael Moore one movie too late.
As veteran fact-checkers of Michael Moore, we should be taking a victory lap in the wake of "Sicko." The liberal icon's latest film has been aggressively fact-checked by major outlets including CNN's Sanjay Gupta, the Associated Press, and USA Today.
However, the media has decided to pounce on Moore just when he seems to be addressing his problems with accuracy. As a result, they have little to say -- indeed, the weakness of the criticism makes Moore look thoughtful and careful with his facts by comparison.
The primary focus of these analyses is to point out that he only tells one side of the story. That is undoubtedly true, as it has been in all his work, but also obvious. Despite Moore's protestations to the contrary on NPR Monday, he is a propagandist. As such, he is under no obligation to present a balanced perspective.
The rest of the critiques -- which focus on discrepancies in health care statistics -- are so minor that even we can hardly protest...
So why are the media going after "Sicko" so aggressively?
... When it comes to fact-checking, the mainstream media tends to wait until the evidence (or the narrative) that someone is a serial dissembler becomes overwhelming... In our experience, outlets that want to seem "objective" rarely fact-check without some sort of strong pretext.
The same logic seems to apply to Moore. With "Stupid White Men" and "Bowling for Columbine," we were one of the few sources documenting his mendacity. However, after "Fahrenheit," several major media outlets began to wake up and question Moore's facts, including Newsweek and USA Today.
With Moore's reputation for dishonesty growing along with his profile, we weren't surprised that CNN, USA Today and the AP thought it was worth devoting their resources to checking the facts in "Sicko." We did the same, assuming Moore would continue his pattern of deception. When we didn't find compelling evidence, we decided not to write about the topic. But the media outlets who had assigned the story apparently felt that they had to write something. As a result, they attempted to critique a movie whose greatest sin was simply being one-sided.
Marx's final paragraph summarizes the problem eloquently:
This sort of thing matters because, in an increasingly contested political landscape and wide-open media environment, there really is a need for fact checking. There is value in forging a consensus across ideological lines that adherence to the facts is a prerequisite for public debate, and the AP is, theoretically, just the sort of institution that can help police politicians who mislead the public. But for the idea of fact checking to have any weight—and any hope of broad credibility—it must mean something more specific than "contesting a statement that we disagree with." When Sarah Palin talks about "Obama's 'death panel,'" she's spreading misinformation that needs to be repudiated. When she talks about being beckoned by purpose, she's being a politician. We need to recognize the difference.
Update 11/23 8:39 AM: The AP trumpeted their success in an internal memo:
It was a literary treasure hunt. Former Alaska Gov. Sarah Palin's "Going Rogue" was perhaps the most anticipated memoir of the year, the pre-order alone placing it at the top of some best-seller lists, and leading newspapers, Web sites and television outlets were clamoring for an early copy...
...[T]he AP was determined to get the first copy.
Finally, they learned that a store had inadvertently placed the book on sale five days before its official Nov. 17 release date.
They bought a copy, ripped it from its spine and scanned it into the system so it could be read and electronically searched. A NewsNow moved within 40 minutes, followed quickly by multiple leads as details were gleaned from the 413-page manuscript.
The story commanded massive play, including The New York Times, Los Angeles Times, Wall Street Journal.com and USA Today, the three major television networks, and major Web sites and portals Yahoo, Google, Huffington Post and Politico. The Washington Post did a separate story about how the publisher's carefully orchestrated rollout was foiled, and Palin herself, not happily, noted the scoop on Facebook.
It's a classic example of how the chase for "breaking news" lowers standards. The AP was determined to be first and devoted a huge amount of resources to beating other outlets to the story. This counts as a win according to the strange rules of journalism in which being first is more important than being right. (The quality and content of the story received no attention in the memo.)