DH: So it sounds like you want to bring actual practice more in line with the deductive ideal in which we test previously stated hypotheses, is that right? Can pre-registration solve these kinds of problems on its own?
BN: Actually, I’m skeptical that preregistration itself — which is starting to come into wider use in development economics as well as experimental political science and psychology — is a solution. As I argued in a white paper for the American Political Science Association Task Force on Public Engagement, it is still too easy for publication bias to creep in to decisions by authors to submit papers to journals as well as evaluations by reviewers and editors after results are known. We’ve seen this problem with clinical trials, where selective and inaccurate reporting persists even though preregistration is mandatory.
I think a better approach is to offer a publishing option in which journals would consider accepting some articles in principle before the results were known based on peer review of the design and analysis plan. Such an approach, which has been formalized by the Registered Reports movement (of which I am a part), would better align author and journal incentives with our goals as scientists.