Perspectives on Politics editor-in-chief Jeffrey C. Isaac and managing editor James Moskowitz share their journal’s policy, established in 2009, including the specific instructions they provide to contributing authors, as well as the guiding principles that inform their approach as journal editors. The goal of the guidelines is to “facilitate the kind of publicity that is at the heart of scholarly communication.” Isaac and Moskowitz emphasize: “Perspectives has long been committed to the highest standards of general research transparency.” Read their full comments and access the supplementary material here.
I’m seeing at least two issues in this thread that I would like to highlight.
The first is that many historians and historical social scientists, including purely narrative or “qualitative” scholars, already observe an ethic of transparency and replicability in their work. By following the citations from footnotes to exact archival locations, it is possible in many, many cases to find the exact document or material object that a scholar was interpreting and analyzing. This is a point that has been missing in much of the debate. Yes, replication of historical work is costly, and yes that process could be improved, but it is frequently done and historical social scientists were, in many respects, well ahead of their peers in this endeavor. I say this as a card-carrying political scientist whose work includes a heavy dose of quantitative and rather technical work.
The second is that [the] point about the costs of digitization are more generalizable. We should always be thinking about who bears the costs of our research policies, and I fear that a too-stringent policy on replication will place the burdens upon younger scholars whose work is often conducted in farflung archives where less material has been digitized. The worry I’ve heard in some circles is that these policies will eventually lead to more and more work being done in a laboratory environment, where the laboratories are housed at wealthier institutions that can foot or subsidize the bill of replication costs. I doubt that process is all-converging and teleological, but I do fear that it will overly standardize the research process.
I think a flexible transparency and replication policy is a good idea. But I worry about a policy that imposes its greatest proportional costs upon younger scholars who are most in need of being able to take risks, which is to say, not those of us who already have tenure and who sit on editorial boards.
Dan Carpenter (Harvard University)
In remarks posted from her research leave in Morocco, Aili Tripp (U of Wisconsin-Madison), writes that while she agrees with the objectives of DA-RT, she worries about how aware reviewers and journal editors are “of the ethical considerations and challenges of doing research in non-democratic contexts.” Furthermore, she is concerned about how data access requirements will affect the quality of field research: “I study women and politics and women’s movements in Africa and I can’t imagine people would want some of the things they say publicly attributed to them or their organization or even to the women’s movement and opponents of the women’s movements. They don’t want their strategies, jealousies, frustrations or weaknesses revealed to their competitors, opponents, or people they are lobbying.”
In order to make our views as clear as possible, we would like to reiterate some central points contained in our letter of November 24, 2015. They include:
*We welcome responses to and comments on this letter, and encourage continued conversations about how best to achieve the goals of research transparency and interpretability in ways that are responsive to the entire APSA membership. . . . We hope that decisions about policies such as DA-RT will be based on deliberation, mutual trust, and shared accommodation.
* We have appointed a publications policy committee comprised of current APSA Council members who will work with the extant membership-based Publications Committee to address a growing need for Council policy on publications. This policy could range from an explicit statement that each editor or editorial team of association-sponsored journals has complete autonomy, to a set of broad policies that association-sponsored journals should follow in order to accord with the APSA Guide to Professional Ethics. The committee will propose a policy to be discussed by the Council in its April 2016 meeting, followed by publication of the proposal and solicitation of members’ views and suggestions. The Council will vote on the (revised) policy in its September 2016 meeting.
* The current president-elect will propose, subject to Council approval, a presidential Task Force on Professional Ethics that will continue and develop ongoing work by current APSA membership-based committees.
*Under long-standing association practice, editors of our journals have wide discretion in editorial standards and procedures. After extensive consultation, the APSR editors have issued a set of draft guidelines for promoting access to evidence and research interpretability. We have therefore not expressed our own views on how these guidelines should be developed and put into practice.
* In our view, all scholars and scholarship benefit from shared engagement around a set of evidence and its possible analyses or interpretations.
We look forward to continued discussion and decision-making about these fascinating and important concerns, and once again, we thank the many political scientists who are involved in thinking through how best to engage in research, teaching, and public engagement.
best to all, Jennifer Hochschild, David Lake, and Rodney Hero
It is regrettable that APSA leadership issued a statement endorsing DA-RT at a time when a broad and very constructive discipline-wide conversation is only now beginning. We think this is premature and, as such, risks preempting rather than facilitating the kind of “deliberation, mutual trust and shared accommodation” that the presidents want to promote.
We do not see much indication in this statement that the presidents have engaged the legitimate concerns raised (e.g., in the thoughtful QMMR newsletter or in the recent petition) by our colleagues who confront very real conundrums in reconciling DA-RT transparency standards with ethical commitments to the protection of human subjects, the ability to publish out of original data sets without being required to share them too early, the logistical burden placed particularly on young scholars and by scholars at non-elite universities, and other concerns. Instead, we note with disappointment that the justification offered for embracing DA-RT relies almost exclusively on a few references to egregious cases of flagrant ethics violations that all of us can of course agree are to be avoided and that in any event are well covered by the existing Ethics guidelines. If there is one thing that the discussion of DA-RT has already made very clear, it is that the issues are a great deal more nuanced and complex than that.
Fortunately, there is a substantive on-going process to deliberate on these issues, investigating what the guidelines should be for issues such as the incentives for young scholars to create their own datasets, and the kinds of research materials, if any, that scholars engaged in immersive research may be required to deposit.
Our approach to this issue is thus different from that of the presidents. Rather than take sides in a process that is underway, we reiterate our request that the journals hold off until some of these issues can be deliberated more adequately in the profession. The Qualitative and Multi-Method Research organized section is launching a wide-ranging and structured deliberation within the profession and we look forward to discussing the results at the next annual meeting. We believe that any significant implementation before such issues have been more fully deliberated is premature and potentially divisive.
Nancy Hirschmann, Mala Htun, Jane Mansbridge, Kathleen Thelen, Lisa Wedeen, and Elisabeth Wood
In a letter posted today on the PS Now website, APSA president Jennifer Hochschild, President-elect David Lake, and former president Rodney Hero affirm that APSR will begin to implement DA-RT in January 2016.
Below is text of the message sent by CPS editors to the board on November 20, 2015:
Dear Members of the CPS Editorial Board:
Our recent conversations have been driven by the goals of promoting research transparency in general and signaling that CPS welcomes qualitative research. Many members of the board expressed support for our efforts – but many also raised important questions and concerns, including a possible tension between both objectives. Not surprisingly these echo concerns colleagues across the discipline have expressed about the DA-RT initiative.
We have decided to do two things. First, we have concluded that it is in the journal’s best interests to put off posting any new guidelines or suggestions pertaining to data access and research transparency, beyond what we already require (deposit of data and code for all statistical analyses a paper presents). We understand that APSA’s QMMR section – as well as others – will be working on these issues over the next year or so, and look forward to hearing from these expert colleagues in the future about best practices.
Second, we have decided to post new “author and reviewer recommendations” that articulate what we regard as central issues in evaluating submissions – of any methodology – to CPS. We do so because many authors (and reviewers) ask us what it takes to get published in CPS, and this list provides a general sense of the qualities that are common to strong papers, whatever the method used. A draft is attached. Our plan is to 1) post these to the CPS submissions page by January 2016 and 2) include these in the email sent to all reviewers. We welcome suggested changes and/or additions or deletions.
Thanks to all who have contributed to this important discussion.
Ben and David