APSA calls for deliberation about Research Transparency and Interpretability, welcomes comments on PSNow website

On the PSNow website, APSA is calling for questions, observations, experiences, and proposals in the hopes of fostering conversation on research transparency and interpretability, and how it should or should not be implemented.

Specific categories include:

  • Confidentiality and anonymity
  • Cost in time or money
  • Transparency or interpretability in archival work
  • Transparency in big data
  • Transparency or interpretability in field work
  • Relevance for political theory or political philosophy
  • Embargos on making public new research materials
  • Editors’ decision-making process
  • Appeals from editors’ decisions
  • Consistency across journals

Martin and Peterson examine TRIP survey results to shed light on views on data sharing

In a new report published on the Monkey Cage, Elizabeth Martin and Susan Peterson examine who shares (what kind of) data, using the “2014 Teaching, Research and International Policy (TRIP) survey of 1,615 international relations (IR) scholars to explore views on data sharing in the discipline.”

Martin and Peterson find “that the discipline’s current divide is not between those scholars who primarily use statistical methods and those who chiefly employ qualitative approaches. Rather, IR scholars—regardless of whether they consider themselves to be primarily quantitative or qualitative researchers—are less likely to share their qualitative data than they are to share their quantitative results.”

In addition, the authors find that: “Sixty-one percent of respondents who shared qualitative data and 56 percent of respondents who shared quantitative data indicated that their sharing included making “ad hoc arrangements with other scholars.” Even quantitatively oriented scholars were more likely to informally share data with other scholars than to make it publicly available through a journal.”

The authors argue that their “findings do not speak directly to the DA-RT debate, but they do suggest that opposition to the new requirement that all data be made available online should not be dismissed as simply qualitative scholars’ resistance to increased transparency.”

Read the whole report here.

Piscopo expresses concerns, under DA-RT, about conducting elite interviews and the widening gulf between the “haves” and the “have-nots”

In this post, Jennifer Piscopo (Occidental College) writes about how a transparency appendix required under DA-RT could  “jeopardize carefully-cultivated networks of trust” needed to gain elite interviews, the “near impossiblity” of replication (since ” elites’ willingness to disclose sensitive information depends on everything from the researcher’s reputation to the phases of the moon”), and her fear about the implications for inequality in the profession (“the best publication opportunities will continue to accrue to highly-resourced researchers, who can rely on teams of graduate students and other paid assistants to help them create the appendices necessary to meet access, transparency, and replicability standards”).

Perspectives on Politics transparency guidelines for authors

Perspectives on Politics editor-in-chief Jeffrey C. Isaac and managing editor James Moskowitz share their journal’s policy, established in 2009, including the specific instructions they provide to contributing authors, as well as the guiding principles that inform their approach as journal editors. The goal of the guidelines is to “facilitate the kind of publicity that is at the heart of scholarly communication.” Isaac and Moskowitz emphasize: “Perspectives has long been committed to the highest standards of general research transparency.” Read their full comments and access the supplementary material here.

Carpenter on SSH editorial board deliberations on DA-RT

I’m seeing at least two issues in this thread that I would like to highlight.

The first is that many historians and historical social scientists, including purely narrative or “qualitative” scholars, already observe an ethic of transparency and replicability in their work. By following the citations from footnotes to exact archival locations, it is possible in many, many cases to find the exact document or material object that a scholar was interpreting and analyzing. This is a point that has been missing in much of the debate. Yes, replication of historical work is costly, and yes that process could be improved, but it is frequently done and historical social scientists were, in many respects, well ahead of their peers in this endeavor. I say this as a card-carrying political scientist whose work includes a heavy dose of quantitative and rather technical work.

The second is that [the] point about the costs of digitization are more generalizable. We should always be thinking about who bears the costs of our research policies, and I fear that a too-stringent policy on replication will place the burdens upon younger scholars whose work is often conducted in farflung archives where less material has been digitized. The worry I’ve heard in some circles is that these policies will eventually lead to more and more work being done in a laboratory environment, where the laboratories are housed at wealthier institutions that can foot or subsidize the bill of replication costs. I doubt that process is all-converging and teleological, but I do fear that it will overly standardize the research process.

I think a flexible transparency and replication policy is a good idea. But I worry about a policy that imposes its greatest proportional costs upon younger scholars who are most in need of being able to take risks, which is to say, not those of us who already have tenure and who sit on editorial boards.

Dan Carpenter (Harvard University)

Rick Wilson on “DA-RT, TOP and Rolling Back Transparency”

Rick Wilson blogs about his thoughts on DA-RT, and brings attention to Transparency and Openness Promotion (TOP) guidelines. He writes: “Personally, I am inclined to sign the DA-RT delay petition because DA-RT does not go far enough.” But he finds it “puzzling that there is resistance to making it clear how one reaches a conclusion.” To him, “All good science is about elucidating the process by which one reaches a conclusion.” You can view his full blog post here.

Kristin Monroe on ethical issues that need to be addressed in order to share data

“All of these people trusted me with personal stories….  Deserving this trust called for great sympathy, sensitivity and – above all – going the extra mile to protect the speaker’s integrity and privacy.”

In this short post, Kristin Monroe  argues that data access requirements must be flexible: ethical and privacy concerns may demand modification of testimony or destruction of data.