APSA calls for deliberation about Research Transparency and Interpretability, welcomes comments on PSNow website

On the PSNow website, APSA is calling for questions, observations, experiences, and proposals in the hopes of fostering conversation on research transparency and interpretability, and how it should or should not be implemented.

Specific categories include:

  • Confidentiality and anonymity
  • Cost in time or money
  • Transparency or interpretability in archival work
  • Transparency in big data
  • Transparency or interpretability in field work
  • Relevance for political theory or political philosophy
  • Embargos on making public new research materials
  • Editors’ decision-making process
  • Appeals from editors’ decisions
  • Consistency across journals

Martin and Peterson examine TRIP survey results to shed light on views on data sharing

In a new report published on the Monkey Cage, Elizabeth Martin and Susan Peterson examine who shares (what kind of) data, using the “2014 Teaching, Research and International Policy (TRIP) survey of 1,615 international relations (IR) scholars to explore views on data sharing in the discipline.”

Martin and Peterson find “that the discipline’s current divide is not between those scholars who primarily use statistical methods and those who chiefly employ qualitative approaches. Rather, IR scholars—regardless of whether they consider themselves to be primarily quantitative or qualitative researchers—are less likely to share their qualitative data than they are to share their quantitative results.”

In addition, the authors find that: “Sixty-one percent of respondents who shared qualitative data and 56 percent of respondents who shared quantitative data indicated that their sharing included making “ad hoc arrangements with other scholars.” Even quantitatively oriented scholars were more likely to informally share data with other scholars than to make it publicly available through a journal.”

The authors argue that their “findings do not speak directly to the DA-RT debate, but they do suggest that opposition to the new requirement that all data be made available online should not be dismissed as simply qualitative scholars’ resistance to increased transparency.”

Read the whole report here.

Perspectives on Politics Editor Jeffrey Isaac on why he opposes DA-RT but did not sign petition to delay it

A BROADER CONCEPTION OF POLITICAL SCIENCE PUBLICITY, OR WHY I REFUSE DA-RT AND YET DID NOT SIGN THE “DELAY DA-RT” PETITION

by Jeffrey C. Isaac December 3, 2015 Uncategorized Leave a comment 772 Views

 

I have served as editor in chief of Perspectives on Politics since June 2009. I have also served as an ex-officio member of the APSA Council since that time. During this time I have worked full-time, with my staff and editorial board, to enhance the intellectual quality and editorial efficiency of Perspectives, and to advocate on behalf of the values that I associate with the journal whose care is in my charge—the values of a vigorous and engaged political science public sphere.

It is only because of my editorial responsibilities that I have come to be involved in the controversy over DA-RT.

I have opposed DA-RT since October 2014, when I sent an open letter to the leaders of the DA-RT initiative, who had assembled in Ann Arbor, MI and drafted a set of principles that later became the DA-RT principles adopted by some 25-plus political science journals.

My letter was deliberately an open letter, and I shared it widely, with my own editorial board, with APSA leaders, and with a great many friends and colleagues who share my value commitments. Some, and perhaps many, of those colleagues who have called for the DA-RT delay first learned about the Ann Arbor meeting, and the DA-RT journal cartel that was taking shape, from my open letter. Since that time many colleagues have shared ideas about the meaning of DA-RT and the ways to best express concerns about it. It was the efforts of some of these colleagues to engage APSA leaders in reasoned discussion that led to a private meeting, in San Francisco, between APSA leaders and DA-RT supporters and critics. This was a productive meeting of fine scholars who are also good and well-intentioned individuals. DA-RT critics expressed their concerns. DA-RT proponents genuinely listened and engaged the concerns. The APSA leaders were pressed by a number of Council members, myself included, to clarify the precise relationship between DA-RT and APSA. It was noted that APSA staff had supported the Ann Arbor meeting, and that the APSA website had for many months featured only pro-DA-RT materials. The question was pressed: when exactly had APSA officially endorsed DA-RT? The answer was clear: APSA had never officially endorsed DA-RT. In 2012 the APSA Council had voted to approve some general professional ethics principles about the responsibilities of scholars to be honest and transparent. But these were general ethical ideas, premised on voluntary acceptance by individual scholars. The Council had never deliberated about “DA-RT,” adopted specific DA-RT principles, or endorsed changes in any journal policies on the basis of DA-RT principles.

At the San Francisco meeting these things were clarified in private conversation.

Many of us left the meeting confident about two things: (1) productive dialogue between qualitative research leaders and DA-RT proponents and affiliated journals would continue, and (2) APSA would clarify, publicly, that DA-RT was an initiative undertaken by many fine disciplinary leaders and journal editors, but it was not an APSA initiative, and APSA would promote a serious dialogue among its members about whether APSA should have an official position on DA-RT.

Weeks and months went by. No further clarification was offered.

During this time a distinguished group of our colleagues organized a letter that eventually went viral as a petition, calling for delay and further deliberation about DA-RT before any changes in editorial policies at major journals were implemented.

Continues on The Plot http://www.the-plot.org/2015/12/03/broader-conception-political-science-publicity-refuse-da-rt-yet-not-sign-delay-da-rt-petition/

 

Piscopo expresses concerns, under DA-RT, about conducting elite interviews and the widening gulf between the “haves” and the “have-nots”

In this post, Jennifer Piscopo (Occidental College) writes about how a transparency appendix required under DA-RT could  “jeopardize carefully-cultivated networks of trust” needed to gain elite interviews, the “near impossiblity” of replication (since ” elites’ willingness to disclose sensitive information depends on everything from the researcher’s reputation to the phases of the moon”), and her fear about the implications for inequality in the profession (“the best publication opportunities will continue to accrue to highly-resourced researchers, who can rely on teams of graduate students and other paid assistants to help them create the appendices necessary to meet access, transparency, and replicability standards”).

Perspectives on Politics transparency guidelines for authors

Perspectives on Politics editor-in-chief Jeffrey C. Isaac and managing editor James Moskowitz share their journal’s policy, established in 2009, including the specific instructions they provide to contributing authors, as well as the guiding principles that inform their approach as journal editors. The goal of the guidelines is to “facilitate the kind of publicity that is at the heart of scholarly communication.” Isaac and Moskowitz emphasize: “Perspectives has long been committed to the highest standards of general research transparency.” Read their full comments and access the supplementary material here.

Carpenter on SSH editorial board deliberations on DA-RT

I’m seeing at least two issues in this thread that I would like to highlight.

The first is that many historians and historical social scientists, including purely narrative or “qualitative” scholars, already observe an ethic of transparency and replicability in their work. By following the citations from footnotes to exact archival locations, it is possible in many, many cases to find the exact document or material object that a scholar was interpreting and analyzing. This is a point that has been missing in much of the debate. Yes, replication of historical work is costly, and yes that process could be improved, but it is frequently done and historical social scientists were, in many respects, well ahead of their peers in this endeavor. I say this as a card-carrying political scientist whose work includes a heavy dose of quantitative and rather technical work.

The second is that [the] point about the costs of digitization are more generalizable. We should always be thinking about who bears the costs of our research policies, and I fear that a too-stringent policy on replication will place the burdens upon younger scholars whose work is often conducted in farflung archives where less material has been digitized. The worry I’ve heard in some circles is that these policies will eventually lead to more and more work being done in a laboratory environment, where the laboratories are housed at wealthier institutions that can foot or subsidize the bill of replication costs. I doubt that process is all-converging and teleological, but I do fear that it will overly standardize the research process.

I think a flexible transparency and replication policy is a good idea. But I worry about a policy that imposes its greatest proportional costs upon younger scholars who are most in need of being able to take risks, which is to say, not those of us who already have tenure and who sit on editorial boards.

Dan Carpenter (Harvard University)

Tripp: transparency is tough for research in non-democratic contexts

In remarks posted from her research leave in Morocco, Aili Tripp (U of Wisconsin-Madison), writes that while she agrees with the objectives of DA-RT, she worries about how aware reviewers and journal editors are “of the ethical considerations and challenges of doing research in non-democratic contexts.” Furthermore, she is concerned about how data access requirements will affect the quality of field research: “I study women and politics and women’s movements in Africa and I can’t imagine people would want some of the things they say publicly attributed to them or their organization or even to the women’s movement and opponents of the women’s movements. They don’t want their strategies, jealousies, frustrations or weaknesses revealed to their competitors, opponents, or people they are lobbying.”