Scientific research these days often involves code as part of the research methods and/or outputs. Whilst it is generally agreed that scientific research needs to be peer-reviewed as part of the publication, this stipulation doesn’t always extend to peer-reviewing code, despite the fact that the code may be an integral part of producing the results, and indeed, bugs in the code may result in incorrect scientific results.
We hope to revive an initiative within the Code is Science project to list popular scientific journals and their policies with regards to code openness and peer review. We aim to build infrastructure that clearly lists not only journal policies with regards to code artefacts, but also compliance with journal policy. This infrastructure would allow contributors to submit and edit entries for journals, including specific evidence with regards to policy compliance levels. Users of this database would be able to search based on specific criteria, such as code or data openness policies, code peer-review policies, and journal policy compliance level. This would be beneficial for people choosing where to submit their papers and/or which journals to review for. In addition, having information clearly available might incentivise other publishers towards open practices.
Work at the Sprint
We propose to expand the current prototype code, building a more robust backend, submission form and data schema. Whilst there wouldn’t be time to complete the entire architecture, we would be able to at least design the structure of the data, such as journal name, open-data policy, open-code policy, policy compliance, proof of compliance (or compliance issues) and so on, as well as mock-ups of what the user interface might look like based on the data design.
We are looking for…
Designers, database admins, coders, UX consultants, and individuals from the publishing industry would all be able to contribute towards the design of our journal policy database prototype. In addition, anyone from the publishing industry, or who interacts with the publishing industry (i.e. researchers), would be able to offer comment on the prototypes and/or user test the prototype.