Time: Day 1, 11:40-13:10

Room: Auditorium A, Level 02

Facilitator: Prof. Ian Gent, University of St. Andrews

Co-ordinator: Graham McDonald

RDF Domain: A

Abstract

Reproducibility is a cornerstone of the scientific method. And in many ways it is easier in computing because we can often just run our programs again and see if we get the same result. So it seems that computer science should be one of the most reproducible of all. This does not seem to be the case. This affects all of us negatively. Not only does it mean we might doubt how valid any given result is, but it often means we cannot build on the state of the art. For example, if you cannot run another researcher’s code from last year, how do you know whether your code is an actual improvement on it? In this workshop we will look at some of the key issues involved in reproducible research, and hope to improve your ability to make reproduce other people’s research and make your own research reproducible.

Content

The session will kick off with an introductory presentation before we begin the group activities. During the workshop, delegates will mostly be working in groups to discuss the pros and cons of reproducibility, including the practicalities of reproducibility, the costs associated to reproducibility and the extrinsic issues, such as legal or ethical issues. The workshop will have a focus on issues that relate to reproducing the work of other researchers and, also, steps that can be taken to make your own research more reproducible. There will also be a re-grouping session, for groups to share their insights, before the workshop concludes with an interactive polling session.

Part 1) Presentation (15 mins).

Part 2) Group discussions on reproducibility (35 mins).

Part 3) Re-grouping to discuss findings from Part 2 (25 mins).

Part 4) Interactive polling session (10 mins).

Learning Outcomes

After attending the workshop, delegates will be able to:

• Identify the benefits of making research reproducible.
• Identify the main issues that can make reproducibility challenging (when making your research reproducible, or when reproducing other people’s experiments).
• Recognise practical steps to overcome the identified challenges and evaluate the pros and cons of any solution.
• Have a high-level understanding of the non-computational issues that relate to reproducible research, such as legal or ethical issues.