This is part 2 of the lesson on scientific reproducibility. Links to other lessons are the end of this post. How can you trust the scientific information you read?
- Understand the causes of irreproducibility, some were identified in the video. Cooking makes a great analogy. My mom and my aunt used to make the same vermicelli pudding. I loved my mom’s pudding but not my aunt’s. Were they using the same ingredients (reagents) and the same recipe (protocol)? If so, how closely were they adhering to the recipe? Were the ingredients from the same brand or was the vermicelli slightly different? Did my aunt’s cooking pot not get hot enough (equipment)? Did my mom secretly add additional ingredients to make the pudding extra delicious (protocol details not shared or unintentional protocol drift) ? Am I biased in thinking that her pudding is better?Add to this the variation inherent in biological systems and exacerbated by environment and you have the perfect conditions for introducing variability in results.
- Remember that popular press often reports the most exciting and novel results, not necessarily those that will go to be reproducible and become part of scientific and medical knowledge.
- Do you see many scientific articles talking about a certain finding or have you seen similar results in multiple papers or using multiple techniques?
- Understand that scientific results can be biased due to many reasons. One way this bias gets introduced in addition the many other ways discussed in the video is p-hacking: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002106
What can you do to improve science reproducibility?
- Ask questions. At meetings, lectures, online in the comments sections of papers.
- Using ReFigure, you can create a roster of experiments that test the finding you are interested in. This not only helps you, it helps society too.
- Post your review of papers. In most sites, there is a comment section but some sites also have a specific place for “post-publication” reviews.
- If you have tried to reproduce a finding yourself, use openreuse.org or ReFigure to link your findings to the published ones. How did you reproduce it? Did you make any changes to the recipe?
- If you are in a position to professionally review a scientist for promotion or accolades or grants, remember to ask about reproducibility.
Here are some ways scientists and institutions are trying to improve the reproducibility of science: https://www.insidehighered.com/views/2017/07/10/introducing-new-series-reproducibility-scientific-research-essay
To discuss this video, ask questions, offer feedback or get assistance on creating your own literature review project, please fill out this form.
Previous links|Introduction | Week 1: Open Access and Introduction to Scientific Papers | Week 2: Let’s start reading papers and creating research outputs! | Week 3: Tools and strategies to find papers and save your literature review | Week 4: Are you too inexperienced to start building a science identity? | Week 5: Reproducibility of scientific findings, Part 1
