Machine learning to help researchers evaluate biases in clinical trials: a prospective, randomized user study

Authors:
Frank Soboczenski School of Population Health and Life Sciences, King's College London, London, UK
Thomas Trikalinos Center for Evidence-based Medicine, Brown University, Providence, USA
Joel Kuiper Vortext Systems, Groningen, Netherlands
Randolph G Bias School of Information, University of Texas at Austin, Austin, USA
Byron C Wallace College of Computer and Information Science, Northeastern University, Boston, USA
Iain J Marshall School of Population Health and Life Sciences, King's College London, London, UK

Structure of the repository

Data folder:

• TimeAnalysis2_1.xlsx (main data file)
  
• UXData1.xlsx (aux data file - mainly including qualitative answers)
  
• sus_calculation.csv (System Usability Scale data)
  
• subset_selfreported.xlsx (subset of TimeAnalysis2_1.xlsx for self reported characteristics)
  
• annotaionschanged.xlsx (subset of TimeAnalysis2_1.xlsx for annotations analysis)
  
• agreement.xlsx (subset of TimeAnalysis2_1.xlsx for self judgement agreement analysis)
  

Jupyter Notebook: Analysis.ipynb
Structure:

1. Main Analysis (distribution of the data, Wilcoxon significance tests, boxplots, scatterplots)
   
2. Descriptives (mean, sd etc of timing data)
   
3. Tukey Ladder of Powers (data transformation and plots)
   
4. Linear Mixed Effects Model Analysis (primary and auxiluiary model analysis)
   
5. Reviewer Judgments & Annoations Analysis (descriptives, self-reported characteristics, agreement data, annotation data)
   
6. Questionnaire Analysis (likert scale analysis, System Usability Score (SUS) evaluation)
   

The System used and evaluated in this study can be found here: RobotReviewer User Study