Skip to content

Latest commit

 

History

History

results

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
vanilla_results
vanilla_results
 
 
README.md
README.md
 
 
gpt_fixes_tests.csv
gpt_fixes_tests.csv
 
 
magicoder_fixes_tests.csv
magicoder_fixes_tests.csv
 
 

README.md

Results

File structures in the directory results:

  • gpt_fixes_tests.csv includes the detailed results of all 500 successfully fixed ID flaky tests from GPT-4. Each line includes Project URL,SHA,Module,Type,Test,Results,Patch.
  • magicoder_fixes_tests.csv includes the detailed results of all 170 successfully fixed ID flaky tests from Magicoder. Each line includes Project URL,SHA,Module,Type,Test,Results,Patch.
  • vanilla_results includes the results for only prompting models with vanilla prompt "This is a flaky test. Can you fix it?"

In the column of Results, F represents Test Failure, P represents Test Pass, and CE represents Compilation Error.