Save data periodically during the experiment

[dreamerie]

I posted a few years earlier about incomplete .csv files being saved from online participants, and what solved the issue was adding a few seconds to the end of the experiment (so participants couldn't immediately close the window or redirect, and the data could be fully transferred.)

I now have an experiment that records audio from the participants, which makes the resulting .csv file much bigger (6 MB according to my tests instead of under 100 KB). I am worried that larger data files may not be fully transferred to the server. Could the .csv file be uploaded periodically during the study? Alternatively, audio is recorded only in the first half of the experiment, could the first half be uploaded as separate file before the second?

[ChristopheBossens]

How are you hosting the experiment and handling data storage? On cognition.run, the data is saved by default after every trial. In our lab we have our own hosting, and what I do to make sure that the data is saved is to have a trial with a continue button at the end of the experiment. The continue button is disabled by default, and the message informs the participant that their data is being sent to the server. The on_start() function of that trial disables the continue button and makes the ajax post request to the server, and when we have a successful response from the server we enable the continue button again.

[dreamerie]

The data is saved to the server as a csv file near the end experiment, using the PHP script in the tutorial: https://www.jspsych.org/7.2/overview/data/#storing-data-permanently-as-a-file

I would like modify it so that the server replies when the data is fully saved, and the participant is allowed to continue afterwards. The code that you have sounds ideal, if you would be willing to share.

[ChristopheBossens]

Below is the code that I use. The idea is to disable the button at the start of the trial and to send the data at the same time. When the server returns a success response the button is enabled and the participant can continue.

    let save_data_trial = {
        type: jsPsychHtmlButtonResponse,
        stimulus: "Waiting for data upload to complete",
        choices:["Continue"],
        prompt: "",
        on_load: function(){
            document.getElementsByClassName("jspsych-btn")[0].disabled = true;
        },
        on_start: function(){
            let json_data = {
                'id'    : 1,
                'data'  : jsPsych.data.get().json()
            };

            let xhr = new XMLHttpRequest();
            xhr.open('POST', 'index.php');
            xhr.setRequestHeader('Content-Type', 'application/json');
            xhr.onload = function() {
                if(xhr.status == 200){
                    document.getElementsByClassName("jspsych-btn")[0].disabled = false;
                }
            };
            xhr.send(JSON.stringify(json_data));
        }
    };