Interruptions on large exports with long queries

[sanjorisa]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Provide the search details that you used by export e.g. usrQuery=TS=(covid-19 vaccin*) AND PY=2020 and databaseID=WOS

My search query is somewhat long, but I am sure it does not have any operator errors:

(((TI=((qubit AND ("quantum sensor" OR "quantum sensing")) OR "quantum-photonic device" OR ("quantum information" NOT ("quantum gis" OR "quantum geographic information system" OR "quantum geographical information system" OR "quantum metrology" OR "quantum communication" OR "quantum sensing" OR "quantum mechanical")) OR "semiconductor superconductor nanowire" OR ("Josephson junction" AND nanowire) OR ("single photon sensor" AND ("photonic crystal" OR time multiplex* OR optical computation*)) OR (quantum AND (photonic* OR optic*) AND interferometer) OR "quantum sensor" OR "quantum sensing" OR "quantum magnetometer" OR "quantum gyroscope" OR "quantum accelerometer" OR "quantum imaging" OR "quantum-enhanced sensing" OR "quantum metrology" OR "quantum gravimeter" OR "quantum thermometer" OR "quantum force sensor" OR "quantum displacement sensor" OR "quantum position sensor" OR "quantum rotation sensor" OR "quantum optical sensing" OR "quantum acoustic sensing" OR "quantum atomic clock" OR "quantum inertial sensor" OR "quantum field sensor" OR "quantum coherence sensing" OR "quantum entanglement sensing" OR "quantum single-photon sensor" OR "quantum noise sensing" OR "quantum-enhanced microscopy" OR "quantum-enhanced interferometry" OR "quantum-enhanced spectroscopy" OR "quantum-enhanced lidar" OR "quantum radar" OR "quantum gravimetry" OR "quantum strain sensor" OR "quantum pressure sensor" OR "quantum flow sensor" OR "quantum vibration sensor" OR "quantum chemical sensor" OR "quantum biosensor" OR "quantum environmental sensor" OR "quantum humidity sensor" OR "quantum gas sensor" OR "quantum nano-sensor" OR "quantum atomic sensor")) OR (AB=((qubit AND ("quantum sensor" OR "quantum sensing")) OR "quantum-photonic device" OR ("quantum information" NOT ("quantum gis" OR "quantum geographic information system" OR "quantum geographical information system" OR "quantum metrology" OR "quantum communication" OR "quantum sensing" OR "quantum mechanical")) OR "semiconductor superconductor nanowire" OR ("Josephson junction" AND nanowire) OR ("single photon sensor" AND ("photonic crystal" OR time multiplex* OR optical computation*)) OR (quantum AND (photonic* OR optic*) AND interferometer) OR "quantum sensor" OR "quantum sensing" OR "quantum magnetometer" OR "quantum gyroscope" OR "quantum accelerometer" OR "quantum imaging" OR "quantum-enhanced sensing" OR "quantum metrology" OR "quantum gravimeter" OR "quantum thermometer" OR "quantum force sensor" OR "quantum displacement sensor" OR "quantum position sensor" OR "quantum rotation sensor" OR "quantum optical sensing" OR "quantum acoustic sensing" OR "quantum atomic clock" OR "quantum inertial sensor" OR "quantum field sensor" OR "quantum coherence sensing" OR "quantum entanglement sensing" OR "quantum single-photon sensor" OR "quantum noise sensing" OR "quantum-enhanced microscopy" OR "quantum-enhanced interferometry" OR "quantum-enhanced spectroscopy" OR "quantum-enhanced lidar" OR "quantum radar" OR "quantum gravimetry" OR "quantum strain sensor" OR "quantum pressure sensor" OR "quantum flow sensor" OR "quantum vibration sensor" OR "quantum chemical sensor" OR "quantum biosensor" OR "quantum environmental sensor" OR "quantum humidity sensor" OR "quantum gas sensor" OR "quantum nano-sensor" OR "quantum atomic sensor"))) AND PY=(2010-2022)) AND DT=(Article)

Database: WOS

Formats: .CSV, Excel and .JSON

I get an error during export sometimes but I am not sure it's the program or the API itself. I have not been able to export large samples without errors, I generally just restart the export, but some exports take 6-hours or so, and it's hard to predict when the error will happen. 1k rows exports are fine, but my exports are usually 10k or more.

3. Attribute selection configuration that you used. Upload your JSON

Expected behavior
A clear and concise description of what you expected to happen.
Export all results from the search query: 14503 without errors. OR if there is a problem with the query or the results, that it would be raised in the validation of the query screen.

Screenshots
If applicable, add screenshots to help explain your problem.

Here's the error message:

Desktop (please complete the following information):

• OS: Windows 11 [e.g. iOS]

Additional context
Add any other context about the problem here.
The folder where the export are being sent to is synchronized with One-Drive, but I don't think that should be a problem.

[gneissone]

@sanjorisa Your query is valid. It results in over 75k results. Perhaps you can try exporting one year at a time to tighten up the search and see if you still get the error? @maloku is out on leave and may not be able to dig into the error you're receiving.

[sanjorisa]

Hi, sorry I didn't follow up on this for a while. I tried to use the API exporter again but it did not work, unfortunately., Yes, I tried to get only one year (around 10k records) but I still got the same error message. I switched to use the WOS to VOSviewer exporter, and it seems to be working okay, so perhaps it is a problem similar to what was described here?

clarivate/wos_api_usecases#23

Thanks for your help!