system crash opening large (compressed) file

[tompropst]

Small description

My system will consistently become unresponsive some time after opening a large data file in bz2 compressed JSON

Some requested data is not available given the outcome of the issue

Expected result

Maintain system control regardless of whether or not vd is functioning

Actual result with screenshot

System will hang indefinitely

Steps to reproduce with sample data and a .vd

I cannot provide the exact data file as it contains sensitive information. If needed, I can try to create a sanitized data set however it will be quite large.

1. Launch vd
2. Navigate to the location of a large data file
   • My case specifically had a 76.5MB bz2 compressed JSON data
   • The file would expand to 2.7GB of JSON
   • I have to mention I was thrilled to see that vd will open the compressed file at all
3. Open the large file
4. Observe that the file appears to open however part way through processing all the rows, everything stops and the system is completely unresponsive

My system is admittedly not up to this task however I would expect either a failure to attempt to load the file or if possible, a partial load of the file.

System:

• Ubuntu 22.04
• Lenovo Thinkpad with Intel I5 processor
• 16GB of memory

Please attach the commandlog (saved with Ctrl-D) to show the steps that led to the issue.

Additional context
Please include the version of VisiData and Python.

• VisiData 3.0.2
• Python 3.10.12