Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG-REPORT] Printing vaex df after sort running out of memory #2398

Open
meta-ks opened this issue Oct 30, 2023 · 0 comments
Open

[BUG-REPORT] Printing vaex df after sort running out of memory #2398

meta-ks opened this issue Oct 30, 2023 · 0 comments

Comments

@meta-ks
Copy link

meta-ks commented Oct 30, 2023

Description
Environment: IPython 8.15.0 + Python 3.10.13
I am encountering an issue where sorting a Vaex DataFrame and then attempting to print it results in running out of memory. I am able to print(just a sample in ipython env) the original df without any issues.

Code:

vdf = vaex.open('large_8GB_arrow_file.arrow', progress=True)
vdf  #works no issue
vdf2 = vdf.sort(['col1', 'col2', ....])   #5cols
vdf2  #hangs and crashes

Software information

  • Vaex version:
'vaex-core': '4.17.1',
'vaex-viz': '0.5.4',
'vaex-hdf5': '0.14.1',
'vaex-server': '0.9.0',
'vaex-astro': '0.9.3',
'vaex-jupyter': '0.8.2',
'vaex-ml': '0.18.3'
  • Vaex was installed via: pip
  • OS: Ubuntu 22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant