Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Would it be possbile to walk it with Datatable.frame type in the future? #510

Open
plusmid opened this issue Apr 1, 2024 · 1 comment
Open
Labels
proposal Vote if you want it Only will be started when enough votes collected.

Comments

@plusmid
Copy link

plusmid commented Apr 1, 2024

https://github.com/h2oai/datatable
It has higher performance reading and manipulating big csv data than pandas/polars/modin. But I can't walk this datatable.frame type.

@ObservedObserver
Copy link
Member

Hi @plusmid, currently, pygwalker does not utilize the pandas API for dataframe-related computations.

When you pass a dataframe to pygwalker, it converts it into either a duckdb format or an in-browser memory JavaScript array. Therefore, the performance remains consistent across various dataframes.

We are exploring more native computation methods using the dataframe API. However, it appears that in most scenarios, using duckDB or integrating pygwalker with Snowflake yields quite good performance.

@ObservedObserver ObservedObserver added Vote if you want it Only will be started when enough votes collected. proposal labels Apr 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
proposal Vote if you want it Only will be started when enough votes collected.
Projects
None yet
Development

No branches or pull requests

2 participants