Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow creating a Trino table using dataframe #524

Open
1 task done
andythsu opened this issue Jan 31, 2025 · 0 comments
Open
1 task done

Allow creating a Trino table using dataframe #524

andythsu opened this issue Jan 31, 2025 · 0 comments

Comments

@andythsu
Copy link
Member

andythsu commented Jan 31, 2025

Describe the feature

Hello, we'd like to support creating a Trino table using dataframe.
API would be simple, something like

conn = connect()
cursor = conn.cursor()
df = pd.Dataframe()
cursor.insert_df("foo", df, 800*1000) # 800k

where

def insert_df(table: str, df: pd.Dataframe, max_size: int):
    :param table: table to create/replace.
    :param df: dataframe to insert.
    :param max_size: max number of characters allowed for the SQL query text. This should be <= query.max-length set on Trino to prevent 
    QUERY_TEXT_TOO_LARGE error.

This should be enough for phase 1. For future iterations, we can think about how to upsert data to an existing table....

Describe alternatives you've considered

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant