Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty results on large queries when using fetchall() #44

Open
gh-dzuluaga opened this issue Sep 28, 2020 · 1 comment
Open

Empty results on large queries when using fetchall() #44

gh-dzuluaga opened this issue Sep 28, 2020 · 1 comment
Labels
bug Something isn't working

Comments

@gh-dzuluaga
Copy link

gh-dzuluaga commented Sep 28, 2020

Hello there.

I've found that these two examples will not always return the same thing on large queries:

  1. fetchall
cur.execute(large_query)
rows = cur.fetchall()
print(rows)
  1. iterator
for r in cur.execute(large_query):
  print(r)

The fetchall approach ends up returning an empty list every time I use it when running large queries, but the iterator approach prints the result successfully

Does anyone know why this may be happening?

I don't know how I should supply a working example to debug this problem, as my queries all contain domain logic of my company. Nevertheless, I can say that I'm connecting around 8 Postgres databases with 100k rows per table and equal schemas, and I union the tables from all databases together with UNION views, like this:

CREATE OR REPLACE VIEW hive.default."table_unioned_view" AS 
SELECT 'database1' AS database, "other_column" FROM database1.public.table
UNION ALL
SELECT 'database2' AS database, "other_column" FROM database2.public.table
UNION ALL
SELECT 'database3' AS database, "other_column" FROM database3.public.table;

And the big query that I'm running uses these type of views

@findepi
Copy link
Member

findepi commented Sep 29, 2020

Would you like to try to come up with a test case reproducing the problem?
See https://github.com/prestosql/presto-python-client/blob/a7438556213bcf954f8a7a5063c7823bbb177edb/integration_tests/test_dbapi.py#L59-L68 for some example test using dockerized Presto

@findepi findepi added the bug Something isn't working label Sep 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

2 participants