Skip to content

Empty results on large queries when using fetchall() #44

Open

Description

@gh-dzuluaga

Hello there.

I've found that these two examples will not always return the same thing on large queries:

  1. fetchall
cur.execute(large_query)
rows = cur.fetchall()
print(rows)
  1. iterator
for r in cur.execute(large_query):
  print(r)

The fetchall approach ends up returning an empty list every time I use it when running large queries, but the iterator approach prints the result successfully

Does anyone know why this may be happening?

I don't know how I should supply a working example to debug this problem, as my queries all contain domain logic of my company. Nevertheless, I can say that I'm connecting around 8 Postgres databases with 100k rows per table and equal schemas, and I union the tables from all databases together with UNION views, like this:

CREATE OR REPLACE VIEW hive.default."table_unioned_view" AS 
SELECT 'database1' AS database, "other_column" FROM database1.public.table
UNION ALL
SELECT 'database2' AS database, "other_column" FROM database2.public.table
UNION ALL
SELECT 'database3' AS database, "other_column" FROM database3.public.table;

And the big query that I'm running uses these type of views

Activity

  1. findepi commented on Sep 29, 2020

    @findepi
    Member

    Would you like to try to come up with a test case reproducing the problem?
    See https://github.com/prestosql/presto-python-client/blob/a7438556213bcf954f8a7a5063c7823bbb177edb/integration_tests/test_dbapi.py#L59-L68 for some example test using dockerized Presto

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions