Skip to content

"Beyond SQL_ACTIVE_STATEMENT limit" issue due to a SELECT query from table/view reflection not being closed #79

Open
@gayleeegayle

Description

@gayleeegayle

I encountered the same issue mentioned in #67. I think I've pinpointed where the error's coming from.

To test, I created a script that only tries to reflect 20 views.

After several tests and checking the logs, noticed the following:

  1. Only 15 out of 20 views are successfully reflected. In the MIDDLE of the SELECT queries to reflect the 16th view, the "Beyond SQL_ACTIVE_STATEMENT limit" error is raised.
  2. ODBC Teradata Driver supports only 16 open result sets.
  3. From the logs, noticed that after executing this particular SELECT query 16 times...
SELECT tablekind 
FROM dbc.tablesV
WHERE DatabaseName (NOT CASESPECIFIC) = CAST(? as VARCHAR(128)) (NOT CASESPECIFIC)
AND TableName=? AND tablekind='V'

...The query after it (shown below) will fail, raising the "Beyond SQL_ACTIVE_STATEMENT limit" error.

SELECT * 
FROM dbc.ColumnsV 
WHERE DatabaseName (NOT CASESPECIFIC) = CAST(? as VARCHAR(128)) (NOT CASESPECIFIC)
AND TableName=?

From the Exception traceback, the SELECT queries used in my reflection is in the sqlalchemy_teradata package's dialect.py.

Checking all the connection.execute syntax used, I noticed that cursor/result set of the FIRST query under the get_columns method may not be closed properly. .rowcount doesn't automatically close the result set.

That's why after the first query (SELECT from dbc.tablesV) is executed the 16th time, there'll be 16 open result sets. When the next query (SELECT from dbc.ColumnsV) is executed, it'll then fail and raise the error.

To address the issue, replaced these lines:

res = connection.execute(stmt, schema=schema, table_name=table_name).rowcount
helpView = (res == 1)

With these:

res = connection.execute(stmt, schema=schema, table_name=table_name)
helpView = (res.rowcount == 1)
res.close()

So the edited get_columns method now looks like this:
image

Afterwards, all 20 views in my test script were successfully reflected.
Would this be an appropriate fix?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions