Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Beyond SQL_ACTIVE_STATEMENT limit" issue due to a SELECT query from table/view reflection not being closed #79

Open
gayleeegayle opened this issue Nov 27, 2018 · 0 comments

Comments

@gayleeegayle
Copy link

I encountered the same issue mentioned in #67. I think I've pinpointed where the error's coming from.

To test, I created a script that only tries to reflect 20 views.

After several tests and checking the logs, noticed the following:

  1. Only 15 out of 20 views are successfully reflected. In the MIDDLE of the SELECT queries to reflect the 16th view, the "Beyond SQL_ACTIVE_STATEMENT limit" error is raised.
  2. ODBC Teradata Driver supports only 16 open result sets.
  3. From the logs, noticed that after executing this particular SELECT query 16 times...
SELECT tablekind 
FROM dbc.tablesV
WHERE DatabaseName (NOT CASESPECIFIC) = CAST(? as VARCHAR(128)) (NOT CASESPECIFIC)
AND TableName=? AND tablekind='V'

...The query after it (shown below) will fail, raising the "Beyond SQL_ACTIVE_STATEMENT limit" error.

SELECT * 
FROM dbc.ColumnsV 
WHERE DatabaseName (NOT CASESPECIFIC) = CAST(? as VARCHAR(128)) (NOT CASESPECIFIC)
AND TableName=?

From the Exception traceback, the SELECT queries used in my reflection is in the sqlalchemy_teradata package's dialect.py.

Checking all the connection.execute syntax used, I noticed that cursor/result set of the FIRST query under the get_columns method may not be closed properly. .rowcount doesn't automatically close the result set.

That's why after the first query (SELECT from dbc.tablesV) is executed the 16th time, there'll be 16 open result sets. When the next query (SELECT from dbc.ColumnsV) is executed, it'll then fail and raise the error.

To address the issue, replaced these lines:

res = connection.execute(stmt, schema=schema, table_name=table_name).rowcount
helpView = (res == 1)

With these:

res = connection.execute(stmt, schema=schema, table_name=table_name)
helpView = (res.rowcount == 1)
res.close()

So the edited get_columns method now looks like this:
image

Afterwards, all 20 views in my test script were successfully reflected.
Would this be an appropriate fix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant