Skip to content

Streaming results leaks memory #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pabloromanelli opened this issue Jul 20, 2017 · 0 comments
Open

Streaming results leaks memory #82

pabloromanelli opened this issue Jul 20, 2017 · 0 comments

Comments

@pabloromanelli
Copy link

AsyncSQLRowStream leeks memory because when making a query that returns a lot of rows, every row is stored inside the result set:

io.vertx.ext.asyncsql.impl.AsyncSQLRowStream

  AsyncSQLRowStream(QueryResult qr) {
    final Option<ResultSet> rows = qr.rows();
    if (rows.isDefined()) {
      rs = rows.get();
      cursor = rs.iterator();

com.github.mauricio.async.db.general.MutableResultSet

private val rows = new ArrayBuffer[RowData]()

And only when the query finishes, the result set is returned and the stream starts.
This behavior makes my application fail by OOM and can't handle streaming of large results.

This is not a feature currently supported by the client, but there is an open issue about this:
mauricio/postgresql-async#108

It would have been very useful for me to have a warning about this problem on the documentation before starting to code. Please consider adding it until fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant