java.sql.SQLNonTransientConnectionException: [Cloudera][JDBC](10060) Connection has been closed.

[jaccoheres]

We are using implyr to connect to impala via JDBC. Usually this works fine, but in ca 25% of the attempt trying to establish the impala connection fails, we get at least three different errors that look a lot like each other:

for(i in 1:100){
  impala <- CreateImpalaConnection(SSLTrustStorePwd = SSLTrustStorePwd)
}

Error in .jcall(conn@jc, "V", "close") : 
  java.sql.SQLNonTransientConnectionException: [Cloudera][JDBC](10060) Connection has been closed.
Error in .jcall(conn@jc, "Ljava/sql/Statement;", "createStatement") : 
  java.sql.SQLNonTransientConnectionException: [Cloudera][JDBC](10060) Connection has been closed.
Error in .jcall(.rJava.class.loader, "[Ljava/lang/String;", "getClassPath") : 
  java.sql.SQLNonTransientConnectionException: [Cloudera][JDBC](10060) Connection has been closed.

I noticed via traceback() that the errors are in different locations:

• The '"[Ljava/lang/String;", "getClassPath")' error is in the .jinit function:
  6: stop(list(message = "java.sql.SQLNonTransientConnectionException: [Cloudera][JDBC](10060) Connection has been closed.", call = .jcall(.rJava.class.loader, "[Ljava/lang/String;", "getClassPath"), jobj = new("jobjRef", jobj = <pointer: 0xac0e2b8>, jclass = "java/sql/SQLNonTransientConnectionException"))) 5: .jcheck() 4: .jcall(.rJava.class.loader, "[Ljava/lang/String;", "getClassPath") 3: .jclassPath() 2: .jinit(classpath = impala_classpath, force.init = TRUE)

• The "Ljava/sql/Statement;", "createStatement" is within the dbConnect function (https://github.com/ianmcook/implyr/blob/master/R/src_impala.R#L126)
  13: .getClassesFromCache(Class) 12: getClassDef(classi, where = where) 11: validObject(.Object) 10: initialize(value, ...) 9: initialize(value, ...) 8: new("jobjRef", jobj = r, jclass = substr(returnSig, 2, nchar(returnSig) - 1)) 7: new("jobjRef", jobj = r, jclass = substr(returnSig, 2, nchar(returnSig) - 1)) 6: .jcall("java/sql/DriverManager", "Ljava/sql/Connection;", "getConnection", as.character(url)[1], as.character(user)[1], as.character(password)[1], check = FALSE) 5: .local(drv, ...) 4: dbConnect(drv, ...) 3: dbConnect(drv, ...) 2: src_impala(drv = drv, ....)

• The "conn@jc, "V", "close"" error is in the db_disconnector function: https://github.com/ianmcook/implyr/blob/master/R/src_impala.R#L506
  and seems to occur less often when I set auto_disconnect to FALSE.

Are these errors known, and what can we do about it? We now wrapped the CreateImpalaConnection function in a try catch loop and try multiple times, but that is not the desired way to do it.

The function CreateImpalaConnection is defined as:

CreateImpalaConnection <- function(SSLTrustStorePwd){
  #Function to create an impala connection
  
  #Imapala settings
  impala_classpath <- "file_path_of_impala_driver"
  
  .jinit(classpath = impala_classpath, force.init = TRUE)
  drv <- JDBC(
    driverClass = "com.cloudera.impala.jdbc4.Driver",
    classPath = impala_classpath,
    identifier.quote = "`"
  )
  impala <- src_impala(
    drv = drv,
    paste0(
      "jdbc:impala://serveradress:21051;",
      "AuthMech=1;KrbRealm=ourdomain;KrbHostFQDN=serveradress;KrbServiceName=impala;",
   "SSL=1;SSLTrustStore=filepath_to_jssecacerts;SSLTrustStorePwd=",SSLTrustStorePwd, ";",
      "CAIssuedCertNamesMismatch=1;"
    ),
    auto_disconnect = FALSE
  )
  return(impala)
}

[ianmcook]

Hi @jaccoheres, thanks for reporting this. A few questions:

• What OS is this running on (Mac, Windows, Linux? If Linux, which distro?)
• What version of the Impala JDBC driver are you using?
• Is it possible to switch to using the odbc package instead of RJDBC? (It's much better.)

[jaccoheres]

Hi @ianmcook thanks for you reply. As answer to your questions:

• We're running on Red Hat Enterprise Linux Server 7.6
• We're using the most recent version of the impala jdbc driver from the cloudera website a few months ago, and more recently my collegue updated to the newest version (I cannot find a way to get the version number out of the jar?), but had the same issues.
• We do not have admin rights so requested our admins to do install the driver. I seems not completely straightforward on Linux systems how to install and configure the driver, but we will try it.