Pure non blocking IMAP request execution

[chibenwa]

What

ImapAsyncSession returns a Java7 future that supports only get() operation.

This forces us to await its value in a fork join pool:

private[command] object ImapSessionExecutor {
  def listen(self: ActorRef, userId: UserId, getResponse: ImapResponses => Response)(logger: Logger)(response: ImapFuture[ImapAsyncResponse]): Unit = {
    listenWithHandler(self, userId, getResponse, _ => ())(logger)(response)
  }

  def listenWithHandler[T](self: ActorRef, userId: UserId, getResponse: ImapResponses => Response, callback: Future[ImapAsyncResponse] => T)(logger: Logger)(response: ImapFuture[ImapAsyncResponse]): T = {
    import collection.JavaConverters._

    callback(Future {
      val responses = response.get()
      val responsesList = ImapResponses(responses.getResponseLines.asScala.to[Seq])
      logger.trace(s"On response for $userId :\n ${responsesList.mkString("\n")}")
      self ! getResponse(responsesList)
      responses
    })
  }
}

Also this leads to many threads being blocked, thread count is proportional to the count of in-flight requests, capped at 256.

Here's what JSTACK has to say regarding this.

"scala-execution-context-global-18" #18 daemon prio=5 os_prio=0 cpu=459.99ms elapsed=166.07s tid=0x00007f54b8a0f800 nid=0x5228 in Object.wait()  [0x00007f54c25e6000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait([email protected]/Native Method)
	- waiting on <no object reference available>
	at com.yahoo.imapnio.async.client.ImapFuture.get(ImapFuture.java:105)
	- waiting to re-lock in wait() <0x00000000dc382340> (a java.lang.Object)
	at com.linagora.gatling.imap.protocol.command.ImapSessionExecutor$.$anonfun$listenWithHandler$1(ImapSessionExecutor.scala:22)
	at com.linagora.gatling.imap.protocol.command.ImapSessionExecutor$$$Lambda$566/0x0000000840537c40.apply(Unknown Source)
	at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:658)
	at scala.concurrent.Future$$$Lambda$269/0x00000008402fe040.apply(Unknown Source)
	at scala.util.Success.$anonfun$map$1(Try.scala:255)
	at scala.util.Success.map(Try.scala:213)
	at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
	at scala.concurrent.Future$$Lambda$270/0x00000008402fe840.apply(Unknown Source)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
	at scala.concurrent.impl.Promise$$Lambda$204/0x0000000840261040.apply(Unknown Source)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec([email protected]/ForkJoinTask.java:1426)
	at java.util.concurrent.ForkJoinTask.doExec([email protected]/ForkJoinTask.java:290)
	at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec([email protected]/ForkJoinPool.java:1020)
	at java.util.concurrent.ForkJoinPool.scan([email protected]/ForkJoinPool.java:1656)
	at java.util.concurrent.ForkJoinPool.runWorker([email protected]/ForkJoinPool.java:1594)
	at java.util.concurrent.ForkJoinWorkerThread.run([email protected]/ForkJoinWorkerThread.java:177)

How to fix

• 1. Contribute the CompletableFuture facade to imapNIO cf API with CompletableFuture/Stage instead of/addition to Future yahoo/imapnio#86
• 2. Adopt it within gatling-imap

[chibenwa]

Before

Huge variability, fron 285ms (one time), 700ms several time, and 3s + mean time some times too. Tests were unreliable.

Backend was partially stressed, results hardly exploitable...

After

Constantly reproduced! Which makes the testing more trustworthy.

Conclusion

This change makes Gatling IMAP meaningfull for larger scale injection scenario that actually really stress the backends.

[Arsnael]

Don't forget to push the PR as I didn't see one regarding this yet :)

[chibenwa]

Don't forget to push the PR as I didn't see one regarding this yet :)

It depends on non released changes...

[chibenwa]

yahoo/imapnio#118 is merged

[chibenwa]

https://mvnrepository.com/artifact/com.yahoo.imapnio/imapnio.core/4.3.8 released. Time to rebase @chibenwa master branch...