Carjump challenge

Hi! Please note this is an org mode document. It reads best in an editor that supports it (like emacs) but since it relies on plain text any other editor will do with some minor limitations.

Dependencies

You will need to install SBT which requires a JDK installation.

I recommend to start the SBT shell with additional JVM options like this:

sbt -J-Xms512M -J-Xmx1024M -J-Xss1M -J-XX:+CMSClassUnloadingEnabled

Run the tests:

sbt test

This will execute unit and integration tests.

Run the application:

sbt run

Now you can send requests to the application:

curl -vvv http://localhost:9000/42

Design decisions

I did not implement the compression as requested.

• Having a Seq[A] in the compression signature requires me to have Seq[A] in memory, so why would I compress it? I am using akka streams to compress the data without loading everything into memory at once, so I changed the signature accordingly.
• Implementing decompress is trivial but I left it as ??? because it is actually not useful for the task at hand. We are searching for an index in the RLE’d data and it makes no sense to unpack the whole thing in memory just to find an element. I implemented a tailrecursive function that finds the element at the requested index in max O(n) and with small memory footprint. The gist is that given we know the size of each RLE chunk it is easy to find a specific index without traversing all of the repeated elements (that’s the point of RLE imho!)

The actor

• I added a supervisor to ensure resilience in case the fetching actor crashes (except for critical cases like out of memory errors, the supervisor will restart the child)
• I didn’t implement a separate actor for cache. It’s more work and IMHO it doesn’t bring any real value.
• I am not testing the actor FSM (could use akka testkit for that) because it is very simple.
• I am not blocking the application until the actor is able to fetch the data. While this could be done there is a case to be made that the application should handle this error gracefully - if we block until we have cached data then the application can be started if and only if the carjump challenge api is working. That’s a lot of coupling. For a production app I would evaluate whether the application can perform other work or if everything is degraded and decide from there. In our challenge case the status controller still works for example.

Tests

I have some unit tests where it makes sense but mainly since the problem domain is so narrow I rely on wiremock tests to ensure that the desired functionality works end to end (esp. the akka streams code). I didn’t write tests for the controller because it’s behaviour is so simple and well-defined.

Exercise text

1. Scheduled non-blocking fetch

Create a service (daemon) which fetches data from our endpoint A at x second intervals and cache results in memory (after each fetch clear the existing cache and populate it with new items)

Our endpoint: GET / A returns a list of items separated by ‘\n’ character

Full URL: http://challenge.carjump.net/A

Constant x should be configurable in reference.conf or application.conf file.

1. HTTP interface

Create HTTP interface that allows clients to access the data at a given index

• use HTTP framework of your choice

GET /(index) return an item at a given index

Please provide instructions in README file how to run your server locally.

(Bonus) 3. Actors

Separate fetching and storage into 2 actors

(Bonus) 4. Compression

Items returned by endpoint A will contain repeated duplicates at high frequencies. Modify your cache to use Run-length encoding (RLE) compression for internal storage. Your compression and decompression should be some concrete implementation of the following trait.

trait Compressor { def compress[A]: Seq[A] => Seq[Compressed[A]] def decompress[A]: Seq[Compressed[A]] => Seq[A] }

sealed trait Compressed[+A] case class Single[A](element: A) extends Compressed[A] case class Repeat[A](count: Int, element: A) extends Compressed[A]

=======================================================================================================================

Constraints

• The only accepted language is Scala.
• There should be no external dependencies except for

– testing, – configuration, – or the HTTP interface.

• Behaviour can be added to provided traits and classes.

Delivery

Response should be in the form of an sbt project, either uploaded to some git repository or emailed back as .zip file (or tarball). In any case, the code should compile preferably without warnings.