Making highly-available and scalable Node.js applications easy to build

[pgte]

Title

Making highly-available and scalable Node.js applications easy to build

Description

Micro-services are a great concept: they make developer's lives easier by splitting each business domain into it's own separate autonomous service. But how do these services handle data persistence? Usually, they do this by using a separate data layer that implements a specific database product.

This approach has many disadvantages, and here are some:

Complexity: this separate data layer is cumbersome to deploy and manage: it has it's own run-time and new set of dependencies, introducing a lot of complexity.

Distance: having a separate data layer means that your application process must access the network every time it requires to read or write data.

Availability: Having the data as a new separate layer introduces at least one new point of failure and a set of new failure modes. Several techniques allow the database to be highly available, but almost always this comes with a big cost of adding significant complexity to the application code.

Aiming to solve these problems, I propose a style of architecture and pure Node.js-based implementation.

Learning objectives

By watching this talk attendees will be familiarized with distributed systems challenges and some solutions. They will learn about strong and eventual consistency, data partitioning, data locality techniques, distributed consensus and the Raft consensus protocol.

City of residence

Funchal, Portugal