init

Author: ourway

README.md

@@ -0,0 +1,16 @@
+# Graft
+Graft offers an Elixir implementation of the raft consensus algorithm, allowing the creation of a distributed cluster of servers, where each server manages a replicated state machine. The `Graft.Machine` behaviour allows users to define their own replicated state machines, that may handle user defined client requests.
+
+In this project's documentation you will find terminology that has been defined in the [raft paper](https://raft.github.io/raft.pdf). The docs do not go into specifics of the raft algorithm, so if you wish to learn more about how raft achieves consensus, the [official raft webpage](https://raft.github.io/) is a great place to start.
+
+## Installation
+To install the package, add it to your dependency list in `mix.exs`.
+
+```elixir
+def deps do
+    [{:graft, "~> 0.1.0"}]
+end
+```
+
+## Documentation
+Find the full documentation as well as examples here.

lib/client.ex

@@ -0,0 +1,10 @@
+defmodule Graft.Client do
+    @moduledoc false
+    def request(server, entry) do
+        case GenStateMachine.call(server, {:entry, entry}) do
+            {:ok, response} -> response
+            {:error, {:redirect, leader}} -> request(leader, entry)
+            {:error, msg} -> msg
+        end
+    end
+end

lib/demo.ex

@@ -0,0 +1,22 @@
+defmodule MyMapMachine do
+    @moduledoc false
+    use Graft.Machine
+
+    @impl Graft.Machine
+    def init([]) do
+        {:ok, %{}}
+    end
+
+    @impl Graft.Machine
+    def handle_entry({:get, key}, state) do
+        {state, Map.get(state, key)}
+    end
+
+    def handle_entry({:put, key, value}, state) do
+        {Map.put(state, key, value), :ok}
+    end
+
+    def handle_entry(_default, state) do
+        {state, :error}
+    end
+end

lib/graft.ex

@@ -0,0 +1,107 @@
+defmodule Graft do
+    @moduledoc """
+    An API of the raft consensus algorithm, allowing for custom client requests
+    and custom replicated state machines.
+
+    ## Example
+
+    Let's create a distributed stack. The first step is to set up the state machine.
+    Here we will use the `Graft.Machine` behaviour.
+
+    ```
+    defmodule MyStackMachine do
+        use Graft.Machine
+
+        @impl Graft.Machine
+        def init([]) do
+            {:ok, []}
+        end
+
+        @impl Graft.Machine
+        def handle_entry({:put, value}, state) do
+            {:ok, [value | state]}
+        end
+
+        def handle_entry(:pop, []) do
+            {:noop, []}
+        end
+
+        def handle_entry(:pop, [response | state]) do
+            {response, state}
+        end
+
+        def handle_entry(_, state) do
+            {:invalid_request, state}
+        end
+    end
+    ```
+
+    Now that we have our state machine, we can define the servers that
+    will make up the raft cluster. Each server must have a unique name.
+
+    ```
+    servers = [:server1, :server2, :server3]
+    ```
+
+    With both the servers and state machine, we can now run the graft funtion,
+    which will start the servers and the consensus algorithm.
+
+    ```
+    {:ok, supervisor} = Graft.start servers, MyStackMachine
+    ```
+
+    `Graft.start` returns the supervisor pid from which we can terminate or restart
+    the servers.
+
+    We can now use `Graft.request` to make requests to our consensus cluster.
+    As long as we know at least one server, we can send requests, since the `Graft.Client`
+    module will forward the request if the server we choose is not the current leader.
+
+    ```
+    Graft.request :server1, :pop
+    #=> :noop
+
+    Graft.request :server1, {:put, :foo}
+    #=> :ok
+
+    Graft.request :server1, :pop
+    #=> :foo
+
+    Graft.request :server1, :bar
+    #=> :invalid_request
+    ```
+
+    That completes the distributed stack.
+    """
+
+    @doc """
+    Starts the raft cluster.
+
+    `servers` - list of server names.
+    `machine_module` - module using the `Graft.Machine` behaviour to define the replicated state machine.
+    `machine_args` - an optional list of arguments that are passed to the `init` function of the machine.
+
+    Returns `{:ok, pid}` where `pid` is the supervisor pid which can be used to supervise the cluster's servers.
+    """
+    @spec start(list(atom), module(), list(any)) :: {:ok, pid()}
+    def start(servers, machine_module, machine_args \\ []) do
+        {:ok, supervisor_pid} = Graft.Supervisor.start_link servers, machine_module, machine_args
+        for server <- servers, do: Supervisor.start_child supervisor_pid, [server, servers, machine_module, machine_args]
+        for server <- servers, do: GenStateMachine.cast server, :start
+        {:ok, supervisor_pid}
+    end
+
+    @doc """
+    Print out the internal state of the `server`.
+    """
+    def data(server), do: GenStateMachine.call(server, :data)
+
+    @doc """
+    Make a new client request to a server within the consensus cluster.
+
+    `server` - name of the server the request should be sent to.
+    'entry' - processed and applied by the replicated state machine.
+    """
+    @spec request(atom(), any()) :: response :: any()
+    def request(server, entry), do: Graft.Client.request server, entry
+end

lib/machine.ex

@@ -0,0 +1,73 @@
+defmodule Graft.Machine do
+    @moduledoc """
+    A behaviour module for implementing a replicated machine for the raft consensus
+    algorithm. Look at the `Graft` module docs for examples on how to create such
+    machines.
+    """
+
+    use GenServer
+
+    @typedoc """
+    The state/data of the replicated machine (similar to the 'state' of GenServer).
+    """
+    @type state :: any
+
+    @typedoc """
+    The entry request sent by the client.
+    """
+    @type entry :: any
+
+    @typedoc """
+    The reply to be sent back to the client.
+    """
+    @type response :: any
+
+    @doc """
+    Invoked when the server starts and links to the machine.
+
+    `args` is a list accepted arguments. Look at `Graft.start` to see how to pass
+    in these optional arguments.
+
+    Returning `{:ok, state}`, will initialise the state of the machine to `state`.
+    """
+    @callback init(args :: list(any)) :: {:ok, state}
+
+    @doc """
+    Invoked when a server in the raft cluster is commiting an entry to its log.
+    Should apply the entry to the replicated machine.
+
+    Should return a tuple of the response for the server along with the new state of the
+    replicated machine.
+    """
+    @callback handle_entry(entry, state) :: {response, state}
+
+    defmacro __using__(_opts) do
+        quote location: :keep do
+            @behaviour Graft.Machine
+        end
+    end
+
+    @doc false
+    @impl GenServer
+    def init([module, args]) do
+        {:ok, state} = module.init args
+        {:ok, {module, state}}
+    end
+
+    @doc false
+    @impl GenServer
+    def handle_call({:apply, entry}, _from, {module, state}) do
+        {reply, state} = module.handle_entry entry, state
+        {:reply, reply, {module, state}}
+    end
+
+    @doc false
+    def register(module, machine_args \\ []) do
+        GenServer.start_link __MODULE__, [module, machine_args]
+    end
+
+    @doc false
+    def apply_entry(machine, entry) do
+        GenServer.call machine, {:apply, entry}
+    end
+end

lib/rpc.ex

@@ -0,0 +1,31 @@
+defmodule Graft.AppendEntriesRPC do
+    @moduledoc false
+    defstruct term: -1,             # leader’s term
+              leader_name: nil,     # so follower can redirect clients
+              prev_log_index: -1,   # index of log entry immediately preceding new ones
+              prev_log_term: -1,    # term of prevLogIndex entry
+              entries: [],          # log entries to store (empty for heartbeat; may send more than one for efficiency)
+              leader_commit: -1     # leader’s commit_index
+end
+
+defmodule Graft.AppendEntriesRPCReply do
+    @moduledoc false
+    defstruct term: -1,             # current_term, for leader to update itself
+              success: false,       # true if follower contained entry matching prev_log_index and prev_log_term
+              last_log_index: -1,   # index of last entry in follower's log
+              last_log_term: -1     # term of last entry in follower's log
+end
+
+defmodule Graft.RequestVoteRPC do
+    @moduledoc false
+    defstruct term: -1,             # candidate’s term
+              candidate_name: nil,  # candidate requesting vote
+              last_log_index: -1,   # index of candidate’s last log entry
+              last_log_term: -1     # term of candidate’s last log entry
+end
+
+defmodule Graft.RequestVoteRPCReply do
+    @moduledoc false
+    defstruct term: -1,             # current_term, for candidate to update itself
+              vote_granted: false   # true means candidate received vote
+end