Merge pull request #8 from edgerun/update-domain-model

Update domain model

Author: thrau

README.md

@@ -30,6 +30,18 @@ You can run the examples we provide in https://github.com/edgerun/faas-sim/tree/
 Where example refers to the specific example package.
 Check out the examples [README](https://github.com/edgerun/faas-sim/tree/master/examples/README.md) for more information.
 
+Run notebooks
+-------------
+
+Notebooks are located in `notebooks`.
+You need to install `faas-sim` in editable state to run the notebooks.
+Inside `notebooks` import modules from `sim`.
+
+To install the project (assuming you already created and activated a virtual environment via `make venv`):
+
+      pip install -e .
+      jupyter notebook
+
 Documentation
 -------------
 

doc/analysis/index.rst

@@ -0,0 +1,49 @@
+.. _analysis:
+
+========
+Analysis
+========
+
+Analysis of simulation results is done by extracting pandas DataFrames upon completion (``sim.env.metrics.extract_dataframe(<name>)``).
+The environment of the simulation contains a ``Metrics`` object used throughout the simulation to log events.
+Those events describe different aspects of a FaaS platform (``FaasSystem``), such as scheduling process, data flow or invocations.
+
+Default logs
+============
+
+The default implementation of a FaasSystem (``DefaultFaasSystem``) logs events of the following processes and can be extracted as dataframe with the associated names:
+
+* Allocation (``'allocation'``)
+* Invocations (``'invocations'``)
+* Scaling (``'scale'``)
+* Scheduling (``'schedule'``)
+* Function Replica Deployment (``'replica_deployment'``)
+* Function Deployments (``'function_deployments'``)
+* Function Deployment (``'function_deployment'``)
+* Function Deployment lifecycle (``'function_deployment_lifecycle'``)
+* Functions (``'functions'``)
+* Flow (``'flow'``)
+* Network (``'network'``)
+* Node utilization (``'node_utilization'``)
+* Function utilization (``'function_utilization'``)
+* Function Execution Times (``'fets'``)
+
+.. hint::
+
+	We provide a basic example in ``examples/analysis/main.py`` and details for each dataframe can be found in the documentation to the corresponding aspect.
+
+Logging
+=======
+
+During the simulation various aspects of the system are being logged.
+Logging happens mainly from the core implementation but some aspects are left to the users.
+Details about those aspects follow later.
+
+``Metrics`` defines a general log function and different out-of-the-box log functions that target specific events in the lifecycle of a FaaS platform.
+
+
+The ``Metrics`` constructor takes a ``RuntimeLogger`` object as initialisation parameter.
+The *logger* stores all records and can be configured by providing a ``Clock`` object, which determines the time of each log event.
+
+.. hint::
+	Checkout ``sim.logging`` for different implementations!

doc/concepts/index.rst

@@ -82,13 +82,47 @@ Think of it like the main API gateway of OpenFaaS or the kube-apiserver of Kuber
 
         def remove(self, fn: FunctionDeployment): ...
 
+        def suspend(self, fn_name: str): ...
+
         def discover(self, fn_name: str) -> List[FunctionReplica]: ...
 
         def scale_down(self, fn_name: str, remove: int): ...
 
         def scale_up(self, fn_name: str, replicas: int): ...
 
-        # ... and several other lookup methods
+        # additional lookup methods:
+        def poll_available_replica(self, fn: str, interval=0.5): ...
+
+        def get_replicas(self, fn_name: str, state=None) -> List[FunctionReplica]: ...
+
+        def get_function_index(self) -> Dict[str, FunctionContainer]: ...
+
+        def get_deployments(self) -> List[FunctionDeployment]:  ...
+
+Conceptually the phases are:
+
+* **deploy**: makes the function invokable and deploys the minimum number of ``FunctionReplica`` instances on the cluster. The number of minimum running instances is configured via ``ScalingConfiguration``.
+
+* **invoke**: the ``LoadBalancer`` selects a replica and simulates the function invocation by calling the ``invoke`` method of the associated ``FunctionSimulator``.
+* **remove**: removes the function from the platform and shutsdown all running replias.
+
+* **discover**: returns all running ``FunctionReplica`` instances that belong to the function.
+
+* **scale_down**: removes the specified number of running ``FunctionReplica`` instances, with respect to the minimum requirement. The current implementation picks the most recent deployed replicas first.
+
+* **scale_up**: deploys the specified number of ``FunctionReplica`` instances but has to respect the maximum number specified in the ``ScalingConfiguration``.
+
+* **suspend**: executes a teardown for all running replicas of a function. (used by ``faas_idler``).
+
+* **poll_available_replica**: repeatedly waits and checks for running replicas of the function.
+
+* **get_replicas**: gets all replicas in the specific state of a function. Returns all replicas in case of ``state == None``.
+
+* **get_function_index**: returns all deployed ``FunctionContainers``.
+
+* **get_deployments**: returns all deployed ``FunctionDeployment`` instances.
+
+.. _Function Simulators:
 
 Function simulators
 ===================
@@ -101,19 +135,19 @@ The FunctionSimulator methods are invoked by the simulator to simulate the the d
 .. code-block:: python
 
     class FunctionSimulator(abc.ABC):
-    
+
         def deploy(self, env: Environment, replica: FunctionReplica):
             yield env.timeout(0)
-    
+
         def startup(self, env: Environment, replica: FunctionReplica):
             yield env.timeout(0)
-    
+
         def setup(self, env: Environment, replica: FunctionReplica):
             yield env.timeout(0)
-    
+
         def invoke(self, env: Environment, replica: FunctionReplica, request: FunctionRequest):
             yield env.timeout(0)
-    
+
         def teardown(self, env: Environment, replica: FunctionReplica):
             yield env.timeout(0)
 
@@ -133,6 +167,7 @@ Conceptually the phases are:
 Each time the simulator creates a new function replica (because of deployment or scaling actions), the SimulatorFactory is called to create or return a FunctionSimulator for that replica.
 The SimulatorFactory can be overwritten to return the same FunctionSimulator every time, create a new instance for each function replica, or any other behavior.
 
+Get more details on function simulators in :ref:`Function Simulator Details` and our examples.
 
 Simulation
 ==========
@@ -166,7 +201,7 @@ Usage example:
 .. code-block:: python
 
     from sim.requestgen import expovariate_arrival_profile, constant_rps_profile
- 
+
     env = ...
     gen = expovariate_arrival_profile(constant_rps_profile(20))
 
@@ -176,7 +211,7 @@ Usage example:
         # send next request
 
 
-.. TODO: upload an example and a cleaned up version of workload_patterns.ipynb
+
 
 The following figure shows several examples and the request patterns the produce:
 
@@ -196,3 +231,9 @@ The second row shows how a constant interarrival distribution can be used to mod
 and how a constant workload profile can be used to model a static workload pattern with randomized interarrivals.
 The last row shows Gaussian random walks (GRW), where each value represents a random sample from a Normal distribution, that is then used as value for :math:`\mu` in the next random sample.
 The request profile can be parameterized with a :math:`\sigma` value that affects the fluctuation over time.
+
+.. hint::
+
+    You can find code examples to generate patterns in our Jupyter Notebook (``workload_patterns.ipynb``) and a
+    simulation example under ``examples/request_gen``.
+

doc/contents.rst

@@ -13,6 +13,9 @@ Documentation for faas-sim
 
    index
    concepts/index
+   system/index
+   analysis/index
+   function_sims/index
    examples/index
 
 Indices and tables

doc/figures/default-faas-system-components.jpg

@@ -0,0 +1,49 @@
+.. _Function Simulator Details:
+
+====================
+Function Simulators
+====================
+
+This section is dedicated to showcase a selection of pre-defined function simulators and gives details on how to implement one yourself.
+
+.. attention::
+
+    Make sure you've familiarized yourself with :ref:`Resources` and :ref:`Function Simulators`.
+
+As our work is heavily influenced by the design and architecture of `OpenFaaS`_, we provide two implementations of `FunctionSimulator` that model the behavior of *forking* and *HTTP* modes (see `Watchdog modes`_).
+
+The implementations are located in ``sim/faas/watchdogs.py`` and can be imported with:
+
+.. code-block:: python
+
+    from sim.faas import ForkingWatchdog, HTTPWatchdog
+
+The abstract class that represents the general Watchdog concept looks like this:
+
+.. code-block:: python
+
+    class Watchdog(FunctionSimulator):
+
+    def claim_resources(self, env: Environment, replica: FunctionReplica, request: FunctionRequest): ...
+
+    def release_resources(self, env: Environment, replica: FunctionReplica, request: FunctionRequest): ...
+
+    def execute(self, env: Environment, replica: FunctionReplica, request: FunctionRequest): ...
+
+
+The ``HTTPWatchdog`` uses a queuing mechanism to simulate works and claims resources after the request received a token (i.e., a worker is available).
+The ``ForkingWatchdog`` claims and executes each request immediately without further delay.
+
+.. attention::
+
+    When using the ``ForkingWatchdog`` make sure that you limit manually the requests due to RAM usage for each fork.
+
+The following figure shows the log events that happen during the execution with the ``HTTPWatchdog`` and also depicts the interaction between different system components.
+
+.. figure:: ../figures/functionsim-invoke-times.png
+    :align: center
+
+
+.. _OpenFaaS: https://docs.openfaas.com/
+.. _Watchdog modes: https://github.com/openfaas/of-watchdog#modes
+

doc/figures/functionsim-invoke-times.png

@@ -0,0 +1,92 @@
+.. _system:
+
+========
+System
+========
+
+In the following we describe the inner workings of our *FaasSystem* implementation.
+The API of the `FaasSystem` is designed around real life requirements and represents typical operations that can be found in a typical API Gateway (such as in `OpenFaaS`_).
+We provide a default implementation of ``FaasSystem``, called ``DefaultFaasSystme`` in ``sim.faas.system.py``.
+The following explains the inner workings of our implementations, which components are used and how you can configure the system.
+
+We recall the methods a ``FaasSystem`` has to implement:
+
+.. code-block:: python
+
+    class FaasSystem(abc.ABC):
+
+        def deploy(self, fn: FunctionDeployment): ...
+
+        def invoke(self, request: FunctionRequest): ...
+
+        def remove(self, fn: FunctionDeployment): ...
+
+        def discover(self, fn_name: str) -> List[FunctionReplica]: ...
+
+        def scale_down(self, fn_name: str, remove: int): ...
+
+        def scale_up(self, fn_name: str, replicas: int): ...
+
+        def suspend(self, fn_name: str): ...
+
+        # ... and several other lookup methods
+
+To implement these functions, our system contains the following state:
+
+.. attention::
+
+    This section provides insights into the current implementation of ``FaasSystem``.
+    Be aware that this is subject to change and using lookup methods is much safer with respect to updates.
+
+
+* ``env: Environment``: used to access global configured components (i.e., ``Metrics``, ``SimulatorFactory``, ``ClusterContext``)
+* ``function_containers: Dict[str, FunctionContainer]``: stores all available function containers from the deployed functions
+* ``replicas: Dict[str, List[FunctionReplica]``: collects all FunctionReplicas under the name of the corresponding FunctionDeployment
+* ``scheduler_queue: simpy.Store``: contains function replicas that need to be scheduled. ``scale_up`` puts replicas into the queue and ``run_schedule_worker`` polls from it.
+* ``load_balancer: LoadBalancer``: called upon ``invoke`` to select replica that handles the invocation. (currently round-robin)
+* ``functions_deployments: Dict[str, FunctionDeployment``: stores the deployed functions and gets modified by ``deploy`` and ``remove``.
+* ``replica_count: Dict[str, int]``: counts the number of active replica per ``FunctionDeployment``
+* ``functions_definitions: Counter``: counts the number of replica per ``FunctionContainer``
+
+.. _OpenFaaS: https://docs.openfaas.com/
+
+
+.. _Resources:
+
+Resources
+=========
+
+Simulation of resources has to be implemented by users due to necessary flexibility regarding the implementation of a ``FunctionSimulator``. In example, the execution of a function can be delayed through queuing.
+Therefore, resources are not immediately used and it's the ``FunctionSimulator's`` responsibility to consume them at the right time.
+
+*faas-sim* offers a standardized interface to manage resources which is based on dictionaries.
+This allows *faas-sim* to implement common componnents (such as resource monitoring for nodes & functions, as well as an implementation of `Kubernetes' HPA`_)
+Resources get added up.
+
+The following code shows an example on consuming resources:
+
+.. code-block:: python
+
+    class CpuConsumingSim(FunctionSimulator):
+
+        def __init__(self, queue: simpy.Resource):
+            self.queue = queue
+
+        def invoke(self, env: Environment, replica: FunctionReplica, request: FunctionRequest):
+            token = self.queue.request()
+            yield token
+
+            # definition of resources is up to users
+            # here we assume that a function call needs 20% cpu usage of the whole call
+            env.resource_state.put_resource(replica, 'cpu', 0.2)
+
+            yield env.timeout(1)
+
+            # release resources
+            env.resource_state.remove_resource(replica, 'cpu', 0.2)
+
+
+The ``Environment`` object contains a resource monitor which continuously collects the momentary resource utilization and puts into the ``MetricsServer`` which can be used to query the average usage of a certain resource.
+
+.. _Kubernetes' HPA: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/
+

doc/function_sims/index.rst

@@ -0,0 +1,43 @@
+import logging
+
+import examples.basic.main as basic
+from examples.custom_function_sim.main import CustomSimulatorFactory
+from sim.faassim import Simulation
+
+logger = logging.getLogger(__name__)
+
+
+def main():
+    logging.basicConfig(level=logging.INFO)
+
+    # prepare simulation with topology and benchmark from basic example
+    sim = Simulation(basic.example_topology(), basic.ExampleBenchmark())
+
+    # override the SimulatorFactory factory
+    sim.create_simulator_factory = CustomSimulatorFactory
+
+    # run the simulation
+    sim.run()
+
+    dfs = {
+        'allocation_df': sim.env.metrics.extract_dataframe('allocation'),
+        'invocations_df': sim.env.metrics.extract_dataframe('invocations'),
+        'scale_df': sim.env.metrics.extract_dataframe('scale'),
+        'schedule_df': sim.env.metrics.extract_dataframe('schedule'),
+        'replica_deployment_df': sim.env.metrics.extract_dataframe('replica_deployment'),
+        'function_deployments_df': sim.env.metrics.extract_dataframe('function_deployments'),
+        'function_deployment_df': sim.env.metrics.extract_dataframe('function_deployment'),
+        'function_deployment_lifecycle_df': sim.env.metrics.extract_dataframe('function_deployment_lifecycle'),
+        'functions_df': sim.env.metrics.extract_dataframe('functions'),
+        'flow_df': sim.env.metrics.extract_dataframe('flow'),
+        'network_df': sim.env.metrics.extract_dataframe('network'),
+        'node_utilization_df': sim.env.metrics.extract_dataframe('node_utilization'),
+        'function_utilization_df': sim.env.metrics.extract_dataframe('function_utilization'),
+        'fets_df': sim.env.metrics.extract_dataframe('fets')
+    }
+
+    logger.info('Mean exec time %d', dfs['invocations_df']['t_exec'].mean())
+
+
+if __name__ == '__main__':
+    main()