started adding unit tests + docs for SPB plugin

docs/experiments/index.md

@@ -0,0 +1,68 @@
+# Experiments
+
+!!! warning "Unfinished Feature" 
+	Note that everything mentioned on this site is still very preliminary and not fully fleshed out.
+
+For accessing data from the SPB experiment at European XFEL see [acces SPB data](spb).  
+*xFrame* supports the definition of *experiments* whose sole purpose is to provide access to XFEL scattering data.
+
+
+!!! warning "Only for developers"
+Conceptually an *experiment* together with its settings file should contain all information necessary to create scattering patterns on a grid in reciprocal space for a given real experimental setup.  
+Such data includes, e.g.:
+
+- X-ray wavelength, detector geometry, distance between sample and detector, detector mask, backround to subtract,  normalization range, ...
+
+A new *experiment* can be created as follows.  
+
+1. Choose a `HOME_FOLDER` as described in the [Framework](../framework/getting_started) section.
+2. Create a custom named folder
+```console
+HOME_PATH/experiments/my_experiment
+```
+3. place an `experiments.py` file in this folder with the following contents:
+
+```py linenums='1'
+from xframe.interfaces import ExperimentWorkerInterface
+from typing import Generator
+
+class DataSelection:
+	def __init__(_dict:dict):
+		pass
+
+class ExperimentWorker(ExperimentWorkerInterface):
+	DataSelection = DataSelection
+	def get_data(self,selection : DataSelection) -> Generator[int,None,None]:
+		pass
+	def get_reciprocal_pixel_grid(self) -> dict:
+		pass
+	def get_geometry(self) -> dict:
+		pass
+	def run(self):
+		pass
+```
+
+`DataSelection` functions as a datastructure that contains all information needed to select a pice of scattering patterns (i.e. pulse_numbers, train_ids ....).  
+`get_data` takes a `DataSelection` object and returns a generator that retrieves chunks of processed scattering patterns.  
+`get_reciprocal_pixel_grid` returns the detector pixel coordinates in reciprocal space.  
+`get_geometry` returns real space information about the detector geometry.  
+
+## Connection to *projects*
+If an experiment has been given, either by calling 
+```py linenums='1'
+import xframe
+xframe.select_experiment('my_experiment','default')
+xframe.import_selected_experiment()
+```
+or by specifying the following command line arguments when a project is started,
+```console
+$ xframe -e my_experiment -eopt default
+```
+one can access an instance of the above defined ExperimentWorker via
+```
+xframe.experiment
+```
+In the above expamle `default` is the name of a settings file placed in either of the following paths  
+`HOME_PATH/settings/experiments/my_experiment/default.yaml`  
+`HOME_PATH/experiments/my_experiment/settings/default.yaml`
+

docs/experiments/spb.md

@@ -0,0 +1,91 @@
+# Getting data form the SPB experiment at EuXFEL
+!!! note "Features"
+	* Access processed chunks of scattering patterns as (16,512,128) numpy arrays
+	* Select patterns by indices or slices of train_ids, cell_ids, pulse_ids and/or frame_ids
+	* Get metadata for each pattern (mask, gain, baseline_shift)
+	* Filter patterns or mask pixels by gain,lit pixels,mean,max,... 
+	* Normalize patterns
+	* Specify regions of interest.
+	* Not implemented yet! Direct data input to the FXS *project*
+
+!!! warning "Assumes xFrame is running on a Maxwell node" 
+!!! warning "Install" 
+	If you have not installed *xFrame* yet and only want to use it to acces SPB data do the following.
+
+    1. Create a conda environment using the commands
+    	 ```
+    	 module load maxwell mamba
+    	 . mamba-init
+    	 mamba create --name xframe python=3.8
+    	 mamba activate xframe
+    	 ```
+    	 You can alternatively call the environment by whatever name you like instead of "xframe"
+     3. Install xframe
+    	```
+    	pip install 'xframe[SPB]'
+		```
+
+	If you already have installed *xFrame* note that the *SPB* experiment requires an additional dependency. In the mamba environment in which *xFrame* is installed you can either call
+	```console
+	$ pip install SharedArray
+	```
+	or 
+	```console
+	$ pip install `xframe[SPB]`
+	```
+
+## Access data
+To get started set a home path for *xFrame*, if you have not done so already, by calling:
+```console
+$ xframe --setup_home HOME_PATH
+```
+where `HOME_PATH` is a custom path to where ever you want *xFrame* to store files by default, if no value for `HOME_PATH` is given `~/.xframe` will be used. 
+
+You should now be able to find the tutorial settings file at 
+```
+HOME_PATH/settings/experiments/SPB/tutorial.yaml
+```
+it has the following contents:
+```yaml linenums="1"
+bad_cells:
+  command: '[0]' # also accepts slice or numpy arrays np.arange(0,1)
+
+sample_distance: 285 # in mm
+detector_origin: [0,0,217] # In laboratory coordinates [mm]
+x_ray_energy: 8000 # In ev
+
+IO:
+  folders:
+    exp_data_base: '/gpfs/exfel/exp/SPB/202202/p003046/'
+```
+To continue change the `exp_data_base` value to the path of an experiment you have access to (don't forget the  '/' at the end on the path).  
+With this setup you can do the following in python.
+```py linenums="1"
+import xframe
+xframe.select_experiment('SPB','tutorial')
+xframe.import_selected_experiment()
+
+exp = xframe.experiment
+run=20
+selection = exp.DataSelection(20,n_frames=2000)
+data_generator = exp.get_data(selection)
+
+for chunk in data_generator:
+	print(chunk.keys())
+	del(chunk)
+```
+After calling the `select_experiment` method the experiments settings and database modules can be accessed via
+```py
+xframe.settings.experiment
+xframe.database.experiment
+```
+As can be seen the result of calling `get_data` is a generator that yields chunks of scattering patterns according to the provided `DataSelection` instance.
+
+If you
+### Data Selection
+The data 
+
+
+## Filters
+## Region of interest
+To get access to 

docs/fxs/getting_started.md

@@ -4,7 +4,7 @@ First lets setup the xframe home directory by calling:
 ```console
 $ xframe --setup_home HOME_PATH
 ```
-and substituting `HOME_PATH` with where ever you want xframe to store files by default, if no value for `HOME_PATH` is given `~/.xframe` will be used. 
+and substituting `HOME_PATH` with where ever you want *xFrame* to store files by default, if no value for `HOME_PATH` is given `~/.xframe` will be used. 
 
 ??? note "Short version of getting started"
 	If you are willing to skip first step of extracting correlations from scattering patterns, the short version of this tutorial consists of executing the commands	
@@ -258,7 +258,6 @@ ccd.h5
 ├── angular_points
 ├── xray_wavelength
 ├── average_intensity
-├── angular_points
 └── cross_correlation
 	└── I1I1 
 ```

docs/fxs/index.md

@@ -7,6 +7,9 @@ The xFrame's `fxs` toolkit provides the following capabilities:
  3. **Reconstruction** of the electron density based on rotational invarants.
  4. **Alignment and Averaging** of reconstructions.
 
+!!! info "No data required"
+	To test the `fxs` toolkit no data is required, after installation simply follow the [getting started](getting_started) guide. 
+
 ## Installation	
 The fxs toolkit comes bundled as part of xFrame. The additional dependencies needed by `fxs` can be automatically installed via:
 

mkdocs.yml

@@ -16,6 +16,9 @@ nav:
         - Data Access: framework/data_access.md
         - Multiprocessing: framework/multiprocessing.md
         - GPU Access: framework/gpu_access.md
+  - Experiments (preliminary):
+      - experiments/index.md
+      - SPB (at EuXFEL): experiments/spb.md
 
 theme:
   name: material