Merge pull request #12 from hotosm/feature/docker

Feature : Docker Installation

.github/workflows/build.yml

@@ -16,6 +16,9 @@ jobs:
         with:
           python-version: 3.8
 
+      - name: Remove solaris
+        run: sudo rm -rf ./docker
+
       - name: Clone Ramp
         run: git clone https://github.com/kshitijrajsharma/ramp-code-fAIr.git ramp-code
 

.github/workflows/upload.yml

@@ -3,7 +3,7 @@ name: Build and Publish
 on:
   push:
     tags:
-      - 'v*'
+      - "v*"
 
 jobs:
   build:
@@ -18,16 +18,20 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@v2
 
+      - name: Remove solaris
+        run: |
+          sudo rm -rf docker
+
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
           pip install setuptools wheel twine
 
       - name: Build package
-        run: python setup.py sdist 
+        run: python setup.py sdist
 
       - name: Publish to PyPI
         env:
           TWINE_USERNAME: __token__
           TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
-        run: twine upload dist/*
+        run: twine upload dist/*

Dockerfile

@@ -0,0 +1,43 @@
+FROM tensorflow/tensorflow:2.9.2-gpu-jupyter
+
+RUN apt-get update && apt-get install -y python3-opencv
+RUN add-apt-repository ppa:ubuntugis/ppa && apt-get update
+RUN apt-get update
+RUN apt-get install -y gdal-bin
+RUN apt-get install -y libgdal-dev
+ENV CPLUS_INCLUDE_PATH=/usr/include/gdal
+ENV C_INCLUDE_PATH=/usr/include/gdal
+
+#install numpy before gdal 
+RUN pip install numpy==1.23.5
+
+# pip install dependencies.
+RUN pip install --global-option=build_ext --global-option="-I/usr/include/gdal" GDAL==`gdal-config --version`
+
+COPY docker/ramp/docker-requirements.txt docker-requirements.txt
+RUN pip install -r docker-requirements.txt
+
+# pip install solaris -- try with tmp-free build
+# COPY docker/ramp/solaris /tmp/solaris
+COPY docker/solaris/solaris  /tmp/solaris/solaris
+COPY docker/solaris/requirements.txt  /tmp/solaris/requirements.txt
+COPY docker/solaris/setup.py /tmp/solaris/setup.py
+
+RUN pip install /tmp/solaris --use-feature=in-tree-build
+
+RUN pip install scikit-fmm --use-feature=in-tree-build
+
+RUN pip install setuptools --upgrade 
+
+ENV RAMP_HOME=/tf
+
+# Install the package in development mode
+COPY setup.py ./setup.py 
+COPY hot_fair_utilities ./hot_fair_utilities
+RUN pip install -e .
+
+# install ramp-fair
+RUN pip install ramp-fair mercantile pandas==1.5.3
+
+## Copy Sample data
+COPY ramp-data ./ramp-data

README.md

@@ -1,94 +1,39 @@
 # hot_fair_utilities ( Utilities for AI Assisted Mapping fAIr )
-Initially lib was developed during Open AI Challenge with [Omdeena](https://omdena.com/). Learn more about challenge from [here](https://www.hotosm.org/tech-blog/hot-tech-talk-open-ai-challenge/)  
 
-## `hot_fair_utilities` Installation
-
-hot_fair_utilities is collection of utilities which contains core logic for model data prepration , training and postprocessing . It can support multiple models , Currently ramp is supported. 
-
-1. To get started clone this repo first : 
-    ```
-    git clone https://github.com/hotosm/fAIr-utilities.git
-    ```
-2. Setup your virtualenv with ```python 3.8``` ( Ramp is tested with python 3.8 )
-
-3. Install tensorflow ```2.9.2``` from [here] (https://www.tensorflow.org/install/pip) According to your os
-
-#### Setup Ramp : 
-
-4. Copy your basemodel : Basemodel is derived from ramp basemodel 
-    ```
-    git clone https://github.com/radiantearth/model_ramp_baseline.git
-    ```
-
-5. Clone ramp working dir 
-
-    ```
-    git clone https://github.com/kshitijrajsharma/ramp-code-fAIr.git ramp-code
-    ```
-
-6. Copy base model to ramp-code 
-    ```
-    cp -r model_ramp_baseline/data/input/checkpoint.tf ramp-code/ramp/checkpoint.tf
-    ```
+Initially lib was developed during Open AI Challenge with [Omdeena](https://omdena.com/). Learn more about challenge from [here](https://www.hotosm.org/tech-blog/hot-tech-talk-open-ai-challenge/)
 
-7. Install native bindings 
-    - Install Numpy 
-        ```
-        pip install numpy==1.23.5
-        ```
-    - Install [gdal](https://gdal.org/index.html) .
-
-        for eg : on Ubuntu 
-        ```
-        sudo add-apt-repository ppa:ubuntugis/ppa && sudo apt-get update
-        sudo apt-get install gdal-bin
-        sudo apt-get install libgdal-dev
-        export CPLUS_INCLUDE_PATH=/usr/include/gdal
-        export C_INCLUDE_PATH=/usr/include/gdal
-        pip install --global-option=build_ext --global-option="-I/usr/include/gdal" GDAL==`gdal-config --version`        
-        ```
-        on conda : 
-        ```
-        conda install -c conda-forge gdal
-        ```
-    - Install rasterio 
+## `hot_fair_utilities` Installation
 
-        for eg: on ubuntu : 
-        ```
-        sudo apt-get install -y python3-rasterio
-        ```
-        on conda : 
-        ```
-        conda install -c conda-forge rasterio
-        ```
+Installing all libraries could be pain so we suggest you to use docker , If you like to do it bare , You can follow `.github/build.yml`
 
-8. Install ramp requirements 
+Clone repo
 
-    Install necessary requirements for ramp  and hot_fair_utilites
-         
-    ```
-    cd ramp-code && cd colab && make install  && cd ../.. && pip install -e .
-    ```
+```
+git clone https://github.com/hotosm/fAIr-utilities.git
+```
 
+Build Docker
 
+```
+docker build --tag fairutils .
+```
 
-## Conda Virtual Environment
-Create from env fle 
+Run Container with default Jupyter Notebook , Or add `bash` at end to see terminal
 
 ```
-conda env create -f environment.yml
+docker run -it --rm --gpus=all  -p 8888:8888 fairutils
 ```
-Create your own
+
+By Default tf is set as Ramp_Home , You can change that attaching your volume to container as tf
 
 ```
-conda create -n fAIr python=3.8
-conda activate fAIr
-conda install -c conda-forge gdal
-conda install -c conda-forge geopandas
-pip install pyogrio rasterio tensorflow
-pip install -e hot_fair_utilities
+-v /home/hotosm/fAIr-utilities:/tf
 ```
 
-## Test Installation and workflow 
+## Test Installation and workflow
+
+You can run `package_test.ipynb` on your notebook from docker to test the installation and workflow with sample data provided , Or open with [collab and connect your runtime locally](https://research.google.com/colaboratory/local-runtimes.html#:~:text=In%20Colab%2C%20click%20the%20%22Connect,connected%20to%20your%20local%20runtime.)
+
+## Get started with development
 
-You can run ```package_test.ipynb``` to your pc to test the installation and workflow with sample data provided 
+Now you can play with your data , use your own data , use different models for testing and also Help me Improve me !

docker/ramp/docker-requirements.txt

@@ -0,0 +1,78 @@
+affine==2.3.1
+albumentations==1.0.3
+attrs==21.4.0
+branca==0.4.2
+brotlipy==0.7.0
+certifi==2021.10.8
+cffi==1.15.0
+charset-normalizer==2.0.4
+click==8.1.0
+click-plugins==1.1.1
+cligj==0.7.2
+cloudpickle==2.0.0
+cryptography==36.0.0
+cycler==0.11.0
+cytoolz==0.11.2
+dask
+efficientnet==1.0.0
+Fiona==1.8.21
+folium==0.12.1.post1
+fonttools==4.31.2
+fsspec==2022.2.0
+GDAL
+geojson==2.5.0
+geopandas==0.10.2
+h5py==3.6.0
+idna==3.3
+image-classifiers==1.0.0
+imagecodecs
+imageio==2.16.1
+imgaug==0.4.0
+Jinja2==3.1.1
+joblib==1.1.0
+Keras-Applications==1.0.8
+kiwisolver==1.4.2
+locket==0.2.0
+mapclassify==2.4.3
+MarkupSafe==2.1.1
+matplotlib==3.5.1
+munch==2.5.0
+munkres==1.1.4
+networkx
+numpy
+opencv-python==4.5.5.64
+packaging==21.3
+pandas
+partd==1.2.0
+Pillow==9.0.1
+pip==21.2.4
+pycosat==0.6.3
+pycparser==2.21
+pyOpenSSL==21.0.0
+pyparsing==3.0.7
+pyproj
+PySocks==1.7.1
+python-dateutil==2.8.2
+pytz==2022.1
+PyWavelets==1.3.0
+PyYAML==6.0
+rasterio==1.2.10
+requests==2.27.1
+Rtree==0.9.7
+scikit-image==0.19.2
+scikit-learn==1.0.2
+scipy
+segmentation-models==1.0.1
+setuptools==58.0.4
+Shapely==1.8.0
+six==1.16.0
+snuggs==1.4.7
+threadpoolctl==3.1.0
+tifffile
+tinydb==4.7.0
+toolz==0.11.2
+tqdm==4.62.3
+unicodedata2==14.0.0
+urllib3==1.26.7
+wheel==0.37.1
+xyzservices==2022.3.0

docker/solaris/.gitattributes

@@ -0,0 +1,3 @@
+*.ipynb filter=nbstripout
+
+*.ipynb diff=ipynb

docker/solaris/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,47 @@
+---
+name: Bug report
+about: Report an error or other suspected bug behavior
+title: "[BUG]: (your description here)"
+labels: 'Status: Review Needed, Type: bug'
+assignees: ''
+
+---
+
+_Thank you for helping us improve `solaris`!_
+
+## Summary of the bug
+A clear and concise description of what the bug is.
+
+## Steps to reproduce the bug
+Please either paste sample code used to generate the buggy behavior below, or provide step-by-step instructions to reproduce the problem.
+
+```
+Paste bug-causing code here
+```
+
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+## Buggy behavior and/or error message
+Please describe the buggy behavior and/or paste output here.
+```
+Paste output here
+```
+
+## Expected behavior
+A clear and concise description of what you expected to happen.
+
+## Screenshots
+If applicable, add screenshots to help explain your problem.
+
+## Environment information
+- OS:
+- `solaris` version:
+- python version:
+- version of any relevant dependencies (optional - we may ask for this information later if not provided)
+
+## Additional context
+Add any other context about the problem here.

docker/solaris/.github/ISSUE_TEMPLATE/documentation.md

@@ -0,0 +1,18 @@
+---
+name: Documentation
+about: Request for additional documentation, clarification of existing documentation, or tutorials
+title: "[DOCS]"
+labels: 'Status: Review Needed, Type: Documentation'
+assignees: ''
+
+---
+
+_Thank you for helping to improve `solaris`!_
+
+## Documentation request summary
+
+A short summary of the changes you'd like made to the documentation.
+
+## Task detail and notes
+
+List any additional information, links, or other content here that is needed to make the changes you're requesting. 

docker/solaris/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: "[FEATURE]: (short description)"
+labels: 'Status: Review Needed, Type: Enhancement'
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

docker/solaris/.github/ISSUE_TEMPLATE/maintenance.md

@@ -0,0 +1,22 @@
+---
+name: Maintenance
+about: Request for maintenance, refactoring, or related tasks
+title: "[MAINT]"
+labels: 'Status: Review Needed, Type: Maintenance'
+assignees: ''
+
+---
+
+## Maintenance request summary
+
+A short summary of the maintenance to perform, including why it needs to be done.
+
+## Task detail and notes
+
+Any additional information:
+
+- What other code will this impact?
+
+- Do new tests need to be written or do existing tests need to be updated? 
+
+- Will this impact documentation?