Skip to content

Self-hostable, open-source engine for reproducible data harmonization, dataset building & exploration

License

Notifications You must be signed in to change notification settings

datasnack/datahub

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Data Hub

The Data Hub is a geographic information system (GIS) featuring a data fusion engine designed for data harmonization, alongside an interactive dashboard for effective data exploration and collaboration. Its key objective is to merge data of multiple formats and sources across temporal and spatial axes, allowing users to combine, analyze, and interpret the data.

In this repository you can explore an example setup of the Data Hub software tailored to data concerning Ghana.

Installation

The recommended way to use the Data Hub is via Docker and to reference the container built from this repository. See the Ghana Data Hub example for installation steps. By doing so it allows for updating the core system independent of you customizations.

To install the Data Hub from source, follow these steps:

  • Install uv
  • Clone this repository and open it in your terminal
  • Create a .env file based on the .env.example file.
  • Install a PostGIS v16.x database (you can use the provided Docker image from the docker-compose.yml, $ docker compose up -d postgis).
  • Create a Python v.3.12.x. virtual environment with uv venv --python 3.12 and activate it with source .venv/bin/activate.
  • Install Python dependencies via uv sync (might be complicated due to GDAL/PROJ dependencies).
  • Run database migrations with python manage.py migrate
  • Run Django with python manage.py runserver

The system is now running and usable at http://localhost:8000/, to use it:

  • Create a new superuser with python manage.py createsuperuser
  • Import your Shapes with python manage.py loadshapes <file>
  • Place your Data Layer source files in src/datalayers/
  • Downloaded data will be placed in data/datalayers/

Attributions

The Data Hub is an open-source software (OSS) developed through the DiDEX project (Digital Data and Exploratory Spaces for Strengthening Infectious Disease Research within the One Health nexus) at the Bernhard Nocht Institute for Tropical Medicine. The project is supported and funded by the Joachim Herz Foundation ("Innovate! Academy" program).

The source code of the Data Hub is informed by the results of the ESIDA project.

License

MIT