huggingface_model_deployment

TODO:

• Upload model to HuggingFace modelhub
• Productionize and modularize code
• Makefile
• Dockerize
  • Integrate poetry
  • Separate the fastAPI backend and gradio frontend services in different containers
  • Handle them with Docker compose
• [IN PROGRESS] CI/CD
  • Use a self-hosted CI/CD runner
  • Use a self-hosted docker image repository
  • Self-host the deployment environment
  • Properly tag docker images
  • Create a release on new changes to the main branch
  • Clean-up old images/containers on the prod server
• Reduce image size by using a lighter version of PyTorch
• Pytest / doctest - unit/system tests
• Nox/Tox
• Mypy - typechecking
• Linting
• Logging
• Prometheus, Grafana, Loki
• Training/Validation/Testing scripts and modularity
• Documentation (MkDocs / Sphinx) + Docstrings
• Kubernetes/Kserve/Helmchart + GPU
• Model tracking, drift, automated retraining
• Airflow/Dagster/Argo
• Create an diagram for the entire architecture (CI/CD, model retraining, etc.)
• OPTIONAL: Deploy in AWS - EC2 or ECS
• OPTIONAL: Terraform infrastructure