|
2 | 2 | 
|
3 | 3 | 
|
4 | 4 |
|
5 |
| -## TODO |
6 |
| -### API |
7 |
| -* Wrapper for DB related functions (w/ error handling) |
| 5 | +Broker was created as a tool for a team of 9 ML practitioners to manage their |
| 6 | +access to a single GPU machine. The idea was to avoid running multiple jobs at |
| 7 | +once (which would cause OOM errors), and to have a dashboard to visualize the |
| 8 | +jobs and their respective states. |
| 9 | + |
| 10 | +According to Wikipedia: |
| 11 | +> A broker's prime responsibility is to bring sellers and buyers together and |
| 12 | +> thus a broker is the third-person facilitator between a buyer and a seller. |
| 13 | +
|
| 14 | +In the same way, the global architecture of the system was designed around the |
| 15 | +Scheduler, the API that connects users with runners. |
| 16 | + |
| 17 | + |
| 18 | + |
| 19 | +Theoretically, the system could scale to multiple runners. But has yet to be |
| 20 | +tested. |
| 21 | + |
| 22 | +## Install |
| 23 | +### Scheduler |
| 24 | +To install the Scheduler on the main host machine (API + Dashboard): |
| 25 | + |
| 26 | +Open `docker/docker-compose-dev.yml` and modify the commented lines. Then, |
| 27 | +using docker-compose, build the two images |
| 28 | +``` |
| 29 | +$ docker-compose -f docker/docker-compose-dev.yml build |
| 30 | +``` |
| 31 | + |
| 32 | +And run the app: |
| 33 | +``` |
| 34 | +$ docker-compose -f docker/docker-compose-dev.yml up |
| 35 | +``` |
| 36 | + |
| 37 | +### Runners |
| 38 | +Runners are single file python scripts, that you can run directly on your |
| 39 | +python env or on a docker container using `docker/runner-dev.Dockerfile`. |
| 40 | +The only dependancy is `requests`. |
| 41 | + |
| 42 | +Using poetry: |
| 43 | +``` |
| 44 | +$ cd runner |
| 45 | +$ poetry run python runner.py --scheduler_ip [IP] --scheduler_port [PORT] |
| 46 | +``` |
| 47 | + |
| 48 | +## Caveats |
| 49 | +This is my first time writing an API / doing web dev. This is a development |
| 50 | +version that has no security at all. |
| 51 | + |
| 52 | +## Aknowledgements |
| 53 | +The dashboard uses this template |
| 54 | +<https://github.com/BootstrapDash/StarAdmin-Free-Vue-Admin-Template> |
8 | 55 |
|
9 |
| -### Frontend |
10 |
| -* Header disappears when using a small window |
|
0 commit comments