Skip to content

Commit

Permalink
Revert "🚀 Dockerize llamacpp (antimatter15#132)"
Browse files Browse the repository at this point in the history
This reverts commit 2af23d3.
  • Loading branch information
anzz1 committed Mar 21, 2023
1 parent 5bdf5dc commit c8bd351
Show file tree
Hide file tree
Showing 8 changed files with 4 additions and 306 deletions.
17 changes: 0 additions & 17 deletions .devops/full.Dockerfile

This file was deleted.

18 changes: 0 additions & 18 deletions .devops/main.Dockerfile

This file was deleted.

46 changes: 0 additions & 46 deletions .devops/tools.sh

This file was deleted.

24 changes: 0 additions & 24 deletions .dockerignore

This file was deleted.

45 changes: 4 additions & 41 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ env:
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}

jobs:
ubuntu-latest-make:
ubuntu-latest:
runs-on: ubuntu-latest

steps:
Expand All @@ -36,27 +36,8 @@ jobs:
run: |
make
ubuntu-latest-cmake:
runs-on: ubuntu-latest

steps:
- name: Clone
uses: actions/checkout@v1

- name: Dependencies
run: |
sudo apt-get update
sudo apt-get install build-essential
- name: Build
run: |
mkdir build
cd build
cmake ..
cmake --build . --config Release
macOS-latest-make:
runs-on: macos-latest
macOS-latest:
runs-on: macOS-latest

steps:
- name: Clone
Expand All @@ -73,25 +54,7 @@ jobs:
run: |
make
macOS-latest-cmake:
runs-on: macOS-latest

steps:
- name: Clone
uses: actions/checkout@v1

- name: Dependencies
run: |
brew update
- name: Build
run: |
mkdir build
cd build
cmake ..
cmake --build . --config Release
windows-latest-cmake:
windows-latest:
runs-on: windows-latest

steps:
Expand Down
61 changes: 0 additions & 61 deletions .github/workflows/docker.yml

This file was deleted.

33 changes: 0 additions & 33 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ Supported platforms:
- [X] Mac OS
- [X] Linux
- [X] Windows (via CMake)
- [X] Docker

---

Expand Down Expand Up @@ -237,38 +236,6 @@ Finally, copy the `llama` binary and the model files to your device storage. Her

https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4

### Docker

#### Prerequisites
* Docker must be installed and running on your system.
* Create a folder to store big models & intermediate files (in ex. im using /llama/models)

#### Images
We have two Docker images available for this project:

1. `ghcr.io/ggerganov/llama.cpp:full`: This image includes both the main executable file and the tools to convert LLaMA models into ggml and convert into 4-bit quantization.
2. `ghcr.io/ggerganov/llama.cpp:light`: This image only includes the main executable file.

#### Usage

The easiest way to download the models, convert them to ggml and optimize them is with the --all-in-one command which includes the full docker image.

```bash
docker run -v /llama/models:/models ghcr.io/ggerganov/llama.cpp:full --all-in-one "/models/" 7B
```

On complete, you are ready to play!

```bash
docker run -v /llama/models:/models ghcr.io/ggerganov/llama.cpp:full --run -m /models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -n 512
```

or with light image:

```bash
docker run -v /llama/models:/models ghcr.io/ggerganov/llama.cpp:light -m /models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -n 512
```

## Limitations

- We don't know yet how much the quantization affects the quality of the generated text
Expand Down
66 changes: 0 additions & 66 deletions download-pth.py

This file was deleted.

0 comments on commit c8bd351

Please sign in to comment.