Skip to content

Commit

Permalink
Fix: Running C++ & Py benchmarks
Browse files Browse the repository at this point in the history
Co-authored-by: Victor Zverovich <[email protected]>
  • Loading branch information
ashvardanian and vitaut committed May 13, 2024
1 parent 6d0aad5 commit b8daf5a
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 46 deletions.
51 changes: 5 additions & 46 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,55 +17,14 @@ All of them implement identical endpoints:
- `resize` - batch-capable image processing operation for Base64-encoded images
- `dot_product` - batch-capable matrix vector-vector multiplication operation

## Echo and Summation

The `echo` example is a simple demonstration of a server that echoes back the request.
To make it a bit more challenging, the same server also contains the `sum` endpoint, which accepts two numbers and returns their aggregate.

- `echo_server_ucall.cpp` - UCall server implementation in C++
- `echo_server_ucall.py` - UCall server implementation in Python
- `echo_server_fastapi.py` - UCall server implementation in FastAPI
- `echo_client.py` - shared client implementation for benchmarking

To run those:

```sh

```

## Redis

Redis example is a basic in-memory key-value store implementation with just two endpoints - `set` and `get`.
The server implementations are available in C++ and Python for UCall.

- `redis_server_ucall.cpp` - UCall server implementation in C++
- `redis_server_ucall.py` - UCall server implementation in Python
- `redis_server_fastapi.py` - UCall server implementation in FastAPI
- `redis_client.py` - shared client implementation for benchmarking

## Multimedia

Multimedia example showcases the more edvanced UCall functionality, like built-in support for NumPy arrays and Pillow images.
It also uses batch endpoints for AI inference and image processing, the functionality not available in FastAPI.

- `multimedia_server_ucall.cpp` - UCall server implementation in C++
- `multimedia_server_ucall.py` - UCall server implementation in Python
- `multimedia_client.py` - shared client implementation for benchmarking


===

The simplest possible endpoint after `hello-world` and `echo`, is probably `sum`.
We would just accept two numbers and return their aggregate.
Packets are tiny, so it is great for benchmarking the request latency.

## Reproducing Benchmarks

### FastAPI

```sh
pip install uvicorn fastapi websocket-client requests tqdm fire
cd examples && uvicorn sum.fastapi_server:app --log-level critical &
cd examples && uvicorn fastapi_server:app --log-level critical &
cd ..
python examples/bench.py "fastapi_client.ClientREST" --progress
python examples/bench.py "fastapi_client.ClientWebSocket" --progress
Expand All @@ -82,14 +41,14 @@ python examples/bench.py "fastapi_client.ClientWebSocket" --threads 8
### UCall

UCall can produce both a POSIX compliant old-school server, and a modern `io_uring`-based version for Linux kernel 5.19 and newer.
You would either run `ucall_example_sum_posix` or `ucall_example_sum_uring`.
You would either run `ucall_example_server_posix` or `ucall_example_server_uring`.

```sh
sudo apt-get install cmake g++ build-essential
cmake -DCMAKE_BUILD_TYPE=Release -B build_release
cmake --build build_release --config Release
build_release/build/bin/ucall_example_sum_posix &
build_release/build/bin/ucall_example_sum_uring &
build_release/build/bin/ucall_example_server_posix &
build_release/build/bin/ucall_example_server_uring &
python examples/bench.py "jsonrpc_client.CaseTCP" --progress
python examples/bench.py "jsonrpc_client.CaseHTTP" --progress
python examples/bench.py "jsonrpc_client.CaseHTTPBatches" --progress
Expand All @@ -99,7 +58,7 @@ kill %%
Want to customize server settings?

```sh
build_release/build/bin/ucall_example_sum_uring --nic=127.0.0.1 --port=8545 --threads=16 --silent=false
build_release/build/bin/ucall_example_server_uring --nic=127.0.0.1 --port=8545 --threads=16 --silent=false
```

Want to dispatch more clients and aggregate more accurate statistics?
Expand Down
3 changes: 3 additions & 0 deletions examples/bench.py
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,10 @@ def main(
progress: bool = False,
):
script_dir = os.path.dirname(os.path.abspath(__file__))
project_dir = os.path.dirname(script_dir)
sys.path.append(f"{script_dir}/login")
sys.path.append(os.path.join(project_dir, "src"))

class_ = locate(class_name)
stats = bench_parallel(
callable=class_(),
Expand Down

0 comments on commit b8daf5a

Please sign in to comment.