Replies: 22 comments 80 replies
-
|
There was another user who posted recently about using openvino on AMD CPUs. Did you try this? How does it compare? |
Beta Was this translation helpful? Give feedback.
-
|
Haven't run it with frigate but standalone performance for inference only, yolov8n_320x320 compares about like this for my system:
With yolov8s_320x320 the difference is 65 fps (GPU) vs 50 fps (CPU), with same resource usage. I suspect higher GPU performance would be possible if multiple images were chained together into the processing queue. Another interesting thing would be to measure power usage, but I don't have equipment at hand for that at this moment. |
Beta Was this translation helpful? Give feedback.
-
|
Having run it for a few days now I have to say it looks really good. For one the performance is nice. Another thing is that with the ultralytics' yolov8s pretrained model I'm getting almost no false positives. While the Coral based default model got me 20-30 false positives every day, I've got 0 (zero) false positives so far with yolov8s. And plenty of true positives. I'm very happy. This is now a usable system. I also implemented a CPU version of the detector using the
|
Beta Was this translation helpful? Give feedback.
-
do we have OOTB models that can be used?
|
Beta Was this translation helpful? Give feedback.
-
|
there may be more things once a PR is put up |
Beta Was this translation helpful? Give feedback.
-
In theory anything cuda (TensorRT) can run, ROCm can run (in f32) -- if they can be exported (or found) in the ONNX format. But in that case also needs pre- and post-processing code. Which currently is mostly hard-coded into every detector separately. Or take something simple from https://github.com/onnx/models?tab=readme-ov-file#object_detection as the default noone really wants. Btw the yolo models with tensorrt, aren't they GPL too? How do you handle it? I don't have much context or idea how things are done these days. As another solution to circumvent GPL, have you thought about having URLs in the configuration: detectors:
rocm:
type: rocm
model:
path: https://github.com/harakas/models/raw/main/ultralytics/yolov8.1/yolov8s_320x320.onnx
labelmap_path: https://github.com/harakas/models/raw/main/ultralytics/yolov8.1/yolov8s_labels-frigate.txt
model_type: yolov8 Could have these url-s downloaded and cached automagically? |
Beta Was this translation helpful? Give feedback.
-
|
For tensorrt you use the env variables to set the models you want downloaded and they are downloaded on startup |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
They indeed are. Take a look at my post. With Ryzen 7940HS (Minisforum UM790 pro), I get inference speed of 3.3ms!! That too with CPU itself, not the TPU. |
Beta Was this translation helpful? Give feedback.
-
|
OpenVINO on Ryzen is cpu only. So that is 38 fps at 400% cpu, no gpu usage. You could add the rocm detector in parallel on the gpu to increase the capacity. Or even a google coral. It can evaluate yolov8m_320x320 at 15.5fps. |
Beta Was this translation helpful? Give feedback.
-
|
Last time i tried I had issues converting the models. But I have to admit that I didn't look into it too much. |
Beta Was this translation helpful? Give feedback.
-
|
I've update the Docker images I published. They now download the models at start instead of bundling them within and place them into 
I have coral, openvino and rocm all working in parallel with yolov8s. There were couple of issues with openvino export: it still uses f32 in many layers, also in the input layer (so needs modification to the openvino detector code). And I had to update the openvino runtime in frigate as it didn't support the file version coming from the ultralytics api. Need to look a bit more into the export to see why some layers are floats. My config is following: model:
labelmap_path: /config/model_cache/yolov8/labels-frigate.txt
model_type: yolov8
detectors:
rocm:
type: rocm
model:
path: /config/model_cache/yolov8/yolov8s_320x320.onnx
coral:
type: edgetpu
device: usb
model:
path: /config/model_cache/yolov8/yolov8s_320x320_edgetpu.tflite
openvino:
type: openvino
model:
path: /config/model_cache/yolov8/yolov8s_320x320_i8_openvino.xml |
Beta Was this translation helpful? Give feedback.
-
|
How great of an advantage is there to running them in parallel? |
Beta Was this translation helpful? Give feedback.
-
|
In theory it should add all the capacities of the detection methods (cpu, gpu, and coral) -- they run in parallel. So I should get about 52+55+38=145fps detection ability. In practice for me or most people it's not really necessary -- just coral or gpu would be enough -- as those save cpu to use for other things and reduce system power consumption. |
Beta Was this translation helpful? Give feedback.
-
|
@adarshahd |
Beta Was this translation helpful? Give feedback.
-
|
#9762 was merged very recently so you can just try this out. |
Beta Was this translation helpful? Give feedback.
-
|
I tried this with a different chipset as in the docs: I get errors when starting: Is this because my chipset isn't supported? |
Beta Was this translation helpful? Give feedback.
-
|
I have no experience with gfx803. Googling for gfx803 I see that it has worked with ROCm and I see gfx803 tagged files in rocm-5.7.3 that is used in the frigate build. So why is it not working? Googling for the same error I found two reports. One of them reported as a solution installing a different version: ROCm/ROCm#2748 -- this seems to use 5.7.0. So maybe it works with 5.7.0 but not 5.7.3? So maybe try with 5.7.0? If you can build you can try this by replacing all occurences of 5.7.3 with 5.7.0 under docker/rocm/ (in Dockerfile, rocm.list, rocm.hcl) -- then type Maybe also try the ROCm is unfortunately quite flaky so lots of hacking involved and it might not work despite all effort -- so don't spend too much time. |
Beta Was this translation helpful? Give feedback.
-
hi, |
Beta Was this translation helpful? Give feedback.
-
|
Hassos does not include Radeon drivers |
Beta Was this translation helpful? Give feedback.
-
|
Is there any way to install this manually in hassos? |
Beta Was this translation helpful? Give feedback.
-
|
I have no experience with hassos but simple google yielded: home-assistant/operating-system#2320 So it'll probably work unless you have an old version? You'll have to use my forked frigate container: |
Beta Was this translation helpful? Give feedback.
-
|
I see lots of stuff has been merged, but I can't seem to pull |
Beta Was this translation helpful? Give feedback.
-
|
there is no stable rocm build |
Beta Was this translation helpful? Give feedback.
-
|
thanks. i see this probably also had an impact? @harakas are there any non YOLO models that could be used to avoid any licensing issues? |
Beta Was this translation helpful? Give feedback.
-
|
I'm just writing to thank you so much @harakas ! I just tested on my Unraid with the docker ghcr.io/harakas/frigate:latest-rocm-gfx900 you provided to test on my 5700G and it worked flawless! Previously had to disable the detection but now it works perfectly. Below the config I'm using: `mqtt: ffmpeg: cameras: Dahua2: Doorbell: record: detect: model: detectors: objects: birdseye: |
Beta Was this translation helpful? Give feedback.
-
|
@harakas model name : AMD Ryzen 7 8845HS w/ Radeon 780M Graphics |
Beta Was this translation helpful? Give feedback.
-
|
@harakas |
Beta Was this translation helpful? Give feedback.
-
|
Here are the details of the amd gpu i have in the acer nitro 17 laptop here is the glxinfo output. looks reasonable and identifies itself as glx1103_r1 |
Beta Was this translation helpful? Give feedback.
-
|
docker-compose.yml config.yml Any thoughts about what I may be doing wrong and why it may be failing ? |
Beta Was this translation helpful? Give feedback.
-
|
I can't tell from this why it hangs. AMD does not officially support any of these iGPU-s. They do work most of the time but sometimes can be quite flaky. What is throwing it off here, I don't know. So you're not doing anything "wrong", the hardware/software is malfunctioning somehow along the way. Trying to figure it out what fails and how to make it work is complicated and requires knowledge of software engineering. And there is no guarantee of success. One starting point would be to see if it works with the simplest application, say https://github.com/harakas/amd_igpu_yolo_v8 -- seeing if it works with this for a single inference, then seeing if it hangs when run in a loop. Trying different software versions (amd driver version, rocm version, linux kernel). There are also a bunch of parameters you can use to tune how rocm runs. You can google them and others (though AMD has been purging them from their documentation, so you might need to query internet archives). Usually they make things unpractically slow though. Say |
Beta Was this translation helpful? Give feedback.
-
|
@harakas Then I tried sudo for the more complex frigate application and it too runs. without the sudo it failed and I fixed it by recursively modifying the permissions of all the volume mount files I added my user id 1000 to all the groups you documented and now it appears to run without sudo Thanks for all the motivation and encouragement |
Beta Was this translation helpful? Give feedback.
-
|
@harakas Are you expecting ROCm detector to continue to be a viable option in mainline frigate as long as we do a volume mount of the model weights? Thanks for your response |
Beta Was this translation helpful? Give feedback.
-
|
RocM still has to have its own images generated which currently does not occur as the CI jobs were removed. I have a branch in progress which adds RocM support for yolonas models |
Beta Was this translation helpful? Give feedback.
-
It might work with ROCm 6.1.3, it might not. One can only find out by trying. If you have hardware it's trivial. But ROCm 6.0 broke gfx900 (I have gfx90c) -- I think they dropped support for it. So I used rocm 5.7.3. |
Beta Was this translation helpful? Give feedback.
-
https://github.com/ollama/ollama/blob/main/docs/gpu.md#amd-radeon I recently tried this with the latest build and it worked fine with my GFX900 (Vega 64) https://github.com/ollama/ollama/blob/main/docs/linux.md#amd-gpu-install curl -L https://ollama.com/download/ollama-linux-amd64-rocm.tgz -o ollama-linux-amd64-rocm.tgz Oct 17 21:51:48 p1ck4x3007 ollama[131473]: time=2024-10-17T21:51:48.683Z level=DEBUG source=common.go:168 msg=extracting runner=rocm_v60102 payload=linux/amd64/rocm_v60102/libggml.so.gz Looks like they are using rocm v60102 before this point my card would just crash when ollama would try to use it. |
Beta Was this translation helpful? Give feedback.
-
|
@NickM-27 The 8th Gen AMD ryzen processors and probably even the 7th Gen like 7940 tout the presence of a 6 TOPS NPU which is probably superior to the original coral tpu. Has anyone had success with using it? |
Beta Was this translation helpful? Give feedback.
-
I'm looking forward to the Strix Point with around 50-60 TOPS :-D but without propper support it doesn't matter. So I hope somehow the support will come along till then. |
Beta Was this translation helpful? Give feedback.
-
|
I had no idea what strix point was till I saw this https://www.theverge.com/2024/7/19/24198605/amd-ryzen-ai-strix-point-vs-apple-intel-qualcomm-event |
Beta Was this translation helpful? Give feedback.
-
|
maybe i spoke to soon radeontop indicates 845M/1954M or roughly 45% in use |
Beta Was this translation helpful? Give feedback.
-
|
I didn't know ollama has a rocm image too |
Beta Was this translation helpful? Give feedback.
-
|
From what I can gather, there is little to no support for Linux. Only Windows drivers. Sad state of affairs. https://github.com/amd/RyzenAI-SW/tree/main/tutorial/hello_world |
Beta Was this translation helpful? Give feedback.
-
|
I recently also took a look at using the GPU on my um780 xtx with frigate. Ran into the same infamous |
Beta Was this translation helpful? Give feedback.
-
|
Just want to say this is working really great so far on my Vega64 ie version gfx900 using that build. Thank you very much for doing this. Can you pull latest changes [1162 commits behind] Is there any chance this will become a PR to the main build? The official documentation says this: ghcr.io/blakeblackshear/frigate:stable-rocm <- doesn't exist? These are my results from the forked code: Detectors Compared to CPU cpu1 Can you use multiple GPU's or select the GPU it is using so I could dedicate 1 to Frigate and the other to Ollama etc? (I have 4 Vega64's would it decrease inference speed if so?) |
Beta Was this translation helpful? Give feedback.
-
|
I found this in image tags image: ghcr.io/blakeblackshear/frigate:3c591ad-rocm and tested it but got really bad results 90ms compared to the prior harakas builds of 13-16ms not sure what I am missing to get it to work like the prior build. |
Beta Was this translation helpful? Give feedback.
-
|
@NonaSuomy |
Beta Was this translation helpful? Give feedback.
-
|
Latest builds here: https://github.com/blakeblackshear/frigate/pkgs/container/frigate "0b9c4c1-rocm" Detector Inference Speed |
Beta Was this translation helpful? Give feedback.
-
|
do you have to install the models? any directions on installation or setup that you can provide @NonaSuomy |
Beta Was this translation helpful? Give feedback.
-
|
Follow the Yolo-NAS directions at the provided link above or here https://github.com/blakeblackshear/frigate/blob/dev/docs/docs/configuration/object_detectors.md#yolo-nas-1 |
Beta Was this translation helpful? Give feedback.
-
|
I get 55-65ms for yolo-nas-s and 75-85ms foro yolo-nas-m on a 780M. On CPU these models are way faster. The image is |
Beta Was this translation helpful? Give feedback.
-
|
Out of curiosity... what's the power consumption difference? |
Beta Was this translation helpful? Give feedback.
-
I modified frigate to created an AMD GPU based detector that can run the ultralytics/yolov8 models:
https://github.com/harakas/frigate
It is based on the AMD/ROCm development environment.
Fresh from the oven but so far appears to work. See https://github.com/harakas/frigate/blob/dev/docs/docs/configuration/object_detectors.md#amdrocm-gpu-detector on how to run it.
I have a home server with an integrated AMD GPU (AMD Ryzen 3 5400U with Radeon Graphics). Can run yolov8n at 140fps, and yolov8s at 60fps.
Edit: changed readme link.
Beta Was this translation helpful? Give feedback.
All reactions