RX 6700xt Best Settings / Parameters and tips for low GPU ❤️ [Outdated]

[Milor123]

I want share my small experience after to test some settings share by the users of this forum
actially i am in windows 11 22H2

edit: 29/06/2023, I've done manually test for a hours...

Solving the: VRAM error (RuntimeError: Could not allocate tensor with bytes. There is not enough GPU video memory available!).

I am working with inpainting in a intel core i7 4790k old hardware xD. 24 GB of ram 1600 MHz, my GPU is have light underclock, you could get better performance than it.

CHINASE lenguage translate : ➡️➡️➡️ CLICK HERE 🚩👈🏻👈🏻👈🏻

Summay and Settings

My best settings for a 6700/6750 xt cards (Tested in Img2Img:

• Sampling steps 20
• 824x512
• CFG 5.2
• Denoising strength 0.65

Prompt a product photo of (beewax:1.3) natural blocks,wildly inventive,and elegant style by Florian Gadsby,changing the industry,impossible,groundbreaking,stunning quality,shot for pottery magazine,canon C300,high resolution,1. 4,50mm,bokeh,product photography,highly detailed,sharp focus,HDR

negative lowres,bad anatomy,bad hands,text,error,missing fingers,extra digit,fewer digits,cropped,worst quality,low quality,normal quality,jpeg artifacts,signature,watermark,username,blurry,bee,

Random Seeds!

Settings OLD ✅:

set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.5,max_split_size_mb:256
set COMMANDLINE_ARGS=--opt-split-attention --upcast-sampling --precision full --always-batch-cond-uncond --disable-nan-check --api  --medvram --no-half-vae --sub-quad-q-chunk-size 1024 --sub-quad-kv-chunk-size 128 --sub-quad-chunk-threshold 85 --use-cpu interrogate clip --no-hashing 
set SAFETENSORS_FAST_GPU=1 

Settings updated New v2 [03/07/2023] ✅:
Remove --upcast-sampling ✅ (need remove it for use --no-half)
Replace it by: --no-half ✅
Why?: Inpainting not work without --no-half, and some upscaling models required it.

set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.5,max_split_size_mb:256
set COMMANDLINE_ARGS=--opt-split-attention --precision full --always-batch-cond-uncond --disable-nan-check --api  --medvram --no-half --no-half-vae --sub-quad-q-chunk-size 1024 --sub-quad-kv-chunk-size 256 --sub-quad-chunk-threshold 75 --use-cpu interrogate clip --no-hashing 
set SAFETENSORS_FAST_GPU=1 

Note if you want generate larges batches, maybe you need --sub-quad-chunk-threshold change it for a value more small, try substract 5, or more, i mean says that you have 75, try with 70, or 65. (It could decrese a few the perfomance.)

fast summary

If you have a AMD gpu lower than 12GB, maybe could try divide the values in 2
--sub-quad-q-chunk-size 1024/2
--sub-quad-kv-chunk-size 128/2 (Or try keep)
--sub-quad-chunk-threshold 85 (if you divide sub-quad-kv-chunk-size, you must increse there by 95-100 )
and about of garbage_collection_threshold:0.5,max_split_size_mb:256, maybe keep it in 256 or divide in 2
Hint: If you divide something, try divide all and keep relation or distance in the values.

KEEP ALWAYS --opt-split-attention ✅
--medvram increses the performance, is good option ✅ (Use --lowvram only if you have problems in you gpu with 8gb of ram)

Extremely important

⚠️⛔ Dont change --sub-quad-kv-chunk-size without change --sub-quad-chunk-threshold | You need set the correct threshold value respect to your KV chunk size or you get VRAM ERROR. THIS IS VERY VERY IMPORTANT. ⚠️⛔

6700xt-Series-Chinase-zh.pdf
6700xt-Series-English-en.pdf

[Milor123]

@lshqqytiger Today. I've updated this post, because the old post have bad settings and need do testing!. Now is a big upgraded according to my test. Try keep it updated if found something.

[xil64]

My understanding of these settings is that attention optimizations are different settings to do the same thing, i.e. the --sub-quad chunk and threshold settings would have no effect unless you are using --opt-sub-quad-attention also. So if you're using --opt-split-attention instead those settings should have no effect. I've spend hours myself changing settings using an AMD card and that's the conclusion I came to.

[Milor123]

Oh no bro, if i do changes while keep --opt-split-attention, I can see and meter the changes using the Quad parameters. You can test by your self, set "--sub-quad-chunk-threshold in 100% ❌" and "set --sub-quad-kv-chunk-size in 512 or doble ❌ "and you get a VRAM error if have a GPU similar to the me.

ALERT DONT DO IT, its only bad example.

commandline argument	explanation
--opt-sdp-attention	May results in faster speeds than using xformers on some systems but requires more VRAM. (non-deterministic)
--opt-sdp-no-mem-attention	May results in faster speeds than using xformers on some systems but requires more VRAM. (deterministic, slightly slower than --opt-sdp-attention and uses more VRAM)
--xformers	Use xformers library. Great improvement to memory consumption and speed. Nvidia GPUs only. (non-deterministic)
--force-enable-xformers	Enables xformers regardless of whether the program thinks you can run it or not. Do not report bugs you get running this.
--opt-split-attention	Cross attention layer optimization significantly reducing memory use for almost no cost (some report improved performance with it). Black magic. On by default for torch.cuda, which includes both NVidia and AMD cards.
--disable-opt-split-attention	Disables the optimization above.
--opt-sub-quad-attention	Sub-quadratic attention, a memory efficient Cross Attention layer optimization that can significantly reduce required memory, sometimes at a slight performance cost. Recommended if getting poor performance or failed generations with a hardware/software configuration that xformers doesn't work for. On macOS, this will also allow for generation of larger images.
--opt-split-attention-v1	Uses an older version of the optimization above that is not as memory hungry (it will use less VRAM, but will be more limiting in the maximum size of pictures you can make).
--medvram	Makes the Stable Diffusion model consume less VRAM by splitting it into three parts - cond (for transforming text into numerical representation), first_stage (for converting a picture into latent space and back), and unet (for actual denoising of latent space) and making it so that only one is in VRAM at all times, sending others to CPU RAM. Lowers performance, but only by a bit - except if live previews are enabled.
--lowvram	An even more thorough optimization of the above, splitting unet into many modules, and only one module is kept in VRAM. Devastating for performance.
*do-not-batch-cond-uncond	Prevents batching of positive and negative prompts during sampling, which essentially lets you run at 0.5 batch size, saving a lot of memory. Decreases performance. Not a command line option, but an optimization implicitly enabled by using --medvram or --lowvram.
--always-batch-cond-uncond	Disables the optimization above. Only makes sense together with --medvram or --lowvram
--opt-channelslast	Changes torch memory type for stable diffusion to channels last. Effects not closely studied.
--upcast-sampling	For Nvidia and AMD cards normally forced to run with --no-half, should improve generation speed.

[gigend]

hello, any recommendation setting for 4 GB VRAM / Radeon RX 570 ? thank you.

[Milor123]

I am not sure because i dont have this gpu for try, but i could suggest you the next variations

Explain of workflow for testing in your gpu

Guide base: dont use it.
set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.5,max_split_size_mb:{VARIATION} Between {128|64}
set COMMANDLINE_ARGS=--opt-split-attention --precision full --always-batch-cond-uncond --disable-nan-check --api --lowvram --no-half --no-half-vae --sub-quad-q-chunk-size 256 --sub-quad-kv-chunk-size 256 --sub-quad-chunk-threshold 75 --use-cpu interrogate clip --no-hashing

--sub-quad-q-chunk-size {VARIATION} Between {512|256|128}

The next table is linked arguments, if you use a value must change the other, example kv-chunk-size 256 -> need use chunk-threshold 75.

Note: This relation values was creates testing with Rx 6700xt
Arguments (LINKED)	Size variation 1	Size variation 2	Size variation 3
--sub-quad-kv-chunk-size {VARIATION}	256	128	64
--sub-quad-chunk-threshold {VARIATION}	75	85	95 or 100

set SAFETENSORS_FAST_GPU=1

• --opt-split-attention or --opt-split-attention-v1 (This arguments are very important, but you should test if you dont have VRAM, I try firts with the most updated -> --opt-split-attention the V1 is OLD implementation )

• --lowvram is vital for you.

Real Examples (It could works for other GPU too)

Test 2 (Copy and paste it):

set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.5,max_split_size_mb:128
set COMMANDLINE_ARGS=--opt-split-attention  --precision full --always-batch-cond-uncond --disable-nan-check --api  --lowvram --no-half --no-half-vae --sub-quad-q-chunk-size 256 --sub-quad-kv-chunk-size 256--sub-quad-chunk-threshold 75 --use-cpu interrogate clip --no-hashing 
set SAFETENSORS_FAST_GPU=1

Test 2 (Copy and paste it):

set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.5,max_split_size_mb:128
set COMMANDLINE_ARGS=--opt-split-attention  --precision full --always-batch-cond-uncond --disable-nan-check --api  --lowvram --no-half --no-half-vae --sub-quad-q-chunk-size 256 --sub-quad-kv-chunk-size 128 --sub-quad-chunk-threshold 85 --use-cpu interrogate clip --no-hashing 
set SAFETENSORS_FAST_GPU=1

Test 3 (Copy and paste it):

set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.5,max_split_size_mb:128
set COMMANDLINE_ARGS=--opt-split-attention  --precision full --always-batch-cond-uncond --disable-nan-check --api  --lowvram --no-half --no-half-vae --sub-quad-q-chunk-size 256 --sub-quad-kv-chunk-size 64--sub-quad-chunk-threshold 95 --use-cpu interrogate clip --no-hashing 
set SAFETENSORS_FAST_GPU=1

For determinate if something is working bad, should generate 1 image or 2 images, and check if get VRAM problems. I really not stay sure if my relationship kv-chunk-size and chunk-threshould is the same or is much diferent in your GPU, i dont have your hardware for test it, but this relation are very important. And in the table are the values for a rx 6700xt

if you want do test, write in https://t.me/Junklont, i could suggest you settings.

[gigend]

Thank you for giving the setting suggestions. Actually I had a problem with upscaling the image/img2img/hi-res fix.

Specifications about the hardware that I use.

• VGA AMD Ellesmere Powercolor RADEON RX 570 4 GB VRAM, GPU Clock 1100 Mhz (underclock), VRAM Clock 2000 Mhz
• RAM 8GB Dual Channel DDR4-2400

The args settings I'm currently using are:

set COMMANDLINE_ARGS=--sub-quad-q-chunk-size 256 --always-batch-cond-uncond --medvram --precision full --upcast-sampling --autolaunch --opt-sub-quad-attention -- disable-nan-check --opt-split-attention-v1
set SAFETENSORS_FAST_GPU=1

which I copied from tanuki33 here https://github.com/lshqqytiger/stable-diffusion-webui-directml/discussions/

• to generate normal normal images.
• supports textual inversion, which usually can't: pictures are item and weird.
• I haven't tested inpaint and other features, but it doesn't seem to work, i think.

the problem I got when running stable diffussion directml 714e97f

• I can only generate images with a resolution below 512 x 512, and when I successfully generate several images, they often run out of memory, and I have to restart the program.
• out of memory when hi-res fix or upscale even if only 1.5-2x via img2img, I tried several tiled diffusion method extensions the result is still OOM.
• upscale extension ultimate sd upscale, linear tiled method, only affects the first tiled or first row, the other tiled are only limited to upscale, even though I changed the denoising strength to a high number.
• If I overclock the GPU to 1,200 mhz and above, when generating images my monitor screen will usually turn off, the fans turn on fast, and I have to restart my PC manually (because the screen is off/disconnected). But when I underclock it to 1100 mhz, it doesn't happen anymore.

I just want to be able to upscale at resolutions over 1024, but right now I'm having this problem. I've tried all three of the recommended settings. But the results are worse, even for generating 512x512 the speed is very slow, and if upscale it will definitely be out of memory.

[yamfun]

What is your speed for generating say, 4 images at 512x512 ?

[Milor123]

@gigend Its complicared, do you try reescaling from extra tabs? without use text2img or img2img. Some models like Real-SR have good results for me (In products for ecommerce), but in some cases it also show me VRAM errors :/ or using external tools like upscalying or Waifu2x-Extension-GUI

in windows you could try use Shark, is very very fast than directml (Here i can get 1it/s, with shark i can get 6it/s, is amazing) but the problem is hard to configure, and have limitations, i dont feel comfortable using it, i need controlnet and other things that have automatic1111

[gigend]

@yamfun i'm just only generate 1 per batch image. My VRAM is always not enough if it's more than that. For speed, if you use --medvram args, it's around 1.9-2.5 it/s if you use --lowvram 3.0-4.5 it/s. Generate a normal 512x512 image with the 'euler a' sampling method. Maybe if you add too many extensions and lora also affects the speed? I didn't pay too much attention either, the speed is clear that much.

@Milor123 Yes, if you use upscaling on the extra tab, there is no problem at all. It's pretty fast and doesn't take up a lot of VRAM. Using ESRGAN doesn't seem to matter if the image is realistic.

But if I want to generate from an anime model (I'm using anylora fp16 pruned btw, around 2-3 GB model), the reason I upscaling with the hi-res fix method, img2img upscaling, tiled upscaling, or controlnet, is to be able to add more detail and improve weird body parts, proportions, or composition. If i use it only from the extras tab, there are still artifacts resulting in less detail. The results are unsatisfactory.

The point is I still often experience out of memory until now and haven't found the best solution, it's still need to be more optimized stable diffussion directml for mid-low range AMD graphics cards.

[ElevenTimes]

Holy shit dude, you're amazing! Before my images were taking 2 minutes to generate a 856x512 on 20 steps, but with your args added they take no more than 20 seconds! I can't believe I only found your post now, you're the GOAT!!!

[scannerfm77]

Thank you. I feel my setup is more stable now.
I'm using RX6700XT.
I use below arguments:

set COMMANDLINE_ARGS=--opt-split-attention --precision full --always-batch-cond-uncond --disable-nan-check --api --medvram --no-half --no-half-vae --sub-quad-q-chunk-size 1024 --sub-quad-kv-chunk-size 128 --sub-quad-chunk-threshold 75 --use-cpu interrogate clip --no-hashing

But still often show error below while generating.

File "C:\stable-diffusion-webui-directml\modules\sd_hijack_optimizations.py", line 268, in split_cross_attention_forward
s1 = einsum('b i d, b j d -> b i j', q[:, i:end], k)
RuntimeError: Could not allocate tensor with 39403520 bytes. There is not enough GPU video memory available!

Do you know where I can improve it?
Thank you.

[Milor123]

Thank you. I feel my setup is more stable now. I'm using RX6700XT. I use below arguments:

set COMMANDLINE_ARGS=--opt-split-attention --precision full --always-batch-cond-uncond --disable-nan-check --api --medvram --no-half --no-half-vae --sub-quad-q-chunk-size 1024 --sub-quad-kv-chunk-size 128 --sub-quad-chunk-threshold 75 --use-cpu interrogate clip --no-hashing

But still often show error below while generating.

File "C:\stable-diffusion-webui-directml\modules\sd_hijack_optimizations.py", line 268, in split_cross_attention_forward
s1 = einsum('b i d, b j d -> b i j', q[:, i:end], k)
RuntimeError: Could not allocate tensor with 39403520 bytes. There is not enough GPU video memory available!

Do you know where I can improve it? Thank you.

Using ROCm on linux or not using AMD bro :/, use AMD is really frustrating for all. I try many things but AMD is very bad in windows, in linux you could have really a great perfomance. Or using SHARK in windows is very very fast but is limiting compare with this webui, and is hard to configure.

[CaniAI]

Thank you for sharing!
Are there any updates?
Getting crazy with out of memory.
Hires.fix is nearly impossible if I try to get 1920x1080.
Using A11 without any changes right now.
RX6750XT + 16GB RAM.

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 8.90 GiB. GPU

[ride5k]

Thank you for sharing! Are there any updates? Getting crazy with out of memory. Hires.fix is nearly impossible if I try to get 1920x1080. Using A11 without any changes right now. RX6750XT + 16GB RAM.

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 8.90 GiB. GPU

i would give up on DML and pivot to ZLUDA. not sure if your card has/needs ROCM libs though...

[CaniAI]

Alrdy using ZLUDA with necessary ROCM libs.
set COMMANDLINE_ARGS= --medvram --opt-split-attention
Mostly I cant upscale (from 896x1152 to 1344x1728) while using VAE.
Probably should just buy NVIDIA.

[ride5k]

Alrdy using ZLUDA with necessary ROCM libs. set COMMANDLINE_ARGS= --medvram --opt-split-attention Mostly I cant upscale (from 896x1152 to 1344x1728) while using VAE. Probably should just buy NVIDIA.

are you not using --use-zluda in your args? that could be a big issue!

regarding other args, once i went to zluda i no longer needed --medvram. also there is no need to specify a cross attention optimization as that is now done in web gui.

[CaniAI]

"Using ZLUDA in C:\Users\Cani\Desktop\Stable Diffusion\stable-diffusion-webui-amdgpu.zluda"
Shouldn't be necessary if I have that in my Terminal while launching or am I wrong?

[ZeroNyte]

tried these setting on my 6750xt, but still having generation problems, like the images come out as deep fried/plasma and not sure what to do.
and when I try to go larger than 512x768 I get not enough vram/"couldnt allocate tensor with XXX bytes"

[CaniAI]

I use it only with set COMMANDLINE_ARGS= --medvram --opt-split-attention, ZLUDA with necessary ROCM libs and Upcast cross attention layer to float32, mostly working for me.

[ZeroNyte]

I just installed SC next with ZLUDA, atm its slow, but most likely because I have debug on, and the GUI isnt great, atleast not the standard one. also not sure what the "Upcast cross attention layer to float32" is.

[CaniAI]

I would stay with Automatic1111 + ZLUDA.
Generating right now with SDXL model and its using 16,4GB RAM, so definitely make sure u have enough RAM otherwise try other models like 1.5.

Edit: Max RAM usage was ~22GB.

[ZeroNyte]

have 32GB of RAM, which should be plenty, and tried the A1111 fork of lshqqytiger, but when I did --use-zluda, it went and used my CPU, but not sure if that was because I didnt have the ROCm libs installed, so gonna try that again

and you're using standard A1111 instead of the fork?

[CaniAI]

You definitely need ROCm libs mostly for RDNA2, use ROCm version 5.7
https://github.com/brknsoul/ROCmLibs
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-AMD-GPUs
https://www.youtube.com/watch?v=n8RhNoAenvM

[Milor123]

Please guys, dont try my settings, it was a old preset test in DirectML, now you have ZLuda and I've bought nvidia because AMD is a headache, I will never update this guide in the future.

[CaniAI]

Any advice for deciding which NVIDIA GPU to buy? Any special specs needed/recommended?

[Milor123]

Any advice for deciding which NVIDIA GPU to buy? Any special specs needed/recommended?

Depends of you needs, or if you need it for a serius and hard work.

I've bought a simple rtx 4070 non Ti version with 12GB of VRAM and all works very nice (LLMs until 12B {It is better each a day with new efficient models}, images models, video models {I have very little experience in this field, probably a gpu with more vram will work better.}), however if you want use some highest models for clothing (not like ipadapter that works very poor for this task) change like IDM VTON that needs 16VRAM (Based in it I could recommend better a RTX 4070 ti super or more if you think use it in serius work)

But after to learn and work in somethings, I think that the best way to start is a 4070 ti super and if you have extra money, maybe a 4090

I also use it in LLM models using LMStudio and Anything LM

But definitely an NVIDIA card is much better than the other brand, which gives headaches (I even got to use ROCm in linux and still my user experience very bad).