Skip to content

ggml : add ggml_scale_bias #14417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Conversation

ngxson
Copy link
Collaborator

@ngxson ngxson commented Jun 27, 2025

Ref discussion: #14400 (comment)

Added ggml_scale_bias(ctx, a, s, b) in this PR, which allows calculating x = a * s + b

I only added Metal kernel for now, just for discussion. @ggerganov does this looks good to you?

TODO: support other backends

@ngxson
Copy link
Collaborator Author

ngxson commented Jun 27, 2025

I hope this won't have a significant impact on the performance

@github-actions github-actions bot added testing Everything test related ggml changes relating to the ggml tensor library for machine learning Apple Metal https://en.wikipedia.org/wiki/Metal_(API) labels Jun 27, 2025
Copy link
Member

@ggerganov ggerganov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think it's a useful extension of the operator.

Comment on lines +3968 to +3971
ggml_vec_scale_f32(nc, (float *) ((char *) dst->data + i1*nb1), s);
if (b != 0.0f) {
ggml_vec_acc1_f32(nc, (float *) ((char *) dst->data + i1*nb1), b);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge these in ggml_vec_mad1_f32(). If you want, you can try to add a GGML_SIMD version using GGML_F32_VEC_FMA - it's quite simple. But also can leave it a basic for loop without SIMD.

@ehoogeveen-medweb
Copy link

Quick question: Is the "scale-bias" nomenclature more appropriate here than "multiply-add"? From an outsider perspective familiar with fused multiply-add ("MAD") operations, I didn't realize that "scale" meant "multiply" and "bias" meant "add" until I took a closer look.

@ngxson
Copy link
Collaborator Author

ngxson commented Jun 29, 2025

multiply-add can be confused because we already had ggml_mul and ggml_add which takes 2 tensors as input

ggml_scale on the other hand, takes a tensor and a scalar value as input

So ggml_scale_bias is the best fit because the naming doesn't clash with "multiply" or "add". Bias meaning we add a scalar value, not a tensor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apple Metal https://en.wikipedia.org/wiki/Metal_(API) ggml changes relating to the ggml tensor library for machine learning testing Everything test related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants