Skip to content

Commit ce71d59

Browse files
authored
Fix a comment typo in flce (linkedin#256)
## Summary os huge -> is huge ## Testing Done - Hardware Type: <BLANK> - [ ] run `make test` to ensure correctness - [ ] run `make checkstyle` to ensure code style - [ ] run `make test-convergence` to ensure convergence
1 parent 58fd2bc commit ce71d59

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/liger_kernel/ops/fused_linear_cross_entropy.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ def fused_linear_cross_entropy_forward(
9797

9898
# gradient of logits_chunk is computed in-place by the above triton kernel.
9999
# Following HuggingFace model source code, we do the forward and backward
100-
# w.r.t. logits in fp32 for numerical stability especially as the num classes (vocab size) os huge.
100+
# w.r.t. logits in fp32 for numerical stability especially as the num classes (vocab size) is huge.
101101
# (reference: https://github.com/huggingface/transformers/blob/v4.42.4/src/transformers/models/llama/modeling_llama.py#L1194)
102102
# Propagating to lm_head's backward, we'll switch back to the original dtype.
103103
logits_chunk = logits_chunk.to(dtype)

0 commit comments

Comments
 (0)