[Bug]: strange repetition issue #788
Labels
bug
Something isn't working
Comments
|
Here is the same query, same model, run on tabbyapi: curl output: tabby output |
|
I will try to repro in TGI and vllm |
|
I've been trying to replicate this, with some success:
{
"id": "chat-93aaefa981c046d497a5699308ad094d",
"object": "chat.completion",
"created": 1729716396,
"model": "DolphinPod/dolphin-2.9.1-llama3.1-8b",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The sky is blue because the color blue is defined by the color of the blue sky is the the the the the the the is the.",
"tool_calls": []
},
"logprobs": null,
"finish_reason": "stop",
"stop_reason": null
}
],
"usage": {
"prompt_tokens": 14,
"total_tokens": 43,
"completion_tokens": 29
},
"prompt_logprobs": null
}
{
"id": "chat-0a08ca7350f74464a6f7a4d0c0ca420a",
"object": "chat.completion",
"created": 1729716528,
"model": "NousResearch/Meta-Llama-3.1-8B-Instruct",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The sky appears blue to us during the daytime because of a phenomenon called scattering. When sunlight enters Earth's atmosphere, it encounters tiny molecules of gases such as nitrogen and oxygen. These molecules scatter the shorter, blue wavelengths of light more than the longer, red wavelengths, due to a principle called Rayleigh scattering. This scattering makes the blue light more visible to our eyes, giving the sky its blue appearance.\n\nHere's a simplified explanation of the process:\n\n1. **Sunlight reaches the Earth**: Sunlight is composed of different colors, with each corresponding to a specific wavelength. The colors range from red (longer wavelengths) to blue (shorter wavelengths).\n\n2. **Scattering occurs**: When sunlight enters Earth's atmosphere, it encounters molecules of gases like nitrogen and oxygen. These molecules scatter or bounce off the light in different ways, depending on their size and the wavelength of light they encounter.\n\n3. **Rayleigh scattering**: The scattering of light by small molecules or particles is known as Rayleigh scattering. This is more pronounced for shorter wavelengths, which are in the blue part of the spectrum. Therefore, the blue light is scattered more than the other colors.\n\n4. **Scattered light reaches the observer**: Because blue light is scattered more, from the observer's perspective, more blue light reaches the eyes than other wavelengths. This results in the sky appearing blue.\n\nIt's worth noting that this effect is especially pronounced during the daytime, when the sun is high in the sky and its light has to travel through more of the atmosphere to reach the observer. The blue visibility can also be maximized with clearer skies, as clouds can scatter light in different ways, altering the apparent color of the sky.\n\nIn the context of different environmental and celestial conditions, the sky's color can be perceived differently. For example, during sunrise and sunset, the color of the sky can change or appear different due to the different angles at which the sun's light enters the atmosphere, scattering the light in various ways. At dawn and dusk, the sky often appears more red or orange because the light has to travel through a greater distance, scattering the shorter wavelengths of blue light and increasing the dominance of longer wavelengths like red and orange.",
"tool_calls": []
},
"logprobs": null,
"finish_reason": "stop",
"stop_reason": null
}
],
"usage": {
"prompt_tokens": 16,
"total_tokens": 461,
"completion_tokens": 445
},
"prompt_logprobs": null
}I'll try and see what's happening with this model in particular. |
|
Whoops sorry misinput. |
|
Any idea what might cause thit? |
|
@AlpinDale I would like to offer a $1,000 USD bounty for this issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Your current environment
The output of `python env.py`
```text Your output of `python env.py` here ```🐛 Describe the bug
There is a strange output that happens in aphrodite, but it doesn't happen in other engines such as tabbyapi.
curl output:
aphrodite output:
The text was updated successfully, but these errors were encountered: